ML Knowledge

When considering neural network training, do vanishing gradients typically happen near the input layers or the output layers?

Machine Learning Engineer

Snap

Dropbox

Microsoft

Whatsapp

Wayfair

Pinterest

Did you come across this question in an interview?

  • When considering neural network training, do vanishing gradients typically happen near the input layers or the output layers?
  • Within a neural network, are vanishing gradients more prone to occur near the beginning or the end of the network?
  • In the operational sequence of a neural network, is the vanishing gradient issue more common at the beginning or the end?
  • Regarding neural network layers, where does the vanishing gradient issue predominantly take place: at the start or the finish?
  • In neural networks, are the initial layers or the deeper layers more often affected by the problem of vanishing gradients?
  • For neural networks, is the vanishing gradient concern more frequent towards the network's input or its output?
  • Is the vanishing gradient phenomenon a more significant issue at the start or at the end of a neural network's layers?
  • At which point in a neural network do vanishing gradients most often present a problem: early on or closer to the output?
  • In the layout of a neural network, are the first layers or the final layers more at risk of experiencing vanishing gradients?
  • Does the phenomenon of vanishing gradients affect the initial stages or the later stages of a neural network's progression?
  • In the context of neural networks, does the issue of vanishing gradients occur closer to the start of the network or towards the end?

Interview question asked to Machine Learning Engineers interviewing at Microsoft, Peek, SAP Concur and others: When considering neural network training, do vanishing gradients typically happen near the input layers or the output layers?.