Weight Initialization Quiz

Answers and explanations appear as you take the quiz.

1. Why is initializing all weights to zero a fatal mistake?
   • All neurons receive identical gradients and never differentiate (symmetry is not broken)
   • It makes training too fast
   • It causes the loss to become negative
   • It uses too much memory
2. He (Kaiming) initialization is specifically designed for which activation function?
   • Sigmoid
   • ReLU
   • Tanh
   • Softmax
3. What is the main goal of principled weight initialization schemes?
   • To make all weights equal
   • To set every weight to one
   • To keep the variance of activations and gradients roughly constant across layers
   • To eliminate the need for data
4. Xavier (Glorot) initialization is best suited for networks using which activations?
   • ReLU and its variants
   • Only linear activations
   • No activation at all
   • Tanh and sigmoid
5. Why does He initialization use a variance of 2 / fan_in rather than 1 / fan_in?
   • To compensate for ReLU setting roughly half of its inputs to zero
   • To make weights larger for no reason
   • To reduce the number of parameters
   • To match the batch size
6. How are biases typically initialized when weights are randomly initialized?
   • To very large positive values
   • To zero, since symmetry is already broken by the random weights
   • To the same value as the learning rate
   • To random values larger than the weights
7. Under Xavier initialization, which quantities determine the weight variance in the guide’s formula?
   • The number of training epochs and batch size
   • The output class count alone
   • Fan-in and fan-out
   • The activation’s test accuracy
8. For a ReLU layer with fan-in 64, what variance does the He formula 2/fan_in specify?
   • 1/64
   • 1/16
   • 2/64 squared
   • 1/32