Batch Normalization Quiz

Answers and explanations appear as you take the quiz.

1. What does batch normalization normalize?
   • The inputs to a layer, across the examples in the current mini-batch
   • The final output labels of the network
   • The learning rate schedule over time
   • The size of the training dataset
2. Why does batch normalization include learnable parameters gamma and beta?
   • To increase the size of the dataset
   • To let the network scale and shift the normalized values, preserving representational power
   • To randomly drop neurons during training
   • To compute the loss function
3. What is a commonly cited practical benefit of batch normalization?
   • It eliminates the need for any training data
   • It guarantees the model cannot overfit
   • It allows higher learning rates and faster, more stable convergence
   • It removes the need for an activation function
4. At inference time (predicting on new data), what statistics does batch normalization use?
   • The mean and variance of the current single input only
   • Randomly generated statistics each time
   • The statistics of the test set computed in advance
   • Running averages of mean and variance accumulated during training
5. Why can batch normalization behave poorly with very small batch sizes?
   • Batch statistics become noisy and unreliable estimates of the true distribution
   • Small batches make the model too large to fit in memory
   • It disables the activation functions
   • Small batches force the use of L1 regularization
6. Which normalization technique is most commonly used in transformer architectures as an alternative to batch normalization?
   • Dropout normalization
   • Layer normalization
   • L2 normalization of the loss
   • Gradient clipping
7. Why does BatchNorm use running averages during inference instead of the current test batch’s statistics?
   • To retrain the model on test data
   • To make all activations exactly zero
   • So a prediction does not depend on which other examples happen to share the batch
   • To eliminate learned gamma and beta
8. Where is BatchNorm commonly placed within a layer block according to the guide?
   • Only after the final prediction
   • Before the input data are loaded
   • Inside the optimizer update
   • Between the linear transformation and its activation function