Activation Functions Quiz

Answers and explanations appear as you take the quiz.

1. Why can stacking only linear layers not learn a curved decision boundary?
   • Each layer deletes all input features
   • The output becomes a probability distribution
   • Linear layers cannot be trained with backpropagation
   • Their composition is still a single linear transformation
2. What is the output of ReLU for an input of −3?
   • −3
   • 0
   • 3
   • A class probability
3. Why can sigmoid and tanh stall learning in deep networks when inputs reach their extremes?
   • Their gradients approach zero in saturated regions
   • They force every weight to be identical
   • They cannot represent bounded values
   • They always produce an undefined output
4. A multiclass classifier needs output values that sum to one across classes. Which function does the guide use?
   • Leaky ReLU
   • Tanh
   • Softmax
   • ReLU
5. What does the “dying ReLU” problem mean in the guide?
   • The training dataset loses labels
   • A unit stays on its zero-output side and receives no useful gradient there
   • The softmax outputs fail to sum to one
   • All positive inputs become negative
6. Which variant is suggested when many ReLU units stay inactive?
   • Softmax
   • A purely linear hidden layer
   • Sigmoid at every hidden layer
   • Leaky ReLU, which allows a small negative-side response
7. What feature of GELU does the guide associate with its use in transformer models?
   • It removes all nonlinear behavior
   • It always yields exact class probabilities
   • A smooth response rather than ReLU’s hard cutoff
   • It fixes every training instability
8. For positive inputs, what ReLU gradient property helps preserve a backpropagated signal?
   • Its slope is 1 on the positive side
   • Its slope is always zero
   • It has no derivative anywhere
   • It normalizes gradients to class probabilities