Gating and Routing in Conditional Computation Quiz

Answers and explanations appear as you take the quiz.

1. What is conditional computation?
   • The network selects which submodules to activate based on the input
   • Every network layer runs identically for every input
   • The model selects its training dataset after inference
   • A human routes every token manually
2. In a mixture-of-experts layer, what does the router do?
   • Trains the ground-truth labels
   • Combines all parameters into one dense layer
   • Scores experts and sends an input or token to selected experts
   • Removes the expert outputs
3. What is active parameter count?
   • The number of parameters stored on disk
   • The parameters used for a particular input, which can be fewer than total model parameters
   • The number of experts in the model
   • The count of training examples per expert
4. Why can routing be computationally efficient?
   • The router removes all dense computations
   • Experts do not contain parameters
   • The model skips every output layer
   • Only selected experts process a given input rather than every expert
5. What is expert load imbalance?
   • Every expert receives exactly the same number of tokens
   • The router has no parameters
   • Some experts receive many tokens while others receive few
   • Tokens are sorted by output length
6. Why add a load-balancing objective?
   • To encourage a more even distribution of tokens among experts
   • To increase the total parameter count
   • To force every token through all experts
   • To make routing independent of input
7. What can happen when an expert reaches a capacity limit?
   • The model automatically doubles the expert’s weights
   • Every other expert is disabled
   • The token becomes a training label
   • Tokens may be dropped or routed according to the system’s overflow policy
8. Does a sparse model’s total parameter count equal its per-token compute?
   • Yes, every stored parameter runs on each token
   • No; only selected experts are active for a particular input
   • No; sparse models contain no parameters
   • Yes, if a router is used