Kuis terkait panduan · Sedang Tingkat

Gating dan Routing dalam Kuis Komputasi Bersyarat

Menguji perutean pakar yang bergantung pada input, komputasi aktif, distribusi beban token, kapasitas pakar, luapan, dan komputasi total versus komputasi yang dieksekusi.

Jalur panduan terkaitGating Dan Routing Dalam Perhitungan Bersyarat
Pertanyaan 1 dari 8

What is conditional computation?

Pratinjau setiap pertanyaan8 pertanyaan

Jawaban dan penjelasan muncul saat Anda mengikuti kuis.

  1. What is conditional computation?
    • The network selects which submodules to activate based on the input
    • Every network layer runs identically for every input
    • The model selects its training dataset after inference
    • A human routes every token manually
  2. In a mixture-of-experts layer, what does the router do?
    • Trains the ground-truth labels
    • Combines all parameters into one dense layer
    • Scores experts and sends an input or token to selected experts
    • Removes the expert outputs
  3. What is active parameter count?
    • The number of parameters stored on disk
    • The parameters used for a particular input, which can be fewer than total model parameters
    • The number of experts in the model
    • The count of training examples per expert
  4. Why can routing be computationally efficient?
    • The router removes all dense computations
    • Experts do not contain parameters
    • The model skips every output layer
    • Only selected experts process a given input rather than every expert
  5. What is expert load imbalance?
    • Every expert receives exactly the same number of tokens
    • The router has no parameters
    • Some experts receive many tokens while others receive few
    • Tokens are sorted by output length
  6. Why add a load-balancing objective?
    • To encourage a more even distribution of tokens among experts
    • To increase the total parameter count
    • To force every token through all experts
    • To make routing independent of input
  7. What can happen when an expert reaches a capacity limit?
    • The model automatically doubles the expert’s weights
    • Every other expert is disabled
    • The token becomes a training label
    • Tokens may be dropped or routed according to the system’s overflow policy
  8. Does a sparse model’s total parameter count equal its per-token compute?
    • Yes, every stored parameter runs on each token
    • No; only selected experts are active for a particular input
    • No; sparse models contain no parameters
    • Yes, if a router is used