引导式测验 · 中等 等级

条件计算测验中的门控和路由

测试依赖于输入的专家路由、主动计算、令牌负载分布、专家容量、溢出以及总计算与执行计算。

相关引导路径条件计算中的门控和路由
问题 1 的 8

What is conditional computation?

预览每个问题8 问题

当您参加测验时，就会出现答案和解释。

  1. What is conditional computation?
    • The network selects which submodules to activate based on the input
    • Every network layer runs identically for every input
    • The model selects its training dataset after inference
    • A human routes every token manually
  2. In a mixture-of-experts layer, what does the router do?
    • Trains the ground-truth labels
    • Combines all parameters into one dense layer
    • Scores experts and sends an input or token to selected experts
    • Removes the expert outputs
  3. What is active parameter count?
    • The number of parameters stored on disk
    • The parameters used for a particular input, which can be fewer than total model parameters
    • The number of experts in the model
    • The count of training examples per expert
  4. Why can routing be computationally efficient?
    • The router removes all dense computations
    • Experts do not contain parameters
    • The model skips every output layer
    • Only selected experts process a given input rather than every expert
  5. What is expert load imbalance?
    • Every expert receives exactly the same number of tokens
    • The router has no parameters
    • Some experts receive many tokens while others receive few
    • Tokens are sorted by output length
  6. Why add a load-balancing objective?
    • To encourage a more even distribution of tokens among experts
    • To increase the total parameter count
    • To force every token through all experts
    • To make routing independent of input
  7. What can happen when an expert reaches a capacity limit?
    • The model automatically doubles the expert’s weights
    • Every other expert is disabled
    • The token becomes a training label
    • Tokens may be dropped or routed according to the system’s overflow policy
  8. Does a sparse model’s total parameter count equal its per-token compute?
    • Yes, every stored parameter runs on each token
    • No; only selected experts are active for a particular input
    • No; sparse models contain no parameters
    • Yes, if a router is used