quiz bu lëkkale ak tegtal · Dëgër Niveau

Parallelism bu xarañ ngir laaj ak tontu bu MoE

Test MoE plasement eksper, token dispatch ak charge imbalance, boole ci ni eksper yiñ tann ak jokkoo di amee ci inference buñ séddale.

Yooni njiit yu jëm ciParallelism Expert ngir Moe di liggéey
Laaj 1 de 8

Lan la parallelism expert bi di séddale ci GPU yi?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan la parallelism expert bi di séddale ci GPU yi?
    • Kopi model bi yépp ngir redondance
    • Bopp yiy bàyyi xel ci benn-benn nit kese
    • Xeeti feed-forward yu bari ci wàllu MoE
    • Layers ci tablo biy dugal
  2. Ban xeetu jokkoo mooy gëna am solo ci paralelismu kàngam yi ci MoE?
    • Ñaari weccoo lépp-ci-ñépp (yonnee ak boole)
    • Benn transmisioŋ
    • Sinkronisaasioŋu gradient poñ ci poñ
    • Amul benn jokkoo buñu soxla
  3. Lan moo waral MoE di tëye xayma ci token bu nekk doonte ay parametre yu bari lañu?
    • Dafay tënk diisaay yi ba 1 bit
    • Dafay teggi wàll yiñ wara bàyyi xel ci
    • Ekspeer yi bokk nañu seen poid yépp
    • Expert yiñ tànn ci kaw ñooy daw ci token bu nekk
  4. Ban jafe-jafe mooy xew ndax routing mingi aju ci done?
    • Degrade buy kalaate
    • Désequilibre de charge, fu yenn expert yi di gëna am jetons
    • Mënu tàggat routeur bi
    • Token yi dañuy ñàkk seen position
  5. Why are capacity limits and padding used in some MoE implementations?
    • To control expert buffer sizes and make computation shapes more predictable
    • To require every expert to process every token
    • To eliminate all token routing
    • To guarantee that every MoE system drops tokens
  6. naka la sistem yi di nëbbee latency jéego yépp-ci-lépp?
    • Sooy daw ci CPU
    • Soo dindi jéego boole
    • Yokk limu jàng
    • Ni ñuy jaxasoo jokkoo ak ordinatër eksper
  7. A routed token chooses two MoE experts on different devices. What must expert-parallel execution do?
    • Run every expert in the model regardless of routing
    • Dispatch its representation to the selected experts and combine their outputs
    • Replace both expert networks with an attention head
    • Drop the token solely because it selected two devices
  8. One MoE expert receives far more tokens than others. Why can the layer’s latency increase?
    • The total expert parameter count automatically doubles
    • The router’s outputs become attention positions
    • The overloaded expert becomes a straggler while other devices wait
    • All other experts are removed from memory