Guide-yakabatana mibvunzo · Zvakaoma Nhanho

Nyanzvi Parallelism yeMoE Kushandira Quiz

Edzai MoE nyanzvi yekuisa, kutumira tokeni uye kusaenzana kwemutoro, kusanganisira kuti nyanzvi dzakasarudzwa uye kutaurirana zvinokanganisa sei kugoverwa.

Nzira dzekutungamira dzinoenderanaNyanzvi Parallelism YeMoe Kushumira
Mubvunzo 1 ye 8

Chii chinonzi parallelism nyanzvi inogovera mhiri kweGPUs?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinonzi parallelism nyanzvi inogovera mhiri kweGPUs?
    • Makopi eiyo yakazara modhi yedundancy
    • Misoro yekutarisa yega yega
    • Nyanzvi dzakasiyana dzekudyisa-mberi dzeMoE layer
    • Matanho etafura yekumisikidza
  2. Ndeipi nzira yekutaurirana iri pakati peMoE nyanzvi parallelism?
    • Kuchinjana kuviri-kune-kwese (kutumira uye kusanganisa)
    • Nhepfenyuro imwe chete
    • Kuwiriranisa kwenzvimbo-kune-inongedzo
    • Hapana kutaurirana kunodiwa
  3. Nei MoE ichichengeta per-token compute yakaderera kunyangwe yakakura yakazara paramita?
    • Inomanikidza uremu kusvika ku1 bit
    • Inodarika mitsara yekucherechedza
    • Vanamazvikokota vanogovera uremu hwavo hwose
    • Chete chepamusoro-k nyanzvi dzakasarudzwa dzinomhanya pane tokeni
  4. Ndeipi dambudziko rinomuka nekuti nzira inotsamira data?
    • Gradient kuputika
    • Kusaenzana kwemutoro, uko dzimwe nyanzvi dzinowana ma tokeni akawanda
    • Iyo router haigone kudzidziswa
    • Tokeni dzinorasikirwa nenzvimbo yadzo yekumisikidza
  5. Why are capacity limits and padding used in some MoE implementations?
    • To control expert buffer sizes and make computation shapes more predictable
    • To require every expert to process every token
    • To eliminate all token routing
    • To guarantee that every MoE system drops tokens
  6. Masisitimu anoviga sei latency yezvese-kune-zvese matanho?
    • Nekumhanya paCPU pachinzvimbo
    • Nekubvisa danho rekubatanidza
    • Nokuwedzera mwero wekudzidza
    • Nekupindirana kutaurirana nenyanzvi computing
  7. A routed token chooses two MoE experts on different devices. What must expert-parallel execution do?
    • Run every expert in the model regardless of routing
    • Dispatch its representation to the selected experts and combine their outputs
    • Replace both expert networks with an attention head
    • Drop the token solely because it selected two devices
  8. One MoE expert receives far more tokens than others. Why can the layer’s latency increase?
    • The total expert parameter count automatically doubles
    • The router’s outputs become attention positions
    • The overloaded expert becomes a straggler while other devices wait
    • All other experts are removed from memory