Mchanganyiko wa Maswali ya Wataalam

Majibu na maelezo huonekana unapojibu maswali.

1. Katika safu ya Mchanganyiko wa Wataalam, ni nini huamua ni wataalam gani wanashughulikia ishara fulani?
   • Kipanga njia kidogo kilichojifunza (mtandao wa lango) ambacho hupata wataalam
   • Jedwali la kuangalia lililowekwa kabla ya mafunzo
   • Urefu wa haraka wa mtumiaji
   • Mchoro wa nasibu kwa kila hatua
2. Kwa nini mfano wa MoE unaweza kuwa na vigezo vya jumla zaidi kuliko mfano mnene bila ongezeko linalolingana la gharama kwa kila ishara?
   • Inasisitiza vigezo katika uzito mmoja ulioshirikiwa
   • Inaruka tabaka za tahadhari kabisa
   • Wataalam wachache tu (juu-k) wameamilishwa kwa kila ishara
   • Inaendesha kila mtaalam lakini wastani wa matokeo
3. Je, ni tatizo gani linaloshughulikia upotevu wa kusawazisha mzigo (msaidizi) wakati wa mafunzo ya MoE?
   • Ishara kuwa ndefu sana kwa dirisha la muktadha
   • Kipanga njia kinawatumia wataalam wachache na kuwafadhaisha wengine
   • Gradients kutoweka katika vichwa vya tahadhari
   • Kutosha kwa seti ya uthibitishaji
4. Mixtral 8x7B huwasha wataalam wake 2 kati ya 8 kwa kila tokeni. Je, hii inaashiria nini kuhusu hesabu yake dhidi ya saizi yake?
   • Inatumia hesabu zote za wataalam 8 kwenye kila tokeni
   • Ina vigezo vichache vya jumla kuliko mfano mnene wa 7B
   • Haiwezi kukimbia kwenye GPU hata kidogo
   • Kokotoo lake amilifu kwa tokeni ni ndogo sana kuliko hesabu yake ya jumla ya vigezo inavyopendekeza
5. Why can a sparse MoE still have substantial memory or storage requirements?
   • Every token necessarily activates all experts
   • Inactive experts cease to have parameters
   • Inactive expert weights still need accessible storage even when not selected for a token
   • The router removes all attention layers
6. Two selected MoE expert outputs are 2 and 6, with combination weights 0.25 and 0.75. What is their weighted output?
   • 5
   • 8
   • 2
   • 0.5
7. Increasing an MoE’s expert count keeps the same top-k and expert size. What can still increase serving cost?
   • The selected expert count must automatically become the total count
   • The model must stop using attention
   • Every inactive weight is automatically deleted
   • Router or communication overhead and the storage required for more experts
8. A sparse MoE calls a learned feed-forward subnetwork an expert. What does that name establish?
   • It is certified to answer only one academic discipline
   • It is one of the networks selected by routing, without proving a named subject specialty
   • It must be a human-written collection of rules
   • It contains no learned parameters