Isbarbardhigga taxanaha iyo Kediska Feejignaanta giraanta

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

1. Dhinacgee cabbirka isbarbar yaaca isku xigxiga u qaybiyay xogta?
   • Cabirka qarsoon/muuqaalka
   • Calaamadda (isku xigxiga) cabbirka
   • Cabbirka dufcadda kaliya
   • Tirada lakabyada
2. Maxay Fiiro gaar ah u leedahay inta u dhaxaysa GPU-yada lagu habeeyay giraanta?
   • Miisaanka model buuxa
   • Optimizer states
   • Logits kama dambaysta ah
   • Qaybaha furaha/qiimaha
3. How can Ring Attention combine partial attention blocks without approximating the full attention pattern?
   • Online softmax with a running maximum and normalizer
   • Randomly dropping unseen key blocks
   • Truncating every distant key
   • Replacing all attention scores with zeros
4. Waa maxay sababta aanu garaaca garaaca uga baahnayn hal GPU si uu u kaydiyo K/V buuxa?
   • Waxay tuurtaa furayaasha intooda badan
   • Waxay isticmaashaa nooc ka yar
   • K/V waxaa lagu shubaa baloogyo waxaana la ururiyaa natiijooyinka qayb ahaan
   • Feejignaanta waa la qiyaasaa, maaha mid sax ah
5. When can Ring Attention reduce the exposed time spent transferring K/V blocks?
   • Whenever the model has any number of parameters
   • Only when every device stores the full sequence
   • By eliminating all transfers between devices
   • When useful block computation overlaps those transfers
6. Waa maxay dhibka ay sababtu u soo bandhigto isku dheeli tirka culayska feejignaanta
   • Waxay ka dhigaysa moodelka mid aan kala duwanayn
   • Calaamadaha dambe waxay galaan furayaal badan, taasoo keenaysa shaqo aan sinnayn halkii GPUba
   • Waxay meesha ka saaraysaa baahida giraanta
   • Waxay labanlaabtaa tirinta cabbirka
7. A new Ring Attention block has larger attention scores than previous blocks. Why rescale the earlier softmax sums?
   • To discard the earlier keys permanently
   • To replace all token embeddings
   • To keep accumulated terms consistent with the updated running maximum
   • Si loo yareeyo cabbirka ereyada
8. Under causal masking with contiguous sequence shards, which tokens can require more attention work?
   • Later tokens that can attend to more preceding positions
   • All tokens require exactly one key
   • Only padding tokens at the start
   • Earlier tokens because they can see the future