quiz bu lëkkale ak tegtal · Dëgër Niveau

Model ak gasoduk paralelism Quiz

Dafay leeral xaajale tensor ak gasoduc, toxal aktivasioŋ, oraaru microbatch, bubble gasoduc, ak kompromis jokkoo.

Yooni njiit yu jëm ciModel ak paralelismu tuyo
Laaj 1 de 8

Ban jafe-jafe bu am solo la paralelism model ak pipeline di saafara?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Ban jafe-jafe bu am solo la paralelism model ak pipeline di saafara?
    • Benn model bu yaatu lool ba mënu ci nekk ci benn mémoire GPU
    • Done yuy yab ci disk bi yeex
    • Degrade yi dañuy kalaate ci diiru tàggat yaram
    • misaali tàggat yaram yu néew
  2. naka la ko paralelism tensor (ci biir couche) di doxee?
    • Dafay jox ay layer yu lëmm ay GPU yu wuute
    • Dafay xaaj math bi ci biir benn couche, melni matrix buy yokk, ci GPU yi
    • Dafay ñaari yoon layer bu nekk ci GPU bu nekk
    • Dafay dox ñaari yoon ci couche bu nekk ngir redondance
  3. Luy 'bubble' bi ci paralelismu gasoduc bu xamul dara?
    • Memoire leak buy màgg lu jamono di dox
    • Ñaari kopi bu staadu optimisatër bi
    • Jamonoy inaktif bi GPU yi ci suuf di xaar etape yi ci kaw ngir génne ay aktivaasioŋ
    • Yokkateg valeur gradient yi
  4. naka lay def ba paralelism ci gasoduc yi wàññi bubble bi?
    • Jëfandikoo taxawaayu jàng bu gëna ndaw
    • Soo dindi yenn couche ci model bi
    • Sooy denc mbir yi ci CPU bi
    • Ni ñuy xaajalee lote bu nekk ci ay micro-lote yuy jaar ci etape yi ci anam wu xaajaloo
  5. Lan moo waral ñuy denc paralelism tensor ci benn node?
    • Ndax limuy faral di wàññi lépp ci njariñu lëkkalekaay yu gaaw yi ci biir node yu melni NVLink
    • Ndax soxlawul benn jokkoo
    • Ndax node yi mënu ñu séddoo ay done
    • Ndax benn GPU kese lay jàppale
  6. Ban kaadar lañu gëna xamee ci paralelism tensor ci tàggat transformatër?
    • scikit-jàng
    • Megatron-LM
    • OpenCV
    • Panda
  7. What data must move between stages in pipeline parallelism during a forward pass?
    • The entire training dataset after each layer
    • Only the optimizer’s learning rate
    • Activation tensors produced by one stage and consumed by the next
    • No data, because stages are independent models
  8. Why does tensor parallelism often require fast communication within its group?
    • It assigns each GPU a separate dataset and never synchronizes
    • It communicates only once when the model is loaded
    • It uses no cross-device operations
    • Devices exchange partial results or activations within layers during computation