Isbarbardhigga Tensor-ka ee Modelyada Waaweyn Kediska

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

1. Waa maxay isbarbardhigga tensor-ku u kala qaybsamaan GPU-yada?
   • Miisaanku waxa uu isku koobaa hal lakab
   • Lakabyada oo dhan ayaa lagu dhejiyay GPU-yo kala duwan
   • Xogta tababarka kala duwan
   • Heerka waxbarashada wanaajinta
2. Kala qaybsanaanta qaabka Megatron ee MLP, sidee loo qaybiyaa labada miisaan miisaan si loo yareeyo isgaarsiinta?
   • Labada tiir-caqli leh
   • Tiirka hore-caqli leh, safka labaad-xikmad leh
   • Labada saf-caqli
   • Haba yaraatee lama kala qaybin
3. Waa maxay sababta isbarbardhigga tensor-ka badanaa loogu hayaa hal nood?
   • Nodes ma socodsiin karaan GPU-yada
   • GPU-yada qanjidhada kala duwan waxay isticmaalaan xisaab kala duwan
   • Waxay u baahan tahay isgaarsiin joogto ah oo badwidh-sare ah oo ay sida ugu fiican ugu adeegto NVLink
   • Waa shuruud sharci
4. Sidee habka feejignaanta caadi ahaan barbar socdaa isbarbardhigga tensor?
   • Adigoo siinaya GPU kasta xisaabinta dareenka buuxda si aan xad lahayn
   • Adigoo gabi ahaanba ka boodaya dareenka
   • Adiga oo dareenka u rogaya isbedbeddel
   • Adiga oo u kala qaybinaya madaxa dareenka, GPU kastaa wuxuu gacanta ku hayaa qayb hoose
5. Waa maxay hawlgalka wadajirka ah ee caadi ahaan isku dara natiijooyinka qayb ahaan isbarbardhigga tensor?
   • Dhammaan-dhimista
   • Nuqul fayl
   • Ku biir database
   • Raadinta DNS
6. Sidee isbarbar yaaca tensor caadi ahaan ugu habboon yahay 'isbarbardhigga 3D'?
   • Waxay beddeshaa dhammaan noocyada kale ee isbarbardhigga
   • Waxaa lagu daraa dhuumaha iyo isbarbardhigga xogta
   • Waxa kaliya oo ay shaqeysaa iyada oo aan wax isbarbar-dhig kale ahi jirin
   • Waxa loo isticmaalaa oo keliya moodooyinka sawirka
7. After the row-wise partition of an MLP’s second matrix, why must GPUs combine partial outputs?
   • The GPUs each hold a different optimizer
   • The result is a vocabulary probability that must be sorted
   • Each GPU computed a partial contribution to the same output, so the pieces must be summed
   • The model needs to average separate datasets
8. Why can interconnect latency limit tensor-parallel scaling across distant nodes?
   • Tensor parallelism never communicates between GPUs
   • The attention mask must be copied after every token
   • Every GPU trains a separate model
   • Collectives are required repeatedly within layers, so communication latency accumulates