Akopọ Communication ati NCCL adanwo

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Kini iṣẹ-ṣiṣe idinku gbogbo ṣe aṣeyọri ni ikẹkọ pinpin?
   • O daapọ awọn iye kọja gbogbo awọn GPUs ati fun gbogbo GPU ni abajade apapọ
   • O npa data laiṣe lati GPU kan
   • O dinku iyara aago GPU lati fi agbara pamọ
   • O firanṣẹ data lati gbogbo awọn GPU si disk ibi ipamọ kan
2. Kini adape NCCL duro fun?
   • Ohunkikan Iṣiro Iṣupọ Layer
   • Network kaṣe Iṣakoso kannaa
   • NVIDIA Collective Communications Library
   • Nomba Convolution Compute Library
3. Why can chunked ring communication be useful for a large tensor?
   • It guarantees constant latency for any number of devices
   • Chunks can be streamed across participating links instead of waiting for one monolithic transfer
   • It makes topology irrelevant
   • It prevents all network contention
4. Iṣe apapọ wo ni o ṣajọ nkan data GPU kọọkan sinu ẹda kikun ti o waye nipasẹ gbogbo awọn GPU?
   • Igbohunsafefe
   • Din-tuka
   • Idena
   • Gbogbo-kó
5. What does a supported network reduction offload such as SHARP do?
   • Guarantee exactly half the traffic for every collective
   • Remove the need for participating ranks
   • Perform some reduction work in compatible network hardware
   • Turn all-gather into a local memory copy
6. Isopọmọra wo ni NCCL maa n lo fun ibaraẹnisọrọ LARIN awọn olupin (awọn apa)?
   • InfiniBand tabi RoCE
   • Okun USB
   • Iho PCIe nikan
   • Wi-Fi
7. Three ranks contribute scalar values 2, 3 and 5 to a sum all-reduce. What value does each receive?
   • Only its own original value
   • 3 for the middle rank and nothing elsewhere
   • The average 3.33 unless requested separately
   • 10
8. Which collective combines contributions but distributes reduced pieces instead of the entire reduced buffer to every rank?
   • Gbogbo-kó
   • Din-tuka
   • Igbohunsafefe
   • A standalone model checkpoint