引導式測驗 · 硬 等級
集體交流和 NCCL 測驗
測試全歸約、全收集、廣播和歸約分散角色、NCCL 身分、網路卸載以及一個小型求和歸約範例。
問題 1 的 8
all-reduce 操作在分散式訓練中能完成什麼?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- all-reduce 操作在分散式訓練中能完成什麼?
- 它組合了所有 GPU 上的值,並為每個 GPU 提供組合結果
- 它從一個 GPU 中刪除冗餘數據
- 它降低 GPU 時脈速度以節省電量
- 它將資料從所有 GPU 傳送到單一儲存磁碟
- 縮寫 NCCL 代表什麼?
- 神經運算集群層
- 網路快取控制邏輯
- NVIDIA 集體通訊庫
- 數值卷積計算庫
- Why can chunked ring communication be useful for a large tensor?
- It guarantees constant latency for any number of devices
- Chunks can be streamed across participating links instead of waiting for one monolithic transfer
- It makes topology irrelevant
- It prevents all network contention
- 哪個集體操作將每個 GPU 的資料收集到所有 GPU 持有的完整副本?
- 廣播
- 減少分散
- 屏障
- 全員聚集
- What does a supported network reduction offload such as SHARP do?
- Guarantee exactly half the traffic for every collective
- Remove the need for participating ranks
- Perform some reduction work in compatible network hardware
- Turn all-gather into a local memory copy
- NCCL 通常使用哪種互連來進行伺服器(節點)之間的通訊?
- InfiniBand 或 RoCE
- USB 連接線
- 僅 PCIe 插槽
- 無線網路
- Three ranks contribute scalar values 2, 3 and 5 to a sum all-reduce. What value does each receive?
- Only its own original value
- 3 for the middle rank and nothing elsewhere
- The average 3.33 unless requested separately
- 10
- Which collective combines contributions but distributes reduced pieces instead of the entire reduced buffer to every rank?
- 全員聚集
- 減少分散
- 廣播
- A standalone model checkpoint