概述
Offloaded bulk data movement can reduce CPU copying and kernel-path overhead; CPUs and software still configure, initiate and manage the communication.
深入探討
Distributed AI training can spend substantial time moving gradients and other tensors between devices and servers. InfiniBand provides a switched fabric with RDMA support and link-level flow control. NDR denotes a 400 Gb/s link mode in the cited NVIDIA hardware documentation; application throughput and latency depend on messages, adapters, topology and workload. RDMA-capable adapters can move bulk data to registered buffers without the repeated host copying of a conventional path. This does not mean software or CPUs have no role: applications create communication objects, register buffers and post work. RoCE carries RDMA over Ethernet, with its own network-design requirements. Select and benchmark a fabric using the actual collective traffic, congestion and failure conditions.
現實世界的實施
在人工智慧超級電腦中連接數千個 GPU,以便梯度資料在分散式訓練期間以微秒的速度在節點之間移動
讓一台伺服器直接讀取另一台伺服器的記憶體 (RDMA) 以加速分散式檔案系統和資料庫,而無需 CPU 開銷
透過 InfiniBand 運行 NCCL all-reduce 操作以跨 GPU 叢集同步模型權重
使用 RoCE 將 RDMA 式低延遲傳輸引入現有乙太網路資料中心網絡
InfiniBand 和 RDMA 網路的未來
Interconnects and adapters continue to evolve, and vendors offer different bandwidth modes and offloads. Teams should compare measured throughput, latency, CPU work, congestion handling and reliability under their own workload. In-network reductions and closer accelerator integration may help particular configurations, but a headline link rate does not determine training performance by itself. Keep hardware, firmware and software versions with benchmark results.
付諸實踐
為什麼這很重要
- 成本與預算. 多年來,架構決策決定著效能和營運成本。
- 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
- 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。
留意
- 優化一項基準測試可以隱藏更廣泛的系統弱點。
- 基礎設施和維護成本常常被低估。
- 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。
後續步驟
- 在實施之前定義延遲、品質和成本目標。
- 在實際負載和資料條件下進行基準測試。
- 儀器監控錯誤、漂移和使用者影響。
- 在擴展之前準備回滾和事件回應路徑。
不斷探索
常見問題
什麼是 InfiniBand 和 RDMA 網路?
InfiniBand is a high-speed switched interconnect used in clusters, and RDMA supports direct transfers between registered memory buffers. Offloaded bulk data movement can reduce CPU copying and kernel-path overhead; CPUs and software still configure, initiate and manage the communication.
What does an RDMA-capable bulk data path aim to reduce?
RDMA offloads data movement but software still sets up and manages communication.
How can an RDMA-capable InfiniBand data path reduce bulk-transfer overhead?
RDMA hardware can move data between prepared buffers with less repeated host copying and kernel-path work. Software still handles setup, registration, permissions, and completion processing.
NDR InfiniBand 提供的每鏈路頻寬大致是多少?
A four-lane NDR400 InfiniBand link is rated at 400 Gb/s. This link-rate figure does not guarantee an application’s effective transfer throughput.
在 RDMA 中,「隊列對」有什麼用?
應用程式將工作請求發佈到發送和接收佇列(佇列對);主機通道適配器讀取它們並執行直接記憶體傳輸。
什麼是RoCE?
RoCE 代表融合乙太網路上的 RDMA,允許在標準乙太網路結構(而不是本機 InfiniBand)上進行低延遲、核心旁路傳輸。