技術指南

關鍵批次大小 (Critical Batch Size)

Critical batch size describes a transition in training efficiency: beyond it, larger batches give diminishing reductions in the number of updates needed to reach a target.

聆聽此指南AI 語音敘述 · ~3 min

本頁閱讀時間3分鐘
  1. 概述
  2. 深入探討
  3. 現實世界的實施
  4. The Future of Critical Batch Size
  5. 付諸實踐
  6. 不斷探索
  7. 常見問題

概述

It is a property of a training problem and performance target, not simply the largest batch that fits in accelerator memory. Choosing a batch also requires measuring elapsed time, total processed examples, and final task performance.

深入探討

A batch is the set of training examples used to form an update. Increasing its size averages over more examples and can reduce gradient noise. With enough parallel hardware, a larger batch may also reduce how many sequential updates are needed to achieve a chosen performance level. But those improvements do not continue proportionally forever. Critical batch size marks a characteristic transition in this tradeoff. In the empirical model of McCandlish and colleagues, batches well below the critical scale are relatively efficient per example. Far above it, adding examples to each update brings diminishing reductions in the number of updates. This is a smooth change in efficiency, not a universal hard limit beyond which training cannot work. The research connects the transition to gradient noise scale and finds that it can shift as training progresses. Define a common target before comparing runs. For an invented example, batch 256 reaches that target in 4,000 updates, processing 1,024,000 examples. Batch 512 takes 2,200 updates but processes 1,126,400 examples. It saves 45% of the updates while using 10% more examples. That could be an attractive elapsed-time tradeoff, but only measured update times reveal the runtime benefit. These two observations alone do not identify a precise critical batch size. Sweep several global batch sizes and tune relevant optimizer settings fairly. Record examples or tokens, updates, elapsed time, and validation performance at the same target. Keep track of whether a learning-rate schedule is expressed in updates or processed data, since changing the batch changes that relationship. Memory capacity, communication, and kernel efficiency impose additional practical limits that the statistical concept alone does not describe.

現實世界的實施

In a hypothetical experiment, batch 256 needs 4,000 updates to reach a target, while batch 512 needs 2,200. The second run uses fewer updates but processes 1,126,400 examples rather than 1,024,000.

A team doubles its batch again but sees almost no reduction in updates to the same target. It checks whether additional parallel work is providing useful optimization progress.

A researcher repeats a batch-size sweep later in training rather than assuming the best early-training batch remains best near the target loss.

Two configurations use the same global batch but different numbers of devices. The team compares elapsed time because communication and per-update execution can differ.

The Future of Critical Batch Size

Adaptive batch schedules may become easier to evaluate as training tools expose gradient statistics and progress measurements. Their value still needs evidence on the actual optimizer, model, and data rather than a borrowed threshold from another workload. Teams can keep a small set of batch experiments alongside learning-rate studies, revisit the choice when the training phase changes, and report both resource use and time to target. A reproducible comparison should preserve the target definition and include unsuccessful configurations so the apparent benefit is not based only on a selected run.

付諸實踐

為什麼這很重要

  • 成本與預算. 多年來,架構決策決定著效能和營運成本。
  • 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
  • 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。

留意

  • 優化一項基準測試可以隱藏更廣泛的系統弱點。
  • 基礎設施和維護成本常常被低估。
  • 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。

後續步驟

  1. 在實施之前定義延遲、品質和成本目標。
  2. 在實際負載和資料條件下進行基準測試。
  3. 儀器監控錯誤、漂移和使用者影響。
  4. 在擴展之前準備回滾和事件回應路徑。

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

Take the Critical Batch Size quiz

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

What is Critical Batch Size?

Critical batch size describes a transition in training efficiency: beyond it, larger batches give diminishing reductions in the number of updates needed to reach a target. It is a property of a training problem and performance target, not simply the largest batch that fits in accelerator memory. Choosing a batch also requires measuring elapsed time, total processed examples, and final task performance.

哪些行為表徵了遠遠超出訓練問題的關鍵批次規模的批次?

臨界規模描述了大批量的最佳化回報遞減,而不是普遍無法訓練。

為什麼記憶體中適合的最大批次不一定是臨界批次大小?

關鍵批量大小關係到訓練效率,而記憶體容量是一個單獨的實際限制。

在工作比較中,將批次從 256 個變更增加到 512 個變更需要將更新從 4,000 次增加到 2,200 次。處理後的範例會發生什麼?

儘管需要的更新較少,但較大的批次可處理 512 × 2,200 = 1,126,400 個範例。

在得出較少的訓練更新可以節省運行時間的結論之前,還需要進行哪些測量?

更新可能會因批次大小和設備排列而花費不同的時間。

所引用的大量研究與最大有用批次範圍相關的統計數據是什麼?

該研究使用梯度雜訊尺度作為臨界批次範圍的經驗預測器。