技術指南

Mixup 與 CutMix 增強

Mixup 和 CutMix 是資料增強技術,它們讓神經網路學習兩個樣本的混合,而非單一範例。

聆聽此指南AI 語音敘述 · ~8 min

本頁閱讀時間 9 分鐘
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. 手動混合一個 CIFAR-10 批次
  5. 現實世界的實施
  6. Mixup 和 CutMix 增強的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

Mixup 對兩張完整影像及其標籤進行平均;CutMix 將一張影像中的矩形貼到另一張影像上,並根據面積混合標籤。兩者幾乎不增加成本,能減少過擬合,且是現代影像分類器訓練流程中的標準做法。

重點摘要

  1. Mixup 透過對兩張影像及其標籤進行加權平均來訓練,混合權重取自由 alpha 控制的 Beta 分佈。
  2. CutMix 將一張影像中的隨機矩形貼到另一張影像上,並根據每張影像的可見面積設定標籤權重。
  3. 在 CutMix 論文中,ResNet-50 在 ImageNet 上的 top-1 錯誤率從 23.68% 降至 21.40%,在同一比較中優於 Cutout 和 mixup。
  4. Mixup 的作者發現,在 ImageNet 上 alpha 介於 0.1 到 0.4 之間有助於提升效果,而較大的值會導致欠擬合,因此 alpha 需要調整。
  5. 這兩種方法增加的計算量微乎其微,因為它們混合的是當前批次中已有的樣例,且 torchvision 提供了現成的 MixUp 和 CutMix 轉換。

深入探討

Data augmentation creates new training examples from existing ones so a model sees more variety and memorizes less. Classic image augmentations transform one picture at a time: flip it, crop it, shift its colors. Mixup and CutMix take a different approach and combine two pictures, along with their labels, into one training example. The label is no longer a single class but a weighted mix, such as 60% dog and 40% cat. This encourages the network to make smooth, proportionate predictions rather than overconfident ones.

Mixup was introduced by Hongyi Zhang of MIT with Moustapha Cisse, Yann Dauphin and David Lopez-Paz of Facebook AI Research in "mixup: Beyond Empirical Risk Minimization" (arXiv October 2017, ICLR 2018). Empirical risk minimization (ERM) is the standard practice of minimizing average loss on the training examples themselves. Mixup instead trains on points in between examples. The procedure has four steps. First, draw a random mixing weight lambda from a Beta distribution with parameter alpha. Second, take two random training examples, often a batch and a shuffled copy of the same batch. Third, form the new input as lambda times the first input plus one minus lambda times the second. Fourth, form the new label the same way from the two one-hot label vectors, and train on that pair with ordinary cross-entropy loss. The authors note it takes a few lines of code and adds minimal computation.

The mixup paper reported gains across several domains. On ImageNet-2012, with 1.3 million training images in 1,000 classes, top-1 error for ResNet-50 trained for 200 epochs fell from 23.6% to 22.1%, and for ResNet-101 from 22.0% to 20.8%. The authors found that alpha between 0.1 and 0.4 helped on ImageNet while larger alpha caused underfitting, and that bigger models and longer training benefited most. They also showed mixup reduced memorization of deliberately corrupted labels, improved robustness to adversarial examples, and stabilized the training of generative adversarial networks.

CutMix came from Sangdoo Yun and colleagues at Naver's Clova AI Research and was published at ICCV 2019. It addressed a weakness the authors saw in two earlier methods. Cutout, from 2017, erases a random square of an image, forcing the model to look beyond the most obvious feature, but wastes those pixels. Mixup uses every pixel, but blended images look unnatural and locally ambiguous, and the CutMix paper reports that mixup lowered weakly supervised localization accuracy on ImageNet and object detection performance after transfer. CutMix replaces the erased rectangle with the matching patch from a second image and sets the label weights in proportion to the area each image occupies. The rectangle's size is drawn from the same Beta distribution, with alpha set to 1 so the area ratio is uniform between 0 and 1, and its position is drawn uniformly.

CutMix's results were strong across tasks. On ImageNet with ResNet-50, top-1 error fell from 23.68% to 21.40%, against 22.93% for Cutout, 22.58% for mixup and 22.50% for Manifold Mixup, a variant that mixes hidden-layer features. ResNet-101 improved by 1.70 percentage points. On CIFAR-100 it reached 14.47% top-1 error, a state-of-the-art result at the time. The paper also reported better weakly supervised object localization, gains when the pretrained network was fine-tuned for detection and captioning, improved robustness and reduced overconfidence.

Compared with alternatives, the two methods complement single-image augmentations rather than replacing them. Flips, crops and color jitter teach invariance to transformations of one image. Cutout and random erasing teach the model to use partial evidence. Mixup and CutMix additionally soften labels in a data-dependent way, which is related to label smoothing, a technique that replaces hard one-hot targets with slightly softened ones; the mixup paper's ablations found mixup beat combinations of input mixing and label smoothing. In practice, training recipes often apply both: libraries such as torchvision provide CutMix and MixUp transforms that can be randomly alternated per batch, and the DeiT recipe for vision transformers used both together with RandAugment and random erasing.

The methods have trade-offs. Because labels become fractional, training accuracy is no longer directly comparable to the usual metric, and the mixup authors found that longer training runs and larger models benefited most, so gains may be small on short schedules. Too strong a mix underfits, which is why alpha is tuned. Mixed images are not realistic, so mixup can hurt tasks that depend on precise spatial evidence, as the CutMix paper's localization and detection numbers show for mixup. CutMix can paste a patch that contains none of the second image's object, such as a patch of background, while the label still assigns that class weight in proportion to area. Both assume label mixing is meaningful, which is natural for classification but needs care for tasks such as segmentation or detection.

The cost is minimal. Both methods operate on a batch already in memory and add only a few tensor operations and a random draw per batch, so training time barely changes; the CutMix authors describe the overhead as negligible. The real cost is tuning: choosing alpha, the probability of applying each method, and the number of training epochs. Labels must be represented as probability vectors over classes rather than integer class indices, which torchvision's transforms handle by converting labels to a batch size by number of classes tensor.

04工作範例

手動混合一個 CIFAR-10 批次

  1. 載入 8 張 CIFAR-10 圖像(尺寸 32×32)及其整數標籤,並將標籤轉換為長度為 10 的 one‑hot 向量。

  2. 對批次進行隨機排列,產生一個打亂的副本,使每張圖像與同一批次中的另一張圖像配對。

  3. 對 mixup,從 Beta(0.2, 0.2) 抽樣 λ,例如 0.7,計算 0.7 倍原圖加 0.3 倍其配對圖,標籤同樣以相同權重混合。

  4. 對 CutMix,從 Beta(1, 1) 抽樣 λ,例如 0.75,得到一個邊長為 32×√0.25,即 16 像素的方框,位置隨機。

  5. 將每張配對圖的 16×16 區域貼到原圖上,若超出邊界則裁剪,並重新計算 λ 為 1 減去貼上區域除以 1,024 的比例。

  6. 顯示兩個混合批次及其軟標籤,並將任一批次送入使用交叉熵訓練的模型。

它顯示了什麼

你會得到兩批增強資料:混合、類似幽靈的圖像,例如標籤為 0.7 飛機與 0.3 船;以及貼合圖像,其標籤權重等於可見區域比例。通過確認每個軟標籤總和為 1 且 CutMix 權重符合每個來源圖像的像素比例來檢查正確性。

現實世界的實施

在 ImageNet 上,原始 2017 年的 mixup 論文將 ResNeXt‑101 32x4d 的 top‑1 錯誤率從 21.3% 降至 20.1%,訓練 200 個 epoch,混合強度 alpha 為 0.4。

CutMix(ICCV 2019)將 ResNet‑50 的 ImageNet top‑1 錯誤率從 23.68% 降至 21.40%,超越 Cutout 的 22.93% 以及同一比較中的 mixup 22.58%。

使用 CutMix 預訓練的模型在其他任務上轉移更好:CutMix 作者報告在 Pascal VOC 物件偵測上提升約 1 mAP,在 MS-COCO 影像說明上提升 2 BLEU 分。

Facebook AI 的 DeiT 訓練視覺 Transformer 於 ImageNet(2021)僅包含 mixup 與 CutMix,並指出這些正則化器能提升效能。

mixup 不僅限於影像:其作者亦將其應用於 Google Speech Commands 資料集(65,000 個一秒語音指令)以及表格 UCI 資料集。

Mixup 和 CutMix 增強的未來

Since 2019, research has produced many mixing variants. Manifold Mixup mixes hidden-layer features rather than pixels, and the CutMix paper itself tested a feature-level CutMix that scored 21.80% top-1 error on ImageNet against 21.40% for pixel CutMix. Transformer-era recipes such as DeiT (2021) combine mixup and CutMix with other augmentations such as RandAugment and random erasing. Open questions remain about why and when mixing works. Mixed labels interact with calibration, how well predicted probabilities match real accuracy; the CutMix authors reported reduced overconfidence, and how far that holds across models and datasets is still studied. Extending label mixing to dense tasks like segmentation and detection, to text, and to multimodal models requires deciding what a mixed label should mean. As training increasingly uses very large datasets and self-supervised pretraining, the relative value of mixing augmentations compared with simply adding more data is an active area of study, and practical guidance still relies heavily on tuning per task.

付諸實踐

為什麼這很重要

  • 成本與預算. 多年來,架構決策決定著效能和營運成本。
  • 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
  • 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。

留意

  • 優化一項基準測試可以隱藏更廣泛的系統弱點。
  • 基礎設施和維護成本常常被低估。
  • 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。

後續步驟

  1. 在實施之前定義延遲、品質和成本目標。
  2. 在實際負載和資料條件下進行基準測試。
  3. 儀器監控錯誤、漂移和使用者影響。
  4. 在擴展之前準備回滾和事件回應路徑。

資料來源與延伸閱讀

  1. arXiv (ICLR 2018)mixup: Beyond Empirical Risk Minimization
  2. arXiv (ICCV 2019)CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
  3. arXivTraining data-efficient image transformers and distillation through attention
  4. PyTorch torchvision documentationHow to use CutMix and MixUp

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

Take the Mixup and CutMix Augmentation quiz

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是 Mixup 和 CutMix Augmentation?

Mixup 和 CutMix 是資料增強技術,透過混合兩個樣本而非單一樣本來訓練神經網路。mixup 平均兩張完整圖像及其標籤;CutMix 將一張圖像的矩形區域貼到另一張圖像上,並按區域比例混合標籤。兩者幾乎不增加成本,能減少過擬合,且已成為現代影像分類器訓練配方的標準。

Mixup 如何建立新的訓練範例?

混合形式 x̃ = λx_a + (1−λ)x_b 並將標籤與從 Beta 分佈中提取的相同 λ 混合。

CutMix 和 Mixup 的核心差異是什麼?

CutMix 將一個矩形區域從一個影像複製到另一個影像中,保持局部連貫的內容,而不是將兩個影像重影在一起。

在 CutMix 中,如何決定標籤的混合權重 (lambda)?

CutMix 中的標籤比例是根據貼框相對於整個影像的面積來設定的,使標籤與可見像素保持一致。

除了輸入影像之外,Mixup 和 CutMix 還修改什麼?

這兩種方法也混合標籤,產生軟目標,其作用類似於依賴資料的標籤平滑形式。

哪一種分佈通常用於對混合係數 lambda 進行取樣?

Mixup 和 CutMix 通常從 Beta 分佈中提取 λ,該分佈控制兩個範例混合的強度。