技術指南

LoRA 專家的混合體

LoRA 專家混合是一系列微調方法,將多個小型低階適配器(稱為 LoRA 專家)附加到凍結語言模型,並使用學習路由器來決定每個適配器對每個輸入貢獻多少。

聆聽此指南AI 語音敘述 · ~7 min

本頁8 分鐘閱讀
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. 計算小型 LoRA 混合物中的參數
  5. 現實世界的實施
  6. LoRA 專家混合的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

它將 LoRA 的低成本與專家混合模型的專業化相結合,因此一個基本模型可以在 1% 的參數下訓練時處理多項任務。

重點摘要

  1. LoRA 專家將多個低階適配器連接到凍結模型,並使用路由器對每個輸入進行加權。
  2. 根據 2021 年的論文,與完全微調的 GPT-3 175B 相比,LoRA 將可訓練參數減少了 10,000 倍,GPU 記憶體減少了三倍。
  3. Cohere 2023 年的研究匹配了完全微調,同時使用輕量級專家對 110 億參數模型進行了不到 1% 的更新。
  4. 動態路由會阻止將適配器合併到基本權重中,因此 LoRA 專家的混合會增加一些推理成本,而單一合併的 LoRA 可以避免這種成本。
  5. 負載平衡損失使路由器無法將幾乎所有令牌發送給少數專家,這種故障稱為專家崩潰。

深入探討

Mixture of LoRA experts combines two established ideas. The first is LoRA, low-rank adaptation, introduced by Edward Hu and colleagues in June 2021. Instead of retraining all the weights of a large model, LoRA freezes them and learns a small correction for selected weight matrices, written as the product of two thin matrices. For GPT-3 175B, the LoRA paper reports 10,000 times fewer trainable parameters and three times less GPU memory than full fine-tuning, with no added inference delay once the correction is merged into the weights. The second idea is the mixture of experts (MoE): a model contains several sub-networks, called experts, and a small gating network, or router, chooses which ones to use for each input. Noam Shazeer and colleagues showed in 2017 that sparsely gated MoE layers with up to thousands of experts could raise model capacity more than 1,000-fold with little extra computation.

A mixture of LoRA experts uses LoRA adapters as the experts. Step by step: a pretrained model is frozen; at chosen layers, usually the attention projections or the feed-forward block, several LoRA adapters are attached side by side; a router, typically one small linear layer followed by a softmax, reads the current token's hidden state and outputs a weight for each adapter; each adapter computes its low-rank correction; and the corrections are added to the frozen layer's output in proportion to the router's weights. With sparse routing, only the top one or two adapters are computed for each token. During training only the adapters and router are updated.

The reason to do this rather than train one bigger adapter is interference. When one adapter is trained on a mixture of very different tasks, such as coding, translation and arithmetic, the tasks compete for the same few parameters. Separate experts let different kinds of input use different parameters. LoRAMoE, posted in December 2023 by Shihan Dou and colleagues, targeted a specific version of this problem: scaling up instruction data can damage the world knowledge a model learned in pretraining. It splits its LoRA experts so some focus on preserving existing knowledge and others on new tasks, and reports gains on downstream tasks while retaining stored knowledge.

The approach developed through several papers. In September 2023 Ted Zadouri and colleagues at Cohere, in "Pushing Mixture of Experts to the Limit", built mixtures of lightweight experts including LoRA-based ones and updated less than 1% of an 11-billion-parameter model, outperforming standard parameter-efficient methods and matching full fine-tuning on instruction tuning. LoraHub, posted in July 2023 by Chengsong Huang and colleagues, took a different route: it combines LoRA modules already trained on separate tasks, choosing fixed mixing weights from a few examples of a new task with a gradient-free search. MixLoRA, from April 2024, added top-k routed LoRA experts inside feed-forward blocks plus separate LoRA adapters on attention, with a load-balancing loss so the router does not overuse a few experts. In April 2024 Xun Wu, Shaohan Huang and Furu Wei described Mixture of LoRA Experts (MoLE), which learns gating over existing trained LoRAs and was tested in both language and vision-and-language settings.

Compared with the alternatives, the trade-offs are clear. Full fine-tuning updates every weight and adapts best, but needs far more memory and produces a full copy of the model per task. A single LoRA adapter is cheapest to train and serve but can struggle with many mixed tasks. Merging several LoRAs by simple weighted addition, a common shortcut, is free at inference but, according to the MoLE paper, can weaken the base model's generation and blur each LoRA's distinct behavior. Keeping separate adapters and switching between them by hand works when the task is known in advance, but requires an outside decision about which adapter to use. A learned router makes that choice automatically, per token, at the cost of extra machinery.

The approach has real limitations. Routers can collapse, sending almost all tokens to one or two experts and leaving the others untrained, which is why methods such as MixLoRA add load-balancing losses. Input-dependent routing removes one of LoRA's main advantages: because the mixture changes token by token, the adapters cannot be merged into the base weights ahead of time, so inference does extra work. Batching requests that use different adapters complicates serving, which is the problem S-LoRA addresses. Reported gains are mostly self-reported on academic benchmarks with specific base models, and results vary with the number of experts, rank, routing method and which layers receive experts.

In practice the costs are modest. Each LoRA expert adds two thin matrices per adapted weight matrix, so even several experts usually total well under the 1% of model parameters reported in the Cohere work. Training fits on the same hardware used for ordinary LoRA fine-tuning, plus a little memory for the extra adapters and router. The main engineering decisions are how many experts to use, their rank, whether routing is dense or top-k, and which layers to adapt. Serving many adapters efficiently benefits from systems built for it; S-LoRA reports serving thousands of concurrent adapters with up to four times the throughput of naive approaches in Hugging Face PEFT and vLLM.

04工作範例

計算小型 LoRA 混合物中的參數

  1. 以大小為 4,096 x 4,096 的注意力投影為例,其中包含約 1,680 萬個凍結權重。

  2. 增加一個 8 級 LoRA 轉接器:A 為 8 x 4,096,B 為 4,096 x 8,提供 65,536 個可訓練參數。

  3. 使其混合 4 個這樣的專家:4 乘以 65,536 就是 262,144 個參數。

  4. 新增一個路由器,將 4,096 維輸入對應到 4 個閘得分:4,096 乘以 4 等於 16,384 個參數。

  5. 總可訓練參數:278,528,約佔凍結矩陣的 1.7%。

  6. 使用 top-2 路由,每個 token 只計算兩個適配器校正,而不是四個,並將額外工作量與基礎矩陣乘法進行比較。

它顯示了什麼

一個具體預算顯示,四個專家加上一個路由器的成本不到一個權重矩陣的 2%,且 top-2 路由將適配器計算量減半。通過在 PyTorch 中構建該層,凍結 W0,並對 requires_grad 設置為 true 的參數使用 numel() 求和來檢查。

現實世界的實施

Cohere 的研究人員於 2023 年證明,訓練於指令上的 LoRA 專家混合只更新 11 億參數模型不到 1%,但表現與完整微調相當。

LoRAMoE(2023 年 12 月)將部分 LoRA 專家專門用於保留模型的已存世界知識,而其他專家學習新任務,從而減少大型指令資料集造成的遺忘。

LoraHub(COLM 2024)將在不同任務上訓練的 LoRA 模組結合起來,從少量示例中解決未見的 Big-Bench Hard 任務,無需梯度或額外參數。

MixLoRA 將 top-k 路由的 LoRA 專家加入凍結模型的前饋層,並報告比標準參數高效方法高約 9% 的多任務準確率。

S-LoRA 是為多個適配器構建的服務系統,在 Hugging Face PEFT 和 vLLM 中,能以高達四倍的吞吐量同時服務數千個 LoRA 適配器。

LoRA 專家混合的未來

Research since 2023 has explored several directions at once: jointly trained LoRA mixtures for instruction tuning (Cohere, September 2023), mixtures that protect pretrained knowledge (LoRAMoE, December 2023), composition of independently trained adapters (LoraHub, published at COLM 2024, and MoLE, April 2024) and sparse top-k LoRA experts with load balancing (MixLoRA, 2024). Serving infrastructure has advanced in parallel, with S-LoRA showing that thousands of adapters can share one base model on the same hardware. Open problems include stable routing without expert collapse, principled choices for the number and rank of experts, fair comparison across papers that use different base models and benchmarks, and keeping latency low when adapters cannot be merged into the base weights. LoraHub's authors proposed a platform where users share trained LoRA modules for reuse on new tasks; whether shared adapter libraries with automatic routing become common practice, or whether single well-trained adapters and larger base models remain simpler and good enough, is still an open question that depends on evidence from production use rather than benchmarks alone.

付諸實踐

為什麼這很重要

  • 成本與預算. 多年來,架構決策決定著效能和營運成本。
  • 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
  • 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。

留意

  • 優化一項基準測試可以隱藏更廣泛的系統弱點。
  • 基礎設施和維護成本常常被低估。
  • 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。

後續步驟

  1. 在實施之前定義延遲、品質和成本目標。
  2. 在實際負載和資料條件下進行基準測試。
  3. 儀器監控錯誤、漂移和使用者影響。
  4. 在擴展之前準備回滾和事件回應路徑。

資料來源與延伸閱讀

  1. arXivLoRA:大型語言模型的低秩適配。
  2. arXiv極度大型神經網絡:稀疏門控混合專家層。
  3. arXiv (Cohere)將混合專家推向極限:極高參數效率的指令微調 MoE
  4. arXivLoRAMoE:通過 MoE 風格插件緩解大型語言模型的世界知識遺忘
  5. arXiv (COLM 2024)LoraHub:通過動態 LoRA 組合實現高效跨任務泛化
  6. arXivLoRA 專家的混合體
  7. arXivMixLoRA:利用基於 LoRA 的混合專家提升大型語言模型微調
  8. arXivS-LoRA:同時服務數千個 LoRA 適配器

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

參加 LoRA 專家混合(Mixture of LoRA Experts)測驗

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是 LoRA 專家混合(Mixture of LoRA Experts)?

LoRA 專家混合是一種微調方法,將多個小型低秩適配器(稱為 LoRA 專家)附加到凍結的語言模型上,並使用學習到的路由器決定每個輸入中每個適配器的貢獻。它結合了 LoRA 的低成本與混合專家模型的專業化,使單一基礎模型能處理多個任務,同時訓練不到其參數的 1%。

Mixture of LoRA Experts 中的「LoRA」部分指的是什麼?

LoRA 代表低秩適應:它凍結基本模型並訓練緊湊的低秩矩陣,以低成本調整行為。

MoLE 中的門控/路由器網路的作用是什麼?

路由器對可用的 LoRA 專家產生權重,根據輸入(通常是每層或令牌)選擇和混合它們。

為什麼 MoLE 通常比在許多混合任務上訓練一個適配器更好?

當一個適應者必須同時學習許多相互衝突的技能時,單獨的專家可以避免災難性的遺忘和任務幹擾。

在 MoLE 訓練期間,基本模型的預訓練權重通常會發生什麼情況?

脊椎骨仍然凍結;只有小型 LoRA 專家和門控網路接收梯度更新。

在擴散影像模型中,MoLE 中的分層/每層門控有助於解決什麼問題?

每層門控了解每個適配器在每一層的貢獻程度,讓多個概念 LoRA 結合在一起,而不會由一個適配器占主導地位。