公司指南

諾斯研究

Nous Research 是一家以 Hermes 系列開放權重模型聞名的 AI 公司,這些模型是通過微調開放基礎模型(如 Meta 的 Llama 3.1)構建的,並且在互聯網上以更少的頻寬進行模型訓練研究。

聆聽此指南AI 語音敘述 · ~8 min

本頁閱讀時間 9 分鐘
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. 估算壓縮更新節省的頻寬
  5. 現實世界的實施
  6. 諾斯研究的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

其 DisTrO 和 DeMo 優化器以及 Solana 協調的 Psyche 網路旨在讓獨立擁有的 GPU 一起訓練模型。

重點摘要

  1. Nous Research 透過微調基礎模型(如 Meta 的 Llama 3.1 405B)並發布權重,構建 Hermes 開放權重模型。
  2. Hermes 3 405B 在 2024 年 8 月發布的報告中被描述為 Llama 3.1 405B 的第一個完整參數微調模型。
  3. Hermes 4 在 2025 年 8 月報告中加入了混合推理功能,並在約 500 萬個樣本和 600 億個 token 上進行訓練。
  4. DeMo 優化器僅共享最大的變換動量分量,報告顯示每個 GPU 的數據量比 AdamW 少高達 85 倍。
  5. Psyche 在 Solana 上協調網際網路規模的訓練,其首次運行針對 400 億參數模型,使用 20 兆 token。

深入探討

Nous Research is an AI research company that describes its mission as creating and spreading open access to intelligence. It is best known for two lines of work. The first is the Hermes series of large language models, which take open base models released by other labs and fine-tune them into chat assistants, publishing the resulting weights for anyone to download. The second is research and infrastructure for decentralized training, meaning training one model across many independently operated computers connected by the ordinary internet rather than inside a single data center. Today the company also offers Hermes Agent, an assistant it describes as self-improving, and a hosted service called Nous Portal.

Fine-tuning is the core of the Hermes work. A base model, trained by another lab on trillions of words, knows a great deal but does not reliably follow instructions or hold a conversation. Fine-tuning continues training on curated examples of the desired behavior, such as multi-turn chats, tool calls and reasoning traces. Hermes models use the ChatML prompt format and add dedicated system prompts for function calling and JSON-only structured output, which makes them convenient for developers building agents. The Hermes 3 report calls it a neutrally aligned generalist instruct and tool-use model, a stance that leaves more of the behavior to the user's and developer's instructions.

The Hermes 3 technical report, by Ryan Teknium, Jeffrey Quesnelle and Chen Guang, was posted on August 15, 2024. Its largest model, built on Meta's Llama 3.1 405B, was described on its model card as the first full-parameter fine-tune of that base model, meaning every weight was updated rather than a small adapter. The report claims state-of-the-art results among open-weight models on several public benchmarks, and the model card describes Hermes 3 as competitive with, and in some areas better than, Meta's own Llama 3.1 Instruct models. The practical cost of such a model is large: the card notes FP16 inference needs over 800 GB of GPU memory, so an FP8 version of about 430 GB is offered.

Hermes 4 followed with a technical report dated August 25, 2025, describing a family of hybrid reasoning models that can either answer directly or reason step by step inside think tags. Its 405B model again builds on Llama 3.1 405B, with smaller 70B and 14B variants. According to its model card, the training set grew to roughly 5 million samples and 60 billion tokens, from about 1 million samples and 1.2 billion tokens for Hermes 3. Nous Research also created RefusalBench, a test of whether a model helps with requests that many models refuse, and the model card says Hermes 4 achieves state-of-the-art results on it. That framing reflects the company's emphasis on user control, which also means deployers must add their own safety policies where needed.

The decentralized training work tackles a specific bottleneck. In standard data-parallel training, every GPU must share its full gradients with every other GPU after each step, which requires the very fast links found inside data centers. In August 2024 Nous Research published a preliminary report on DisTrO, a family of distributed optimizers it says reduces inter-GPU communication by three to four orders of magnitude. The underlying research, DeMo (Decoupled Momentum Optimization), was posted in November 2024 by Bowen Peng, Jeffrey Quesnelle, Diederik P. Kingma and colleagues. Each GPU keeps its own momentum, converts it with a discrete cosine transform, and shares only the largest components. On 300M and 1B-parameter language models, the paper reports up to 85 times less data per GPU than standard AdamW training with comparable loss and accuracy.

Psyche, announced on May 14, 2025, turns those optimizers into a network. A coordinator runs as a smart contract on the Solana blockchain, tracking participants, assigning data and synchronizing training steps, while GPUs exchange compressed updates directly peer to peer. Psyche adds overlapped training, so nodes begin the next step while results transfer, and 1-bit sign quantization of updates. Its first run, Consilience, is a 40-billion-parameter dense model using the Multi-head Latent Attention design from DeepSeek V3, trained on 20 trillion tokens drawn from FineWeb, FineWeb-2 and The Stack V2. Nous described it as the largest pretraining run over the internet to that date. A December 2024 test trained a 15-billion-parameter model.

The company's funding is tied to this crypto-coordinated approach. In April 2025 SiliconANGLE reported a 50 million dollar Series A almost entirely funded by the crypto-focused firm Paradigm, following seed rounds totaling 20 million dollars, and cited a Fortune report that the round implied a 1 billion dollar token valuation. Cofounder Karan Malhotra described contributed compute as a transaction rather than a donation, with crypto rewards compensating contributors.

Compared with large closed labs, Nous Research works mostly on top of other organizations' base models and publishes weights openly, so its models are inexpensive to obtain but inherit the licenses and limits of their bases; the Hermes 3 and Hermes 4 405B models carry the Llama 3 license. Compared with other open fine-tuners, its distinctive bets are the neutral alignment stance and decentralized training. The limitations are concrete. Running the largest Hermes models still needs hundreds of gigabytes of GPU memory. Decentralized training remains early, and the Psyche announcement itself notes that verifying that participants submitted honest results is an open problem. Benchmark claims in technical reports come from the authors and are best checked against independent evaluations for a given use.

04工作範例

估算壓縮更新節省的頻寬

  1. 以一個 10 億參數的模型為例,假設標準資料並行處理每步為每個參數發送一個 16 位元的值。

  2. 計算標準負載:1e9 x 2 位元組 = 每個工作節點每步 2 GB。

  3. 應用 DeMo 論文報告的每個 GPU 最多減少 85 倍資料的最佳情況。

  4. 計算壓縮後的負載:2 GB / 85,每步約 23.5 MB。

  5. 比較 1 Gbps 連結(約每秒 125 MB)上的傳輸時間:約 16 秒對比約 0.19 秒。

  6. 注意這忽略了什麼:延遲、協定開銷以及品質在更大規模下是否匹配。

它顯示了什麼

該估算顯示了為什麼壓縮更新使得透過普通網際網路連結進行訓練變得可行,將 16 秒的交換時間縮短到不到一秒。透過使用您自己的連結速度和模型大小重新進行計算來進行驗證,並將 85 倍視為論文報告的上限數字,而非保證。

現實世界的實施

使用諸如 vLLM 或 Transformers 等工具運行 Hermes 3 或 Hermes 4 開放權重聊天模型;Hermes 3 405B 被描述為 Llama 3.1 405B 的首次全參數微調。

使用 Hermes 4 的混合推理模式,在此模式下,模型可以在透過聊天範本旗標或系統提示啟用時,於 think 標籤內進行深思熟慮,然後直接回答。

透過 DeMo(這是來自 Nous 研究人員與 Diederik Kingma 的最佳化工具)在遠端資料中心或普通乙太網路連結上進行訓練,在已報導的 10 億參數測試中,其每個 GPU 傳送的資料量比 AdamW 少高達 85 倍。

貢獻 GPU 時間給 Nous Research 的分散式訓練網路 Psyche,其首次運行「Consilience」旨在訓練一個在 20 兆個 Token 上訓練的 400 億參數模型。

建構依賴 Hermes 模型 ChatML 格式的開發者代理程式,並配備用於函式呼叫與純 JSON 結構化輸出的專用系統提示,正如 Hermes 3 模型卡片中所記錄的。

諾斯研究的未來

Nous Research's stated direction combines open-weight models, agents and decentralized infrastructure. Its 2025 releases, Hermes 4 in August and the Psyche network in May with the 40-billion-parameter Consilience run, show the two tracks advancing together, and the company now also promotes Hermes Agent and its Nous Portal service. The central open questions are technical and economic. Whether bandwidth-light optimizers like DeMo and DisTrO can match centralized training quality at frontier scale has not been shown in public results, which so far cover models up to tens of billions of parameters. Psyche's own announcement names verification of honest work as unsolved, and token-based incentives add their own uncertainties. On the model side, a neutral alignment stance gives developers control but places responsibility for safeguards on whoever deploys the model. How these approaches perform will be clearer as Consilience and later runs publish evaluations, and claims about future capability should be judged against those published results.

付諸實踐

為什麼這很重要

  • 供應商策略. 供應商路線圖會影響您的團隊接下來可以建立的功能。
  • 成本與預算. 商業條款和部署選項會影響長期成本和風險。
  • 風險與安全. 公司激勵措施塑造了產品預設、安全態勢和開放性。

留意

  • 發佈公告可能會超過實際生產工作流程的穩定性。
  • API 定價或政策轉變可能會在一夜之間打破假設。
  • 單一供應商依賴性增加了鎖定和遷移成本。

後續步驟

  1. 使用您自己的任務和資料集評估提供者。
  2. 在整合之前查看隱私、安全和法律條款。
  3. 維護跨模型或供應商的後備計劃。
  4. 監控發行說明,以便路線圖的變更不會讓團隊感到意外。

資料來源與延伸閱讀

  1. Nous Research諾斯研究
  2. arXiv (Teknium, Quesnelle, Guang)Hermes 3 技術報告
  3. Hugging FaceNousResearch/Hermes-3-Llama-3.1-405B
  4. Hugging FaceNousResearch/Hermes-4-405B
  5. arXiv (Peng et al.)DeMo:解耦動量最佳化 (Decoupled Momentum Optimization)
  6. GitHubNousResearch/DisTrO
  7. Nous Research民主化 AI:Psyche 網路架構
  8. SiliconANGLENous Research 獲得由 Paradigm 領投的 5000 萬美元分散式 AI 訓練資金

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

參與 Nous Research 測驗

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是諾斯研究?

Nous Research 是一家以 Hermes 開放權重模型系列而聞名的 AI 公司,這些模型是透過微調開放基礎模型(例如 Meta 的 Llama 3.1)所建構的,並以透過網際網路以遠低於以往的頻寬訓練模型的研究所聞名。其 DisTrO 與 DeMo 最佳化工具以及由 Solana 協調的 Psyche 網路,旨在讓獨立擁有的 GPU 能夠共同訓練模型。

Nous Research 在早期崛起時最出名的是什麼?

Nous 通過將 Llama 和 Mistral 等開放模型微調到流行的 Hermes 系列中而獲得了認可。

Nous 最常對哪些基本模型進行微調?

Nous 的 Hermes 和 Capybara 系列基於開放式底座模型,特別是 Llama 和 Mistral。

Nous Research 的 DisTrO 專案解決了什麼問題?

Nous 將 DisTrO 描述為一系列旨在降低 GPU 之間通訊需求的分散式訓練最佳化工具。

什麼是 Psyche 網路?

Psyche 是 Nous 的去中心化訓練網絡,探索如何匯集分散的 GPU 來訓練大型模型。

Nous 通常會使用什麼技術來調整其微調模型?

Nous 使用精選資料集在開放基礎權重之上應用監督微調和偏好優化。