Bedrijven GIDS
Nous Onderzoek
Nous Research is an AI company known for the Hermes family of open-weight models, built by fine-tuning open base models such as Meta's Llama 3.1, and for research on training models over the internet with far less bandwidth.
Luister naar deze gidsAI-verteld · ~8 min
Op deze pagina9 minuten lezen
Overzicht
Its DisTrO and DeMo optimizers and its Solana-coordinated Psyche network aim to let independently owned GPUs train models together.
Belangrijkste inzichten
- Nous Research builds the Hermes open-weight models by fine-tuning base models such as Meta's Llama 3.1 405B and publishing the weights.
- Hermes 3 405B, released with an August 2024 report, was described as the first full-parameter fine-tune of Llama 3.1 405B.
- Hermes 4, reported in August 2025, adds hybrid reasoning and trained on roughly 5 million samples and 60 billion tokens.
- The DeMo optimizer shares only the largest transformed momentum components and reported up to 85 times less data per GPU than AdamW.
- Psyche coordinates internet-scale training on Solana, and its first run targets a 40-billion-parameter model on 20 trillion tokens.
Diepe duik
Nous Research is an AI research company that describes its mission as creating and spreading open access to intelligence. It is best known for two lines of work. The first is the Hermes series of large language models, which take open base models released by other labs and fine-tune them into chat assistants, publishing the resulting weights for anyone to download. The second is research and infrastructure for decentralized training, meaning training one model across many independently operated computers connected by the ordinary internet rather than inside a single data center. Today the company also offers Hermes Agent, an assistant it describes as self-improving, and a hosted service called Nous Portal.
Fine-tuning is the core of the Hermes work. A base model, trained by another lab on trillions of words, knows a great deal but does not reliably follow instructions or hold a conversation. Fine-tuning continues training on curated examples of the desired behavior, such as multi-turn chats, tool calls and reasoning traces. Hermes models use the ChatML prompt format and add dedicated system prompts for function calling and JSON-only structured output, which makes them convenient for developers building agents. The Hermes 3 report calls it a neutrally aligned generalist instruct and tool-use model, a stance that leaves more of the behavior to the user's and developer's instructions.
The Hermes 3 technical report, by Ryan Teknium, Jeffrey Quesnelle and Chen Guang, was posted on August 15, 2024. Its largest model, built on Meta's Llama 3.1 405B, was described on its model card as the first full-parameter fine-tune of that base model, meaning every weight was updated rather than a small adapter. The report claims state-of-the-art results among open-weight models on several public benchmarks, and the model card describes Hermes 3 as competitive with, and in some areas better than, Meta's own Llama 3.1 Instruct models. The practical cost of such a model is large: the card notes FP16 inference needs over 800 GB of GPU memory, so an FP8 version of about 430 GB is offered.
Hermes 4 followed with a technical report dated August 25, 2025, describing a family of hybrid reasoning models that can either answer directly or reason step by step inside think tags. Its 405B model again builds on Llama 3.1 405B, with smaller 70B and 14B variants. According to its model card, the training set grew to roughly 5 million samples and 60 billion tokens, from about 1 million samples and 1.2 billion tokens for Hermes 3. Nous Research also created RefusalBench, a test of whether a model helps with requests that many models refuse, and the model card says Hermes 4 achieves state-of-the-art results on it. That framing reflects the company's emphasis on user control, which also means deployers must add their own safety policies where needed.
The decentralized training work tackles a specific bottleneck. In standard data-parallel training, every GPU must share its full gradients with every other GPU after each step, which requires the very fast links found inside data centers. In August 2024 Nous Research published a preliminary report on DisTrO, a family of distributed optimizers it says reduces inter-GPU communication by three to four orders of magnitude. The underlying research, DeMo (Decoupled Momentum Optimization), was posted in November 2024 by Bowen Peng, Jeffrey Quesnelle, Diederik P. Kingma and colleagues. Each GPU keeps its own momentum, converts it with a discrete cosine transform, and shares only the largest components. On 300M and 1B-parameter language models, the paper reports up to 85 times less data per GPU than standard AdamW training with comparable loss and accuracy.
Psyche, announced on May 14, 2025, turns those optimizers into a network. A coordinator runs as a smart contract on the Solana blockchain, tracking participants, assigning data and synchronizing training steps, while GPUs exchange compressed updates directly peer to peer. Psyche adds overlapped training, so nodes begin the next step while results transfer, and 1-bit sign quantization of updates. Its first run, Consilience, is a 40-billion-parameter dense model using the Multi-head Latent Attention design from DeepSeek V3, trained on 20 trillion tokens drawn from FineWeb, FineWeb-2 and The Stack V2. Nous described it as the largest pretraining run over the internet to that date. A December 2024 test trained a 15-billion-parameter model.
The company's funding is tied to this crypto-coordinated approach. In April 2025 SiliconANGLE reported a 50 million dollar Series A almost entirely funded by the crypto-focused firm Paradigm, following seed rounds totaling 20 million dollars, and cited a Fortune report that the round implied a 1 billion dollar token valuation. Cofounder Karan Malhotra described contributed compute as a transaction rather than a donation, with crypto rewards compensating contributors.
Compared with large closed labs, Nous Research works mostly on top of other organizations' base models and publishes weights openly, so its models are inexpensive to obtain but inherit the licenses and limits of their bases; the Hermes 3 and Hermes 4 405B models carry the Llama 3 license. Compared with other open fine-tuners, its distinctive bets are the neutral alignment stance and decentralized training. The limitations are concrete. Running the largest Hermes models still needs hundreds of gigabytes of GPU memory. Decentralized training remains early, and the Psyche announcement itself notes that verifying that participants submitted honest results is an open problem. Benchmark claims in technical reports come from the authors and are best checked against independent evaluations for a given use.
04Uitgewerkt voorbeeld
Estimating bandwidth saved by compressed updates
Take a 1-billion-parameter model and assume standard data parallelism sends one 16-bit value per parameter per step.
Compute the standard payload: 1e9 x 2 bytes = 2 GB per worker per step.
Apply the DeMo paper's reported best-case reduction of up to 85 times less data per GPU.
Compute the compressed payload: 2 GB / 85, about 23.5 MB per step.
Compare transfer times on a 1 gigabit per second link (about 125 MB per second): about 16 seconds versus about 0.19 seconds.
Note what this ignores: latency, protocol overhead and whether quality matches at larger scale.
Wat het laat zien
The estimate shows why compressed updates make training over ordinary internet links plausible, turning a 16-second exchange into a fraction of a second. Check it by redoing the arithmetic with your own link speed and model size, and treat 85 times as the paper's reported upper figure, not a guarantee.
Implementatie in de echte wereld
Running Hermes 3 or Hermes 4 open-weight chat models with tools such as vLLM or Transformers; Hermes 3 405B was described as the first full-parameter fine-tune of Llama 3.1 405B.
Using Hermes 4's hybrid reasoning mode, in which the model can deliberate inside think tags when enabled through a chat-template flag or system prompt, then answer directly.
Training across distant data centers or ordinary Ethernet links with DeMo, an optimizer from Nous researchers and Diederik Kingma that sent up to 85 times less data per GPU than AdamW in reported 1B-parameter tests.
Contributing GPU time to Psyche, Nous Research's distributed training network, whose first run, Consilience, targets a 40-billion-parameter model trained on 20 trillion tokens.
Building developer agents that rely on Hermes models' ChatML format with dedicated system prompts for function calling and JSON-only structured output, as documented on the Hermes 3 model card.
De toekomst van Nous-onderzoek
Nous Research's stated direction combines open-weight models, agents and decentralized infrastructure. Its 2025 releases, Hermes 4 in August and the Psyche network in May with the 40-billion-parameter Consilience run, show the two tracks advancing together, and the company now also promotes Hermes Agent and its Nous Portal service. The central open questions are technical and economic. Whether bandwidth-light optimizers like DeMo and DisTrO can match centralized training quality at frontier scale has not been shown in public results, which so far cover models up to tens of billions of parameters. Psyche's own announcement names verification of honest work as unsolved, and token-based incentives add their own uncertainties. On the model side, a neutral alignment stance gives developers control but places responsibility for safeguards on whoever deploys the model. How these approaches perform will be clearer as Consilience and later runs publish evaluations, and claims about future capability should be judged against those published results.
Het in de praktijk brengen
Waarom het ertoe doet
- Leveranciersstrategie. Roadmaps van leveranciers beïnvloeden welke functies uw team vervolgens kan bouwen.
- Kosten en budget. Commerciële voorwaarden en implementatieopties zijn van invloed op de kosten en risico's op de lange termijn.
- Risico en veiligheid. Bedrijfsprikkels bepalen productgebreken, veiligheidshouding en openheid.
Pas op voor
- Lanceringsaankondigingen kunnen de stabiliteit in echte productieworkflows overtreffen.
- API-prijzen of beleidswijzigingen kunnen van de ene op de andere dag de aannames doorbreken.
- De afhankelijkheid van één leverancier verhoogt de lock-in- en migratiekosten.
Volgende stappen
- Evalueer providers met behulp van uw eigen taken en datasets.
- Controleer de privacy-, beveiligings- en juridische voorwaarden vóór de integratie.
- Onderhoud een noodplan voor alle modellen of leveranciers.
- Houd de release-opmerkingen in de gaten, zodat wijzigingen in de routekaart teams niet verrassen.
Bronnen en verdere literatuur
- Nous ResearchNous Onderzoek
- arXiv (Teknium, Quesnelle, Guang)Hermes 3 Technical Report
- Hugging FaceNousResearch/Hermes-3-Llama-3.1-405B
- Hugging FaceNousResearch/Hermes-4-405B
- arXiv (Peng et al.)DeMo: Decoupled Momentum Optimization
- GitHubNousResearch/DisTrO
- Nous ResearchDemocratizing AI: The Psyche Network Architecture
- SiliconANGLENous Research raises $50M for decentralized AI training led by Paradigm
Blijf verkennen
Gratis nieuwsbrief
Ontvang de dagelijkse AI-briefing
Drie verifieerde AI-verhalen elke weekdagmorgen, geschreven in eenvoudig Engels. Gratis voor altijd, geen advertenties.
Eén e-mail elke weekdag. Afmelden met één klik. We verkopen of delen je adres nooit.
Test jezelf
Take the Nous Research quiz
Directe feedback op elke antwoord, en een deelbare certificaat met een verifieerbare ID zodra je een cursus hebt afgerond.
Ondersteun gratis AI-onderwijs. AI Understanding is een 501(c)(3) non-profit — geen advertenties, geen betaalmuur, nooit. Doe een donatie
Veelgestelde vragen
What is Nous Research?
Nous Research is an AI company known for the Hermes family of open-weight models, built by fine-tuning open base models such as Meta's Llama 3.1, and for research on training models over the internet with far less bandwidth. Its DisTrO and DeMo optimizers and its Solana-coordinated Psyche network aim to let independently owned GPUs train models together.
Waar is Nous Research het meest bekend om in zijn vroege opkomst op de voorgrond?
Nous kreeg bekendheid door open modellen als Llama en Mistral te verfijnen tot de populaire Hermes-familie.
Welke basismodellen heeft Nous het meest verfijnd?
De Hermes- en Capybara-series van Nous zijn gebouwd op open basismodellen, vooral Llama en Mistral.
What problem does Nous Research’s DisTrO project address?
Nous describes DisTrO as a family of distributed-training optimizers designed to reduce inter-GPU communication requirements.
Wat is het Psyche-netwerk?
Psyche is het gedecentraliseerde trainingsnetwerk van Nous dat onderzoekt hoe verspreide GPU's kunnen worden samengevoegd om grote modellen te trainen.
Welke techniek gebruikt Nous doorgaans om zijn verfijnde modellen op elkaar af te stemmen?
Nous past gecontroleerde verfijning en voorkeursoptimalisatie toe bovenop open basisgewichten met behulp van samengestelde datasets.
Blijf leren
Gerelateerde gidsen
Er zijn meer handleidingen voor dit onderwerp geselecteerd