What happened
Baidu’s ERNIE team announced the release of ERNIE‑5.5‑Flash‑Base on November 5, 2026. The checkpoint is a 280‑billion‑ mixture‑of‑experts (MoE) model that activates 14 billion parameters per token. It is provided in FP8 and BF16 safetensors formats and can be fine‑tuned for domain adaptation, instruction tuning, or reinforcement‑learning research. The model card details a 43‑layer architecture with Heavily Compressed Attention, Multi‑head Latent Attention, and a 512‑expert MoE block (10 experts active per token). Pre‑training used 33.5 trillion tokens, the Muon optimizer, and FP8 mixed precision, with context windows up to 128 K tokens and inference support for up to 1 million tokens. Baidu also released the FlashMask repository with GPU kernels for Hopper and Blackwell GPUs, claiming up to 175 % faster forward passes and 74 % faster backward passes versus FlashAttention‑4 MaskMod on SM100 hardware. The release includes a custom license that forbids use, modification, or distribution in the EU, UK, and South Korea, and requires a “Powered by ERNIE” attribution for commercial users with annual revenue under $50 million.
On November 5, 2026 Baidu’s ERNIE team published the ERNIE‑5.5‑Flash‑Base checkpoint, a 280‑billion‑ text‑only model that uses a mixture‑of‑experts architecture. Only 10 of the 512 routed experts are activated per token, resulting in roughly 14 billion active parameters per token. The model’s 43 layers include 37 Heavily Compressed Attention layers that cache one entry per 128‑token block and six Multi‑head Latent Attention layers that retain full token‑level context via an indexer.
The checkpoint is offered in both FP8 and BF16 safetensors formats and can be downloaded for non‑commercial use at no charge. Commercial users with annual revenue below $50 million must provide a notice to Baidu and display a “Powered by ERNIE” attribution; larger commercial users need prior written permission. The license explicitly excludes the European Union, United Kingdom, and South Korea, and bars the use of the model or its outputs to improve unrelated AI systems, limiting distillation and synthetic‑data pipelines.
Technical documentation includes a FlashMask repository that provides optimized attention kernels for NVIDIA Hopper and Blackwell GPUs. Baidu reports up to 175 % speed‑up in forward passes and 74 % in backward passes compared with FlashAttention‑4 MaskMod on SM100 hardware. The Muon optimizer, used during pre‑training on 33.5 trillion tokens, combines dense and MoE synchronization into a single collective operation, improving scaling efficiency.
Baidu evaluated ERNIE‑5.5‑Flash‑Base against publicly released checkpoints from DeepSeek, Hy3, and MiMo using a shared evaluation harness. The model leads on MMLU and HumanEval+ benchmarks but trails on FullStackBench and InfiniteBench, indicating strong exam‑style and coding performance but mixed results on broader system‑level tasks.
Why it matters
The ERNIE‑5.5‑Flash‑Base release gives researchers a high‑capacity, long‑context foundation model without the need to repeat Baidu’s massive pre‑training effort. Its MoE design and compressed‑attention mechanisms aim to keep per‑token compute roughly constant as context length grows, addressing a key bottleneck for applications such as code‑base reasoning, software‑engineering agents, and retrieval‑augmented generation. By publishing the checkpoint, Baidu provides a common baseline for evaluating new training techniques, instruction‑tuning pipelines, and reinforcement‑learning‑from‑human‑feedback (RLHF) methods. The licensing terms, however, restrict adoption in major markets (EU, UK, South Korea) and impose commercial thresholds, which could limit the model’s impact on open‑source ecosystems and enterprise deployments. The release also showcases Baidu’s hardware‑aware optimizations (FlashMask, Muon system) that may influence future model‑training stacks, especially on NVIDIA’s Hopper and Blackwell GPUs.
Providing a publicly downloadable 280‑billion‑ MoE checkpoint lowers the barrier for academic and industry researchers who want to experiment with long‑context, high‑capacity models without incurring the petaflop‑scale compute costs of pre‑training. This can accelerate research in areas such as code‑level reasoning, software‑engineering agents, and retrieval‑augmented generation that require handling of very long sequences.
The model’s compressed‑attention and expert‑routing design directly addresses the quadratic scaling problem of traditional transformers, offering a path to constant‑time per‑token computation as context length grows. If the claimed speed gains hold across diverse hardware, they could influence the design of future LLMs and the development of training frameworks that prioritize long‑context efficiency.
The restrictive license may fragment the open‑source AI landscape. By barring use in the EU and limiting commercial deployment, Baidu could push European and large‑scale enterprises toward alternative models that have more permissive terms, potentially reducing the model’s real‑world impact despite its technical merits.
The release also serves as a showcase for Baidu’s hardware‑aware software stack (FlashMask, Muon). Other organizations may adopt or adapt these components, shaping the broader ecosystem of GPU‑optimized training pipelines, especially as NVIDIA’s Hopper and Blackwell GPUs become more widely available.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
Which component of an AI application is the machine-learning model itself?
What to watch next
Future updates from Baidu, including a post‑trained ERNIE‑5.5‑Flash model, will reveal how the base checkpoint performs after instruction tuning or RLHF. The community’s response to the geographic and commercial licensing restrictions will be important for adoption in Europe and large‑scale enterprises. Benchmark comparisons with competing MoE models such as DeepSeek‑V4‑Flash‑Base, Hy3‑preview‑Base, and MiMo‑V2.5‑Base will help gauge the practical trade‑offs of Baidu’s attention and expert routing designs. Finally, the availability of the FlashMask kernels and Muon optimizer may spur third‑party tooling and influence GPU‑vendor roadmaps for long‑context training.
The upcoming post‑trained ERNIE‑5.5‑Flash model will demonstrate how well the base checkpoint can be adapted for instruction following, RLHF, or domain‑specific fine‑tuning. Performance gains or regressions relative to the base will be closely monitored.
Community and enterprise reactions to the geographic and commercial licensing restrictions will indicate whether the model gains traction outside Baidu’s home market. Legal challenges or policy discussions in the EU could arise if the restrictions are viewed as anti‑competitive.
Benchmark updates that compare ERNIE‑5.5‑Flash‑Base against other MoE and dense LLMs on a broader set of tasks (e.g., reasoning, coding, multi‑modal) will help clarify the practical trade‑offs of Baidu’s architecture choices.
Adoption of the FlashMask kernels and Muon optimizer by third‑party frameworks (e.g., PyTorch, DeepSpeed) could broaden the impact of Baidu’s performance innovations beyond its own models.