What happened
The b10776 pre-release of llama.cpp adds support for NVIDIA’s Nemotron-3-Puzzle-75B-A9B model and updates the conversion and GGUF-loading paths needed for its per-layer mixture-of-experts configuration.
According to the release notes, llama.cpp now handles Nemotron 3 Puzzle’s variable per-layer expert feed-forward sizes and top-k routing across its mixture-of-experts layers. The implementation supports scalar or per-layer values in existing GGUF keys, broadcasts scalar values for uniform models, and uses indexed accessors when layers differ.
The release also adds converter support for NemotronHPuzzleForCausalLM. It parses per-block configuration, handles the model’s MTP-related tensors, accepts the official NVIDIA BF16 checkpoint’s model.* tensor naming, normalizes the alternate router-bias name, and registers the architecture. The notes say the official index contains 42,683 keys and that every indexed tensor resolves after normalization.
Nemotron 3 Puzzle is explicitly kept out of llama.cpp’s NemotronH MTP export path. The release notes state that --mtp and --no-mtp therefore fail at the command line for Puzzle, while the model’s head is handled through its distinct per-block configuration. The release is labeled Pre-release on GitHub and lists build targets for macOS, iOS, Linux, Android, Windows, and other platforms, with some target-specific entries marked disabled.
Why it matters
This change could make a large, locally runnable AI model easier to use through llama.cpp, but the source documents compatibility work rather than independent evidence of speed, accuracy, or reliability.
For developers running models locally, support for a new architecture can be as important as a model announcement: without correct per-layer expert metadata, a converter or inference engine may misinterpret routing and feed-forward dimensions. Reusing existing GGUF keys also preserves compatibility with uniform MoE models while allowing Puzzle’s heterogeneous configuration.
The practical audience is llama.cpp users and developers who can obtain or build the b10776 pre-release and provide a compatible checkpoint. The source does not establish general availability beyond the GitHub pre-release, does not provide a price, and does not report independent benchmarks, hardware requirements, inference quality, memory use, or production readiness.
The update may also help model developers convert the officially distributed BF16 checkpoint without manually renaming its tensors. That benefit remains a compatibility claim in the release notes; the source contains no independently reported conversion or inference test.
What to watch next
Watch for stable-release status, successful conversion and inference reports, and practical performance across the listed hardware targets.
The main unknown is whether the new path works reliably across the full official checkpoint and supported backends. The release notes describe tensor-map coverage and implementation changes, but do not include test results, throughput measurements, memory measurements, or failure rates.
Users should watch whether a later stable release changes the current pre-release status, adds or clarifies MTP support, and documents which downloadable artifacts or build configurations are usable for Nemotron 3 Puzzle. The listed platform targets do not by themselves establish that every target can run this model.
Further documentation may clarify the model’s actual resource requirements and whether per-layer routing produces materially different performance from uniform MoE models. None of those outcomes can be concluded from this release alone.