What happened
NVIDIA reported benchmark results claiming that its Vera Rubin NVL72 system can deliver up to 30 times more agentic-AI throughput per megawatt than GB300 NVL72 under a specific long-context coding-agent workload. The company also reported substantial efficiency gains for GB300 over H200 systems.
In a technical blog post dated August 24, NVIDIA described results from AgentX, an agentic-coding benchmark in SemiAnalysis’s open-source InferenceX suite. The benchmark replays prerecorded Claude Code sessions turn by turn using the AIPerf client. According to the post, those sessions preserve the original input and output lengths, reasoning intervals, tool-call latency and accumulated context. NVIDIA presents this as a way to test serving behavior that fixed prompt-and-response workloads do not capture.
NVIDIA says AgentX measures several user-experience and efficiency variables at different concurrency levels. These include throughput per provisioned megawatt, end-to-end normalized interactivity, standard interactivity, end-to-end latency and time to first token. The company says the benchmark is intended to account for long-context prefill, reuse of previously processed context through the KV cache, interactive decoding, gaps caused by tool execution and distributed mixture-of-experts execution.
The main new claim concerns a Vera Rubin NVL72 preview result on the AgentX DeepSeek V4-Pro workload. NVIDIA says that at a target of 160 output tokens per second per user, Vera Rubin NVL72 delivered up to 30 times higher AI-factory throughput per megawatt than GB300 NVL72. The post explicitly says these results were measured by NVIDIA using the SemiAnalysis workload and are pending SemiAnalysis review. The source does not provide the underlying chart values in text beyond the stated comparison.
The post also reports results for the existing Blackwell GB300 NVL72 platform. NVIDIA says GB300 achieved up to 15 times the throughput per megawatt of H200 NVL8 on DeepSeek V4 Pro 1.6T, up to 10 times lower cost per million tokens in the same comparison and roughly 80 times the throughput per megawatt of H200 NVL8 on the Kimi K3 2.8T AgentX workload. NVIDIA attributes these results to a combination of serving runtimes, expert-parallel execution, DeepGEMM-based kernels, MXFP4 and MXFP8 mixed-precision formats, the Dynamo serving stack, and the NVLink fabric connecting 72 GPUs.
Read the primary source: developer.nvidia.com ↗
Why it matters
The claims focus on the power and serving costs of AI agents, whose multi-step interactions can require repeated model calls, growing context, tool-use pauses and concurrent sessions. If independently confirmed, the results could affect how operators compare infrastructure for agentic workloads rather than relying on fixed-length chatbot benchmarks.
The source’s central argument is that agentic AI changes the economics of inference. A conventional chatbot request can often be modeled as one prompt followed by one response. An agentic session may make multiple model calls, invoke tools, delegate work and carry an expanding history from one turn to the next. NVIDIA cites an OpenRouter report saying average prompt tokens per request grew roughly fourfold across 100 trillion tokens of usage and that a single agentic request consumes 15 times the tokens of ordinary chat. Those figures are claims cited by NVIDIA, not independently established in this source.
Power efficiency matters because the benchmark measures useful interactive output against the power provisioned for an AI factory. A higher token-per-megawatt result could allow an operator to serve more concurrent agent sessions within a fixed power budget, or to provide the same capacity with lower infrastructure use. The practical value depends on whether the reported interactivity and latency remain acceptable, since a high aggregate throughput figure would be less useful if individual agent turns wait too long.
The technical details also show why system-level comparisons may matter more for agents than accelerator specifications alone. NVIDIA links the claimed performance to cache-aware routing, separate prefill and decode workers, communication overlap, expert parallelism and high-bandwidth connections across a rack-scale system. These are presented as NVIDIA’s explanation for the benchmark results, not as an independently verified causal analysis.
What to watch next
The central caveat is that NVIDIA measured the Vera Rubin results and says they are pending SemiAnalysis review. The source does not establish independent replication, commercial availability, pricing, deployment scale or performance across other models and workloads.
The most immediate question is whether SemiAnalysis reviews and confirms the Vera Rubin measurements. Until that review occurs, the 30x figure should be treated as a company-reported preview result rather than a settled industry benchmark. The source gives no independent test results, raw measurements, confidence intervals or details about how the comparison systems were configured beyond the named platforms and workloads.
The comparison is narrow. It uses prerecorded agentic-coding sessions and specific model workloads, including DeepSeek V4-Pro and Kimi K3 2.8T. The post does not show whether the same efficiency differences would hold for other models, different agent frameworks, different tool latencies, other concurrency levels or workloads outside coding. It also does not establish how the systems perform on reliability, quality, failed tool calls or the total cost of operating a complete production service.
The source does not state when Vera Rubin NVL72 will be generally available, how much it will cost, where it has been deployed or what customers have achieved with it. It also does not provide a direct comparison of energy use, cooling, networking and software costs outside the benchmark’s provisioned-megawatt measure. Those details will be important for organizations deciding whether benchmark-level efficiency translates into real operating savings. That distinction is important when interpreting the headline result, because the reported figure describes a particular benchmark comparison and not a universal measure of agentic-AI performance. The available description therefore supports a cautious reading of the result alongside the stated methodological caveats.
