What happened
Forbes reports that SemiAnalysis published AgentX, an open-source benchmark that replays anonymized coding-agent sessions rather than using fixed prompts. In the configurations covered, SemiAnalysis found Nvidia hardware had up to a fivefold cost-efficiency advantage over AMD at a specified operating point, with larger gaps on some open serving stacks. The supplied source does not independently confirm the benchmark or reproduce its results.
Forbes reports that SemiAnalysis published AgentX on August 24 as a trace-replay benchmark for production-style inference. Instead of sending uniform prompts and output lengths, AgentX replays the structure of real coding-agent sessions, including long histories, multiple turns and pauses while an agent uses tools. Forbes says SemiAnalysis built a corpus of more than 8,000 sessions and 610 billion tokens from intercepted Claude Code and Codex requests involving its own staff. The public version 1.0 subset contains 393 Claude Code sessions and is released under Apache 2.0. The source does not provide independent verification of those figures.
Forbes reports that the public traces are converted into session-scoped chained hash blocks of 64 tokens, preserving prefix relationships while removing original prompts and code. SemiAnalysis reported a median input length of 142,000 tokens, a median output of 444 tokens and a median 3.84-second gap between turns. Forbes also says 175 of the 393 public sessions spawned at least one subagent. Those characteristics are intended to test whether serving systems retain and reuse key-value cache state across turns, route a session to the worker holding that state and avoid unnecessary reprocessing of repeated context.
At one reported operating point, Forbes says SemiAnalysis measured Nvidia hardware as up to five times more cost-efficient than AMD when running GLM 5.3 through the open-source SGLang serving stack at 150 output tokens per second per user. The article says a B200 comparison with AMD’s MI355X showed an Nvidia advantage of about 57% at 108 tokens per second per user and 247% at 141 tokens per second per user. On Qwen 3.5 through SGLang, Forbes reports that Nvidia was more than 20 times ahead at 90 tokens per second per user. These are configuration-specific results, not a universal hardware ratio.
Read the primary source: forbes.com ↗
Why it matters
The report suggests that AI-inference performance for agents depends heavily on software, cache reuse, routing and long-context serving—not only accelerator specifications. That could affect infrastructure purchasing, cloud pricing and the competitiveness of AMD and Nvidia. The findings are limited to the tested hardware, models, software versions and workload traces.
The practical significance is that agent workloads can spend far more time processing or reusing context than generating final answers. Forbes reports a median AgentX input of 142,000 tokens against only 444 output tokens, a pattern unlike many conventional benchmarks. If similar traffic is common in production, the ability to reuse cached context and route follow-up turns efficiently could influence the cost and responsiveness of coding agents, research assistants and other tool-using systems. The supplied source does not establish how representative the traces are of the broader market.
Forbes attributes much of Nvidia’s reported advantage to the software layer. The article describes cache management, routing, scheduling, specialized kernels and boundary-aware incremental tokenization in Nvidia’s TensorRT-LLM stack. Forbes says incremental tokenization matched full-tokenization results across 1,087 tested transitions in one Qwen 3.5 trace while reducing mean processing time per turn from 185.1 milliseconds to 11.3 milliseconds. If accurate, such improvements can make an older or otherwise comparable accelerator more competitive by reducing repeated work. They also mean benchmark rankings may change quickly when serving software changes.
The report matters for competition because it challenges the assumption that a second accelerator supplier will automatically narrow Nvidia’s position through hardware pricing alone. Forbes says AMD’s ATOM engine beat a GB300 NVL72 system running vLLM across part of one Kimi K3 latency curve, and that AMD’s MI355X matched B200 on DeepSeek V4 with SGLang before later Nvidia and Inferact optimizations were merged upstream. That reversal illustrates both AMD’s potential and the importance of software adoption. Forbes also reports that ATOM has limited production use and that AMD backends were not listed for some vLLM context-parallel paths, but those software conditions can change.
What to watch next
The key follow-up is whether AMD’s ATOM optimizations and support for long-context serving are adopted more broadly in mainstream software such as vLLM and SGLang. Buyers should also examine production prefix-cache hit rates, host-memory use, session routing and turn-level latency. SemiAnalysis is expected, according to Forbes, to update the benchmark within a month.
The most important follow-up is whether the reported gap persists after software updates. Forbes says SemiAnalysis scheduled an AgentX update within a month, and the article notes that an upstream change on August 21 altered the DeepSeek V4 comparison. Future versions should show whether AMD’s ATOM capabilities reach commonly used serving stacks, whether vLLM and SGLang add more mature AMD support for long-context parallelism and offload, and whether Nvidia’s lead remains after comparable optimizations are applied to both sides.
Infrastructure buyers should ask providers for workload-specific evidence rather than rely on aggregate tokens-per-second figures. Forbes recommends examining sustained prefix-cache hit rates at production concurrency, the amount of host memory used to back accelerator memory, and whether session routing keeps a conversation with the worker holding its cached prefix. The article reports that SemiAnalysis found high-bandwidth-memory hit rates of 91% on one B300 configuration and 73% on a B200 configuration under different concurrency levels, with additional host-memory reuse. Those measurements are not directly comparable enough to establish a general rule, but they show which operational details can affect cost and latency.
The benchmark also leaves several material unknowns. Forbes says AgentX replaces anonymized trace content with synthetic tokens, which can distort speculative-decoding acceptance, and that SemiAnalysis used an acceptance length taken from SPEED-Bench rather than measuring it live. The traces come from coding harnesses with heavy context, while a leaner harness produces a different input distribution. AgentX’s closed-loop design can also change the workload mix as faster systems complete more requests. Google TPUs are absent, and later Nvidia Rubin hardware and AMD’s MI455X are outside the comparison described. Vendor responses, independent replications and results on non-coding agent workloads are not provided in the source.
