What happened
NVIDIA describes a collaboration in which Siemens used NeMo Gym to evaluate and refine the Siemens Fuse EDA , which orchestrates tools for standard-cell characterization. The team verified final Liberty files against reference artifacts and used those results to revise the agent’s skills and Model Context Protocol tool descriptions.
NVIDIA says Siemens built a custom NeMo Gym evaluation environment for the Solido Characterization Suite agent. The workflow covers tool setup, SPICE simulation, distributed computation, Liberty-file generation, debugging, and final validation. The agent accesses characterization tools through MCP servers and skills exposed by the Siemens Fuse EDA .
The evaluation used the open-source GF180MCU process design kit, covering 229 standard cells and 36 process, voltage, and corners. For tuning, each run was restricted to two cells across those corners, took approximately 1.5 to 2 hours, and used about 25 rollouts with Claude Opus 4.8 in Claude Code as the fixed model and harness.
The team created a verifier that checked coverage, validity, relative validation, integrity, and the proportion of values within tolerance. NVIDIA says end-to-end checks exposed missing cells and operating conditions, incorrect slew thresholds, and timing or power values outside tolerance. After simplifying overlapping interfaces and adding explicit JSON schemas to tool inputs, the reported mean composite score rose from approximately 55% to 95%. Agent turns fell from 155 to 53, while output tokens fell from about 81,000 to 29,000.
Source details: developer.nvidia.com ↗
Why it matters
The source presents a practical method for testing AI agents whose errors can remain plausible while propagating through complex engineering workflows. Evaluating final artifacts, rather than isolated tool calls, could help chip-design teams identify failures that ordinary intermediate checks miss. However, the reported improvement is a source claim from NVIDIA and Siemens, not an independently reproduced result, and the post does not establish production-wide reliability or general performance across other models, designs, or EDA workflows.
Chip-design agents operate across dependent stages, so an early configuration or tool-use error can produce a plausible but unusable final artifact. The source’s central contribution is an evaluation pattern that treats the final engineering output as the object of verification and uses rollout traces to locate the cause of failure.
The reported score increase suggests that tool descriptions and agent skills can materially affect reliability and efficiency, not only the underlying model. This is a useful practical implication for EDA and other engineering teams building long-running tool-using agents.
The evidence has important limits. The source reports results from one agent workflow, one fixed model and harness, a restricted task subset, and approximately 25 rollouts. It provides no independent replication, statistical uncertainty, comparison with alternative models, or evidence that the measured score corresponds to production-level correctness.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
crm_get_transaction(id='4092').An agent must create a draft calendar event for Tuesday at 2 p.m. Which evidence would establish the requested result?
What to watch next
The key unknowns are whether the reported gains transfer beyond the restricted benchmark, whether they hold with other models and agent harnesses, and whether the workflow has been deployed in production. The source does not document commercial access, pricing, or a broader availability program for the evaluated setup.
Whether Siemens or NVIDIA publishes the verifier, benchmark configuration, rollout traces, or reproducible implementation details needed for outside evaluation.
Whether the method improves reliability on the full 229-cell dataset, additional process design kits, broader characterization tasks, or other EDA and computer-aided engineering workflows.
Whether the optimized agent is available to customers, under what licensing or service terms, and whether the reported reductions in turns and tokens translate into lower operational cost in production.
How teams handle cases where reference artifacts, tolerances, or domain-specific validity gates are incomplete or unavailable.