Chuyện gì đã xảy ra
Forbes reports that Nvidia’s AVO software harness enabled Anthropic’s Claude Opus 5 to complete every level in ARC-AGI-3’s public benchmark set without retraining or the model. The article attributes the improvement to persistent memory, supervision, iterative planning and error recovery, and swappable tools around the model.
Forbes reported on Aug. 24 that Nvidia’s AVO system helped Anthropic’s Claude Opus 5 move from completing about 30% of ARC-AGI-3’s public-set tasks to completing 100% of them. The article says the model itself was not retrained or fine-tuned and that no model weights were changed. The reported improvement therefore came from software operating around the model. Forbes presents the result as a continuation of Nvidia’s previously reported AVO benchmark showing, but adds details about the underlying model and the reported action count.
According to Forbes, ARC-AGI-3 places an agent in game-like environments without explicit instructions, rules or stated goals. The agent must infer what each environment requires through trial and error. Forbes says the public set contains 25 environments, with six to 10 levels each and 183 levels overall. It reports that AVO completed the public levels in 6,624 actions, approximately 12% fewer than the previous most efficient system. These figures describe the public benchmark set; the article says the private competition sets remain a more difficult test.
Forbes describes AVO as an “agentic variation operator” system, or harness, with four main elements: persistent memory, a supervisor, an iterative work loop and swappable tools. The memory carries forward what the agent has tried and learned. The supervisor can intervene when progress stalls. The loop repeatedly inspects the situation, plans, acts and evaluates the result. The tools can be changed for different tasks. Forbes says Nvidia first used the system to optimize GPU code and later replaced its code tools with game controls, framing the benchmark result as evidence that the architecture is not limited to one task.
The supplied Forbes article is a secondary report by a contributor whose page describes the work as independent expert analysis and insight. The benchmark scores, action count, system design and Claude Opus 5 deployment are therefore attributed to Forbes in this article and are not independently confirmed by the supplied source material. The source does not provide a reproducible evaluation log, a paper, a complete implementation, or independent testing of the claimed result.
Tại sao nó quan trọng
The report illustrates how agent performance can depend on the software system surrounding a model, not only on model weights or benchmark scores. If reproduced, the result could make memory, planning, tool use and recovery central engineering priorities for organizations deploying AI on long-running tasks.
The report focuses attention on a distinction that matters for deployed AI: a model’s isolated response quality is not the same as an agent’s ability to complete a long task. A model may produce useful individual steps yet fail because it forgets earlier attempts, repeats errors, loses track of a goal or cannot recover after an unsuccessful action. Forbes argues that AVO targets these system-level failure modes through memory, supervision and repeated evaluation. That is a practical engineering claim, not proof that the approach will work equally well outside the benchmark.
If Forbes’s account is accurate, the result suggests that software orchestration can unlock capabilities already present in a model without the expense or delay of training a new frontier model. This could shift competition toward the teams that build agent runtimes, memory systems, tool interfaces, monitoring and recovery mechanisms. It also means that comparing models without specifying the surrounding harness may give an incomplete picture of what users actually experience. The relevant unit of performance may increasingly be the model-plus-system combination.
The public impact remains uncertain. ARC-AGI-3 is designed to test adaptation to unfamiliar environments, which is relevant to general-purpose agents, but success in game-like settings does not establish dependable performance in workplaces, public services or safety-critical operations. Forbes connects the benchmark to enterprise adoption problems and cites an MIT NANDA estimate that only about 5% of examined enterprise AI initiatives produced measurable business value despite $30 billion to $40 billion in spending. Those figures and the article’s explanation of the causes are reported by Forbes but are not independently verified here.
The result may also increase, rather than reduce, demand for computing. A persistent agent that repeatedly plans, acts, evaluates and recovers can make many more model calls than a short chatbot exchange. More reliable completion could make additional AI use economically attractive, but the supplied source does not quantify AVO’s inference cost, latency, memory requirements or energy use. Without those figures, it is not possible to determine whether the reported efficiency gain translates into lower total operating cost.
Cơ chế tương tác: Nó thực sự hoạt động như thế nào
Khám phá công nghệ cơ bản đằng sau sự phát triển này một cách tương tác.
crm_get_transaction(id='4092').What is the most accurate way to describe what AI Agents can do today?
Xem gì tiếp theo
The key questions are whether the result can be independently reproduced, how much of the gain comes from AVO rather than Claude Opus 5 itself, and whether the approach transfers to practical work. The supplied source does not establish private-set performance, general enterprise reliability, operating cost, or broad availability.
Independent replication is the first priority. Reviewers should look for the exact Claude Opus 5 configuration, the AVO implementation, tool permissions, memory contents, stopping rules, action accounting and benchmark protocol. A 100% completion rate on a public set can be meaningful while still being sensitive to setup choices. The supplied Forbes article does not establish whether outside researchers have reproduced the result or whether the benchmark environments and evaluation procedures were independently audited.
The contribution of each component also needs to be separated. The report attributes the improvement to the complete harness, but it does not show ablation results for persistent memory, the supervisor, the iterative loop or the swappable tools. It is therefore unknown which elements matter most, whether the system adds substantial inference overhead, and whether similar gains would appear with other models. Comparisons should distinguish the base model, the model with a simple agent loop and the full AVO system.
Performance on the private ARC-AGI-3 set and on unrelated tasks will be important. Forbes says the public environments are not the final proving ground, but it provides no private-set score. Future evidence should test unfamiliar software workflows, research tasks and other settings where goals, state and feedback are more complicated than in the benchmark. The most useful results would report both success rates and failure costs, including actions taken, time, human intervention and model calls.
Finally, the source does not establish product availability, licensing, customer deployments or enterprise outcomes. Nvidia’s reported internal use of AVO for GPU-code optimization is not the same as a generally available product. Readers should also be cautious about the article’s broader investor and enterprise conclusions, including its discussion of companies positioned to benefit from agent software. Those are Forbes’s analysis, not independently demonstrated market results. The central unresolved question is whether a benchmark-leading harness can become a dependable, affordable and auditable system for real work.