What happened
NVIDIA and CrowdStrike evaluated an agentic cybersecurity system designed to connect simulated offensive attacks with defensive detection engineering in a continuous loop. The system used Nemotron 3 Ultra for defensive orchestration, a customized Nemotron 3 Super for detection generation and repair, and CrowdStrike Falcon sensors for telemetry. The testing took place in an isolated environment modeled on NVIDIA’s accelerated-computing infrastructure.
NVIDIA says the project combined an offensive red-agent harness with a defensive blue-agent harness inside an isolated representative environment. The red agent selected and executed an attack path, while Falcon endpoint sensors recorded telemetry and the system retained an action trace. The blue agent then used those records and contextual information about available data sources to identify which events could be reconstructed, which existing detections fired and where visibility gaps remained.
The defensive workflow used six controls: a schema knowledge base to constrain agents to supported Falcon fields and query syntax; telemetry grounding to tie outputs to observed events; a specialized detection-authoring model; automated artifact linting; replay against captured attack telemetry; and independent review. Candidate detections could be rejected for syntax errors, unsupported fields, failure to match the recorded attack or reliance on environment-specific strings such as particular hosts or IP addresses. Failed checks returned structured feedback for another attempt.
For the optimized open-model configuration, Nemotron 3 Ultra handled orchestration, including reconstructing attack sequences, planning detection steps and invoking tools. A customized Nemotron 3 Super handled detection writing and repair. NVIDIA says CrowdStrike built this specialist from Nemotron 3 Super using continual cybersecurity pretraining, supervised fine-tuning and reinforcement learning with verifiable rewards. The training set contained 9,349 detection-generation and multistep-repair examples covering 59 programmatically generated error types.
The reinforcement-learning process used NeMo Gym to execute generated queries in Falcon LogScale. Invalid queries received engine errors and up to five repair attempts; unresolved attempts received no reward. Valid generated and reference queries ran against the same synthetic logs, and the F1 overlap between returned events supplied the reward signal. NVIDIA says NeMo RL then supported group relative policy optimization. The source describes this as rewarding matching query results, not merely syntactically valid queries.
The evaluation proceeded from backtesting to live-fire testing. Backtesting measured whether detections matched the recorded attack, while live-fire testing deployed every passing detection verbatim against eight unseen attacks from the same scenario family. NVIDIA says the optimized open pipeline averaged a 41.9% detection rate across six authoring sessions, compared with 16.5% for Nemotron 3 Ultra using the default harness across eight sessions. The source attributes the improvement to the complete pipeline, not to an isolated model change.
Source details: developer.nvidia.com ↗
Why it matters
The work offers a concrete example of AI being used for a high-consequence operational task: repeatedly testing attack paths and turning observed gaps into machine-checked detection rules. It also illustrates a system-level approach in which model specialization, tools, replay tests and independent review are treated as part of the safety and reliability design rather than relying on a language model’s output alone.
The practical significance is the attempt to shorten a traditionally manual red-team and blue-team cycle. In the workflow described by NVIDIA, an attack creates telemetry, a defensive agent proposes a detection, deterministic checks and replay test it, and a newly seeded attack probes whether the detection generalizes. That feedback loop could allow security teams to examine more variations than periodic exercises permit, at least in bounded test environments.
The reported live-fire results are notable but should be read as company-reported findings from a narrow case study. NVIDIA says 11 backtest-passing detections from the Nemotron pipeline and 35 from a complete frontier system were tested against eight unseen attacks. Five open-model detections, or 45%, detected at least one attack, compared with 10 frontier detections, or 29%. The open pipeline averaged 2.6 detections per detection, compared with 1.1 for the frontier system.
Detection was not the final quality measure. To qualify as gold, a detection had to remain quiet on available test traffic and pass independent review for behavioral grounding, use of multiple signals and absence of environment-specific strings. NVIDIA says four of the five firing open detections and nine of the 10 firing frontier detections stayed quiet. The open detections covered all eight attacks and the frontier detections covered seven. After review, three open detections qualified as gold and none from the frontier system did.
The results suggest that a smaller or open model can be competitive when paired with a carefully designed harness, domain training and strong validation. That is different from showing that Nemotron alone is more capable than a proprietary model. The optimized configuration changed the model stack, training, context, tools and validation process, and the source explicitly says the comparison is system-level.
The approach also matters for governance and auditability. The system records traces, grounds queries in telemetry, rejects unsupported artifacts and uses a separate judge to review detections. Those mechanisms create checkpoints that human security teams can inspect. They do not remove the need for human oversight, however, particularly when a validated rule could affect live monitoring, incident response or access controls.
What to watch next
The main limitation is scope. NVIDIA describes one scenario family, small detection sets and limited benign traffic, so the results do not establish production-level false-positive rates or broad generalization. Future evidence should show performance across different environments, attack techniques and normal enterprise workloads, along with the rate and consequences of harness failures.
The most important unresolved question is generalization beyond the tested scenario family. NVIDIA says the evaluation used one family of scenarios and small detection sets. It does not establish how the system performs against unrelated attack techniques, different network architectures, cloud environments, identity systems or new telemetry schemas. Independent evaluations across several organizations would be needed to determine whether the reported advantage persists.
Production noise is another open issue. The source says the benign-traffic test was limited and therefore does not represent production false-positive performance. A detector that catches more attacks but generates excessive alerts could increase workload or obscure real incidents. Future reports should publish precision, recall, alert volume, analyst review burden and the conditions under which detections are withdrawn or revised.
Reliability of the agent harness also requires scrutiny. Three of eight live-fire runs experienced harness failures, although NVIDIA says they produced complete telemetry and were retained. The source does not explain the failure modes, whether they affected attack selection or detection authoring, or how such failures would be handled in an operational deployment. Auditable failure reporting and repeatable recovery procedures will be important if these systems move beyond isolated testing.
The comparison with the frontier system has meaningful unknowns. The source does not identify the frontier model, provide a full model or harness configuration, or present an ablation separating the effects of model choice, domain context, tools and validation. It also reports CrowdStrike’s separate claim that its Blue Solano model was more accurate than a leading proprietary frontier model at 99% lower cost in internal evaluations, but provides no supporting methodology in this article. That claim should not be generalized from the evidence presented here.
NVIDIA and CrowdStrike describe the system as a pattern for specialized agents: one model orchestrates, another performs a bounded expert task, and the harness supplies context and checks. What to watch next is whether this pattern produces reproducible gains under independent testing, whether organizations can safely connect validated outputs to production systems, and how operators verify that automated detections remain appropriate as infrastructure, attacker behavior and normal activity change.