Evaluating AI Agent Skill Performance with NVIDIA SkillEvaluator
NVIDIA SkillEvaluator measures the impact of verified skills on AI agent performance through a three-tier evaluation process.
Read story Verified source: developer.nvidia.com
Updated daily1421 verified stories
AI News.Without the noise.
Source-checked AI coverage of product launches, policy shifts, safety research, and industry moves, explained in plain English by a nonprofit education team.
Verified sourcing
Every story links to the strongest available evidence: original sources when available, otherwise clearly attributed reporting.
Plain English
What happened, why it matters, and what to watch — no jargon tax.
No filler
When the signal is thin, we publish nothing rather than padding the feed.
AI news archive — Page 117
A growing stream of verified perspectives for people who need to understand AI without chasing hype.
Paper Introduces "Shadow Evaluations": AI Agents Did the Engineering but Failed Two Research Questions
A 24-author preprint had frontier AI agents attempt the central research questions of two unpublished NeurIPS 2026 submissions, then had the papers' own authors grade the results. The agents handled the engineering unaided over six days but were unambiguously rejected on the research.
arxiv.org
Warp Opens Early Access to "Factories," a Config-as-Code System for Running Fleets of Coding Agents
Warp is taking early-access requests for Warp Factories, which defines fleets of coding agents as code — repos, models, permissions and human checkpoints in one YAML file, driven by CLI, API, SDK and MCP. Its automation and cost figures are vendor claims: no pricing, general availability date or independent testing.
warp.dev
More stories
9 storiesBenchmark Says Top Multimodal Models Score Under 10% at Reading Words From Pen Sounds and Hand Motion
A new arXiv paper introduces a test in which models must infer a written word from pen-scratch audio and hand-movement video, with no ink visible. The authors report humans above 80% ordered letter accuracy and leading models below 10% — and that giving models both modalities often made results worse.
arxiv.orgPaper Says Agent-Aware Cache Management Cuts First-Token Delay Up to 45% in Multi-Agent Serving
A new arXiv preprint describes CacheScout, a layer built on the open-source vLLM server that decides what to keep in a model's key-value cache based on which agent is likely to run next. The authors report double-digit latency and throughput gains; the workloads, models, and hardware are not stated in the abstract.
arxiv.orgAudit of an OpenAI AI-Generated Proof Finds a Reversed Condition, and Publishes a Repair
Two researchers say a lemma proof in Chapter 6 of OpenAI's mathematics document has a polarity error: a test in terms of average success where the next step needs a large conditional failure. They give a counterexample and a corrected proof, and caution that this is not verification of the chapter's main theorem.
arxiv.orgReplication Study Says FLOPs Still Mispredict AI Runtime, and the Proposed Fix Fails on Newer Hardware
A preprint by two researchers reproduces an earlier study on why equal FLOP counts do not mean equal execution time. It confirms the underlying claim but reports that the α-FLOPs correction formula generally underestimates runtime on newer hardware, which shows jumps and oscillations the formula does not capture.
arxiv.orgBenchmark Paper Finds Four Ways to Query Enterprise Data With LLMs All Score Under 26%
A new arXiv preprint pits four architectures for natural-language querying of enterprise databases against each other on a synthetic bilingual benchmark. None answered more than about a quarter of cases correctly, and the design that scored highest was not the safest or the cheapest.
arxiv.orgPaper Proposes Grading AI Security Agents Without Labels by Measuring Convergence to a Stronger Model
A new arXiv preprint argues security teams can judge whether a memory- or retrieval-equipped AI agent is learning by measuring how far it closes the gap to a stronger "teacher" model, rather than on labeled benchmarks that are often scarce or stale. Judging by a similarly powered model gave no usable signal.
arxiv.orgNew Benchmark Tests Whether AI Assistants Can Remember a Year of Phone Use
A 17-author technical report posted to arXiv introduces MobileMem, a benchmark and framework for on-device long-term memory built from a year-scale collection of mobile experiences. The abstract describes the design but reports no scores, and key details about the underlying data remain undisclosed.
arxiv.orgPaper Reports Brain-Like Modular Organization Emerging Inside Large Language Models
A new arXiv preprint says large language models develop functionally specialized internal structure that lines up with distinct human brain networks, based on circuit analyses across 46 tasks in four cognitive domains. The abstract page leaves key methodological details unstated.
arxiv.orgPaper Finds Late Layers of a Mixture-of-Experts Model Tolerate Heavy Expert Masking
A preprint reports that disabling low-magnitude experts in the last five layers of a 35-billion-parameter Mixture-of-Experts model preserved far more usable code-translation outputs than spreading the same cuts across all layers. It covers one model and one benchmark, and the abstract reports no unmasked baseline.
arxiv.org
One useful briefing each week
Keep up with AI without living in the feed.
Get the week’s verified AI news, original data, useful tools, learning picks, and fresh AI jobs.
Build with our audience
Hiring an AI professional or launching a useful AI product? Put it in front of people who came here to learn and act.
Post an AI job Submit an AI tool