Release-claim watchlist · Method 1.0.0

Announced. But available?

AI Release Tracker follows product release claims in our published news archive. Use it to revisit an announcement, inspect its recorded sources, and decide what needs checking next.

Snapshot assembled . Latest matching article: . This is not a source-access or availability-check timestamp.

Release-claim articles
223
Availability unchecked
223
7-day checks due
210
30-day checks due
84
Availability not independently checked. Every entry is an automatically selected claim, including stories that say “available now.” Unknown availability does not mean unavailable. No access tests or completed follow-ups are recorded here.

Where every claim stands

Follow-up status

Suggested check dates fall 7 and 30 days after each article was published. “Due” means the date has passed; no check has been completed for any claim.

7-day suggested checks

223 claims
  • Due; not completed 210
  • Upcoming; not scheduled 13
  • Date unknown 0

30-day suggested checks

223 claims
  • Due; not completed 84
  • Upcoming; not scheduled 139
  • Date unknown 0

Watchlist mentions

Articles mentioning
  • OpenAI50
  • Google / DeepMind43
  • NVIDIA36
  • Anthropic32
  • Meta27
  • Alibaba / Qwen21
  • Microsoft16
  • DeepSeek13

Mentions, not confirmed ownership: one article can mention several organizations.

The watchlist

Find a release claim

“Major” means names on our explicit watchlist, not a verified importance ranking. Organization matches are mentions, not confirmed product ownership. Dates are UTC.

Reset filters

223 matching articles · page 1 of 12. Export includes every matching record, not just this page.

Article published · Product

Reflection AI launches Beam, second‑best US open‑weight model on Artificial Analysis index

Availability not independently checked

Reflection AI’s new 501‑billion‑parameter mixture‑of‑experts model, Beam, scores 30 on Artificial Analysis’s Intelligence Index, placing it second among U.S. open‑weight models behind Meta’s Muse Spark Open.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • officechai.comhttps://officechai.com/ai/reflection-ai-releases-beam-becomes-second-best-us-open-model-on-artificial-analysis-intelligence-index/Recorded: Unknown · Article source field
  • officechai.comhttps://officechai.com/ai/reflection-ai-releases-beam-becomes-second-best-us-open-model-on-artificial-analysis-intelligence-index/Recorded: · Source history

Article published · Product

Llama.cpp adds integrated Hugging Face model discovery and download management

Availability not independently checked

The latest llama.cpp release introduces a native 'Discover Models' interface, allowing users to browse, search, and download GGUF models directly from the Hugging Face Hub within the application.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “adds”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • github.comhttps://github.com/ggml-org/llama.cpp/releases/tag/b11714Recorded: Unknown · Article source field
  • github.comhttps://github.com/ggml-org/llama.cpp/releases/tag/b11714Recorded: · Source history

Article published · Product

DeepSeek releases V4.1‑Flash‑1017 model, touts faster, leaner performance

Availability not independently checked

DeepSeek announced its V4.1‑Flash‑1017 model on Oct 17, saying it delivers stronger benchmark scores, quicker task completion and smaller output sizes than its predecessor, while remaining aggressively priced.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (4)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • straitstimes.comhttps://www.straitstimes.com/world/deepseek-pushes-out-updated-ai-model-it-says-is-faster-leanerRecorded: Unknown · Article source field
  • officechai.comhttps://officechai.com/ai/deepseek-v4-1-flash-1017-benchmarks-cost/Recorded: · Source history
  • finance.biggo.comhttps://finance.biggo.com/news/f993a5f3-fbbf-4070-917f-7a292fa8f58dRecorded: · Source history
  • straitstimes.comhttps://www.straitstimes.com/world/deepseek-pushes-out-updated-ai-model-it-says-is-faster-leanerRecorded: · Source history

Article published · Product

Nvidia unveils RTX Spark laptops and desktops for on‑device AI coding agents

Availability not independently checked

Nvidia’s new RTX Spark systems combine the Grace CPU, Blackwell GPU and up to 128 GB of unified memory, letting developers run coding agents, fine‑tune models and compile code on a single workstation.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “unveils”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • developer.nvidia.comhttps://developer.nvidia.com/blog/code-test-and-deploy-on-one-machine-with-nvidia-rtx-spark/Recorded: Unknown · Article source field
  • developer.nvidia.comhttps://developer.nvidia.com/blog/code-test-and-deploy-on-one-machine-with-nvidia-rtx-spark/Recorded: · Source history

Article published · Enterprise

Anthropic opens Singapore office and signs MoU to expand Claude deployments in public sector

Availability not independently checked

Anthropic announced a new commercial office in Singapore and a memorandum of understanding with the Ministry of Digital Development and Information, formalising collaborations that already see Claude used by GovTech, Grab, Sea, DBS and other regional firms.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “opens”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • artificialintelligence-news.comhttps://www.artificialintelligence-news.com/news/singapore-partners-with-anthropic-on-public-sector-ai-deployments/Recorded: Unknown · Article source field
  • artificialintelligence-news.comhttps://www.artificialintelligence-news.com/news/singapore-partners-with-anthropic-on-public-sector-ai-deployments/Recorded: · Source history

Article published · Product

Microsoft announces MAI-Base-2, a 2.8 trillion parameter AI model

Availability not independently checked

Microsoft revealed MAI-Base-2, a 2.8 trillion parameter mixture-of-experts model trained on 16,000 Nvidia GB200 GPUs, claiming significant efficiency gains over DeepSeek-V4-Pro-Base.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “announces”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • finance.biggo.comhttps://finance.biggo.com/news/628565eb-7cc6-46b5-a61f-8be6fcbebc5bRecorded: Unknown · Article source field
  • finance.biggo.comhttps://finance.biggo.com/news/628565eb-7cc6-46b5-a61f-8be6fcbebc5bRecorded: · Source history

Article published · Product

Meta releases Muse Spark Open, an 816‑billion‑parameter model for Apple Mac Studios

Availability not independently checked

Meta unveiled Muse Spark Open, an 816‑billion‑parameter open‑weight model that runs on two Apple M5 Ultra Mac Studios and is available via major cloud marketplaces.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (5)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • techrepublic.comhttps://www.techrepublic.com/article/news-meta-muse-spark-open-apple-mac-studio/Recorded: Unknown · Article source field
  • startupfortune.comhttps://startupfortune.com/meta-released-an-open-ai-model-so-big-it-needs-1tb-of-mac-memory/Recorded: · Source history
  • tradersunion.comhttps://tradersunion.com/news/companies/show/3835582-meta-muse-spark-open-coding-ai/Recorded: · Source history
  • eu.36kr.comhttps://eu.36kr.com/en/p/4027793714991746Recorded: · Source history
  • techrepublic.comhttps://www.techrepublic.com/article/news-meta-muse-spark-open-apple-mac-studio/Recorded: · Source history

Article published · Product

NVIDIA releases Nemotron 3.5 Super VL for multimodal enterprise agents

Availability not independently checked

NVIDIA says its open-weight Nemotron 3.5 Super VL model combines multimodal reasoning, tool use and enterprise customization for long-running AI agents.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • developer.nvidia.comhttps://developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specialized-agents-with-multimodal-reasoning/Recorded: Unknown · Article source field
  • developer.nvidia.comhttps://developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specialized-agents-with-multimodal-reasoning/Recorded: · Source history

Article published · Product

Intel launches tuned inference containers for Xeon processors

Availability not independently checked

Intel says its new Inference Microservices automatically select processor-specific settings for running supported language models behind an OpenAI-compatible API.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • intel.comhttps://www.intel.com/content/www/us/en/newsroom/news/artificial-intelligence/intel-inference-microservices-tuned-llm-processors.htmlRecorded: Unknown · Article source field
  • intel.comhttps://www.intel.com/content/www/us/en/newsroom/news/artificial-intelligence/intel-inference-microservices-tuned-llm-processors.htmlRecorded: · Source history

Article published · Security

Google DeepMind and Singapore announce on-device scam detection model

Availability not independently checked

Google DeepMind and Singapore’s GovTech say they have developed an on-device AI model that detects manipulative scam conversations while keeping message analysis on users’ smartphones.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “announce”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • deepmind.googlehttps://deepmind.google/blog/on-device-ai-scam-detection-mddi/Recorded: Unknown · Article source field
  • deepmind.googlehttps://deepmind.google/blog/on-device-ai-scam-detection-mddi/Recorded: · Source history

Article published · Product

Google broadly releases Gemini 4 Argon with introductory API pricing

Availability not independently checked

Gulf News reports that Google opened Gemini 4 Argon to developers and businesses on October 13, with temporary API prices and claims about long-horizon tasks and prompt-injection resistance.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (3)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • gulfnews.comhttps://gulfnews.com/technology/google-launches-gemini-4-argon-ai-model-for-developers-businesses-1.500709608Recorded: Unknown · Article source field
  • venturebeat.comhttps://venturebeat.com/technology/google-makes-gemini-4-argon-broadly-available-with-limited-time-low-api-pricing-of-2-10-per-1m-tokenRecorded: · Source history
  • gulfnews.comhttps://gulfnews.com/technology/google-launches-gemini-4-argon-ai-model-for-developers-businesses-1.500709608Recorded: · Source history

Article published · Innovation

Alibaba’s Qwen introduces QwenGyre for long-horizon agent training

Availability not independently checked

MarkTechPost reports that Alibaba’s Qwen team introduced QwenGyre, an elastic reinforcement-learning framework designed to train coding agents on tasks lasting hours and involving hundreds of model and tool interactions.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “introduces”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • marktechpost.comhttps://www.marktechpost.com/2026/10/12/alibaba-qwen-introduces-qwengyre-elastic-reinforcement-learning-rl-for-xlong-horizon-agents/?amp=1Recorded: Unknown · Article source field
  • marktechpost.comhttps://www.marktechpost.com/2026/10/12/alibaba-qwen-introduces-qwengyre-elastic-reinforcement-learning-rl-for-xlong-horizon-agents/?amp=1Recorded: · Source history

Article published · Product

Meta launches Muse Wonder and Muse Wonder Full models with subscription plans

Availability not independently checked

Meta unveiled two new AI models, Muse Wonder and Muse Wonder Full, positioning them as its most capable offerings yet and tying them to paid subscription services for its personal AI agent and coding tool.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Upcoming; not scheduled
30-day suggested check
· Upcoming; not scheduled
Recorded source links (3)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • economictimes.indiatimes.comhttps://economictimes.indiatimes.com/tech/artificial-intelligence/meta-releases-muse-wonder-ai-models-to-challenge-anthropic-and-openai/articleshow/134901136.cmsRecorded: Unknown · Article source field
  • cnbc.comhttps://www.cnbc.com/2026/10/12/meta-muse-wonder-ai-model-agent.htmlRecorded: · Source history
  • economictimes.indiatimes.comhttps://economictimes.indiatimes.com/tech/artificial-intelligence/meta-releases-muse-wonder-ai-models-to-challenge-anthropic-and-openai/articleshow/134901136.cmsRecorded: · Source history

Article published · Product

Developer releases ArtCraft, an open-source AI-built alternative to Adobe suite

Availability not independently checked

Developer Brandon Thomas has released ArtCraft, a free, open-source suite of seven applications designed to mimic Adobe's creative software, built using Claude Opus 5.5. While the tools are functional, testing reveals significant gaps in performance and feature parity compared to Adobe's mature products.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • finance.biggo.comhttps://finance.biggo.com/news/73561259-ed7b-4ea5-800d-cadb1acafcffRecorded: Unknown · Article source field
  • finance.biggo.comhttps://finance.biggo.com/news/73561259-ed7b-4ea5-800d-cadb1acafcffRecorded: · Source history

Article published · Product

Qwen Code Desktop v0.25.1 adds Linux aarch64 support and managed agent improvements

Availability not independently checked

The latest Qwen Code Desktop release introduces Linux aarch64 support and significant infrastructure updates for managed AI agent sessions and workspace runtime reliability.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “adds”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • github.comhttps://github.com/QwenLM/qwen-code/releases/tag/desktop-v0.25.1Recorded: Unknown · Article source field
  • github.comhttps://github.com/QwenLM/qwen-code/releases/tag/desktop-v0.25.1Recorded: · Source history

Article published · Product

Anthropic launches Claude Managed Agents dynamic workflow beta

Availability not independently checked

Anthropic has opened public beta access to Claude Managed Agents, a dynamic workflow feature allowing a main agent to orchestrate up to 1,000 sub-agents. In internal testing, this multi-agent approach detected 66 out of 70 embedded bugs in 116,000 lines of code, significantly outperforming single-agent modes.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • eu.36kr.comhttps://eu.36kr.com/en/p/4020977816539010Recorded: Unknown · Article source field
  • eu.36kr.comhttps://eu.36kr.com/en/p/4020977816539010Recorded: · Source history

Article published · Product

Google unveils enterprise Gemini agent with dedicated Workspace account

Availability not independently checked

Google Cloud has launched a unified Gemini-based enterprise agent that operates with its own Google Workspace account and email address, enabling autonomous task execution across internal systems.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “unveils”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • finance.biggo.comhttps://finance.biggo.com/news/7b42b30a-2d29-473b-85f0-8a63f8bc22ebRecorded: Unknown · Article source field
  • finance.biggo.comhttps://finance.biggo.com/news/7b42b30a-2d29-473b-85f0-8a63f8bc22ebRecorded: · Source history

Article published · Industry

Chinese firms accelerate AI development with 16 new model releases in one month

Availability not independently checked

Chinese developers, led by DeepSeek and Xiaomi, have released 16 new AI models in a single month, emphasizing a strategy of providing cheaper, open-weight systems amid intensifying global competition.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “releases”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • hokanews.comhttps://www.hokanews.com/2026/10/chinese-firms-release-16-ai-models-in.htmlRecorded: Unknown · Article source field
  • hokanews.comhttps://www.hokanews.com/2026/10/chinese-firms-release-16-ai-models-in.htmlRecorded: · Source history

Article published · Product

Vannor launches autonomous AI marketing agents that claim 30‑fold boost in Google rankings

Availability not independently checked

Vannor unveiled an agentic marketing platform that uses autonomous AI agents on Nvidia hardware to automate research, content creation and SEO, reporting a 554% rise in search impressions and a 30‑times increase in first‑page rankings within a month.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (2)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • pluang.comhttps://pluang.com/en/news-feed/vannor-luncurkan-pemasaran-agentik-setelah-ai-agen-meningkatkan-ranking-googleRecorded: Unknown · Article source field
  • pluang.comhttps://pluang.com/en/news-feed/vannor-luncurkan-pemasaran-agentik-setelah-ai-agen-meningkatkan-ranking-googleRecorded: · Source history

Article published · Product

Microsoft launches Decision-1 model for fast decision scoring

Availability not independently checked

Microsoft announced Decision-1, a purpose‑built decision‑scoring model that claims top accuracy and latency on structured decision tasks, now available via Microsoft Foundry and OpenRouter.

Above: AI Understanding’s article text, not a quotation from a linked source. Automatic title match: “launches”. Announcement date: unknown. Article updated: .

7-day suggested check
· Due; not completed
30-day suggested check
· Upcoming; not scheduled
Recorded source links (4)

Source types are unclassified. Links have not been opened by this tracker; source publication and access dates are unknown. A recorded date only indicates when the archive added the link.

  • commandline.microsoft.comhttps://commandline.microsoft.com/microsoft-decision-1-model-foundry/Recorded: Unknown · Article source field
  • finance.biggo.comhttps://finance.biggo.com/news/4941336a-143f-4f17-9a27-3c8cdeb1c9a1Recorded: · Source history
  • eu.36kr.comhttps://eu.36kr.com/en/p/4019383942434688Recorded: · Source history
  • commandline.microsoft.comhttps://commandline.microsoft.com/microsoft-decision-1-model-foundry/Recorded: · Source history

Open method · 1.0.0

Methodology & limits

01Selection

Title must contain release wording and either product wording or have the Product category. A watchlist name must appear in the title or excerpt. Mentions are not verified product ownership. This is an automatic candidate filter, not independent verification or an importance ranking.

02Unit

One canonical published article matching the watchlist and title rules; not one unique product or launch.

03Follow-ups

Suggested checks at 7 and 30 UTC calendar days after article publication (a proxy, not the announcement date). Due means the suggested date has arrived, not that a check ran. No reminders or checks are scheduled. Updated articles retain their original publication anchor.

04Coverage

Only articles returned by getAllNewsArticles, including its existing publication, suppression and media filters. That reader can silently fall back to bundled articles when the database is unavailable. Feed completeness cannot be established here; this is not a census of AI releases.

05Reproducibility

Download and retain the JSON snapshot: article text and evidence links may change with editorial updates. generatedAt is dataset assembly time, not a source access or verification time. Filters affect records, not the full-watchlist summary. Records sort newest first, undated last, then by canonical URL.

06Data source

Data source: AI Understanding’s published article archive (database and bundled articles). The existing reader does not expose which origin supplied each record or whether fallback occurred. Linked evidence is unclassified, not automatically primary.

  • Keyword selection can include rumors, commentary, comparisons and non-launch announcements, and can miss releases using other wording or names.
  • Several articles can concern one release; one article can concern several products. Mentioned organizations can be competitors rather than the releasing organization.
  • Region, account tier, waitlist, rollout stage, price and actual access remain unknown. This tracker does not measure model quality or run workflow tests.

What would count as confirmed? A separately documented human access check with time, region, account tier, product version, steps and evidence of a successful representative action. It would confirm only those conditions. This version records none; source claims, including first-party announcements, never become confirmation automatically.

The JSON contains the selection patterns, full watchlist, matched wording, dates, source provenance, unchecked status and suggested follow-ups. Missing dates remain null. Report a mismatch or missing source.

Protocol only · No results yet

Can three tools turn a source packet into usable nonprofit work?

Our first repeatable test design compares 3 tools × 10 tasks × 3 repetitions = 90 planned runs. It covers factual extraction, briefings, arithmetic, outreach, planning, privacy and uncertainty using the same synthetic source packet.

  1. Before running: select three accessible tools, freeze their versions and settings, publish the task packet and answer key, verify current prices, and approve a total cash and human-time cap.
  2. Run each task in a fresh session three times with identical inputs and rotated tool order. Preserve failures, refusals, outputs, time and cost. Stop at the approved cap.
  3. A human scores correctness, completeness, instruction compliance and usability from 0–2 each. A pass needs at least 7/8, full correctness and compliance, and no critical error. A second reviewer checks every proposed pass and critical error.
  4. Publish all denominators, disagreements, missing runs and limitations. Do not report results until runs and human review exist. Three repetitions cannot establish broad model superiority.

The full execution protocol, ten tasks, shared fixture, answer key, scoring and budget preflight live in the repository document docs/AIU-ORIGINAL-TEST-PROTOCOL.md. Tool selection, funding and execution are still pending.

Get the weekly AI Understanding briefing