What happened
OpenAI identified a coordinated campaign that sought to extract protected reasoning from its models – internal “thought‑process” data that is normally hidden from users. The campaign began in early July, peaked on July 24‑25 with roughly 16,000 requests from over 4,000 users, and involved more than 15,000 accounts before being fully disrupted by July 28. OpenAI attributes a core cluster of the activity to individuals associated with Moonshot AI, the developer of the Kimi chatbot. The operators did not breach encryption or databases; instead they manipulated model interactions to reproduce hidden reasoning in ways that violated OpenAI’s terms of service. In response, OpenAI deployed account enforcement, technical controls, and partner coordination, including bans on fraudulent accounts, strengthened signup and infrastructure safeguards, and new checks that detect streamed output that could expose reasoning. The company also shared findings with the Frontier Model Forum and government channels to help other developers defend against similar attacks.
OpenAI’s security team detected a coordinated effort to extract protected reasoning from its models beginning the first week of July. The activity escalated on July 24‑25, generating about 16,000 requests that used a specific extraction pattern across more than 4,000 user accounts.
The campaign did not involve breaking encryption or accessing stored user conversations. Instead, attackers manipulated model interactions to reproduce hidden reasoning, a technique known as adversarial , which violates OpenAI’s terms of service.
OpenAI attributes a core cluster of the activity to actors linked to Moonshot AI, the developer of the Kimi chatbot, though it cannot confirm whether all observed operators were part of a single coordinated group.
In response, OpenAI enforced account bans, tightened signup and infrastructure controls, added detection for streamed outputs that might expose reasoning, and closed a pathway that allowed replay of another user’s encrypted reasoning. The company also collaborated with third‑party service providers to disrupt related accounts.
Findings were shared with the Frontier Model Forum and relevant government channels to help other developers and public‑sector partners strengthen their defenses against similar attacks.
Why it matters
The incident highlights a growing security threat known as adversarial , where attackers harvest a model’s internal reasoning to train or improve other models without inheriting the original safety controls. Extracted reasoning could be repurposed for dual‑use applications, raising safety and national‑security concerns as frontier models become more capable. By publicly detailing the attack vector and mitigation steps, OpenAI signals the seriousness of the risk, encourages industry‑wide defenses, and provides a template for coordinated response. The attribution to Moonshot AI underscores that competitive pressures can drive actors to seek cheaper ways to replicate advanced capabilities, potentially accelerating an arms race in AI capability replication. Understanding and curbing such campaigns is essential for maintaining the integrity of measures and for informing policymakers about emerging threats.
Adversarial threatens the safety envelope of AI systems by allowing attackers to harvest internal reasoning without inheriting the original model’s safeguards, potentially enabling the creation of less‑controlled replicas.
Extracted reasoning can be used for dual‑use purposes, including applications that may have national‑security implications, especially as models advance in domains like bio‑technology or autonomous systems.
The public disclosure of the attack vector and mitigation steps serves as a warning to the broader AI community, encouraging proactive defenses and coordinated industry response.
Attribution to Moonshot AI illustrates how competitive pressures can motivate actors to pursue illicit shortcuts for replicating advanced capabilities, raising concerns about an emerging AI capability arms race.
OpenAI’s collaboration with industry partners and government entities demonstrates a growing recognition that security challenges of this scale require collective action beyond any single organization.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
Why can ethical evaluation not be reduced to one model score?
What to watch next
Future adversarial‑ attempts are likely to become more sophisticated as models improve, so monitoring for novel ‑pattern attacks will be critical. Watch for additional disclosures from OpenAI or other frontier labs about similar campaigns, as well as any regulatory or industry initiatives aimed at standardising defenses against reasoning extraction. The effectiveness of OpenAI’s new technical controls and partner‑sharing mechanisms will also be a key indicator of how well the ecosystem can collectively mitigate this class of threat.
The evolution of adversarial‑ techniques, especially as frontier models become more capable and their internal reasoning more valuable.
Potential disclosures from other AI labs about similar extraction attempts, which would indicate whether this threat is widespread.
Regulatory or industry initiatives aimed at standardising defenses against reasoning extraction, including possible mandates for transparency or reporting of such incidents.
Effectiveness of OpenAI’s newly deployed technical controls, such as the streamed‑output checks and account‑enforcement mechanisms, in preventing future campaigns.
Any escalation in the scale or sophistication of coordinated campaigns, particularly those linked to competitive actors like Moonshot AI.