What happened
Attackers breached Fireworks AI storage systems, accessing Notion AI chat transcripts stored for the Insights feature. Fireworks confirmed the incident on November 9, noting that while core inference services remained unaffected, stored data was compromised. Notion advised customers to treat the data as exposed.
VentureBeat reports that attackers breached Fireworks AI storage, specifically accessing systems that held Notion AI chat transcripts. The breach occurred via a third-party cloud provider that hosted Fireworks R&D machines, with unauthorized access beginning on October 23 and leading to additional customer data impact by October 24.
Fireworks AI stated that its core Inference, Training SDK, and Nexus services, which operate under a default (ZDR) policy, were not impacted. However, the breach reached storage systems used for features that do store customer content, such as stored responses, batch jobs, and training data. CEO Lin Qiao clarified that batch inference was excluded from the 'no impact' statement, and the company disabled self-serve dataset uploads and downloads during the investigation.
Notion confirmed that fewer than 1% of its workspaces may have been affected. The compromised data included Notion AI chat transcripts used for the 'Insights' feature, which Notion used for classification jobs to help customers understand their AI usage. Notion advised customers to treat this data as if it were exposed, noting that for the vast majority of impacted workspaces, the transcripts were unattributable.
Fireworks attributed the attack to a financially motivated group aiming to sell tokens from unauthorized LLM access to illicit routing gateways, a practice known as 'LLMJacking.' The company noted that environment variables, including API keys and tokens, were captured in certain affected accounts, and a subset of incomplete model weights and datasets were accessed.
Source details: venturebeat.com
Why it matters
This incident highlights a critical security gap in AI infrastructure where '' defaults do not apply to all features, such as batch jobs and stored responses. Organizations relying on AI vendors must audit specific feature configurations, as default settings may leave sensitive data vulnerable to third-party breaches. The breach underscores the need for precise vendor questionnaires regarding data storage and encryption practices.
The incident exposes a significant discrepancy between AI vendors' marketing of '' and the actual storage practices of specific features. While Fireworks' default policy rejects requests that persist customer content, features like batch inference and stored responses explicitly store data. This means that organizations assuming all data is ephemeral may be leaving sensitive information vulnerable in storage systems.
The breach demonstrates how a third-party compromise can cascade through AI infrastructure, affecting downstream customers like Notion. This highlights the importance of supply chain security in AI deployments, where the security posture of a vendor's underlying cloud providers and development environments directly impacts the confidentiality of end-user data.
Security experts, including Wiz and the Cloud Security Alliance, have emphasized the need for more detailed vendor questionnaires. Organizations should ask specific questions about how customer secrets are encrypted, who can decrypt them, and whether internal staff or automated jobs can read plaintext secrets. This incident serves as a case study for the limitations of generic security assurances in AI services.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
Why can ethical evaluation not be reduced to one model score?
What to watch next
Monitor for further disclosures on the scope of affected data and whether other AI vendors have similar storage vulnerabilities. Watch for regulatory responses regarding AI vendor security standards and data retention transparency.
Monitor for further disclosures from Fireworks and Notion regarding the specific types of data accessed and the full scope of the breach. Additional details on the 'LLMJacking' attack vector may emerge as threat intelligence groups continue to analyze the incident.
Watch for regulatory or industry responses to the incident, particularly regarding standards for AI vendor data retention transparency. The Australian government's recent move to require AI firms to report cyber breaches may be influenced by incidents like this one.
Observe how other AI infrastructure providers respond to similar security concerns, potentially leading to industry-wide changes in how data retention policies are communicated and implemented.