What happened
AI application startups are increasingly adopting open-weight models to mitigate rising API costs from OpenAI and Anthropic. Harvey, a legal AI unicorn, saw its gross margin drop to -50% in June due to high usage of OpenAI's GPT-4. In response, Harvey released a self-owned model supported by Moonshot AI's Kimi K3, which restored positive margins. Other companies, including Abridge, Decagon, and Ramp, are also developing or customizing their own models, with some migrating up to 80% of traffic to self-owned systems. This shift is driven by aggressive pricing strategies from model providers and the improving performance of open-source alternatives.
Harvey, a legal AI startup valued at $15.6 billion, experienced a significant drop in gross margin from 50% to -50% in June due to high API costs from OpenAI's GPT-4. To address this, Harvey released a self-owned model in August, supported by Moonshot AI's Kimi K3, which offers performance close to Anthropic's best products at a fraction of the cost, restoring positive margins.
The trend is spreading across multiple industries. Abridge is building a foundational model for clinical scenarios using NVIDIA's , while Decagon processes 80% of its query requests with its self-owned model. In fintech, Ramp and Rogo are exploring self-training, and in programming tools, Cursor and Cognition have released customized models.
OpenAI and Anthropic have shifted to charging enterprise users extra for model usage, punishing high-volume 'token-maximizing' patterns. Additionally, both companies are recruiting talent in startup core tracks and launching industry-specific plugins, creating direct competition with their customers. The risk of access suspension, as seen with Cursor after its acquisition by SpaceX, further incentivizes startups to seek model independence.
Despite the shift, most startups do not expect to fully abandon OpenAI and Anthropic. Harvey still relies on Anthropic's Claude Opus for complex tasks, indicating a mixed usage strategy. Challenges remain, including high talent costs for engineers, the need for proprietary data, and significant infrastructure expenses for managing open-weight models.
Why it matters
This trend signals a structural shift in the AI industry's cost dynamics, as application-layer companies seek to reduce dependence on expensive proprietary APIs. For OpenAI and Anthropic, which are preparing for IPOs, the outflow of enterprise customers to open-weight models could erode their primary revenue streams. The move also highlights the growing viability of open-source models as a cost-effective alternative, potentially reshaping the competitive landscape and forcing model providers to adjust their pricing and business strategies to retain enterprise clients.
The adoption of open-weight models by major AI startups represents a significant cost-saving measure that could alter the revenue dynamics for OpenAI and Anthropic, especially as they prepare for IPOs. This shift may force model providers to reconsider their pricing strategies and enterprise offerings to maintain customer loyalty.
The improving performance of open-source models, such as Moonshot AI's Kimi K3, is making them a viable alternative to proprietary APIs for many use cases. This development could accelerate the democratization of AI capabilities and reduce the barrier to entry for new AI applications, fostering innovation in sectors like legal, healthcare, and finance.
The trend also highlights the growing importance of data and infrastructure in AI development. Startups must invest in proprietary data and computing resources to train effective models, which could create new opportunities for data providers and cloud infrastructure companies while posing challenges for smaller firms with limited resources.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
What is the best response when AI Models Explained makes a mistake in production?
What to watch next
Monitor the financial performance of OpenAI and Anthropic as they approach their IPOs, particularly any changes in API pricing or enterprise retention strategies. Track the adoption rates of open-weight models among major AI application startups and the development of new self-owned models in sectors like healthcare, finance, and legal services. Also, observe the impact of talent and infrastructure costs on the feasibility of self-training models for smaller startups.
Watch for any changes in OpenAI and Anthropic's API pricing or enterprise contracts in response to the growing adoption of open-weight models. Their ability to retain enterprise customers will be crucial for their IPO valuations and long-term sustainability.
Monitor the performance and adoption of new self-owned models from startups like Harvey, Abridge, and Decagon. Success in these areas could validate the open-source approach and encourage further migration away from proprietary APIs.
Observe the impact of talent and infrastructure costs on the feasibility of self-training models. If these barriers remain high, smaller startups may continue to rely on a mix of open and closed-source models, while larger firms may fully commit to self-owned solutions.