アプリケーションガイド
Gross Margins and Inference Costs in AI Businesses
An AI business's gross margin is its revenue minus the direct cost of delivering the product, divided by revenue.
このページでは4 分で読めます
概要
For AI software that cost includes inference, the compute spent every time a model answers a request. Inference grows with usage instead of staying close to zero for each extra user, so AI products often have lower and less predictable margins than classic SaaS. That affects how they set prices, how they raise money and whether they survive.
ディープダイブ
Gross margin is revenue minus cost of revenue, divided by revenue. In classic SaaS, cost of revenue is mostly hosting, customer support and third-party software. Serving one more user costs almost nothing, so mature SaaS companies commonly report gross margins of 70 to 80 percent or higher. AI products add a cost that behaves more like a raw material. Every prompt uses GPU time. The company pays for it either per token to an API provider such as OpenAI, Anthropic or Google, or through GPUs it owns or rents. A heavy user can cost many times more than a light user on the same subscription. A widely read 2020 Andreessen Horowitz essay, "The New Business of AI", argued that many AI companies looked like a mix of software and services, with gross margins pulled down by cloud compute and humans in the loop. Human work still counts: labeling data, reviewing outputs and handling escalations often sit in cost of revenue alongside moderation, retrieval infrastructure such as vector databases, and evaluation runs. Companies protect margins with a familiar set of levers: - **Model routing** sends easy requests to small, cheap models and keeps large models for hard ones. - **Caching** reuses work. Providers can cache repeated prompt prefixes, and a product can store answers to identical queries. - **Smaller or distilled models**, sometimes fine-tuned for one task, can replace a general model for high-volume work. - **Leaner requests** help too: tighter prompts, limits on output length, and batch processing for jobs that can wait. - **Pricing** is the last lever. Usage-based tiers, credits or caps tie revenue to cost. A common misconception is that falling token prices will fix margins on their own. Prices for a given level of capability have dropped sharply. But products often use up the savings by moving to newer models, longer contexts and agents that make many calls per task. Margin comes from engineering and pricing choices, not from some fixed property of AI.
戦略的影響
ビルドの選択
AI が実際の成果を向上させるかどうかは、アプリケーション レベルの設計によって決まります。
チームとワークフロー
ワークフローを適切に統合すると、ユーザーが信頼できる生産性が向上します。
リスクと安全性
適切な範囲のユースケースにより、変更の疲労と実装のリスクが軽減されます。
The Future of Gross Margins and Inference Costs in AI Businesses
Inference costs per unit of capability have been falling. Hardware, serving software and smaller models keep improving, and competition among model providers holds prices down. That trend should help, but it will not automatically raise margins, because longer contexts, reasoning models and multi-step agents use up much of the saving. More companies are likely to treat cost engineering as a core discipline and build in routing, caching and task-specific models from the start. Many will likely move from unlimited flat pricing to plans that account for usage. A business that depends entirely on one upstream provider will keep facing margin risk from that supplier's pricing decisions.
現実世界の実装
A customer-support chatbot startup sends simple FAQ questions to a small, cheap model and passes only complex tickets to a frontier model. This lowers its average cost per conversation without hurting answers on the hard cases.
A coding assistant sold at a flat monthly price per seat finds that a small group of power users accounts for a large share of its inference bill. It adds usage caps and a premium tier so revenue follows cost.
A contract-summarization tool puts its long system prompt and standard clause library at the start of every request. Provider prompt caching then bills those repeated tokens at a discounted rate.
A legal research company sees that one type of query dominates its traffic. It fine-tunes a smaller open-weight model for that task and stops paying a general-purpose API for it.
リスクとガードレール
壊れたプロセスを自動化すると、既存の問題がさらに拡大する可能性があります。
チームが過剰に自動化し、必要な人間の判断を排除してしまう可能性があります。
出力が継続的に評価されないと、品質が変動する可能性があります。
実装ロードマップ
現在のワークフローをマッピングし、最も摩擦が大きいステップを特定します。
完全自動化の前に人間によるチェックポイントを定義します。
プロンプト、エスカレーション パス、品質基準についてユーザーをトレーニングします。
タスクレベルの結果を追跡して、持続的な価値を確認します。
探検を続けましょう
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gross Margins and Inference Costs in AI Businesses quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
よくある質問
What is Gross Margins and Inference Costs in AI Businesses?
An AI business's gross margin is its revenue minus the direct cost of delivering the product, divided by revenue. For AI software that cost includes inference, the compute spent every time a model answers a request. Inference grows with usage instead of staying close to zero for each extra user, so AI products often have lower and less predictable margins than classic SaaS. That affects how they set prices, how they raise money and whether they survive.
According to the guide, why does serving an extra user usually cost more for an AI product than for classic SaaS?
In classic SaaS, an extra user adds almost no cost. In AI products, every request uses compute, paid per token or through GPUs, so cost rises with usage and behaves more like a raw material.
What does model routing do?
Routing matches each request to the cheapest model that can handle it well. That lowers average cost per request. Storing outputs for repeated questions is a different lever: caching.
What did the 2020 Andreessen Horowitz essay "The New Business of AI" argue?
The essay argued that cloud compute costs and human involvement, such as labeling and review, gave many AI companies lower gross margins than typical software businesses.
Why don't falling token prices automatically fix AI gross margins?
Cheaper tokens help. But when products adopt larger models, longer contexts or multi-call agents, total spend per task can stay flat or even rise.
To get the most out of provider prompt caching, how should you structure a prompt?
Caching works on a stable, repeated prefix. Keeping unchanging material at the start and variable content at the end means more of each request matches the cached prefix.
学び続ける
関連ガイド
このトピックのために選ばれたその他のガイド