Gross Margins and Inference Costs in AI Businesses Quiz

Answers and explanations appear as you take the quiz.

1. According to the guide, why does serving an extra user usually cost more for an AI product than for classic SaaS?
   • AI products must buy a separate software license for each user
   • AI products need far larger customer databases
   • Every prompt consumes GPU time, a variable cost that grows with usage
   • AI products require more salespeople per customer
2. What does model routing do?
   • Sends easy requests to small, cheap models and keeps large models for hard ones
   • Sends every request to the most capable model to guarantee quality
   • Spreads requests evenly across several providers to get volume discounts
   • Stores model outputs so identical questions are never answered twice
3. What did the 2020 Andreessen Horowitz essay "The New Business of AI" argue?
   • AI companies would soon have higher gross margins than any SaaS business
   • Inference costs were too small to affect AI company finances
   • AI companies should avoid using cloud providers entirely
   • Many AI companies looked like a mix of software and services, with lower margins from compute and humans in the loop
4. Why don't falling token prices automatically fix AI gross margins?
   • Token prices have not actually fallen for any capability level
   • Products often use up the savings by moving to newer models, longer contexts and agentic workflows
   • Gross margin does not include compute costs
   • Providers raise prices again every time they cut them
5. To get the most out of provider prompt caching, how should you structure a prompt?
   • Put the user's question first so the model reads it before anything else
   • Randomize the order of instructions on each call to avoid stale answers
   • Put static instructions, tool definitions and reference documents first, and variable user content last
   • Split every prompt into many small calls so each one is cached separately
6. Why does the guide recommend tracking contribution margin by customer cohort instead of relying on averages?
   • Averages overstate the cost of light users
   • Cohort tracking is required by accounting standards
   • Averages make routing impossible
   • Averages hide the power users whose heavy usage can wipe out profit
7. How does a routing cascade work?
   • It tries a small model first and escalates to a larger one when a validation check or confidence signal fails
   • It sends each request to every available model and picks the longest answer
   • It always starts with the largest model and falls back to smaller ones when budgets run out
   • It stores each answer and replays it to future users with different questions
8. Why should cost be modeled per task rather than per API call?
   • Providers bill by task, not by token
   • An agent may make many calls for a single user action, so per-call figures understate true cost
   • Per-call costs are always zero for cached prompts
   • Tasks and calls always cost exactly the same