AI Glossary Term
What does “Compute” mean?
The processing resources required to train and run models, often measured in FLOPS or GPU hours.
How Compute works
Example
Training a large language model can occupy thousands of GPUs for weeks, while answering one question takes a tiny fraction of that effort. Providers pay for both, and the totals show up in their hardware and electricity bills.
Why it matters
Compute is a main limit on what AI labs can build and who can afford to build it. It also drives energy use and costs that end up in product pricing.
Common questions
How is compute measured?
Often in FLOPs (floating-point operations) for total work, or in GPU hours for rented time on hardware.
Why does compute matter for AI progress?
Research on scaling laws found that, within limits, performance tends to improve as compute, data, and model size grow together.
Is compute only needed for training?
No. Running a trained model, called inference, also needs compute, and for popular products it can add up to more than training.