AI Glossary Term

What is Inference?

Definition

The runtime phase where a trained model generates predictions or outputs.

Related terms

Inference Endpoint

A deployed API interface that receives model requests and returns predictions in production.

Human-in-the-Loop

A workflow where humans review, guide, or override AI outputs.

Constitutional AI

A training and behavior-shaping approach where model outputs are guided by a fixed set of written principles.

Distilled Model

A smaller model trained to imitate a larger model's behavior while using less compute at inference.

Usage-Based Billing

Pricing where costs scale with API calls, tokens, inference time, or consumed compute.

Speculative Decoding

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

Learn more in our free guides

AI Inference AI Inference Optimization LLM Inference Routing and Load Balancing Seldon Core and Inference Graphs

Browse the full AI Glossary Explore all free AI guides