What does Inference mean? Definition | AI Understanding

Definition

The runtime phase where a trained model generates predictions or outputs.

Related terms

Inference Endpoint

A deployed API interface that receives model requests and returns predictions in production.

Human-in-the-Loop

A workflow where humans review, guide, or override AI outputs.

Constitutional AI

A training and behavior-shaping approach where model outputs are guided by a fixed set of written principles.

Distilled Model

A smaller model trained to imitate a larger model's behavior while using less compute at inference.

Usage-Based Billing

Pricing where costs scale with API calls, tokens, inference time, or consumed compute.

Speculative Decoding

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

Learn more in our free guides

AI Inference AI Inference Optimization LLM Inference Routing and Load Balancing Seldon Core and Inference Graphs

See also

In-Context Learning Inference-Time Compute Hyperparameter Instruction Tuning Intent Classification Hallucination Jailbreak Guardrails

Browse the full AI Glossary Explore all free AI guides