AI Glossary Term

What does “Inference Endpoint” mean?

Definition

A deployed API interface that receives model requests and returns predictions in production.

Browse the full AI Glossary Explore all free AI guides

How Inference Endpoint works

Example

A developer deploys a model and receives a URL. The mobile app sends text to that URL and receives the model's prediction back, without the app itself containing the model.

Why it matters

Endpoints put models behind a stable interface that scales and can be monitored. They also define access control and cost.

Common questions

Endpoint vs API?

An endpoint is a specific URL within an API where requests are sent.

How is it secured?

With authentication keys or tokens, network rules, and rate limits.

Can it scale?

Yes, many services add or remove servers based on traffic.

API (Application Programming Interface)A structured way for one software system to send requests to and receive responses from another system.InferenceThe runtime phase where a trained model generates predictions or outputs.Usage-Based BillingPricing where costs scale with API calls, tokens, inference time, or consumed compute.Inference-Time ComputeThe amount of processing power consumed while producing each response.Decision TreeA model that makes predictions through a sequence of if-then feature splits.EnsembleCombining predictions from multiple models to improve robustness or accuracy.

Learn more in our free guides

AI Inference AI Inference Optimization LLM Inference Routing and Load Balancing Seldon Core and Inference Graphs

Test yourself

AI Inference Quiz AI Inference Optimization Quiz LLM Inference Routing and Load Balancing Quiz