What does On-Device AI mean? Definition | AI Understanding

Definition

AI inference performed locally on user hardware rather than in a remote cloud service.

Related terms

Causal Inference

Methods for estimating cause-and-effect relationships rather than simple correlations.

Semantic Search

Search that matches meaning rather than exact keyword overlap, often using embeddings.

The runtime phase where a trained model generates predictions or outputs.

Inference-Time Compute

The amount of processing power consumed while producing each response.

Intent Classification

Predicting the user's purpose from a text query to route it correctly.

Model Quantization

Reducing numeric precision of model weights to decrease memory and inference cost.

See also

Model Registry Output Parser Latent Space Prompt Template Knowledge Base Retrieval Precision Inference Endpoint Safety Case

Browse the full AI Glossary Explore all free AI guides