Free AI glossary

The AI Glossary

Plain-language definitions of the AI terms you keep hearing — for students, professionals, and anyone curious about how AI works.

213terms
25letters covered
A–ZSearch + alphabet filter
100%Plain-language definitions

A19 terms

Agentic Loop

An iterative cycle where an AI agent observes, plans, acts, and reflects until it completes a goal or hits a stop condition.

Agentic Workflow

A multi-step process where an AI system plans, executes, checks results, and iterates toward a goal.

AGI (Artificial General Intelligence)

A hypothetical AI system that can perform most intellectual tasks at a human level across many domains.

AI Act

The European Union's risk-based regulatory framework for AI systems and providers.

AI Agent

A software system that can observe, reason, and take actions to achieve a goal, often using tools and memory.

AI Alignment

The work of making AI systems behave according to human intentions, values, and safety constraints.

AI Governance

Policies, standards, and oversight mechanisms that guide how AI is developed and used in society.

AI Safety

A field focused on reducing harmful behavior, failures, and misuse risks in AI systems.

Algorithm

A defined set of rules or steps that a computer follows to solve a problem or complete a task.

Algorithmic Bias

Systematic unfairness in model outputs caused by skewed data, assumptions, or modeling choices.

Algorithmic Transparency

How clearly an AI system's logic, data sources, and limitations are documented and understandable.

ALiBi

A positional bias method that penalizes attention scores based on token distance, helping models extrapolate to longer contexts.

Alignment Tax

The extra cost in time, compute, or product velocity required to make systems safer and more controllable.

ANN (Approximate Nearest Neighbor)

Algorithms that find vectors close to a query without exhaustive comparison, trading exactness for speed.

Annotation

Human-added labels or metadata used to train or evaluate machine learning models.

API (Application Programming Interface)

A structured way for one software system to send requests to and receive responses from another system.

Artificial Intelligence (AI)

The broad field of building systems that perform tasks requiring pattern recognition, reasoning, language, or decision-making.

Attention Mechanism

A model component that dynamically focuses on relevant parts of an input when producing an output.

Autonomous System

A system that can make decisions and act with limited or no direct human control in real time.

B10 terms

Backpropagation

The core training algorithm that updates model weights by propagating prediction errors backward through the network.

Baseline Model

A simple reference model used to compare whether more complex approaches actually improve results.

Beam Search

A decoding algorithm that keeps the top several candidate sequences at each step to find higher-probability outputs.

Benchmark

A standardized test or dataset used to measure and compare model performance.

Benchmark Leakage

When benchmark test examples or close variants are present in training data, inflating reported performance.

Bi-Encoder

A model that encodes queries and documents into separate vectors so they can be compared quickly at scale.

Bias

A consistent pattern of error or unfairness in data or model behavior.

Big Data

Very large and complex datasets that require scalable storage and processing techniques.

Black-Box Model

A model whose internal reasoning is difficult to interpret directly by humans.

BPE (Byte Pair Encoding)

A subword tokenization algorithm that merges the most frequent character pairs into reusable tokens.

C16 terms

Calibration

How well a model's confidence scores match actual correctness probabilities.

Causal Inference

Methods for estimating cause-and-effect relationships rather than simple correlations.

Chain-of-Thought

A reasoning style where an AI model decomposes a problem into intermediate steps.

Citations

References to source passages or documents included in a model's response to support its claims.

Classification

A task where a model assigns an input to one or more predefined categories.

Classifier

A model designed specifically for classification tasks.

CLIP

A multimodal model architecture that learns shared representations between text and images.

Compute

The processing resources required to train and run models, often measured in FLOPS or GPU hours.

Computer Vision

The branch of AI that extracts meaning from images and video.

Confidence Interval

A statistical range that likely contains the true value of a measured model metric.

Constitutional AI

A training and behavior-shaping approach where model outputs are guided by a fixed set of written principles.

Context Window

The maximum amount of input tokens a language model can process at once.

Continual Learning

Training approaches that let a model keep learning from new data without forgetting prior knowledge.

Convolutional Neural Network (CNN)

A neural architecture optimized for processing grid-like data such as images.

Cross-Encoder

A model that scores a query and document together in one pass for high-accuracy relevance judgments.

Cross-Entropy Loss

A common objective function used to train classification models by penalizing incorrect probabilities.

D15 terms

Data Augmentation

Techniques that create modified training examples to improve model generalization.

Data Drift

A shift in real-world input data over time that can degrade model performance.

Data Labeling

The process of assigning tags or target outputs to raw data for supervised learning.

Data Lineage

A record of where data came from, how it was transformed, and where it is used.

Data Provenance

The documented origin, ownership, and history of a dataset or model artifact.

Dataset

A collection of structured or unstructured examples used for training, validation, or testing.

Decision Boundary

The surface in feature space that separates classes predicted by a classifier.

Decision Tree

A model that makes predictions through a sequence of if-then feature splits.

Deep Learning

A subset of machine learning that uses many-layer neural networks for representation learning.

Differential Privacy

A privacy technique that adds statistical noise so individual records cannot be reliably inferred from outputs.

Diffusion Model

A generative architecture that learns to reverse noise to synthesize images, audio, or other content.

Distillation

Compressing knowledge from a large teacher model into a smaller student model.

Distilled Model

A smaller model trained to imitate a larger model's behavior while using less compute at inference.

Domain Adaptation

Methods that transfer a model trained in one domain to perform better in another domain.

DPO (Direct Preference Optimization)

A training method that fine-tunes models directly on preference pairs without needing a separate reward model.

E7 terms

Embedding

A numeric vector representation that captures semantic meaning of text, images, or other data.

Embedding Model

A model specialized for converting data into vectors used for semantic search, clustering, and retrieval.

Encoder

The component of a model that transforms input into latent representations.

Ensemble

Combining predictions from multiple models to improve robustness or accuracy.

Eval Harness

A repeatable evaluation framework that runs prompts, datasets, and scoring logic across model versions.

Evaluation Set

A held-out dataset used to measure model quality after training.

Explainability

The degree to which a model's behavior can be interpreted and explained to humans.

F13 terms

Factuality

How accurately a model's claims match verifiable real-world information.

False Negative

An incorrect prediction where a model misses a true positive case.

False Positive

An incorrect prediction where a model incorrectly flags a negative case as positive.

Feature

An input variable used by a model to make predictions.

Feature Engineering

Designing or transforming input variables to make learning easier and more effective.

Feature Extraction

Converting raw data into informative features that a model can use.

Feature Store

A managed system for storing and serving validated ML features consistently for training and inference.

Few-Shot Learning

Learning or adapting behavior from only a small number of examples.

Fine-Tuning

Continuing training on domain-specific data to adapt a pre-trained model to a specific task.

Flash Attention

An optimized attention algorithm that reduces memory use and speeds up transformer training and inference.

Foundation Model

A large pre-trained model that can be adapted to many downstream tasks.

Frequency Penalty

A decoding setting that reduces the probability of tokens proportionally to how often they've appeared so far.

Function Calling

A model capability to generate structured calls that trigger external tools or APIs.

G10 terms

GAN (Generative Adversarial Network)

A generative setup where a generator and discriminator train against each other.

Generalization

How well a model performs on new, unseen data outside the training set.

Generative AI

AI systems that produce new content such as text, images, audio, video, or code.

Gradient

A vector showing how much each parameter should change to reduce loss.

Gradient Descent

An optimization method that updates parameters in the direction that reduces error.

Ground Truth

Trusted reference labels used to train or evaluate model outputs.

Groundedness

The degree to which an AI response is supported by source data or retrieved evidence.

GSM8K

A benchmark of grade-school math word problems used to evaluate step-by-step reasoning in language models.

Guarded Decoding

A generation strategy that constrains output tokens to valid structures or policy-compliant choices.

Guardrails

Rules, checks, and controls that limit unsafe or undesired model behavior.

H7 terms

Hallucination

When a model generates fluent but false or unsupported information.

HNSW

A graph-based index structure for fast approximate nearest-neighbor search over high-dimensional vectors.

Human Preference Model

A model trained on human rankings to predict which responses users are likely to prefer.

Human-in-the-Loop

A workflow where humans review, guide, or override AI outputs.

HumanEval

A benchmark of Python programming problems used to measure code-generation correctness via unit tests.

Hybrid Search

A retrieval approach that combines keyword (lexical) search with vector (semantic) search for better recall and precision.

Hyperparameter

A configuration value set before training, such as learning rate, batch size, or depth.

I6 terms

In-Context Learning

A model's ability to follow patterns from examples provided directly in the prompt.

Inference

The runtime phase where a trained model generates predictions or outputs.

Inference Endpoint

A deployed API interface that receives model requests and returns predictions in production.

Inference-Time Compute

The amount of processing power consumed while producing each response.

Instruction Tuning

Fine-tuning a model on instruction-response pairs to improve task following.

Intent Classification

Predicting the user's purpose from a text query to route it correctly.

J1 term

Jailbreak

A prompt technique intended to bypass a model's safety constraints.

K5 terms

Knowledge Base

A curated collection of documents or records used for retrieval, support automation, or grounding responses.

Knowledge Cutoff

The latest point in time reflected in a model's training data.

Knowledge Distillation

Training a smaller model to imitate the outputs of a larger model.

Knowledge Graph

A graph structure of entities and relationships used for reasoning or retrieval.

KV Cache

Stored key and value tensors from previous tokens that let transformers generate new tokens without recomputing past attention.

L8 terms

Label Smoothing

A regularization method that softens hard labels to improve generalization.

Large Language Model (LLM)

A language model trained on massive text corpora to generate and analyze text.

Latency

The time between sending a request and receiving the model's output.

Latent Space

A compressed representational space where similar concepts are positioned near each other as vectors.

Learning Rate

A training hyperparameter controlling how much parameters change each update step.

LLM-as-Judge

Using a language model to score or compare outputs from other models during evaluation.

LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning method that adds low-rank adapter matrices.

Loss Function

A mathematical objective that quantifies prediction error during training.

M13 terms

Machine Learning (ML)

Methods that allow systems to learn patterns from data and improve over time.

MCP (Model Context Protocol)

An open protocol that lets AI applications connect to external tools, data sources, and context providers in a standard way.

Memory (Agent Memory)

Stored context an AI agent uses across steps or sessions to improve continuity.

Mid-training

An intermediate training phase between pretraining and post-training, often used for capability or domain adjustments.

Mixture of Experts (MoE)

An architecture with specialized subnetworks where only selected experts run per input.

MMLU

A benchmark testing language models across 57 academic and professional subjects using multiple-choice questions.

Model Card

Documentation describing a model's intended use, metrics, limitations, and risks.

Model Drift

Performance degradation over time as real-world conditions diverge from training assumptions.

Model Quantization

Reducing numeric precision of model weights to decrease memory and inference cost.

Model Registry

A central catalog for versioning, approving, and tracking models across environments.

Multi-Head Attention

A transformer mechanism that runs several attention operations in parallel to capture different types of relationships.

Multi-Query Retrieval

A retrieval technique that rewrites the user query into several variants to improve recall.

Multimodal Model

A model that can process or generate multiple data types such as text, image, and audio.

N4 terms

Named Entity Recognition (NER)

An NLP task that identifies entities such as people, places, dates, or organizations.

Natural Language Processing (NLP)

The branch of AI focused on understanding and generating human language.

Neural Network

A layered computational model inspired by biological neurons and synapses.

Normalization

Transforming values to a consistent scale to improve optimization stability.

O5 terms

OCR (Optical Character Recognition)

Technology that converts text in images or scans into machine-readable text.

On-Device AI

AI inference performed locally on user hardware rather than in a remote cloud service.

Open-Source Model

A model released with public weights or code for inspection, adaptation, and reuse.

Output Parser

Logic that validates and converts model output into strongly typed, machine-usable structures.

Overfitting

When a model memorizes training data and performs poorly on unseen inputs.

P16 terms

Parameter

A learned weight inside a model that influences its outputs.

Parameter-Efficient Fine-Tuning (PEFT)

Methods that adapt models by training a small subset of added parameters.

Parent-Document Retrieval

A retrieval pattern that searches small chunks but returns their larger parent documents for richer context.

Pass@k

A code-evaluation metric measuring the chance that at least one of k generated samples passes the tests.

Perplexity

A language-model metric measuring how surprised the model is by true next tokens.

Pipeline

An ordered workflow of preprocessing, model steps, and postprocessing stages.

Positional Encoding

Information added to token embeddings so transformers can distinguish sequence order.

Post-training

Training steps applied after pretraining, such as instruction tuning, preference optimization, and safety tuning.

Precision

The proportion of predicted positives that are actually correct.

Presence Penalty

A decoding setting that reduces the probability of tokens that have appeared at all, encouraging new topics.

Pretraining

Initial large-scale model training on broad data before downstream adaptation.

Prompt

The input instructions and context provided to a generative model.

Prompt Engineering

Designing prompts to improve output quality, reliability, and controllability.

Prompt Injection

An attack pattern where malicious instructions are inserted into model inputs or retrieved content.

Prompt Template

A reusable prompt pattern with variables, formatting rules, and task-specific instructions.

Pruning

Removing less important model weights or neurons to reduce size and compute.

Q2 terms

QLoRA

A fine-tuning technique that combines 4-bit weight quantization with LoRA adapters to reduce memory needs.

Quantization

Converting model weights to lower precision formats such as 8-bit or 4-bit.

R15 terms

RAG (Retrieval-Augmented Generation)

A method that retrieves external knowledge and feeds it into generation at inference time.

RAG-Fusion

A retrieval method that generates multiple query variants, retrieves results for each, and fuses the rankings.

ReAct

A prompting pattern that interleaves reasoning steps with tool-use actions to solve tasks more reliably.

Recall

The proportion of actual positives that a model correctly identifies.

Recommendation System

A model pipeline that predicts user preferences for ranking content or products.

Red Teaming

Stress-testing an AI system with adversarial prompts to reveal failures and risks.

Reinforcement Learning

Training by reward signals where an agent learns actions that maximize long-term return.

Reinforcement Learning from Human Feedback (RLHF)

A training method that uses human preference signals to shape model behavior.

Repetition Penalty

A decoding setting that lowers the probability of tokens the model has already produced to reduce loops.

Reranker

A model that reorders an initial set of retrieved results to put the most relevant items at the top.

Retrieval

Finding relevant documents or records from a knowledge source for a query.

Retrieval Precision

The proportion of retrieved items that are relevant to the user's query.

Reward Model

A model that scores outputs based on preference signals, often used in RLHF pipelines.

Robustness

A model's ability to maintain performance under noise, shifts, or adversarial inputs.

RoPE (Rotary Positional Embeddings)

A positional encoding method that rotates query and key vectors to encode relative token positions.

S17 terms

Safety Case

A structured argument, supported by evidence, that an AI system is safe for a defined context of use.

Safety Filter

A moderation layer that blocks or rewrites unsafe model inputs or outputs.

Scaling Law

An empirical relationship showing how performance improves with model size, data, or compute.

Self-Play

A training setup where a model improves by generating data through interactions or competitions with copies of itself.

Self-Supervised Learning

Learning representations from unlabeled data by predicting masked or transformed parts.

Semantic Search

Search that matches meaning rather than exact keyword overlap, often using embeddings.

SentencePiece

A language-agnostic tokenizer that learns subword units directly from raw text without pre-splitting on whitespace.

Sentiment Analysis

An NLP task that classifies emotional tone or opinion in text.

Shadow Deployment

Running a model in parallel with production traffic without affecting user-facing decisions.

Sliding Window Attention

An attention pattern where each token attends only to a fixed-size window of nearby tokens to reduce compute.

Small Language Model (SLM)

A compact language model optimized for lower latency, cost, or on-device usage.

Sparse Model

A model where many parameters are zero or inactive to reduce computation.

Speculative Decoding

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

Structured Output

Model output constrained to a defined schema such as JSON, tool arguments, or typed fields.

Supervised Learning

Training a model with labeled examples that map inputs to known outputs.

Synthetic Data

Artificially generated data used to augment, simulate, or protect sensitive training data.

System Prompt

A high-priority instruction that sets behavior, policy, and response style for a model.

T12 terms

Temperature

A sampling setting controlling randomness in generated outputs.

Test-Time Compute

Additional inference computation used during response generation to improve quality or reasoning.

Token

A chunk of text processed by language models, such as a word piece or symbol.

Tokenization

The process of splitting text into tokens for model input.

Tool Use

A model's ability to call external tools such as search, calculators, or APIs.

Top-k Sampling

A decoding strategy that samples only from the k most likely next tokens.

Top-p (Nucleus) Sampling

A decoding strategy that samples from the smallest token set whose probabilities sum to p.

Training Loss

The model error value computed during training and optimized downward over time.

Transfer Learning

Applying knowledge learned in one task or domain to improve another task.

Transformer

A neural architecture that uses attention to model relationships across sequences in parallel.

Tree of Thoughts

A reasoning approach where a model explores multiple branching solution paths and selects the most promising ones.

Trust Calibration

Aligning user confidence in AI outputs with the system's actual reliability in each task.

U2 terms

Unsupervised Learning

Learning patterns from unlabeled data without explicit target outputs.

Usage-Based Billing

Pricing where costs scale with API calls, tokens, inference time, or consumed compute.

V3 terms

Validation Set

A dataset used during development to tune models and prevent overfitting.

Vector Database

A database optimized for storing and querying high-dimensional embedding vectors.

Vision-Language Model (VLM)

A multimodal model that jointly processes visual and textual information.

W4 terms

Watermarking

Embedding a detectable signal in AI-generated text or media so it can later be identified as machine-produced.

Weak Supervision

Using noisy, heuristic, or partial labels to train models when clean labels are scarce.

Weight

A learned numeric value that scales signals passing through a neural network.

Word Embedding

A dense vector representation of words capturing semantic relationships.

X1 term

XAI (Explainable AI)

Techniques and practices for making AI predictions more transparent and understandable.

Z2 terms

Zero Data Retention

A policy where request/response payloads are not stored after processing beyond short-lived operational windows.

Zero-Shot Learning

Solving tasks without task-specific examples by relying on prior general knowledge.

Free AI glossary

213 terms · Plain-language definitions

Explore the full guide library