AI Glossary Term

What is Positional Encoding?

Definition

Information added to token embeddings so transformers can distinguish sequence order.

Related terms

RoPE (Rotary Positional Embeddings)

A positional encoding method that rotates query and key vectors to encode relative token positions.

A positional bias method that penalizes attention scores based on token distance, helping models extrapolate to longer contexts.

Parameter-Efficient Fine-Tuning (PEFT)

Methods that adapt models by training a small subset of added parameters.

AGI (Artificial General Intelligence)

A hypothetical AI system that can perform most intellectual tasks at a human level across many domains.

A software system that can observe, reason, and take actions to achieve a goal, often using tools and memory.

Human-added labels or metadata used to train or evaluate machine learning models.

Learn more in our free guides

Positional Interpolation for Long Context Instant-NGP Hash Encoding Byte-Pair Encoding Tokenization and Byte Pair Encoding

Browse the full AI Glossary Explore all free AI guides