What does RoPE (Rotary Positional Embeddings) mean? Definition | AI Understanding

Definition

A positional encoding method that rotates query and key vectors to encode relative token positions.

Related terms

Positional Encoding

Information added to token embeddings so transformers can distinguish sequence order.

A positional bias method that penalizes attention scores based on token distance, helping models extrapolate to longer contexts.

A retrieval method that generates multiple query variants, retrieves results for each, and fuses the rankings.

ANN (Approximate Nearest Neighbor)

Algorithms that find vectors close to a query without exhaustive comparison, trading exactness for speed.

RAG (Retrieval-Augmented Generation)

A method that retrieves external knowledge and feeds it into generation at inference time.

Reinforcement Learning from Human Feedback (RLHF)

A training method that uses human preference signals to shape model behavior.

Learn more in our free guides

Audio Embeddings and Representation Learning Sentence-BERT Embeddings X-Vector Speaker Embeddings HyDE Hypothetical Document Embeddings

See also

Multi-Head Attention Sliding Window Attention Flash Attention BPE (Byte Pair Encoding)QLoRA SentencePiece DPO (Direct Preference Optimization)Tree of Thoughts

Browse the full AI Glossary Explore all free AI guides