What is Parameter-Efficient Fine-Tuning (PEFT)?

Continuing training on domain-specific data to adapt a pre-trained model to a specific task.

A parameter-efficient fine-tuning method that adds low-rank adapter matrices.

Training steps applied after pretraining, such as instruction tuning, preference optimization, and safety tuning.

Fine-tuning a model on instruction-response pairs to improve task following.

A training hyperparameter controlling how much parameters change each update step.

A training method that fine-tunes models directly on preference pairs without needing a separate reward model.