Tuning bu gaaw

Prompt tuning is a way to teach an existing language model a new job without changing the model itself. A normal text prompt is a string of words, such as "Classify the sentiment of this review". A soft prompt is different: it is a short list of numeric vectors that live in the same space as the model's word embeddings (the vectors the model uses to represent each token), but they do not have to correspond to any real word. The model's billions of weights stay frozen. Training adjusts only the soft prompt, using ordinary gradient descent on labeled examples, until the frozen model produces the right outputs when the prompt sits in front of each input.

The mechanism has four steps. First, choose a prompt length p, typically between 5 and 100 virtual tokens, and create a matrix of p vectors, each with the same width e as the model's embeddings. Second, initialize those vectors, either randomly, by copying embeddings of common vocabulary words, or by copying embeddings of the class label words such as "positive" and "negative". Third, for each training example, convert the input text to embeddings as usual, concatenate the prompt matrix in front of them, and run the frozen model forward. Fourth, compute the task loss on the model's output and backpropagate it only into the prompt matrix. At inference time the learned prompt is attached to new inputs in exactly the same way. Because the prompt is the only task-specific artifact, a single deployed model can handle many tasks by swapping prompts per request.

The technique was introduced by Brian Lester, Rami Al-Rfou and Noah Constant of Google in "The Power of Scale for Parameter-Efficient Prompt Tuning", submitted to arXiv in April 2021 and published at EMNLP 2021. Its central finding is about scale. Using T5 models from small to XXL (11 billion parameters), the authors showed that prompt tuning lags behind full fine-tuning on small models but closes the gap as models pass a few billion parameters. They also report that GPT-3's few-shot prompting scored 71.8 on SuperGLUE against 89.3 for a fine-tuned T5-XXL, even though GPT-3 is 16 times larger. Google's February 2022 research blog summarizes the headline: on T5-XXL, prompt tuning matches model tuning while training about 20,000 parameters to steer an 11-billion-parameter model.

The paper's ablations give practical guidance. Lengthening the prompt beyond about 20 tokens generally gave only small gains, and the XXL model still gave strong results with a single-token prompt. Initializing from class-label embeddings beat random initialization on small and medium models, but the difference vanished at XXL size. A surprising detail concerned pretraining: T5's original span-corruption objective, where the model fills in masked spans with sentinel tokens, made frozen models poor prompt targets. The authors continued training T5 1.1 checkpoints with a plain language-modeling objective for up to 100,000 steps and released these "LM-adapted" checkpoints, which became the standard starting point for prompt-tuning experiments.

Prominent alternatives sit on a spectrum of how much of the model they touch. Prompt design, or hand-written prompting, trains nothing and relies on the model following instructions and a few examples. Prefix tuning, proposed by Xiang Lisa Li and Percy Liang in January 2021, learns continuous vectors that are inserted at every transformer layer rather than only at the input, training about 0.1% of the parameters of GPT-2 and BART. P-Tuning v2, by Xiao Liu and colleagues at Tsinghua (ACL 2022), applies deep prompts to every layer and reports matching fine-tuning across model sizes and harder tasks such as sequence labeling, with 0.1% to 3% of parameters tuned. LoRA, from Microsoft researchers in June 2021, instead adds small low-rank matrices to the weight updates and reports 10,000 times fewer trainable parameters than full fine-tuning of GPT-3 175B. Full fine-tuning updates every weight and usually gives the strongest per-task result, at the cost of storing a full model copy for each task.

The main appeal is robustness and economy. Lester and colleagues trained on SQuAD and tested on out-of-domain question-answering datasets from the MRQA 2019 shared task; prompt tuning beat full fine-tuning on most of them, with a 12.5-point F1 gap on TextbookQA. Transferring a paraphrase detector from the QQP dataset to MRPC, prompt tuning scored 76.3% accuracy against 73.1% for full fine-tuning. The authors interpret this as evidence that freezing the model limits overfitting to quirks of the training set. Storage is the other benefit: with a five-token prompt on T5-XXL, each task needs 20,480 parameters instead of another 11-billion-parameter copy, which the paper notes takes 42 GiB on disk.

The method has clear limits. On models below roughly a billion parameters, plain prompt tuning trails full fine-tuning by a wide margin on SuperGLUE, which is why P-Tuning v2 and prefix tuning add trainable vectors at deeper layers. Training can be slow to converge and sensitive to learning rate; the Google work used a learning rate of 0.3, far higher than typical fine-tuning rates. Learned prompts are hard to interpret, since their vectors rarely match real words; the paper found nearest-neighbor words formed loose semantic clusters, not readable instructions. Prompts also consume context: every virtual token occupies a position the input could have used. Finally, a prompt is tied to the exact model it was trained on, so upgrading the base model means retraining every prompt.

In practice the costs are modest. Training requires a forward and backward pass through the full frozen model, so GPU memory for activations is similar to fine-tuning, but optimizer state is needed only for the prompt, which removes most of the memory for gradients and Adam statistics. Hugging Face's PEFT library exposes the method through a PromptTuningConfig object where the user sets the number of virtual tokens and the initialization, either random, sampled vocabulary or a text string. Its documented example on a 559-million-parameter BLOOMZ model trains 8,192 parameters, about 0.0015% of the total, and the resulting checkpoint is a few kilobytes.