Nyelvi AI ÚTMUTATÓ

Prompt hangolás

Prompt tuning adapts a frozen language model to a new task by learning a short sequence of trainable vectors, called a soft prompt, that is prepended to every input.

Hallgassa meg ezt az útmutatótAI-narrált · ~8 min

Ezen az oldalon9 perc olvasás
  1. Áttekintés
  2. Főbb tanulságok
  3. Mély merülés
  4. Training a complaint classifier with a soft prompt in Hugging Face PEFT
  5. Valós megvalósítás
  6. A Prompt Tuning jövője
  7. A gyakorlatba ültetés
  8. Források és további olvasmányok
  9. Folytassa a felfedezést
  10. Gyakran ismételt kérdések

Áttekintés

Only those vectors change during training, so one shared model can serve many tasks while each task stores a few thousand numbers instead of a full copy of the model.

Főbb tanulságok

  1. Prompt tuning trains only a short matrix of virtual-token embeddings that is prepended to the input, while every weight of the language model stays frozen.
  2. The 2021 Google paper by Lester, Al-Rfou and Constant showed prompt tuning matches full fine-tuning on SuperGLUE once T5 models reach about 11 billion parameters.
  3. A five-token soft prompt on T5-XXL holds 20,480 parameters, compared with 11 billion for a fully fine-tuned copy of the same model.
  4. Prompt tuning beat full fine-tuning on most out-of-domain question-answering tests in the original study, including a 12.5-point F1 gain on TextbookQA.
  5. On models smaller than about a billion parameters, deeper variants such as prefix tuning and P-Tuning v2 usually work better than input-only soft prompts.

Mély merülés

Prompt tuning is a way to teach an existing language model a new job without changing the model itself. A normal text prompt is a string of words, such as "Classify the sentiment of this review". A soft prompt is different: it is a short list of numeric vectors that live in the same space as the model's word embeddings (the vectors the model uses to represent each token), but they do not have to correspond to any real word. The model's billions of weights stay frozen. Training adjusts only the soft prompt, using ordinary gradient descent on labeled examples, until the frozen model produces the right outputs when the prompt sits in front of each input.

The mechanism has four steps. First, choose a prompt length p, typically between 5 and 100 virtual tokens, and create a matrix of p vectors, each with the same width e as the model's embeddings. Second, initialize those vectors, either randomly, by copying embeddings of common vocabulary words, or by copying embeddings of the class label words such as "positive" and "negative". Third, for each training example, convert the input text to embeddings as usual, concatenate the prompt matrix in front of them, and run the frozen model forward. Fourth, compute the task loss on the model's output and backpropagate it only into the prompt matrix. At inference time the learned prompt is attached to new inputs in exactly the same way. Because the prompt is the only task-specific artifact, a single deployed model can handle many tasks by swapping prompts per request.

The technique was introduced by Brian Lester, Rami Al-Rfou and Noah Constant of Google in "The Power of Scale for Parameter-Efficient Prompt Tuning", submitted to arXiv in April 2021 and published at EMNLP 2021. Its central finding is about scale. Using T5 models from small to XXL (11 billion parameters), the authors showed that prompt tuning lags behind full fine-tuning on small models but closes the gap as models pass a few billion parameters. They also report that GPT-3's few-shot prompting scored 71.8 on SuperGLUE against 89.3 for a fine-tuned T5-XXL, even though GPT-3 is 16 times larger. Google's February 2022 research blog summarizes the headline: on T5-XXL, prompt tuning matches model tuning while training about 20,000 parameters to steer an 11-billion-parameter model.

The paper's ablations give practical guidance. Lengthening the prompt beyond about 20 tokens generally gave only small gains, and the XXL model still gave strong results with a single-token prompt. Initializing from class-label embeddings beat random initialization on small and medium models, but the difference vanished at XXL size. A surprising detail concerned pretraining: T5's original span-corruption objective, where the model fills in masked spans with sentinel tokens, made frozen models poor prompt targets. The authors continued training T5 1.1 checkpoints with a plain language-modeling objective for up to 100,000 steps and released these "LM-adapted" checkpoints, which became the standard starting point for prompt-tuning experiments.

Prominent alternatives sit on a spectrum of how much of the model they touch. Prompt design, or hand-written prompting, trains nothing and relies on the model following instructions and a few examples. Prefix tuning, proposed by Xiang Lisa Li and Percy Liang in January 2021, learns continuous vectors that are inserted at every transformer layer rather than only at the input, training about 0.1% of the parameters of GPT-2 and BART. P-Tuning v2, by Xiao Liu and colleagues at Tsinghua (ACL 2022), applies deep prompts to every layer and reports matching fine-tuning across model sizes and harder tasks such as sequence labeling, with 0.1% to 3% of parameters tuned. LoRA, from Microsoft researchers in June 2021, instead adds small low-rank matrices to the weight updates and reports 10,000 times fewer trainable parameters than full fine-tuning of GPT-3 175B. Full fine-tuning updates every weight and usually gives the strongest per-task result, at the cost of storing a full model copy for each task.

The main appeal is robustness and economy. Lester and colleagues trained on SQuAD and tested on out-of-domain question-answering datasets from the MRQA 2019 shared task; prompt tuning beat full fine-tuning on most of them, with a 12.5-point F1 gap on TextbookQA. Transferring a paraphrase detector from the QQP dataset to MRPC, prompt tuning scored 76.3% accuracy against 73.1% for full fine-tuning. The authors interpret this as evidence that freezing the model limits overfitting to quirks of the training set. Storage is the other benefit: with a five-token prompt on T5-XXL, each task needs 20,480 parameters instead of another 11-billion-parameter copy, which the paper notes takes 42 GiB on disk.

The method has clear limits. On models below roughly a billion parameters, plain prompt tuning trails full fine-tuning by a wide margin on SuperGLUE, which is why P-Tuning v2 and prefix tuning add trainable vectors at deeper layers. Training can be slow to converge and sensitive to learning rate; the Google work used a learning rate of 0.3, far higher than typical fine-tuning rates. Learned prompts are hard to interpret, since their vectors rarely match real words; the paper found nearest-neighbor words formed loose semantic clusters, not readable instructions. Prompts also consume context: every virtual token occupies a position the input could have used. Finally, a prompt is tied to the exact model it was trained on, so upgrading the base model means retraining every prompt.

In practice the costs are modest. Training requires a forward and backward pass through the full frozen model, so GPU memory for activations is similar to fine-tuning, but optimizer state is needed only for the prompt, which removes most of the memory for gradients and Adam statistics. Hugging Face's PEFT library exposes the method through a PromptTuningConfig object where the user sets the number of virtual tokens and the initialization, either random, sampled vocabulary or a text string. Its documented example on a 559-million-parameter BLOOMZ model trains 8,192 parameters, about 0.0015% of the total, and the resulting checkpoint is a few kilobytes.

04Működő példa

Training a complaint classifier with a soft prompt in Hugging Face PEFT

  1. Load a small causal language model such as bigscience/bloomz-560m and its tokenizer with the transformers library.

  2. Create a PromptTuningConfig with task_type CAUSAL_LM, prompt_tuning_init set to TEXT and an initialization string such as 'Classify if the tweet is a complaint or no complaint.'

  3. Set num_virtual_tokens to the token length of that string so every virtual token starts from a real word embedding.

  4. Wrap the model with get_peft_model and call print_trainable_parameters to confirm only a few thousand parameters are trainable.

  5. Format each labeled tweet as input text followed by the label word, then train for a few epochs with a relatively high learning rate.

  6. Save the adapter, which writes only the prompt matrix, and reload it on top of the untouched base model for inference.

Mit mutat

The saved adapter is a few kilobytes and contains roughly 8,192 numbers. Check it by generating labels for held-out tweets and comparing accuracy with an untuned base model given the same text prompt.

Valós megvalósítás

Google's 2021 study tuned soft prompts on a frozen T5-XXL model with 11 billion parameters and matched full fine-tuning on the SuperGLUE language-understanding benchmark while training a tiny fraction of the weights.

A support team can train one soft prompt to classify tweets as complaints or not complaints on bigscience/bloomz-560m using Hugging Face PEFT, which reports about 8,192 trainable parameters against 559 million in the base model.

A service hosting many customers can keep one frozen model in memory and batch requests from different tasks together, swapping only the small prompt matrix attached to each request.

Researchers built cheap ensembles by training five prompts for the same SuperGLUE task on a single frozen T5-XXL and taking a majority vote, which beat the average single prompt on every task.

The SPoT method from Google (ACL 2022) first learns a prompt on source tasks and reuses it to initialize a prompt for a new target task, matching or beating full fine-tuning on SuperGLUE.

A Prompt Tuning jövője

Research since 2021 has focused on making soft prompts reusable and less dependent on model scale. SPoT (Vu and colleagues, ACL 2022) showed that a prompt learned on one or more source tasks can initialize a prompt for a new task, and its study covered 26 NLP tasks in 160 source and target combinations, reporting results that match or beat full fine-tuning on SuperGLUE with up to 27,000 times fewer task-specific parameters. P-Tuning v2 (ACL 2022) extended competitive results to smaller models by adding prompts at every layer. Open problems remain. Soft prompts are tied to one base model, so transferring them across model versions or architectures is an active question. Interpretability is limited because learned vectors rarely map to words. In industry, low-rank methods such as LoRA (2021) have become the more common choice for adapting open models, partly because their weights merge into the base model and add no inference tokens, while prompt tuning remains attractive where many tasks must share a single frozen model in one batch. Whether newer, larger models make soft prompts even more competitive, as the 2021 scaling result suggests, is still being studied on current architectures.

A gyakorlatba ültetés

Miért számít

  • Sebesség és méretarány. A nyelvi munkafolyamatok gyorsabban haladhatnak a következetesség feláldozása nélkül.
  • Hozzáférés és elérés. Kibővíti a hozzáférést a nyelvek és a kommunikációs stílusok között.
  • Tisztább döntések. A csapatok több időt tölthetnek az ítélkezéssel, míg az automatizálás kezeli az ismétlést.

Vigyázz

  • A hallucinált tények csendben bekerülhetnek a jelentésekbe, a támogatási folyamatokba vagy a kutatási eredményekbe.
  • Az azonnali érzékenység inkonzisztens eredményeket eredményezhet a hasonló kérések között.
  • Ha a hozzáférés-szabályozás gyenge, az érzékeny szöveges adatok megjelenhetnek.

Következő lépések

  1. A kiadás előtt határozza meg a kimeneti formátumot, hangszínt és minőségi szabványokat.
  2. Földelje a válaszokat megbízható forrásokból, amikor a pontosság számít.
  3. Tartson emberi ellenőrzési pontot a nagy tétű kimenetekhez.
  4. Kövesse nyomon a meghibásodási mintákat, és rendszeresen tanítsa át az utasításokat vagy a munkafolyamatokat.

Források és további olvasmányok

  1. arXiv (EMNLP 2021)The Power of Scale for Parameter-Efficient Prompt Tuning
  2. Google ResearchGuiding Frozen Language Models with Learned Soft Prompts
  3. arXiv (ACL 2021)Prefix-Tuning: Optimizing Continuous Prompts for Generation
  4. arXiv (ACL 2022)P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
  5. arXiv (ACL 2022)SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
  6. arXivLoRA: Low-Rank Adaptation of Large Language Models
  7. Hugging Face PEFT documentationPrompt tuning

Folytassa a felfedezést

Ingyenes hírlevél

Szerezze meg a napi AI összefoglalót

Minden hétköznap reggel három ellenőrzött AI történet, egyszerű angol nyelven. Örökké ingyen, hirdetések nélkül.

Minden hétköznap egy e-mail. Egy kattintással lemondhat. Soha nem értékesítjük vagy osztjuk meg az Ön címét.

Teszteld magad

Take the Prompt Tuning quiz

Azonnali visszajelzés minden válaszra, és megosztható tanúsítvány egy ellenőrizhető azonosítóval, miután sikeresen teljesít egy kurzust.

Kezdő kvíz

Támogassa az ingyenes AI oktatást. AI Understanding egy 501(c)(3) nonprofit – nincs hirdetés, nincs fizetőfal, soha. Tegyen adományt

Gyakran ismételt kérdések

Mi az a prompt hangolás?

Prompt tuning adapts a frozen language model to a new task by learning a short sequence of trainable vectors, called a soft prompt, that is prepended to every input. Only those vectors change during training, so one shared model can serve many tasks while each task stores a few thousand numbers instead of a full copy of the model.

Mit jelent a „puha prompt” az azonnali hangolásban?

A lágy promptok tanítható folytonos vektorok a térbe ágyazva, nem pedig ember által olvasható szavak.

Hol adja hozzá a gyorshangolás a betanítható vektorait?

A prompt hangolás csak a bemeneten fűzi a lágy promptokat, ami miatt könnyebb, mint az előtag hangolása.

Milyen meglepő hatást emelt ki az eredeti cikk a modellskálával kapcsolatban?

A lap címe „a méretarány erejét” hangsúlyozta: a nagyobb modellek lehetővé teszik a rivális teljes finomhangolást.

Ki vezette be az azonnali hangolást?

Brian Lester, Rami Al-Rfou és Noah Constant (Google) 2021-ben közzétették az azonnali hangolási tanulmányt.

Hogyan teszi lehetővé az azonnali hangolás sok feladat kiszolgálását egy modellről?

Csak egy kis soft prompt feladatspecifikus, így egy lefagyott modell sok feladatot képes fogadni a promptok felcserélésével.