quiz bu lëkkale ak tegtal · Diggante Niveau

Wéyal tàggat yaram vs quiz de réglage

Dafay saytu done yi ak wuute yi am ci digganté wéyal tàggat yaram ak ajustement bu baax, fàtte bu metti ak delloosi, ak kañ lay gëna méngoo.

Laaj 1 de 8

Lan mooy wuute gi gëna mag ci done yi diggante wéyal tàggat-yaram ak ajustement buñ saytu?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan mooy wuute gi gëna mag ci done yi diggante wéyal tàggat-yaram ak ajustement buñ saytu?
    • Both use only labeled question and answer pairs
    • Continued pretraining uses raw domain text; fine-tuning uses curated labeled examples
    • Fine-tuning uses more data than continued pretraining
    • Continued pretraining uses only images
  2. Ci fine-tuning buñ saytu, fan lañuy faral di xayma perte bi?
    • On every token including the prompt
    • Only on the prompt tokens
    • On randomly chosen tokens
    • Only on the response tokens
  3. Luy replay ci wéyal tàggat yaram?
    • Mixing some general data into training to limit forgetting
    • Running the same batch twice
    • Restarting training from scratch
    • Replaying user conversations at inference time
  4. naka lañu tabaxee kodu Llama?
    • From scratch on code only
    • By prompt engineering Llama 2
    • By continuing to train Llama 2 on a large code corpus
    • By fine-tuning on a few hundred coding questions
  5. Ci wàllu mbir yuy faral di soppeeku, ban gis-gis la guide bi digle moo gëna baax ci defar ay mbir yu baax?
    • Continued pretraining every day
    • Retrieval of up-to-date documents
    • Raising the sampling temperature
    • Training a larger tokenizer
  6. Which statement about BloombergGPT is correct according to the guide?
    • It was continued pretraining of Llama 2
    • It was created only with LoRA
    • It was a prompt-engineered GPT-3
    • It was trained from scratch on financial and general data
  7. What did the 2020 paper Don't Stop Pretraining show?
    • A second phase of pretraining on in-domain text improved RoBERTa on domain tasks
    • Pretraining should always stop after one epoch
    • Fine-tuning is useless for classification
    • Domain text harms every model
  8. Why re-warm the learning rate at the start of continued pretraining?
    • To reset the tokenizer
    • To freeze the base weights
    • Resuming at a high rate destabilizes training while too low a rate learns little
    • To skip the need for domain data