PANDUAN Teknikal

Pralatihan Berterusan lwn Penalaan Halus

Continued pretraining keeps training a base model with the same next-token objective on large amounts of raw domain text, while fine-tuning trains on a smaller curated set of labeled examples, such as instruction and response pairs, to shape behavior.

Dengar panduan iniAI-diceritakan · ~3 min

Pada halaman ini4 minit membaca
  1. Gambaran keseluruhan
  2. Menyelam dalam
  3. Pelaksanaan Dunia Sebenar
  4. The Future of Continued Pretraining vs Fine-Tuning
  5. Mempraktikkannya
  6. Teruskan Meneroka
  7. Soalan lazim

Gambaran keseluruhan

The choice matters because the first mainly shifts what a model knows about a domain and its language, the second mainly shifts how it responds, and the data and compute needed differ greatly.

Menyelam dalam

Continued pretraining, also called domain-adaptive pretraining, picks up where the original pretraining stopped. The model sees raw text from the target domain, such as medical papers, source code or legal filings, and learns to predict every next token, exactly as in pretraining. Gururangan and colleagues showed in the 2020 paper Don't Stop Pretraining that a second phase of pretraining on in-domain text improved RoBERTa on tasks in biomedical, computer science, news and review domains. It usually needs large volumes of text, often billions of tokens, and full-parameter training, so it costs far more than fine-tuning. Fine-tuning, usually supervised fine-tuning, uses prompt and response pairs. Loss is typically computed only on the response tokens, so the model learns what to produce given an input. Datasets range from a few thousand to hundreds of thousands of examples, and parameter-efficient methods such as LoRA make it affordable on modest hardware. It is good at teaching format, style, task behavior and instruction following. Preference methods such as RLHF or DPO often follow. A common pipeline is base model, then continued pretraining, then supervised fine-tuning, then preference tuning. Continued pretraining has a known risk: catastrophic forgetting, where general skills degrade. Applied to a chat model, it can also erode instruction following, so teams often start from a base model and redo instruction tuning afterward. Two misconceptions stand out. First, fine-tuning is not a reliable way to add many new facts; research such as Gekhman and colleagues (2024) found models learn new facts slowly through fine-tuning and that doing so can increase hallucination. For fast-changing facts, retrieval is often the better tool. Second, BloombergGPT is sometimes cited as continued pretraining, but it was trained from scratch on a mix of financial and general data. A rough rule: use continued pretraining when domain language differs a lot from web text and unlabeled data is plentiful; use fine-tuning when you need a behavior or format.

Pelaksanaan Dunia Sebenar

Code Llama was built by continuing to train Llama 2 on a large corpus of code, and some of its variants were then further trained for Python and for following instructions.

A legal-tech company with millions of unlabeled contracts and filings runs continued pretraining to improve the model's handling of legal language, then fine-tunes on a few thousand annotated clause-extraction examples.

A bank wants a chatbot that answers in a fixed JSON format with a set tone; supervised fine-tuning, or even careful prompting, is enough because the model already knows the general content.

EPFL's Meditron models adapted Llama 2 to medicine through continued pretraining on medical papers and clinical guidelines before any task-specific fine-tuning.

The Future of Continued Pretraining vs Fine-Tuning

The steady release of capable open base models has made continued pretraining practical for more organizations, though it still requires substantial data and compute. Active research areas include reducing forgetting, making domain adaptation cheaper, and combining weights from separately trained models. In practice, many teams will keep pairing light fine-tuning with retrieval rather than full continued pretraining, reserving the heavier option for domains whose language is poorly covered by general web data.

Mempraktikkannya

Mengapa ia penting

  • Kos dan bajet. Keputusan seni bina memacu prestasi dan kos operasi selama bertahun-tahun.
  • Keputusan yang lebih jelas. Pendidikan teknikal membantu pasukan memilih timbunan yang betul, bukan hanya yang terbaharu.
  • Kawalan kualiti. Pilihan kejuruteraan yang lebih baik mengurangkan insiden kebolehpercayaan dalam pengeluaran.

Berhati-hati untuk

  • Mengoptimumkan satu penanda aras boleh menyembunyikan kelemahan sistem yang lebih luas.
  • Kos infrastruktur dan penyelenggaraan sering dipandang remeh.
  • Jurang keselamatan dan pemerhatian boleh berkembang apabila sistem menjadi lebih kompleks.

Langkah seterusnya

  1. Tentukan sasaran kependaman, kualiti dan kos sebelum pelaksanaan.
  2. Penanda aras di bawah beban realistik dan keadaan data.
  3. Pemantauan instrumen untuk ralat, drift dan kesan pengguna.
  4. Sediakan laluan balik dan tindak balas insiden sebelum penskalaan.

Teruskan Meneroka

Newsletter percuma

Dapatkan taklimat AI harian

Tiga cerita AI yang disahkan setiap pagi hari bekerja, ditulis dalam Bahasa Inggeris biasa. Percuma selamanya, tiada iklan.

Satu e-mel setiap hari bekerja. Berhenti melanggan dalam satu klik. Kami tidak pernah menjual atau berkongsi alamat anda.

Uji diri anda

Take the Continued Pretraining vs Fine-Tuning quiz

Maklum balas segera pada setiap jawapan, dan sijil yang boleh dikongsi dengan ID yang boleh disahkan sebaik sahaja anda lulus kursus.

Mulakan kuiz

Sokong pendidikan AI percuma. AI Understanding ialah pertubuhan bukan keuntungan 501(c)(3) — tiada iklan, tiada bayaran dinding, selamanya. Buat derma

Soalan lazim

What is Continued Pretraining vs Fine-Tuning?

Continued pretraining keeps training a base model with the same next-token objective on large amounts of raw domain text, while fine-tuning trains on a smaller curated set of labeled examples, such as instruction and response pairs, to shape behavior. The choice matters because the first mainly shifts what a model knows about a domain and its language, the second mainly shifts how it responds, and the data and compute needed differ greatly.

Apakah perbezaan data utama antara pralatihan berterusan dan penalaan halus diselia?

Pralatihan berterusan meramalkan token seterusnya pada teks mentah, manakala penalaan halus belajar daripada pasangan segera dan respons.

Dalam penalaan halus diselia, di manakah kerugian biasanya dikira?

Menyamarkan gesaan bermakna model mempelajari perkara yang perlu dihasilkan input yang diberikan, bukan untuk menghasilkan semula input.

Apakah main semula dalam pralatihan berterusan?

Memasukkan data umum membantu mengekalkan kemahiran umum daripada merendahkan semasa model mempelajari domain.

Bagaimanakah Kod Llama dibina?

Kod Llama ialah contoh pralatihan berterusan, dengan beberapa varian dilatih lagi untuk Python dan arahan.

Untuk fakta yang sering berubah, pendekatan manakah yang dicadangkan oleh panduan itu selalunya lebih baik daripada penalaan halus?

Penalaan halus mempelajari fakta baharu secara perlahan dan boleh meningkatkan halusinasi, jadi pengambilan semula selalunya lebih baik untuk menukar maklumat.