DPO aṣetunṣe ati Idanwo Tuning Iyanfẹ Ayelujara

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Kini DPO yago fun ti RLHF ibile (PPO) nbeere?
   • Ikẹkọ lọtọ fojuhan ère awoṣe
   • Eyikeyi ikẹkọ data
   • Lilo nẹtiwọki nkankikan
   • Awọn ayanfẹ eniyan patapata
2. Kini idi ti aṣetunṣe (online) DPO nigbagbogbo dara julọ ju ṣiṣe DPO lẹẹkan lori ipilẹ data ti o wa titi?
   • Ko nilo awọn akole ayanfẹ
   • O tọju data ikẹkọ lori eto imulo ati alabapade bi awoṣe ṣe yipada
   • O yọ awoṣe itọkasi
   • O ṣiṣẹ nikan fun awọn aworan
3. Kini ipa wo ni apẹẹrẹ itọkasi ṣe ninu pipadanu DPO?
   • O ṣe agbejade abajade olumulo ikẹhin
   • O ṣe aami awọn ayanfẹ
   • O da eto imulo naa silẹ nipasẹ ipin iṣeeṣe log, iṣakoso iyapa
   • O rọpo tokenizer
4. Ninu aṣetunṣe ẹyọkan ti DPO ori ayelujara, kini ọkọọkan aṣoju?
   • Pa awoṣe rẹ, lẹhinna tun ṣe ikẹkọ lati ibere
   • Nikan relabel atijọ data lai iṣapẹẹrẹ
   • Kọ CNN lori awọn idahun
   • Ayẹwo awọn idahun titun, awọn ayanfẹ idajọ, lẹhinna ṣiṣe imudojuiwọn DPO kan
5. Kini hyperparameter beta ni iṣakoso DPO?
   • Bawo ni agbara eto imulo ṣe yapa lati awoṣe itọkasi
   • Awọn fokabulari iwọn
   • Awọn nọmba ti GPUs
   • Ipinnu aworan
6. Kini imọran bọtini kan lẹhin 'ẹsan-ara-ẹni' aṣetunṣe DPO?
   • Awoṣe naa ko lo awọn ayanfẹ lailai
   • Awoṣe naa ṣe ipilẹṣẹ ati ṣe idajọ awọn orisii idahun tirẹ lati ṣẹda data ikẹkọ
   • Awọn eniyan gbọdọ fi aami si gbogbo bata meji
   • O nikan nlo awọn ere laileto
7. Who can label response preferences in an online DPO iteration?
   • Only the tokenizer
   • Only the reference policy’s embedding layer
   • Human raters or an AI judge such as a reward model
   • A discriminator that never sees responses
8. What additional work does iterative DPO require compared with using a fixed preference dataset once?
   • Training no model parameters
   • Removing all evaluation from the pipeline
   • Storing every possible response in advance
   • Repeated response generation and preference labeling between updates