引導式測驗 · 中等 等級
比值比偏好優化測驗
解釋 ORPO 的無參考模型偏好目標、優勢比懲罰以及組合監督和偏好訓練。
問題 1 的 8
與 DPO 等方法相比,ORPO 的主要實際優勢是什麼?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- 與 DPO 等方法相比,ORPO 的主要實際優勢是什麼?
- 它消除了對單獨參考模型的需要
- 它根本不需要訓練數據
- 它僅適用於圖像模型
- 它保證零幻覺
- ORPO 中的「比值比」有何不同?
- 訓練速度與準確性
- 所選響應與拒絕響應的幾率
- 兩種不同的學習率
- 模型大小與資料集大小
- ORPO 偏好範例包含什麼形式?
- Only a single unlabeled response
- A prompt with two identical completions
- A prompt with one chosen response and one disfavored response
- A reward score with no associated responses
- ORPO 如何將回應機率 p 轉換為機率比?
- One minus p divided by p
- The square root of p
- The logarithm of the prompt length
- p divided by one minus p
- ORPO 損失最好描述為哪一種組合?
- SFT 交叉熵加上加權優勢比偏好項
- 只有獎勵模型分數
- 純粹的強化學習回報
- 隨機噪音注入
- Which response receives the supervised fine-tuning component of ORPO?
- Only the rejected response
- The chosen or preferred response
- A response sampled after deployment
- No response; ORPO has no language-modeling term
- Which parameter controls the weight of ORPO’s preference term relative to the supervised loss?
- 分詞器詞彙量大小
- 注意力頭數量
- The preference-loss coefficient
- The prompt’s character count
- Why should an ORPO result report its preference data and evaluation setup?
- ORPO results are identical across datasets
- The objective removes the need for evaluation
- The number of model layers determines every preference
- Outcomes depend on the dataset, coefficient, base model, and evaluation