引導式測驗 · 硬 等級
XLNet 排列建模測驗
檢查排列因式分解、雙流注意力、Transformer-XL 重現,以及基準聲明如何依賴可比設定。
問題 1 的 8
XLNet 使用什麼預訓練目標?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- XLNet 使用什麼預訓練目標?
- 排列語言建模
- 掩碼語言建模
- 替換令牌偵測
- 下一句話預測
- XLNet 是專門為了解決哪些 BERT 弱點而設計的?
- BERT 太小了
- [MASK] 令牌導致預訓練/微調不匹配,並假定屏蔽令牌獨立
- BERT 無法處理短句子
- BERT 缺乏位置編碼
- XLNet的兩流self-attention分離了什麼?
- 編碼器和解碼器層
- 訓練和推理圖
- 內容流和查詢流
- 標記和句子嵌入
- XLNet 是否會物理地打亂句子中的單字?
- 是的,它隨機地重新排序實際的令牌
- 是的,但僅限於微調期間
- 不,它根本不會改變任何順序
- 不,它使用注意掩模排列因式分解順序,同時保持位置
- 哪種架構作為 XLNet 的遠端上下文骨幹網路?
- 變形金剛-XL
- 長短期記憶網絡
- 原始香草變壓器
- 卷積網絡
- How does XLNet combine bidirectional context with autoregressive prediction?
- It predicts all tokens independently with no attention mask
- It predicts in sampled factorization orders without inserting a [MASK] input token, using attention masks
- It physically shuffles the sentence before every prediction
- It replaces autoregressive prediction with a denoising autoencoder
- For a token sequence A B C, suppose a sampled factorization order predicts C before A and B. What context is available when predicting C?
- The tokens A and B must always be provided first
- Only the next sentence is available
- No earlier tokens in that sampled order, although C remains at its original input position
- The input sequence is physically reordered to C A B
- How does XLNet prevent the target token from leaking into its own prediction query?
- It removes the target from the entire training sequence
- It uses the target’s label as an input embedding
- It replaces the token with a [MASK] symbol
- The query stream represents the target position without its token content