引導式測驗 · 中等 等級

變壓器感應頭測驗

查看歸納前綴匹配、前置驅動資訊、跨層電路、訓練觀察和上下文學習機制的限制。

相關引導路徑變壓器中的感應頭
問題 1 的 8

感應頭本質上執行什麼規則?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. 感應頭本質上執行什麼規則?
    • 預測語言中最常見的標記
    • 始終複製前一個令牌不變
    • 隨機打亂上下文標記
    • 之前查看[A][B],然後再次在[A]上預測[B]
  2. What capability was linked to induction-head behavior in the cited 2022 study?
    • In-context learning in the examined models, with broader large-model claims framed as hypotheses
    • A proof that every form of reasoning uses only one identical circuit
    • Changing the tokenizer vocabulary during every prompt
    • Permanent weight updates whenever a repeated token appears
  3. 「前令牌頭」在感應電路中扮演什麼角色?
    • 它產生最終的softmax
    • 它刪除了位置訊息
    • 它向前複製每個令牌的前任訊息,以便可以進行匹配
    • 它規範了注意力權重
  4. 規範感應電路通常涉及多少個注意力頭?
    • 正好一個
    • 二、跨層組合
    • 單層數百個
    • 零;它只是前饋
  5. What training relationship did the induction-head study report in the models it examined?
    • Head formation coincided with changes in loss behavior and increased in-context learning
    • All heads were fully formed before any training
    • Induction behavior could only appear after every parameter was frozen forever
    • The same exact training transition must occur in every future model
  6. 感應頭最直接處理哪一項任務?
    • 計算數值梯度
    • 壓縮模型的權重
    • 加密輸入提示
    • 透過複製先前匹配標記後面的內容來重複序列
  7. What roles do query-key matching and the output-value path play in the canonical induction circuit?
    • Query-key computations generate the final text filename; output-value computations train the tokenizer
    • Query-key computations select the matching context position; output-value computations promote the token information there
    • Both paths only count the total number of context words
    • The output-value path selects a position without any attention pattern
  8. Does a demonstrated two-head induction circuit prove that every large-model in-context task uses that mechanism alone?
    • Yes; every architecture and prompt has already been causally tested
    • Yes; no other network computation can influence a prediction
    • No; the wider explanation remains a hypothesis needing model- and task-specific evidence
    • Only the number of parameters determines whether the proof is universal