注意力机制测验

当您参加测验时，就会出现答案和解释。

1. What does token attention combine when forming an output representation?
   • Permanent copies of the entire conversation on disk
   • Value information from allowed positions using computed weights
   • Only manually chosen keywords with no learned scores
   • A translation dictionary that removes every other token
2. 2017 年哪一篇具有里程碑意义的论文介绍了 Transformer 架构？
   • “图像识别的深度残差学习”
   • “通过深度强化学习玩 Atari”
   • “注意力就是你所需要的”
   • “生成对抗网络”
3. 为每个注意力标记计算的三个向量是什么？
   • 查询、键和值
   • 输入、输出和偏置
   • 平均值、中位数和众数
   • 编码器、解码器和损失
4. 公式softmax(QK^T / √d_k) V中，为什么要除以√d_k？
   • 将结果转换为整数
   • 删除负数
   • 使模型在 GPU 上运行
   • 为了防止点积分数在 softmax 之前变得太大
5. Why does dense self-attention involve growing pairwise work for very long token sequences?
   • 因为它必须从互联网下载新数据
   • 因为较长的文本总是包含较罕见的单词
   • 因为成本随着序列长度的平方而增长 (O(n²))
   • 因为它切换到较慢的编程语言
6. What capacity does multi-head attention provide through separate learned projections?
   • A guarantee that every head has one unique named meaning
   • Several attention representations that can capture different relationships in parallel
   • Removal of every token embedding
   • A reduction of vocabulary size by definition
7. In a scalar attention example, values two and six receive weights three-quarters and one-quarter. What is their weighted output?
   • 八
   • 四
   • 两个
   • 三
8. What does a causal attention mask prevent in an autoregressive text decoder?
   • Using later token positions when forming the current position's representation
   • Using any previous token
   • Computing any query or key vectors
   • Learning all model parameters