স্লাইডিং উইন্ডো মনোযোগ কুইজ

আপনি কুইজ নেওয়ার সাথে সাথে উত্তর এবং ব্যাখ্যাগুলি উপস্থিত হয়।

1. With fixed window W, how does the pairwise attention term N times W scale as sequence length N increases?
   • Quadratically in N despite W being fixed
   • Exponentially in the vocabulary size by definition
   • It never changes as any tokens are added
   • Linearly in N
2. How can stacked local attention layers propagate information beyond one layer’s direct neighborhood?
   • By guaranteeing perfect recall of every earlier token
   • Through intermediate representations passed across successive layers
   • By randomly moving the original input’s coordinates
   • By eliminating the need for all learned weights
3. For a purely local causal attention layer, why can cache policy discard keys older than its active window?
   • Those keys are outside that layer’s allowed direct attention links
   • Every model layer must therefore have an identical whole-model memory bound
   • Older tokens are always irrelevant to every possible task
   • The model’s original weights have been deleted
4. What attention pattern does the Gemma 2 report describe across its layers?
   • Only one permanently disconnected input token
   • No positional or attention computation anywhere
   • Alternating local sliding-window and global attention
   • A mandatory single identical local neighborhood for all models
5. W আকারের একটি কার্যকারণ স্লাইডিং উইন্ডোর জন্য, আমি যে অবস্থানে উপস্থিত থাকব সেখানে কোন কীগুলি জিজ্ঞাসা করতে পারে?
   • আমি পরে সব ভবিষ্যত অবস্থান
   • অবস্থান i বিয়োগ W প্লাস 1 থেকে i
   • শুধুমাত্র অবস্থান 0
   • সমস্ত অবস্থানের একটি এলোমেলো উপসেট
6. In the Mistral report’s rolling cache, which slot stores the keys and values for timestep i with buffer size W?
   • A new permanent slot numbered i+W with no reuse
   • Only slot zero for every timestep
   • The slot at i modulo W
   • A slot chosen by the output token’s alphabetical order
7. Which context_len value is listed in the original Mistral 7B architecture table, separate from its larger theoretical stacked attention span?
   • 4096 tokens
   • 131072 tokens
   • 32 tokens
   • 8192 tokens
8. In Longformer’s stated symmetric global-attention design, how does a selected global token connect to other sequence tokens?
   • It attends to them and they can attend to it
   • It can receive no attention from any other token
   • It must ignore every token outside its local window
   • It requires all non-global tokens to be deleted