RWKV Linear Attention Quiz

Answers and explanations appear as you take the quiz.

1. What is the headline property of RWKV?
   • It trains like a Transformer but runs inference like an RNN
   • It can only be trained on images
   • It requires a quadratic key-value cache
   • It cannot be parallelized at all
2. What do the letters in RWKV stand for?
   • Read, Write, Keep, Verify
   • Receptance, Weight, Key, Value
   • Recurrent, Wide, Kernel, Vector
   • Random, Weighted, Knowledge, Variation
3. How does RWKV keep memory constant during generation?
   • By deleting the model weights
   • By storing every token on disk
   • By carrying a small fixed-size hidden state forward instead of a growing cache
   • By limiting outputs to ten tokens
4. What role does the time-decay weight (W) play?
   • It sets the learning rate
   • It counts the number of tokens
   • It picks the output language
   • It controls how quickly older information loses influence
5. Compared to softmax attention, what does RWKV's linear-attention recurrence avoid?
   • The quadratic all-pairs token comparison
   • Any use of keys and values
   • The need for embeddings
   • The use of neural networks
6. Why is RWKV especially well suited to streaming generation?
   • Because it must see the whole input before any output
   • Because each new token depends only on the previous fixed-size state
   • Because it cannot generate text
   • Because it needs a GPU cluster for every token
7. In the guide’s RWKV description, what does the receptance gate control?
   • How many image patches are generated
   • The size of the model vocabulary
   • How much of the accumulated state is read out
   • The amount of GPU memory available
8. What training property does RWKV retain from Transformers?
   • Full quadratic key-value caching at inference
   • A separate generator and discriminator
   • A fixed top-k token router in every layer
   • Parallelizable training across sequence positions