XLNet Permutation Modeling Quiz

Answers and explanations appear as you take the quiz.

1. What pretraining objective does XLNet use?
   • Permutation language modeling
   • Masked language modeling
   • Replaced token detection
   • Next sentence prediction
2. Which BERT weakness was XLNet specifically designed to address?
   • BERT was too small
   • The [MASK] token causes a pretrain/fine-tune mismatch and assumes masked-token independence
   • BERT could not handle short sentences
   • BERT lacked positional encodings
3. What does XLNet's two-stream self-attention separate?
   • Encoder and decoder layers
   • Training and inference graphs
   • A content stream and a query stream
   • Token and sentence embeddings
4. Does XLNet physically shuffle the words in a sentence?
   • Yes, it randomly reorders the actual tokens
   • Yes, but only during fine-tuning
   • No, it never changes any ordering at all
   • No, it permutes the factorization order using attention masks while keeping positions
5. Which architecture serves as XLNet's backbone for long-range context?
   • Transformer-XL
   • LSTM
   • Original vanilla Transformer
   • Convolutional network
6. How does XLNet combine bidirectional context with autoregressive prediction?
   • It predicts all tokens independently with no attention mask
   • It predicts in sampled factorization orders without inserting a [MASK] input token, using attention masks
   • It physically shuffles the sentence before every prediction
   • It replaces autoregressive prediction with a denoising autoencoder
7. For a token sequence A B C, suppose a sampled factorization order predicts C before A and B. What context is available when predicting C?
   • The tokens A and B must always be provided first
   • Only the next sentence is available
   • No earlier tokens in that sampled order, although C remains at its original input position
   • The input sequence is physically reordered to C A B
8. How does XLNet prevent the target token from leaking into its own prediction query?
   • It removes the target from the entire training sequence
   • It uses the target’s label as an input embedding
   • It replaces the token with a [MASK] symbol
   • The query stream represents the target position without its token content