Language Modeling Quiz

Answers and explanations appear as you take the quiz.

1. In the stated autoregressive language-model formulation, what is predicted from the preceding token context?
   • A probability distribution for the next token
   • The original file's alphabetical rank
   • Only a temperature measurement from the device
   • A mandatory image label with no sequence information
2. Which limitations should be considered for a short-history n-gram language model?
   • A requirement that all n-gram models reject every unseen sequence even with smoothing
   • A guarantee of unlimited history
   • A requirement to process only images
   • Limited context and possible zero probabilities for unseen continuations without smoothing
3. Why can next-token language-model pretraining obtain targets without a separate class annotation for every position?
   • The model has no training objective
   • The target next tokens are already present in the observed text
   • All training text must have been generated by the same model
   • Every position requires an external image classifier
4. How does autoregressive text generation use a selected continuation token?
   • It predicts every future token independently in one step
   • It discards every selected token before the next prediction
   • It appends the token and uses available earlier output as context for later predictions
   • It changes the training labels to future deployment outcomes
5. Which objective is commonly minimized for the stated autoregressive next-token training setup?
   • Squared numerical distance between alphabetical token ID values
   • Only the number of tokens in an output filename
   • Only hinge loss on image-pixel locations
   • Cross-entropy or negative log-likelihood for observed target tokens
6. What do temperature and top-p choices affect during sampled language-model generation?
   • How continuation tokens are selected from model scores
   • The number of trained layers after every token
   • A guarantee that all output claims become true
   • The label written in every observed training sequence
7. During teacher-forced next-token training, where does the preceding context for a target token come from?
   • Only the model's freely sampled earlier output
   • A randomly generated replacement corpus at every position
   • The observed training sequence
   • The final evaluation labels supplied after deployment
8. A stated two-token prefix has probabilities 0.5 for the first token and 0.2 for the second given the first. What is its joint prefix probability?
   • 0.7
   • 0.1
   • 0.4
   • 0.3