WordPiece Tokenization Quiz

Answers and explanations appear as you take the quiz.

1. What does the '##' prefix indicate in WordPiece output?
   • The token is a continuation piece inside a word, not word-initial
   • The token is a special separator like [SEP]
   • The token has the highest frequency in the vocabulary
   • The token is an unknown out-of-vocabulary marker
2. Roughly how large is the WordPiece vocabulary used by the original BERT?
   • About 256 tokens
   • About 30,000 tokens
   • About 1 million tokens
   • About 500 tokens
3. How does WordPiece's merge criterion differ from standard Byte-Pair Encoding?
   • It merges the rarest pair to reduce vocabulary
   • It never merges and uses only characters
   • It merges by maximizing training-data likelihood rather than raw frequency
   • It merges pairs at random during training
4. What does a BERT-style WordPiece tokenizer return if it cannot segment a word into vocabulary pieces?
   • It always falls back to individual Unicode characters
   • A random subword selected from the vocabulary
   • The original word encoded as one new token
   • The unknown-token marker, such as [UNK]
5. At inference time, how does WordPiece choose how to split a word?
   • It matches the longest subword in the vocabulary greedily, left to right
   • It picks the shortest possible pieces first
   • It tries every possible split and averages them
   • It uses a neural network to predict splits
6. Which well-known model family is WordPiece most associated with?
   • GPT models
   • BERT and its derivatives
   • LLaMA models
   • Stable Diffusion
7. What kind of vocabulary did multilingual BERT use across its training languages?
   • A completely separate vocabulary for every language
   • A vocabulary made only of complete words
   • One shared WordPiece vocabulary
   • A list of byte sequences with no subwords
8. Which tokenizer class in Hugging Face Transformers constructs a BERT tokenizer that uses WordPiece?
   • GPT2Tokenizer
   • RobertaTokenizer
   • CLIPTokenizer
   • BertTokenizer