Tokenizer-Free Byte-Level Models Quiz

Answers and explanations appear as you take the quiz.

1. What does a tokenizer-free byte-level model read as its input units?
   • Raw bytes (UTF-8 values 0-255)
   • Word-piece subwords from a fixed vocabulary
   • Whole words separated by spaces
   • Pre-computed embeddings only
2. What is the main practical drawback of feeding raw bytes to a standard transformer?
   • Bytes cannot represent non-English text
   • The sequences become much longer, raising compute cost
   • There are too many possible byte values to embed
   • Bytes lose all word meaning permanently
3. How does the Byte Latent Transformer (BLT) decide where to group bytes into patches?
   • By splitting on every space character
   • Using a fixed patch size of 4 bytes
   • Based on byte-level prediction entropy (uncertainty)
   • Randomly during each forward pass
4. Why can a fixed subword vocabulary use more tokens for some languages than others?
   • Subword tokenizers only accept ASCII
   • All languages have identical token boundaries
   • They cannot represent numerals
   • Its token inventory may reflect the language distribution in the data used to build it
5. What does removing a fixed tokenizer guarantee about byte-level input?
   • The same byte alphabet can represent text from different scripts, while learned language ability still depends on data
   • The model understands every language without training data
   • It guarantees equal performance for all scripts
   • The model no longer needs a vocabulary or learned embeddings
6. Which earlier model demonstrated byte-level processing but was criticized for being slow?
   • BERT
   • ByT5
   • Word2Vec
   • GPT-2
7. What representations does BLT’s large global Transformer process?
   • Individual words from a fixed subword vocabulary only
   • Image pixels
   • Dynamically sized byte patches
   • Robot joint-angle commands
8. Which components surround the global Transformer in BLT?
   • A reward model and a value network
   • A speech vocoder and a spectrogram
   • A camera and a lidar sensor
   • A local byte encoder and a local byte decoder