แบบทดสอบที่เชื่อมโยงกับคำแนะนำ · ง่าย ระดับ

เหตุใด LLM จึงดิ้นรนเพื่อนับจดหมายแบบทดสอบ

ทดสอบว่าโทเค็นซ่อนตัวอักษรแต่ละตัวอย่างไร เหตุใดจึงทำให้เกิดข้อผิดพลาดในการสะกดและการนับ และวิธีแก้ปัญหาที่เชื่อถือได้ เช่น การสะกดคำหรือโค้ด

เส้นทางแนะนำที่เกี่ยวข้องเหตุใด LLM จึงดิ้นรนเพื่อนับตัวอักษร
คำถาม 1 ของ 8

What is the main reason language models miscount letters in a word?

ดูตัวอย่างทุกคำถาม8 คำถาม

คำตอบและคำอธิบายจะปรากฏขึ้นเมื่อคุณทำแบบทดสอบ

  1. What is the main reason language models miscount letters in a word?
    • They have too little training data about English
    • They process text as tokens, chunks of characters, rather than individual letters
    • They deliberately round numbers to save computation
    • They cannot store numbers larger than ten
  2. What does a tokenizer produce from a piece of text?
    • A list of integer IDs, each standing for a chunk of text
    • A summary of the text's meaning
    • A translation into a universal language
    • An image of the text for the model to read
  3. Which prompt change most helps a model count letters correctly?
    • Asking the question in all capital letters
    • Telling the model the answer is important
    • Asking it to spell the word out with separators before counting
    • Asking it to answer as quickly as possible
  4. Byte pair encoding builds its vocabulary by doing what?
    • Assigning one token to every dictionary word
    • Repeatedly merging the most frequent adjacent pairs of symbols into new tokens
    • Splitting text only at spaces and punctuation
    • Randomly grouping characters into fixed-length chunks
  5. Why might 'Strawberry' and ' strawberry' be handled differently by a model?
    • The model ignores capital letters completely
    • Only one of them is in the training data
    • Capitalization and leading spaces can change how the text is split into tokens
    • Models treat the first word of a sentence as a command
  6. According to the guide, what is a common misconception about letter-counting failures?
    • That tokenizers exist at all
    • That they prove models cannot reason in general, when the limit is mostly the input format
    • That different models use different tokenizers
    • That code can count letters exactly
  7. For a word game that needs exact letter counts every time, what is the most reliable approach?
    • Ask the model three times and take the most common answer
    • Use a larger model
    • Compute the counts with ordinary code and use the model for other parts
    • Ask the model to double-check its answer
  8. Why do newer reasoning models often get the strawberry question right?
    • They were given a list of all letter counts
    • They tend to spell the word out step by step before answering
    • They use no tokenizer at all
    • They refuse to answer counting questions