PANDUAN Asas

Mengapa LLM Berjuang Mengira Surat

Model bahasa yang besar bergelut untuk mengira huruf kerana mereka tidak membaca teks satu aksara pada satu masa.

Dengar panduan iniAI-diceritakan · ~3 min

Pada halaman ini4 minit membaca
  1. Gambaran keseluruhan
  2. Menyelam dalam
  3. Pelaksanaan Dunia Sebenar
  4. The Future of Why LLMs Struggle to Count Letters
  5. Mempraktikkannya
  6. Teruskan Meneroka
  7. Soalan lazim

Gambaran keseluruhan

They read tokens, which are chunks of characters, so the individual letters inside a word are often never directly visible to the model. This one fact explains a whole family of odd mistakes with spelling, counting, reversing and rhyming, and it tells you how to work around them.

Menyelam dalam

Before a language model sees any text, a tokenizer splits it into tokens. Most modern tokenizers use a method such as byte pair encoding, which learns common character sequences from a large body of text. Frequent words often become a single token, and rarer words are split into a few pieces. The model then receives each token as a number and turns it into a vector. It never receives the letters inside that token as separate inputs. So when you ask how many r's are in 'strawberry', the model is not scanning ten characters. It sees perhaps two or three chunks, and it has to recall, from patterns learned during training, which letters those chunks contain. That knowledge exists but it is indirect and fuzzy, like being asked how many times the letter e appears in a word you have only ever seen as a picture. The strawberry question became a widely shared example in 2024 precisely because the answer is obvious to people and surprisingly hard for chatbots. The same cause produces related errors: misspelling when asked to spell out loud, reversing words wrongly, failing letter-based constraints like acrostics or lipograms, and miscounting characters for length limits. Exact splits differ between tokenizers, so one model may get a given word right while another fails. A common misconception is that this shows models cannot reason or are simply careless. The limit is mostly in the input format, not in general reasoning. When a model is prompted to spell the word out first, each letter tends to become its own token, and counting becomes a far easier task. Newer reasoning models often do this spelling step on their own, which is one reason they get these questions right more often. For anything where exactness matters, running a short piece of code is still the most reliable method.

Pelaksanaan Dunia Sebenar

Asked how many times the letter r appears in 'strawberry', a chatbot answers two instead of three, because the word reaches the model as a few multi-letter chunks rather than ten separate letters.

A teacher asks a model to write a sentence where every word starts with the letter s, and it slips in words that do not, since it has to infer each word's first letter instead of seeing it.

A developer asks for the word 'encyclopedia' spelled backwards and gets a version with swapped or missing letters; asking the model to first list the letters one per line, then reverse the list, fixes it.

A puzzle app needs exact letter counts for a word game, so it computes them with a single line of ordinary code and uses the model only to write the hints.

The Future of Why LLMs Struggle to Count Letters

Researchers are exploring models that work directly on bytes or characters, and hybrid designs that group bytes dynamically, which could make character-level tasks more natural. These approaches have tradeoffs, because longer input sequences cost more compute. Meanwhile, reasoning models that spell words out step by step and models that call code tools already reduce many of these errors in practice. It is reasonable to expect fewer embarrassing letter-counting failures over time, but for tasks that need exact character counts, using code rather than trusting a model's recall will likely remain good practice.

Mempraktikkannya

Mengapa ia penting

  • Keputusan yang lebih jelas. Ia membantu anda memisahkan tuntutan teknikal yang jelas daripada bahasa pemasaran.
  • Kos dan bajet. Anda boleh bertanya soalan pelaksanaan yang lebih baik sebelum menghabiskan wang atau masa.
  • Pasukan dan aliran kerja. Pasukan yang berkongsi pemahaman membuat keputusan produk, dasar dan pembelajaran yang lebih baik.

Berhati-hati untuk

  • Pasukan yang berbeza mungkin menggunakan istilah yang sama secara berbeza, jadi tentukan skop lebih awal.
  • Penanda aras boleh kelihatan kukuh manakala prestasi dunia sebenar tidak sekata.
  • Mengabaikan kualiti data dan rancangan penilaian sering menghasilkan hasil yang rapuh.

Langkah seterusnya

  1. Mulakan dengan definisi bahasa biasa hasil yang anda perlukan.
  2. Pilih satu metrik kejayaan dan satu keadaan kegagalan sebelum ujian.
  3. Jalankan juruterbang kecil dengan data perwakilan, bukan set demo yang digilap.
  4. Document where Why LLMs Struggle to Count Letters helps and where simpler methods are better.

Teruskan Meneroka

Newsletter percuma

Dapatkan taklimat AI harian

Tiga cerita AI yang disahkan setiap pagi hari bekerja, ditulis dalam Bahasa Inggeris biasa. Percuma selamanya, tiada iklan.

Satu e-mel setiap hari bekerja. Berhenti melanggan dalam satu klik. Kami tidak pernah menjual atau berkongsi alamat anda.

Uji diri anda

Take the Why LLMs Struggle to Count Letters quiz

Maklum balas segera pada setiap jawapan, dan sijil yang boleh dikongsi dengan ID yang boleh disahkan sebaik sahaja anda lulus kursus.

Mulakan kuiz

Sokong pendidikan AI percuma. AI Understanding ialah pertubuhan bukan keuntungan 501(c)(3) — tiada iklan, tiada bayaran dinding, selamanya. Buat derma

Soalan lazim

What is Why LLMs Struggle to Count Letters?

Large language models struggle to count letters because they do not read text one character at a time. They read tokens, which are chunks of characters, so the individual letters inside a word are often never directly visible to the model. This one fact explains a whole family of odd mistakes with spelling, counting, reversing and rhyming, and it tells you how to work around them.

Apakah sebab utama model bahasa salah mengira huruf dalam perkataan?

Model menerima token, yang selalunya mengandungi beberapa aksara. Huruf di dalam token bukanlah input yang berasingan, jadi model mesti mengingatnya secara tidak langsung.

Apakah yang dihasilkan oleh tokenizer daripada sekeping teks?

Tokenizer membahagikan teks kepada kepingan daripada perbendaharaan kata tetap dan memetakan setiap satu kepada ID integer, yang kemudiannya bertukar menjadi vektor.

Perubahan segera yang manakah paling membantu model mengira huruf dengan betul?

Menulis perkataan dengan pemisah seperti s-t-r-a-w biasanya meletakkan setiap huruf dalam tokennya sendiri, yang menjadikan pengiraan tugas yang lebih mudah.

Pengekodan pasangan bait membina perbendaharaan kata dengan melakukan apa?

BPE bermula daripada bait atau aksara dan terus menggabungkan pasangan bersebelahan yang paling kerap, jadi rentetan biasa menjadi token tunggal.

Mengapakah 'Strawberi' dan 'strawberi' boleh dikendalikan secara berbeza oleh model?

Tokenizer selalunya memasukkan ruang utama dan kotak dalam token, jadi perubahan permukaan yang kecil boleh menghasilkan pemisahan token yang berbeza.