Word Error Rate Explained

Automatic speech recognition turns audio into written words. Word error rate measures how much an output differs from a human reference after aligning the two word sequences. A substitution replaces one reference word, a deletion omits one and an insertion adds an extra word. Add those counts and divide by the number of words in the reference. NIST’s speech-evaluation tools and documentation use these error categories. The denominator is not the number of predicted words, so insertions can make WER exceed 100 percent on a short reference. Suppose a reference says “send five boxes” and the system says “send nine boxes now.” One substitution changes five to nine; one insertion adds now. With three reference words, the WER is two divided by three. Yet that fraction cannot tell whether the wrong number was safety-critical, whether the extra word was harmless or whether a different paraphrase was useful. WER treats words as edit units, not their consequences. Evaluation choices matter. The reference transcript may itself be uncertain in noisy speech. Consistent rules are needed for contractions, punctuation, capitalization, numbers, disfluencies and spelling. A system output “twenty one” may be judged differently against “21” depending on normalization. A single overall WER can also conceal worse performance for particular accents, languages, age groups, microphones or background noise. Report slices with enough examples and inspect where errors occur, not only their aggregate count. For a product, pair WER with task outcomes and human review where words carry special meaning. Names, medication amounts or negations may deserve explicit checks. Real-time systems also need latency metrics; a low final WER does not mean the words appeared promptly or remained stable as the person spoke. Keep reference data independent of model tuning and describe the normalization procedure so the score can be reproduced.