Word error rate (WER) is the share of words a speech recognition system gets wrong compared with a correct reference transcript. It is the standard accuracy metric for automatic speech recognition (ASR), so it is often the first number you check when you build or evaluate a voice agent.
For example, suppose a caller says “charge fifty dollars to card ending nine,” and the system transcribes “charge fifteen dollars to card ending nine.” One of the seven words is wrong, so the WER is 1 / 7, or about 14.3%. That looks like a small error, but the amount a voice agent would act on is now wrong.
Lower WER is better. Because extra inserted words also count as errors, WER can go above 100%.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why word error rate matters for voice agents
A voice agent can only act on what it hears. If the transcript is wrong, every later step works from bad input.
WER is useful for:
- Comparing ASR systems: Measure two recognizers on the same audio and reference transcripts.
- Catching regressions: Check whether a model update or audio pipeline change made transcription worse.
- Finding weak conditions: Break WER down by speaker, microphone, noise, or utterance length to see where recognition fails.
- Separating failure sources: Tell whether a voice agent’s mistake started in transcription or in a later reasoning step.
WER treats every word equally, though. A wrong filler word and a wrong account number count the same. That is why you should pair WER with checks on what the agent actually did.
For more on agents that work through speech, see voice agent.
How word error rate works
WER compares two pieces of text:
- Reference: The correct transcript, usually written by a person.
- Hypothesis: The transcript the ASR system produced.
First, the hypothesis is aligned to the reference word by word, using the fewest possible edits. Then each edit is counted as one of three types:
- Substitution: A reference word was replaced by a different word.
- Deletion: A reference word is missing from the hypothesis.
- Insertion: The hypothesis contains an extra word that is not in the reference.
The Hugging Face audio course evaluation chapter covers this alignment and the metric in detail. WER is also the default evaluation reported for ASR models in the Transformers ASR documentation.
How to calculate word error rate
Add up the three kinds of errors and divide by the number of words in the reference.
Formula:
WER = (S + D + I) / N
Where:
- S is the number of substitutions.
- D is the number of deletions.
- I is the number of insertions.
- N is the number of words in the reference transcript.
N always comes from the reference, never the hypothesis. Using the wrong denominator is the most common calculation error.
Two worked calculations
In the opening example, the reference has seven words (N = 7). Only fifty changed, to fifteen. So S = 1, D = 0, and I = 0.
WER = (1 + 0 + 0) / 7 ≈ 0.143 = 14.3%
Now consider a longer example with two error types. The reference is “please move five hundred dollars from checking to savings” (N = 9). The hypothesis is “please move five hundred dollar checking savings.”
Here dollars became dollar, which is one substitution. The words from and to are missing, which are two deletions. There are no insertions.
WER = (1 + 2 + 0) / 9 ≈ 0.333 = 33.3%
Calculate WER in Python
The following function aligns two transcripts with a word-level edit distance, then walks back through the alignment to count each error type. It uses only the standard library. The third example shows how insertions can push WER above 100%.
def wer_counts(reference, hypothesis):
"""Return (S, D, I, N) from a minimum-edit word alignment."""
ref, hyp = reference.split(), hypothesis.split()
rows, cols = len(ref) + 1, len(hyp) + 1
cost = [[0] * cols for _ in range(rows)]
for i in range(rows):
cost[i][0] = i
for j in range(cols):
cost[0][j] = j
for i in range(1, rows):
for j in range(1, cols):
sub = cost[i - 1][j - 1] + (ref[i - 1] != hyp[j - 1])
cost[i][j] = min(sub, cost[i - 1][j] + 1, cost[i][j - 1] + 1)
# Walk back through the table to count each edit type.
S = D = I = 0
i, j = len(ref), len(hyp)
while i > 0 or j > 0:
if i > 0 and j > 0 and cost[i][j] == cost[i - 1][j - 1] + (ref[i - 1] != hyp[j - 1]):
S += ref[i - 1] != hyp[j - 1]
i, j = i - 1, j - 1
elif i > 0 and cost[i][j] == cost[i - 1][j] + 1:
D += 1
i -= 1
else:
I += 1
j -= 1
return S, D, I, len(ref)
examples = [
("charge fifty dollars to card ending nine",
"charge fifteen dollars to card ending nine"),
("please move five hundred dollars from checking to savings",
"please move five hundred dollar checking savings"),
("yes cancel my order",
"yes uh um please cancel all of my order"),
]
for reference, hypothesis in examples:
S, D, I, N = wer_counts(reference, hypothesis)
print(f"S={S} D={D} I={I} N={N} WER={(S + D + I) / N:.1%}")
Output:
S=1 D=0 I=0 N=7 WER=14.3%
S=1 D=2 I=0 N=9 WER=33.3%
S=0 D=0 I=5 N=4 WER=125.0%
In the third example, five inserted words against a four-word reference give a WER of 125%. For production use, an established evaluation library is a safer choice than hand-written code.
Why normalization changes the score
Before you calculate WER, both transcripts must go through the same text normalization. Lowercasing, punctuation removal, number formatting, and contraction rules all change the score.
The following code compares a reference that has a capital letter and a period with a hypothesis that has neither. It lists the words that differ before and after normalization.
import string
def normalize(text):
text = text.lower()
return text.translate(str.maketrans("", "", string.punctuation))
reference = "Charge fifty dollars to card ending nine."
hypothesis = "charge fifty dollars to card ending nine"
def mismatches(ref, hyp):
return [(r, h) for r, h in zip(ref.split(), hyp.split()) if r != h]
print("raw: ", mismatches(reference, hypothesis))
print("normalized:", mismatches(normalize(reference), normalize(hypothesis)))
Output:
raw: [('Charge', 'charge'), ('nine.', 'nine')]
normalized: []
Without normalization, those two formatting differences would count as two substitutions, a WER of 2 / 7, or about 28.6%. The recognizer heard every word correctly. If normalization differs between reference and hypothesis, you are measuring formatting, not recognition.
How to measure WER well
Fix the normalization pipeline first and apply it to both sides. Then report more than one number.
- Report the error breakdown: Show S, D, and I separately, not only the total.
- Watch insertions: A recognizer that invents words during silence or cross-talk inflates WER differently from one that substitutes words.
- Slice by condition: Break results down by microphone, speaker, background noise, and utterance length. A single average hides where recognition fails, including on a multimodal model‘s audio path.
- Share the pipeline: State your normalization rules so others can compare results.
These follow ordinary evaluation metrics practice.
Check what the agent did, not only what it heard
For a voice agent, add a layer that WER cannot see. Trace the conversation and compare the transcript with the extracted entities, such as the amount, date, name, and confirmation. Then check whether the action matched the caller’s intent.
The Arize AX documentation covers tracing and evaluating audio applications this way.
Suppose one transcript has 5% WER but corrupts the only amount in the sentence. Another has 15% WER, and all its errors are filler words. The first is worse for the task.
That gap between string similarity and task correctness is the main limitation to keep in view. For more on the full evaluation picture, see voice agent evaluation.
WER vs. character error rate
Character error rate (CER) uses the same formula at the character level. N is then the number of characters in the reference.
CER is the better choice in a few cases:
- Languages written without spaces: Word boundaries are ambiguous or missing.
- Morphologically rich languages: A single word can carry many units of meaning.
- Spelling diagnostics: Near-miss spellings matter and you want partial credit.
WER remains the default for word-based languages such as English and for comparing ASR systems on shared benchmarks.
The two answer different questions, so one does not replace the other. Neither measures whether the recognized text leads to the right action.
Limitations
WER is a useful first check, but it measures a narrow thing.
- Ignores importance: Names, numbers, negations, and instructions count the same as articles.
- Depends on normalization: Cross-team comparisons are meaningless unless the pipeline is shared and stated.
- Misses other qualities: WER says nothing about diarization (who spoke), punctuation quality, or whether the transcript supports the task.
- Has no ceiling: Insertions can push WER above 100%, which surprises teams expecting a bounded percentage.
Low WER shows the recognizer is close at the word level. It does not show the conversation succeeded. That is why transcript quality belongs inside broader AI observability for voice systems.
Frequently asked questions
Can WER be higher than 100%?
Yes. Insertions add errors without adding reference words. For example, five inserted words against a four-word reference give a WER of 125% before any other errors.
What is a good WER?
There is no universal threshold. A good WER depends on the language, audio conditions, vocabulary, and the cost of an error in your task. Compare systems on the same audio, reference transcripts, and normalization, and judge acceptability by downstream task errors.
Why is WER low but my voice agent still makes mistakes?
WER weights every word equally. One wrong number or name can be the only error in a sentence and still reverse the action the agent takes. Evaluate entity and task accuracy on traces in addition to WER.
Should I use WER or CER?
Use WER for word-level comparison in languages with clear word boundaries. Use CER where word boundaries are ambiguous or character-level near-misses matter. Many teams report both, with the normalization pipeline stated.
Is WER calculated on the reference or the hypothesis?
The denominator N is always the number of words in the reference transcript. The hypothesis length affects only the error counts.
Putting word error rate to work
When you report WER, include the S, D, and I breakdown, the reference word count, and your normalization rules. Add slices for the audio conditions you care about.
For voice agents, don’t stop at WER. Check the extracted entities and the final action on real conversations.
Tracing helps you connect the two. Arize AX can capture the transcript, the extracted entities, and the agent’s action in one trace, so you can see when a single wrong word changed the outcome.
