An AI system can understand every word someone says and still judge their answer differently because of their accent. New research led by the University of Washington highlights this gap, raising questions for businesses using audio AI to assess candidates, presentations or spoken English.
The study evaluated six audio language models across American, British, Chinese, Indian and Nigerian English. Several models gave Chinese- and Nigerian-accented speakers lower delivery scores, even when the content was identical and the speech was accurately transcribed. Across three widely used models, scores also declined as speech became acoustically more distant from American English.
For teams integrating AI into decision-making workflows, the findings expose a critical distinction: recognizing speech accurately and evaluating a speaker fairly require separate evidence.
The same words, different scores
Researchers used synthetic voices to deliver identical scripts across five accents, then validated their findings with 52 human speakers. The tasks included interview responses, presentations and English proficiency assessments.
The models generally favored American and British accents, placed Indian accents in the middle, and rated Chinese and Nigerian accents lowest. The differences affected delivery scores and sometimes extended to content, despite the words remaining unchanged. Model explanations also described some speakers as unclear while successfully transcribing their speech.
That combination matters. A system might pass a transcription accuracy test while producing an unreliable ranking of candidates. Its written explanation could then make the ranking appear justified, even when the explanation conflicts with what the model demonstrably understood.
What the research does—and does not—establish
The researchers tested general-purpose models rather than auditing the proprietary systems used by specific hiring platforms or examination providers. The findings therefore identify risks in potential building blocks, without establishing how any particular commercial assessment product performs.
The origins of the bias also remain uncertain. Researchers suggest possible contributors include uneven representation of accents in training data and learned assumptions about what professional speech should sound like. These are proposed explanations rather than proven causes.
For a business choosing an AI component, this means vendor benchmarks alone cannot establish whether the finished application is suitable for its intended users.
What this means for engineering teams
Our engineering takeaway is that evaluation needs to follow the decision the software will support. If a system ranks interview answers, testing whether it produces accurate transcripts covers only one part of its behavior.
A practical validation process should include:
- Comparable inputs across accents. Check whether equivalent answers receive comparable scores.
- Separate checks for content and delivery. Investigate whether accent changes assessments of knowledge, reasoning or relevance.
- Results broken down by user group. Look beyond overall averages to identify where performance differs.
- Reviewable decisions. Preserve enough information for reviewers to examine scores and challenge unsupported explanations.
- Regression checks after changes. Repeat relevant evaluations when models, prompts or scoring rules change.
Human review also needs a clear purpose. Reviewers should be able to inspect the underlying answer and question the assessment, with the authority to correct it.
Reliability includes who the system works for
For Control F5 Software, this research reinforces the importance of validating AI within the workflow it will influence. Model selection, evaluation criteria, review mechanisms and ongoing maintenance all shape the reliability of the resulting system.
When software helps determine who progresses to an interview or passes an assessment, quality includes whether equivalent inputs receive consistent treatment. That requires evidence about the people using the system—and a process for addressing failures when they appear.
We have helped 20+ companies in industries like Finance, Transportation, Health, Tourism, Events, Education, Sports.