research notes · 01

How an AI hears your accent

When you speak English, you carry a signature. It isn't a mistake or a flaw — it's the sound system of your first language gently bending the sound system of your second. Spanish speakers navigate English with five native vowels where English uses more than a dozen. Japanese speakers negotiate consonant clusters their language never needed. The result is systematic, consistent, and — it turns out — perfectly audible to a machine.

A map of pronunciation, learned from listening

Accent Sherlock is built on a self-supervised speech model — a neural network that learned about speech by listening to enormous amounts of it, without being told what any of it meant. Models like this (ours builds on Microsoft's open-source WavLM) internally organize what they hear into layers. The fascinating part is where accent lives: not in the layers that capture what you said, and not in the layers that capture who you are — but in the middle layers, which encode how you pronounce. Pull out those middle-layer representations and you get a kind of coordinate system for pronunciation, where two speakers who shape their vowels the same way land close together, whatever they said.

top layers — what you said words · meaning middle layers — how you say it vowel shades · consonant habits · rhythm bottom layers — raw sound pitch · texture · noise accent lives here
A speech model organizes what it hears in layers. Strip away the top (the words) and the bottom (the raw sound), and the middle is left holding pronunciation itself — the accent.

The silence lesson

An early version of the system had a strange bug: quiet recordings drifted toward the wrong continent. The cause was embarrassingly simple. We were averaging the model's representation over the whole clip — including the silence. Silence has no accent, and averaging it in diluted the signal until the guess followed background noise instead of speech. The fix: detect which moments actually contain a voice, and listen only to those. It's a good reminder that in audio machine learning, what you don't filter out speaks as loudly as what you keep.

ignored ignored kept kept kept silence has no accent
The fix for the silence bug: detect which moments actually contain a voice and pool the model's representation over those alone. The quiet stretches between words carry no accent — averaging them in only dilutes the evidence.

Ordinary words are enough

People expect an accent test to demand tongue-twisters. It doesn't need them. An accent isn't hiding in exotic words — it's distributed across everything you say: the exact shade of every vowel, the way a final consonant is released or dropped, the rhythm your first language taught you. That's why our test is a short, simple passage of everyday words. If English is the language you're learning, you shouldn't have to fight the text to find out how you sound.

General first, particular second

The system reasons the way a careful detective would: broad strokes before fine ones. First it places an accent in a region of the world, where the acoustic evidence is strongest. Only when the signal is clear does it venture a country within that region. And when two origins genuinely sound alike — as some do — it says so, instead of picking one with false swagger.

Confidence you can trust

Every guess ships with a confidence level, and the system is deliberately built never to claim certainty — its display literally caps below 100%. That isn't modesty theater; it reflects the truth of the problem. Accents are similarities, not passports. A guess is "your English sounds like speakers from here," never "you are from here" — and the tool is not, and will never be, a way to identify individuals.

The best way to understand it is to try it: read one short passage and watch where you land.

try the accent guesser →