Articles · Literacy · AI in education · Equity
How AI can listen to a child read: speech recognition explained
A plain-language guide to how speech recognition listens to children read aloud, where it goes wrong, and what school leaders should ask before using it.
By Dr. William Hickman · 5 min read
Since 2020, I have been building reading technology that uses artificial intelligence, natural language, and speech recognition to help children learn to read. The question I hear most from principals and reading teachers is a fair one: how can a machine know whether a six-year-old read a word correctly?
The short answer is that it listens, compares what it heard to what was on the page, and makes its best guess. The longer answer matters more, because the guess is only as good as the design behind it, and the children most likely to be misheard are often the ones who most need a patient listener.
Why listening to a child read matters
Reading aloud has always been the window into a beginning reader’s head. A teacher listening to a child read a short passage can hear which sounds she knows, which patterns she guesses at, where she self-corrects, and where she stalls. Running records and oral reading fluency checks are built on that window.
The constraint has never been knowledge. It has been time. One adult and a room full of children means each child reads aloud to a skilled listener for only a few minutes a week, and many children read aloud to no one at home. Practice volume matters. Koedinger and colleagues found in 2023 that students learn at remarkably similar rates for each opportunity to practice, so the number of opportunities makes much of the difference. The promise of a listening tool is more practice, with something paying attention.
What happens in the seconds after a child reads
Here is the basic pipeline, without the jargon.
1. Capture. A microphone records the child’s voice. The quality of everything that follows depends on this step more than most buyers realize.
2. Recognition. An acoustic model turns the sound into the most likely sequence of speech sounds and words. This is the part trained on recordings of many people talking.
3. Alignment. Here reading tools have an advantage over open dictation: they know what the child is supposed to say. The system lines up what it heard against the words on the page, one word at a time.
4. Scoring. Each word gets a mark: read correctly, substituted, skipped, added, repeated, or self-corrected. Timing adds more, such as long pauses, reading rate, and the familiar measure of words correct per minute.
5. Response. Then the tool decides what to do. Wait. Offer a hint. Model the word. Encourage the child to try again. Log the pattern for the teacher.
Natural language processing adds one more layer. It lets a tool ask a child what happened in the story and make sense of an answer given in her own words, instead of only checking a box.
Where the listening goes wrong
A reading tool can fail a child in several predictable ways. Every leader should know them before buying one.
Children’s voices are harder. Young children have higher-pitched voices and still-developing articulation, and many speech systems were built largely on adult speech. A system that works well for adults can stumble on a first grader.
Dialect and accent. In 2020, a Stanford team led by Allison Koenecke tested speech recognition systems from five of the largest technology companies. On average, the systems got about 35 percent of words wrong for Black speakers and about 19 percent for white speakers. Those were adults talking, not children reading, but the lesson carries straight into a reading tool. A dialect feature is not a reading error. A tool that cannot tell the difference will mark a capable reader as struggling.
Language transfer. Picture a first-year English learner who decodes well in Spanish and carries Spanish vowel sounds into English words. A tool may read that pattern as a gap in phonological skills. What she may actually need is explicit instruction on a few English vowel patterns and plenty of reading. A teacher’s ear can catch that. A score alone may not.
Speech differences. A child with an articulation difference may read a word correctly in her own speech. If the tool counts it as a decoding error, it is measuring the wrong thing.
Noise. A classroom microphone picks up the whole room, and fluency estimates from noisy audio can swing wildly from one day to the next. A quieter corner and short reads of a minute or two often fix more than any software update.
What a good listener does with uncertainty
When I build tools that assess students, I follow a few rules I wish every vendor followed.
First, recruit testers from the edges, early. For a reading tool, that means students with dyslexia and students learning English. For anything with voice, it means children with different accents and speech patterns, from the first prototype rather than after launch.
Second, never display false precision. If a system cannot tell a dialect feature from a misread word, it has no business printing a confident number. Show uncertainty plainly, and let the teacher play the recording behind any mark.
Third, match the threshold to the consequence. During practice, a missed error costs little; the child reads again tomorrow. In a screening decision, a false error can attach a label to a child. The bar for acting on a flag should rise with what the flag will cause to happen.
A child’s voice is sensitive data
A recording of a child reading is personal. Families will reasonably ask whether someone is recording their child, who can listen, how long the recordings are kept, and whether they are used to train a company’s models. Get those answers in writing in the vendor agreement, then turn them into a short paragraph that goes home with every permission form. Recordings kept for instruction may become part of a student’s education record, so bring in your privacy lead and your district’s counsel before you launch.
A practice partner, not a diagnosis
A listening tool can give children more practice and give teachers more evidence. It cannot diagnose dyslexia, a speech disorder, or a language disability, and it should never be the only voice in a decision about a child. If the data suggest a concern, that is a reason for the school team, which may include a reading specialist, a speech-language pathologist, and a school psychologist, to take a closer look together with the family.
The next play
- Ask for results by group. How does the tool perform for young children, English learners, speakers of different dialects, and children with speech differences? If the vendor has not tested, require a pilot with your own students.
- Listen yourself. During a pilot, have a reading specialist score twenty recordings by ear and compare her marks with the tool’s.
- Fix the room. Set up a quieter recording spot before you draw any conclusions from the data.
- Write the family paragraph about recordings: who hears them, how long they are kept, and whom to ask.
- Keep a person in every placement. No fluency number alone should move a child into or out of an intervention.
How speech systems and other AI tools fail the people outside their default user is the subject of Built for Someone Else.