Skip to content
Speaking is much faster than typing, and while it’s an increasingly convenient way to interact with computers, it’s hardly private. Providing speech privacy in a way we haven’t seen before is this prototype tongue-reading system that uses machine learning and ultrasound to read tongue movements and turn them into decoded speech. Not only can a user speak without emitting a sound, since it doesn’t read sound waves it’s completely immune to noisy environments.
Tongues are a far richer source of speech data than reading lip and mouth movements.
It turns out that tongue movements are a very rich source of information about speech, and an ultrasound probe under the chin takes very clear video of a tongue. With a dataset consisting of only around 50 hours of training data, the system has a 15.6% error rate and generalizes across different speakers (as long as they speak with similar accents).
That error rate may seem high at first glance, but keep in mind this is for a prototype system built in a month around a relatively small training dataset. All indications are that better results are just a matter of better training.
