We trained a text-to-speech model with voice cloning that fits into 100 MB (fp16) and runs on-device for synthesis, from smartphones to a Raspberry Pi. The model is in a file, generation happens on the device. No API, no cloud backend at inference time.
Early tests on a budget Android phone hit faster-than-realtime on a single core. Full benchmarks with numbers and different devices are coming in a separate post. Here is how we got there.
The model is about 45M parameters (a language model plus an audio decoder). For comparison, typical server-side TTS systems are hundreds of MB to several GB of weights. We trained both stages from scratch and packaged them into a single binary.
The road here was not smooth. This post is a short architecture overview (without it, it is hard to explain what broke) and two concrete stories of what broke and how we fixed it. If you end up training something similar, this should save you a few surprises.
Architecture in two paragraphs
