I built on-device workout rep counting in Flutter — here's what actually worked

I'm building TrainWiz, a Flutter app that turns real exercise into a pet-raising game: you do squats or push-ups, your phone counts the reps, and a little creature levels up and evolves. The core technical problem sounds trivial and absolutely is not: count reps from the camera, on-device, without uploading a single frame.

Here's what broke along the way, and what finally worked.

Why on-device

Two reasons: privacy and latency. A fitness camera that streams your body to a server is a non-starter for most people, and rep feedback has to feel instant or the whole "game" loop dies. So everything runs locally with tflite_flutter + an on-device pose model — no footage ever leaves the phone.

Naive attempt #1: joint-angle thresholds