I wanted to add a background-removal tool to my site's image cluster that stayed true to the 100% client-side processing principle I already use for PDFs and image conversions. The path there was anything but linear: a library dropped over a licensing problem, a carefully chosen model that turned out more limited than expected, and a bug that froze the entire page — not just the tool — during computation.
Here's the full build, including the parts that didn't work the first time.
The starting problem: what's actually feasible for free?
The initial idea was broad: remove backgrounds, and maybe unwanted objects too. The two tasks have very different difficulty levels.
Removing objects requires inpainting — plausibly reconstructing the erased area — which in practice still means heavy generative models, impractical to run client-side with good quality on an average device.
