Most background-removal and object-removal tools online work the same way: you upload an image, a server runs a model on it, you download the result. Convenient, but your photo left your device and you have no idea what happened to that copy afterward.
I wanted to see how far I could push the "everything stays client-side" constraint for a full image editor — background removal, object removal, filters, format conversion, compression — using only what a browser can do natively: Canvas, WebAssembly, and an on-device ML model. This is APIC-Web, and here's what I learned building the two hardest parts.
Background removal: category masks vs. confidence masks
The obvious approach is MediaPipe's ImageSegmenter with outputCategoryMask: true — it hands you a hard 0/1 label per pixel (background or foreground). It works, but the output has a visible jagged, stair-stepped edge around hair and shoulders. There's no way to fix that after the fact with a blur, because the information needed to know how much a boundary pixel belongs to the foreground is already gone — it got rounded to a hard integer.
The fix is outputConfidenceMasks: true instead, which returns a float 0–1 probability per pixel before it gets thresholded. That single change unlocks a proper edge pipeline:
