The wager: an OS in a browser tab
I started HELIOS with three constraints: no native code, no head-mounted display, and no integration that requires a user to install a runtime. The browser would be the OS, the webcam would be the sensor, and WebGL would be the display. That was the whole pitch.
The first prototype was ugly. I rendered a single flat plane in Three.js, put a fake window texture on it, and used MediaPipe’s hand landmarks to push a cursor around. It felt like a tech demo with a lot of lag. The breakthrough came when I stopped emulating a mouse and started modeling gestures. The first time I pinched two fingers together, dragged a plane sideways, and watched it move as a physical object while the camera tracked my hand, the wager felt real. No Bluetooth controller. No QR code setup. No proprietary SDK. Just a Chrome tab, a laptop camera, and a few dozen lines of TypeScript.
The technical bet inside that wager is that WebXR is the wrong abstraction for spatial computing on devices people already own. WebXR is great when you want to strap on a headset and leave the room. But most spatial interactions happen while you are sitting in front of a screen, and the most durable spatial sensor you already have is a camera.
