A few weekends ago I got annoyed at typing prompts into a terminal and decided the fix was, obviously, to control my AI agent with hand gestures instead. This is the story of building that, and the two hours I lost fighting a GPU crash that had nothing to do with my code.
The idea: a webcam watches your hand, MediaPipe tracks the landmarks, and three gestures map to three actions on an Anthropic-powered coding agent.
Pinch (thumb and index touching) - the agent writes code
Spinning your index finger in a circle - the agent brainstorms an idea
Two fingers "running" up and down - it runs whatever code it just wrote
