We’re done with simple text generation. The new era of agentic AI isn't about spitting out strings anymore; it’s about models actually using your mouse and keyboard. As of mid-2026, OpenAI Operator and Claude Computer Use dominate the conversation. They’ve ditched the API-first mindset. Now, they treat your desktop like a browser.
How it works under the hood
Claude Computer Use is basically a tool-calling wrapper slapped onto the Claude 3.5 Sonnet architecture. It snaps screenshots, stares at the pixels, and figures out where to click. You hand it an image and some environment data; it spits back JSON with coordinates and actions.
This is huge for legacy apps. You don't need a REST API to automate a garbage enterprise portal from 2012. If a button renders on your screen, the model can click it. That said, latency is a killer. Because it has to ingest a frame, reason about your UI, and then trigger an action, a single click takes 3 to 10 seconds. You’re trading a two-line Python script for expensive, high-compute visual reasoning.
OpenAI Operator: The Agent approach
