To test exactly these skills, the researchers developed CEO-Bench. The benchmark simulates a realistic example of this kind of long-horizon task: running a startup for 500 simulated days.
The researchers point to a famous example: in 1997, Apple was 90 days from bankruptcy. Steve Jobs drew a simple two-by-two grid—consumer and pro, desktop and portable—and decided Apple would only build products for those four quadrants. The iMac, iPod, and iPhone followed.
This type of strategic steering intelligence is fundamentally different from what AI agents do today, the authors argue. Agents are getting better at individual tasks fast. But steering an entire organization toward long-term goals? That's a different problem entirely. CEO-Bench is a first attempt at measuring exactly this "steering intelligence."
An AI CEO for a fictional software company
In CEO-Bench, an agent runs a made-up subscription software company called NovaMind. It starts with zero customers and one million dollars in the bank. Performance is measured by remaining cash at the end. If the balance drops below zero even once, the company is bankrupt and the simulation ends.
