The Edit That Fixed 4 Tasks and Broke 1

AgentSelfEdit is an open-source sidecar that rewrites its own system prompt from execution feedback. It A/B tests edits and promotes only statistically-proven winners. Code: github.com/deghosal-2026/agent-self-edit

An LLM proposed a prompt edit. It fixed 4 classification tasks. It broke 1. The A/B test showed a real improvement. The gate rejected it.

I inspected every task, every output, every delta. Here's what actually happened — and why "mostly right" isn't good enough when you're building a self-improving system.

The Baseline: A Simple Prompt That Gets Half the Tasks Wrong

The baseline prompt was 2 lines:

The Edit That Fixed 4 Tasks and Broke 1

Related reading

My Self-Improving Agent Still Couldn't Improve. That Was the Breakthrough.

I Built an AI That Rewrites Its Own Prompts — Its Safety Gate Rejected Every…

I Thought This Was a Classification Problem. It Wasn't.

I Compared a Local 4B, a Better Cloud Model, and Role Separation. The Results…

I Thought Role Separation Would Fix the Optimizer. It Didn't.

I Thought the Optimizer Was the Product. I Was Wrong. The Gate Was.