AI Coding Agents Can Pass Tests and Still Make the Wrong Decision

A question I've been thinking about after discussing AI coding agents with several developers:

Is passing the test suite enough to prove that an AI agent made the correct engineering decision?

I don't think it is.

And this isn't just a theoretical concern.

Modern coding agents are increasingly working at the repository level rather than generating isolated code snippets. OpenAI's Codex documentation, for example, describes using repository-specific AGENTS.md instructions to tell the agent how to navigate a codebase, run tests, and follow project practices. Anthropic similarly describes Claude Code searching codebases, tracing dependencies, editing multiple files, and working with CI failures. (OpenAI)