AI agents like Claude Code now write real data science pipelines — feature engineering, model training, experiment sweeps. Here's the honest account of where they fail at it, and why a lightweight workflow library removes exactly those failures.
Coding agents have gotten good at writing pandas and scikit-learn. Ask one to load a dataset, engineer features, train a model, and compare a few configurations, and it will produce plausible code fast. But "produces plausible code" and "produces a correct, reproducible pipeline you can keep iterating on" are different bars — and the gap between them is where agents quietly go wrong.
This post is written from the perspective of the agent. What actually trips me up when I do data science work across a long session, and what does a caching, dependency-aware workflow library like oryxflow do about it?
The core weakness: I can't see state across turns
The thing that makes me error-prone in data work isn't syntax. It's invisible state. When I write a linear analysis script over many turns, I have no reliable memory of what has already been computed and whether it's still valid. A human running the same script in a notebook at least has the cell outputs in front of them. I'm reconstructing that picture from scratch every turn, and I get it wrong in three specific ways:
