An AI data agent turns a plain English question into a query, runs it, and hands back an answer. The demo is always impressive. The part that decides whether it survives contact with a real company is not the query generation at all, it is what happens on either side of it.
Most teams evaluating these tools grade them on the wrong thing. They ask whether the agent got the SQL right on a question they already knew the answer to. The question that matters is whether they would forward the agent's answer to someone who is about to spend money on it.
Query Generation Is The Easy Half
A capable model turns "which campaign drove the most revenue last quarter, broken down by channel" into a join across campaign and revenue tables, filtered to a date range, grouped by channel. That has been reliable for a while now, and it is the part every vendor demos.
What it does not tell you is whether the revenue table double counts refunds, whether the campaign table has rows with a null channel that silently drop out of the join, or whether last quarter is missing three days because an ingestion job failed on a holiday weekend.
