Google Cloud Summit came to London last week, and we took the opportunity to sit down with database execs Sailesh Krishnamurthy (VP engineering) and Yasmeen Ahmad (product executive Agentic Data Cloud).The event was wall-to-wall agentic AI, and true to the theme, Ahmad told us that "we're putting agents at the center ... with the goal that humans are not going to be using data platforms in the next three to five years. It’s going to be humans orchestrating agents, and agents actually doing the work."One of the key AI-driven changes, Krishnamurthy said, is that when retrieving data "it’s not so much about getting the exact results, but getting the best results."
For developers skilled in crafting SQL queries that get precise results in the most efficient way, the notion of inexact queries that go through some sort of non-deterministic and compute-expensive parsing may seem like a step backwards.
"If you have exact questions, you need to be able to provide exact answers," Krishnamurthy told us. "But I think inexact questions are what people are also going to expect. When you think about agentic workloads and operational databases, you want to be able to ask more flexible questions." An example might be a natural language query that takes into account context, such as previous interactions.Krishnamurthy described "AI native infrastructure," including vector indexing, text indexing, and graph technology where "you combine structured and unstructured data, you have to be operating in terms of inexact results and data quality."The company is also investing in the "knowledge catalog," formerly called Dataplex, which is enterprise search now also treated as context for LLMs (large language models). Knowledge catalog aggregates organization data across multiple sources including structured and unstructured sources.Krishnamurthy said that exact SQL queries are not going away, and that sometimes a "fuzzy question in natural language" might generate an SQL query with exact results.How do you verify that AI-generated SQL is producing the results you want? “The answer is the same, not just about SQL, but about many AI-related things," said Krishnamurthy. “The answer is a set of evals you have to maintain ... you might start with something where some results work well and some don’t. And then you have to keep iterating on your blueprints and other pieces of context until your eval set is 100 percent working well."By eval set, Krishnamurthy means "a set of questions that are representative tests that users may have, and what is the right query that is generated associated with it, and then a determination of is this query, is this answer correct or not?"Google SQL as used in its distributed Spanner database, PostgreSQL-compatible AlloyDB, and in the BigQuery data warehouse engine now has AI functions such as AI.IF, which evaluates a condition described in natural language and returns true or false. The prompt value is evaluated using a Gemini LLM; and could return an error or null if the model fails such as when unavailable or out of quota. The inefficiency of functions like AI.IF is a problem, but there are possible solutions. One is the idea of proxy models, which Krishnamurthy described as "a tiny model in the database." A proxy model is trained on the fly, based on a small sample of the data. The query engine evaluates the results from the proxy model, and if good enough, uses it for inference in place of a call to the LLM. According to a paper on the subject proxy models "consume about 400x less tokens, and the latency goes down by 30x-100x."
