Quick answer: What are Agentic Vector Databases? Agentic vector databases are vector database systems adapted for AI agents: they support iterative search, memory, tool use, and knowledge retrieval across multi-step workflows. Instead of only returning relevant chunks to an LLM, they help agents decide what to retrieve, what to remember, and how to act on changing information.TL;DR: Vector databases are moving beyond passive retrieval. In agentic systems, they support iterative search, memory, and knowledge compilation, helping agents retrieve, update, and act on information over time. Chroma, Weaviate, Qdrant, Milvus, and Pinecone show different versions of this shift.Vector databases have been around for a long time – from the early vector-space model in information retrieval, through decades of Nearest Neighbour (NN) and Approximate Nearest Neighbour (ANN) research, and the deep learning era that turned semantic meaning into dense embeddings. Systems like FAISS, Milvus, Pinecone, Weaviate, Qdrant, and Chroma have built vector databases as a solid stack that remains an essential part of the LLMs workflows. Without them models had no relevant knowledge to retrieve via Retrieval-Augmented Generation (RAG).But times are changing. We are entering the Agentic Era, and the rules are changing with it. The needs of agentic AI are different, and we need to revisit what once worked perfectly.Vector database infrastructure is already transforming for these needs: retrieval becomes part of the reasoning process, search becomes more iterative and multi-stage, memory transforms into a more dynamic layer that stores and updates an agent’s experience, and the data itself becomes much more specialized for agents.Today, we are going to break down each of these aspects with illustrative solutions from Chroma and Weaviate (companies that have been working on vector databases for a long time) and look at a new and very interesting case: a new knowledge engine from Pinecone that builds a completely new layer on top of vector databases specifically for agents. This is a must-read if you want to learn how to restructure your retrieval loop to work with agents.In today’s episode:How vector databases work in classic scenarioWhat changes in the Era of AgentsAgentic search or Agentic RAGChroma’s Context-1 search subagentAgentic Memory as a Retrieval LayerEngram: Weavite’s memory layerPinecone’s Nexus: A New Knowledge Engine LayerConcluding thoughtsSources and further readingHow vector databases work in classic scenarioIn their classical form, vector databases solved a practical retrieval problem: given a query represented as a vector, find the most similar stored vectors and return the relevant content to an LLM.The usual pipeline was straightforward. Raw documents were split into chunks. An embedding model converted each chunk into a dense vector, meaning that most dimensions carried some numerical information. The database stored this vector together with the chunk text, identifiers, and metadata. At query time, the system embedded the user’s question, searched for nearby vectors, applied filters when needed, and returned a ranked set of chunks that could be inserted into the model’s context.But vector search was never one universal stack. Different embedding models create different vector spaces, and the geometry of those spaces depends on how the representations are learned. This means the best similarity measure depends on the model, the data, and the task.The most common ways to measure vector similarity are:Cosine similarity which measures how much two vectors point in the same direction.Inner product search (or dot-product search) that “cares” not only about direction but also about magnitude – how large the vectors are.Euclidean distance that shows the straight-line distance between two vectors in space.Vector databases became an infrastructure category because scale changed the problem. When systems have to work with millions or billions of vectors, the database cannot simply store vectors. It also has to make similarity search fast, efficient, and reliable. This is where tools such as Milvus, Pinecone, Weaviate, Chroma, Qdrant, and others became part of the AI infrastructure stack, giving models a way to retrieve relevant context, external knowledge, and grounded sources.Many of these systems are also moving toward hybrid search, combining semantic retrieval with lexical methods such as BM25, sparse vectors, metadata filtering, and reranking.For a deeper breakdown of where vector databases came from, how they work, and how to choose between them, read our guide to vector databases in FMOps.Now, as agents become part of the workflow, the problem is becoming more complex again. This is where the next stage of database evolution begins. As we will see, new layers are already being built on top of familiar vector databases. It’s quite fascinating. What changes in the Era of AgentsFirst of all, the agentic era changes the emphasis. Agents plan the workflow, perform tasks, check out what works and what doesn’t. Everything has moved to the multi-layered reasoning, and agents started to gain practical knowledge. Where to store this and how to enable systems to function under constant change and accumulated experience?In standard LLM workflows, the database mostly acts as a passive retrieval layer. In agentic systems, we have no choice but to make retrieval a part of the reasoning process itself and a part of the memory stack, where a system can write what happened, retrieve what matters, consolidate what should persist, and constrain what should never be reinforced.But there is another notable change: agents are emerging as AI users alongside humans. Raw data and constant updates are a challenge for agents, so we need to provide them navigation in this data world and conditions for proper functioning.DimensionStandard LLM workflowsAgentic systemsDatabase rolePassive retrieval layerActive part of reasoning and memoryMain actionRetrieve relevant chunks for one queryRetrieve, write, update, and reuse knowledge over timeStored informationDocuments, embeddings, metadataTask history, decisions, failures, preferences, constraints, and learned patternsRetrieval purposeAdd context to an LLM responseSupport planning, action, self-correction, and continuityMain challengeFind the right informationDecide what to remember, forget, update, or block from reinforcementUsersHumans and LLM applicationsHumans, applications, and agents navigating changing data environmentsNew layerVector databases and RAGAgentic search, agentic RAG, memory, and knowledge enginesSo it makes sense that many infrastructure companies are now building agentic search, agentic RAG, agentic memory, and knowledge-engine layers on top of existing vector database systems.Let’s look at how the main vector database players are adapting to this shift.Agentic Search or Agentic RAGIn classical RAG, the database mostly acts as a passive retrieval layer. But some questions need a deeper analysis →Don’t settle for shallow articles. Learn the basics and go deeper with us. Truly understanding things is deeply satisfying. Join Premium members from top companies like Microsoft, NVIDIA, Google, Hugging Face, OpenAI, a16z, plus AI labs such as Ai2, MIT, Berkeley, .gov, and thousands of others to really understand what’s going on in AI. The system has to find one clue, use it to search for the next clue, compare documents, ignore distractors, and only then return useful evidence. A single database query is not enough for that. Chroma, Qdrant, Milvus and many others see the solution in agentic search, or agentic RAG, which significantly changes the role of vector databases. In agentic systems, retrieval becomes part of the reasoning process itself and turns into an iterative process. This way, the model/agent can behave more like a researcher and break the problem into smaller sub-questions, search multiple times, evaluate what it found, notice gaps, reformulate queries, continue searching until it has enough context to answer reliably, or even decide whether the retrieval is needed.Image Credit: “What is Agentic RAG? Building Agents with Qdrant” blog postQdrant, Milvus or any other vector databases’ role here is the retrieval layer or a tool that a model can call. It stores vectors, metadata, and sometimes sparse representations, then gives the agent search tools. So the agent might use vector databases for:semantic vector searchsparse / keyword-style searchhybrid search, which is especially important, because agents work with various types of informationmetadata filteringquery expansionresult-quality checkingChroma’s Context-1 search subagentHowever, if you use an entire model for each retrieval step, the process becomes too messy and too expensive. Chroma has recently proposed an interesting development – Context-1 which follows the idea of the division of labor. It is trained as a specialized search subagent with a narrow job:Take a complex queryDecompose it into subqueriesRun iterative searches through the corpusRead documentsAnd return a ranked list of documents that are likely to help a stronger model answer to synthesize the final answer.Context-1 additionally has a self-editing part: it prunes irrelevant chunks from its own context, managing its working memory during search. So only useful documents stay in the prompt.This is Chroma’s attempt to separate two tasks, search and reasoning, to optimize them separately and make search much cheaper and faster.But anyway, in the agentic workflows, retrieval, reasoning, memory, and planning blur together into the same infrastructure layer. This forms a feedback loop around retrieval, and turns everything into a multi-step search loops where agent is constantly deciding what to search, which retrieval method to use, whether the results are sufficient, and what information is still missing.But what to do with the results of agent’s functioning?Agentic Memory as a Retrieval LayerWhen optimization and overall token efficiency matter so much, we can’t afford agents going through the same actions over and over again. This is too expensive. So another important retrieval layer is agentic memory. Agentic memory is where you store all the past experience of agent’s workflow like user preferences, successful retrieval strategies, previous tool calls, intermediate reasoning steps, or facts discovered during earlier interactions. It is like a database but for actions and agents own history, not just external information. Most systems separate this memory into a few broad categories:Semantic memory stores general facts.Procedural memory stores behavior patterns like “if you need to do X, then do Y,” “X should be done before Y,” etc.Episodic memory stores artifacts from previous runs, such as useful search results, successful plans, tool calls and others.Every memory is stored as a record with metadata: type, timestamp, confidence, source, usage frequency, or relevance to different execution phases. During a new run, the agent retrieves memories in the same way it retrieves documents during RAG.So in agentic workflows, vector database functions more like a persistent cognitive layer for the agent itself. And what is even more interesting, memory quality starts to be as important as model quality. Engram: Weaviate’s memory layer An interesting case here is Weaviate’s Engram – a managed memory layer built on top of Weaviate vector database. It turns agent’s interaction data into memory records which you can search, update and reuse. Here is how it works:You send raw data like chat messages, events, or pre-extracted memories to Engram.Engram runs an asynchronous pipeline. It processes memory updates in the background, while the system keeps running normally. This pipeline extracts useful memories, checks them against existing ones and rewrites, merges, keeps, or deletes records. This comparison of new and old data opens up a possibility to store more complex facts with their dynamic changes, for example: “The user used to work as a machine learning engineer, but has now been promoted to CEO.”The final memory objects are stored in Weaviate database.Image Credit: Engram: Memory by Weaviate blog postWhen the agent needs context, it searches the memory store semantically through Weaviate.Engram organizes memory around different topics: user preferences, stable facts about the user, work patterns, conversation summaries, tools the agent used, user feedback and others. It also uses scopes to define who a memory belongs to and which data is allowed to affect it.So vector databases like Weaviate provide the retrieval substrate, but the real work now happens in the pipeline around it, performed by additional layers like Engram and Context-1 and the full reconstructions of the search and memory for the agentic needs.The first generation of vector databases solved some foundational problems. The next challenge is broader: helping systems remember, update, reason over, and act on knowledge over time. Agents now require this. That is what pushed Pinecone to create a new knowledge-engine layer.Pinecone’s Nexus: A New Knowledge-Engine LayerGive humans a set of raw retrieved files, and they usually know what to do with them. They can scan, compare, summarize, ignore irrelevant parts, and bring in judgment from outside the retrieval system. Give the same files to agents, and the problem changes. In the best case, they spend a lot of tokens moving through repeated retrieval and analysis steps. In the worst case, they get stuck, miss the point, or hallucinate. Retrieval reasoning remains one of the weaker parts of agentic workflows.This is why Pinecone is trying to move more reasoning to an earlier stage: from retrieval at query time to knowledge compilation before the agent even asks for information.Since Pinecone has just released this layer, we went straight to the company and asked a few questions about what this development means for vector databases. Their view is that while “vector primitives and their management are essential, the retrieval patterns agents require are fundamentally different from what humans need.”The result is Nexus, a knowledge engine that Jeff Zhu frames as a new infrastructure category built on top of the database. Instead of giving an agent raw files, Nexus prepares task-optimized representations in advance. These can include artifacts: structured forms of information that an AI agent can use directly in planning, reasoning, and action.Nexus has two components: Image Credit: Pinecone Nexus blog postThe context compiler is the main one that reads the raw data, builds task-specific context and constructs artifacts. Each agent gets only that information which suits its specialization and that it needs to understand the workflow and perform the concrete task or group of tasks. For example, a finance agent will get artifact with billing schedules, usage thresholds, expansion signals, while a CEO agent will be provided with data about product milestones and hiring velocity. This knowledge is completely reusable. The context compiler is iterative, trying and evaluating different representations to find what works best for each agent. That strategy is then formalized as a curate() function.Here are a couple of words about the structure of the artifacts:❝The key difference in our approach is that artifact shape isn't predetermined by a fixed schema or graph ontology. The Pinecone Nexus context compiler discovers the right structure, granularity, and construction strategy for each domain based on evaluation signals. It then generates the artifacts from raw data based on the strategy that scored the highest in the evaluations.The artifact representation can range from markdown files to extracted entities to tables. These artifacts are then indexed for retrieval within Pinecone’s database which support both semantic, sparse, and full text search capabilities so that the right artifacts are retrieved during query time and then composed for the final structured output response from Nexus.Jeff Zhu, VP of Product at PineconeThe composable retriever then passes these structured artifacts to the agent after they query information.This gives lower latency because everything is already prepared in advance for the agent. Pinecone also claims Nexus reduces token use by up to 90%. But what if the data is constantly changing and the system needs to react in real time? Indeed, that was one of the major advantages of traditional vector databases.❝When source data changes, curate() function runs incrementally over just the new or updated content and refreshes the affected artifacts. We use deep citations in the artifacts to help us identify the artifacts that need to be updated when sources change,explains Jeff ZhuAnd if the task changes, you can update the evaluations to adjust the context compiler, and it will recompile artifacts to match the new task.The most costly part is the process of finding the initial strategy to structure domain-specific knowledge for agents. Then it is fixed and easily used and reused during execution, which also means that Nexus approach can run on small and cheaper language models.❝This, in addition to the “compile once, read many” patterns that will be common in agentic systems, allows us to provide significantly better cost economics rather than asking the agent to reason over raw data at run time,says Jeff ZhuIn addition to Nexus, Pinecone has also introduced KnowQL – a declarative query language for AI agents that standardizes how agents request, retrieve, and consume knowledge across heterogeneous data sources. It is like “SQL for agent knowledge infrastructure.” Agents can specify six primitives: the intent of the query, required filters, output structure, provenance/citations, confidence thresholds, and compute budgets. Everything is made for the new-coming infrastructure.Concluding thoughtsThis is a useful glance into how one of AI’s most basic layers – data and knowledge – is changing. Some companies are integrating databases more directly into agentic workflows. Others are building new layers on top of existing database infrastructure. Databases are not going away – they are following the transformation of the field. And we don’t know yet what approach will prove to be the best.The retrieval stack is becoming more orchestration-oriented, and memory is moving beyond passive chat history. It is becoming part of the reasoning loop. But this also creates new engineering problems: deciding what is worth remembering, handling conflicting memories, reducing retrieval noise, ranking relevance over time, versioning memories, and forgetting outdated information.Infrastructure is now being reshaped for human users with more complex needs, and for agents that need to navigate data, use context, and act on knowledge. Some tasks do make sense to delegate to agents, and in some cases agents may handle them better than humans. What makes this shift especially interesting is that it is happening at the lower layers of the stack. The system is being rebuilt so agents can move through it, retrieve what they need, and operate more reliably. And what we see is the whole stack is transforming layer by layer. Amazing.Sources and further readingEngram: Memory by Weaviate | Blog postWhat is Agentic RAG? Building Agents with Qdrant | Blog postAgentic RAG with Milvus and LangGraph | Blog postRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks | PaperResources From Turing Post:FAQWhat is an agentic vector database?An agentic vector database is a vector database adapted for AI-agent workflows. It supports iterative search, memory retrieval, metadata filtering, hybrid search, and context selection, so agents can find, reuse, and act on knowledge across multi-step tasks.Agentic RAG vs traditional RAG: what is the difference?Traditional RAG usually retrieves relevant chunks once and sends them to an LLM as context. Agentic RAG makes retrieval iterative. The agent can break a task into sub-questions, search multiple times, evaluate results, reformulate queries, and decide whether it has enough evidence to continue.Why do AI agents need memory?AI agents need memory because they operate across tasks, tools, and repeated interactions. Memory helps them avoid repeating the same steps, preserve useful context, store preferences and constraints, and reuse successful patterns from previous runs.What is a knowledge engine?A knowledge engine is an infrastructure layer that prepares information for agents before retrieval happens. Instead of giving an agent raw files, it can compile, structure, update, and serve task-specific knowledge that the agent can use more directly.Are vector databases still needed in agentic AI?Yes. Vector databases still provide the retrieval substrate for semantic search, hybrid search, metadata filtering, and memory lookup. What is changing is the layer around them: retrieval is becoming more dynamic, memory-aware, and integrated into agent workflows.