Whenever someone asks me to add search to an application, I try to find ways around it. The implementation itself isn't the problem, it's everything that comes with it: extra components to manage, more failure points, and the constant challenge of keeping data in sync. For the past few years, I've worked a lot with DynamoDB and with the introduction of Vector Search I feel a lot more comfortable to add this type of functionality. I wrote last week about why AWS released another vector store and where DynamoDB Vector Search fits in the landscape. In this post I want to show you how you can take an existing DynamoDB table and add vector search to it.
The starting point
The current API uses a serverless setup. It includes SAM for infrastructure, API Gateway in front, Lambda functions behind, and DynamoDB for storage. The API contains plain CRUD operations to manage recipes: create, read, update, delete, and list.
The complexity arises when you want to add filters to query exactly for what you need. In DynamoDB this means you need to add Global Secondary Indexes (GSI) for every permutation... this is really not scalable. So the only option until now was to have a data pipeline to index the data separately and provide search. That is not the case anymore! With vector search in DynamoDB, we can store vector embeddings alongside our data and search them directly. Now, users can search our recipes using natural language queries and find recipes based on the meaning, not just exact keyword matches.
