Over the last year, building a RAG application has become incredibly easy. But if you’ve tried to move that prototype from a Jupyter Notebook to a production-grade CI/CD pipeline, you’ve probably hit a brick wall.
Testing vector search in a GitHub Actions runner without melting the server is a logistical nightmare. When setting up pytest for an app that relies on ChromaDB or Qdrant, engineering teams usually fall into one of two traps.
Trap 1: The Mocking Illusion
You decide to mock the database client using unittest.mock. Your test asserts that collection.query() was called with the right parameters.
Why it fails: You aren't actually testing anything useful. The entire point of a RAG pipeline is semantic search, the cosine similarity thresholds, embedding dimensions, and chunk retrieval. If you mock the vector database, you are flying blind. You’ll deploy to production only to realize your distance metrics were completely wrong.
