When building local RAG (Retrieval-Augmented Generation) applications, edge agents, or serverless AI pipelines, developers usually hit a wall with standard vector stores: memory overhead.
Running a dedicated vector database locally often demands hundreds of megabytes—or gigabytes—of RAM just to keep indices warm. On the flip side, lightweight local options like scanning raw JSON files or querying SQLite don't scale well when vector dimensions climb into the thousands (1536d+).
We built Bitweave to solve this exact trade-off: a zero-copy, SIMD-accelerated hybrid retrieval engine in Rust (with Python bindings) that handles categorical filtering and vector search while locking its active heap footprint under 1.1 MB RSS.
The Architecture: How Bitweave Achieves Sub-Millisecond Speed at <1.1 MB RAM
Bitweave relies on a 3-part design to maximize search speed while keeping memory consumption negligible:
