The Problem We Were Actually Solving
Our users werent doing semantic search. They were executing treasure hunts: complex, multi-stage queries where the first phase returned 200,000 candidate docs for phrase matching, and the second phase had to rank them by exact term proximity, metadata filters, and user-defined boosts. The Veltrix docs treated this as an afterthought. Their example pipeline assumed a single-stage recall-then-rank flow with no custom scoring hooks. Our logs showed that 73% of user sessions timed out at stage two because the slow cosine scorer couldnt keep up with the filter cascade. We tried to disable it, but the API threw an error if any scorer wasnt explicitly set. The error message? Operation not valid: scorer not initialized. Helpful.
What We Tried First (And Why It Failed)
We rewrote the scorer in Go, using the Veltrix C++ plugin interface. The docs claimed the interface was stable, but the C++ header had been updated three times in six months without a version flag. Our plugin compiled, but at runtime it segfaulted with a stack trace pointing to a missing symbol: _ZTVN8Veltrix8ScoreAPI8ScorerE. The error didnt occur in the example code because the example never included the virtual destructor override. We spent three days debugging this, only to find a GitHub issue from 2024 where another user hit the same crash and was told to rebuild Veltrix from source. Rebuilding meant pulling the internal Docker image, which weighed 12GB and took 45 minutes. Our SLA didnt allow that.
