"The reports of my death are greatly exaggerated." — Mark Twain, if he were a search engine.
For a few years there, it looked like the future of search belonged to the upstarts. Pinecone, Weaviate, Milvus, Qdrant—specialized vector databases born in the LLM era, promising semantic search at the speed of thought. Meanwhile, the venerable Apache Lucene (and its flagship offspring, Elasticsearch) was written off as a "legacy keyword engine" with some vector features bolted on the side.
That narrative, it turns out, was premature.
Between 2025 and 2026, Lucene underwent a hardware-native revolution that rewrote its vector search engine from the silicon up. Elasticsearch leveraged these foundations to launch a serverless architecture that decouples compute from storage, and introduced DiskBBQ—a vector format that sustains 15ms query latencies in 100 MB of RAM. Enterprise adoption of hybrid search (combining lexical + dense vector + sparse neural retrieval) tripled in a single quarter, while standalone vector databases lost market share.
This isn't just a comeback story. It's a fundamental architectural shift. Let's dig into the engineering.
