Party is over: regularizing ColBERT models to fix efficient ANN methods

Back to Articles

Efficient ANN methods like MUVERA and SMVE promise to simplify ColBERT infrastructure, but they fell short with modern ColBERT model. We trace the root cause to embedding geometry, show that mean-centering is a strong but insufficient fix, and introduce STE-based regularization that directly optimizes the model for the target projection space. The surprise: the fix doesn't make embeddings more isotropic but concentrates them into fewer dimensions, the opposite of what intuition predicts, but what random projections need. The regularization transfers across methods, seeds, and hyperparameters without degrading full MaxSim retrieval.

Table of Contents

Late Interaction: Powerful, but Expensive

The Solutions