Back to Articles
A field report on multi-vector mining, MaxSim geometry, knowledge distillation, and the experiments that did not work.
I started with a narrow question: Do hard negatives mined by a multi-vector retriever produce a better late-interaction model than negatives mined by a dense retriever? Pffftt, Obviously!
I introduce GLInt, which reaches 57.43 mean nDCG@10 on the 15 BEIR Tasks, starting from the unsupervised LateOn checkpoint. It outperforms the 57.22 result of LateOn, making it the strongest retriever under 300M parameters in this comparison. It also sets the highest Quora and HotPotQA score among models below 7B parameters.
The model and training data are available here:
