NVIDIA Introduces X-Token: Projection-Guided Cross-Tokenizer KD That Outperforms GOLD by +3.82 Average Points on Llama-3.2-1B

NVIDIA proposes X-Token, a projection-guided cross-tokenizer knowledge distillation method using P-KL and H-KL loss formulations

Raccontata damarktechpost.com