The paradox of LLM self-distillation: Faster reasoning, weaker generalization - TechTalks

Optimizing LLMs for concise answers can destroy their ability to explore alternative solutions on difficult problems. New study reveals the hidden cost of self-distillation.

Raccontata dabdtechtalks.com