TL;DR— Low-rank adapters are treated as a cheaper stand-in for full-parameter fine-tuning, but the rank bottleneck reshapes the optimization landscape itself. For preference optimization and distillation, this means LoRA doesn't just learn a compressed version of the same thing full fine-tuning would learn— it learns something structurally different, often favoring surface style over the deeper distribution shifts alignment actually requires. Choosing a post-training method is a modeling decision, not a compute-budget decision.
Everyone treats LoRA as a budget lever. Full fine-tuning is expensive, LoRA is cheap, so teams default to LoRA and assume they're getting a slightly worse version of the same result. That assumption is wrong in a way that matters most precisely where teams reach for LoRA the most: preference optimization and distillation.
The mistake is thinking of low-rank adaptation as compression of a fine-tuning run. It isn't. It's a constraint on the optimization problem itself. When you restrict weight updates to a low-rank subspace, you aren't approximating the gradient full fine-tuning would take— you're forcing the model to solve a different problem: find the best update that happens to live in a small subspace. Those are not the same optimum, and the gap between them is not uniform across tasks.
