Home
Storia in 1 fonti
LoRA: I Trained <1% of a 1.5B Model and Matched a Full Fine-Tune
Part 2 of a 4-part series. How LoRA works (the low-rank trick), a working PEFT config, and three real GPU walls I hit — the FP16 unscale error, an OOM, and a 2-GPU speed mystery.
Raccontata da
dev.to
Timeline cronologica
- ·
I Fine-Tuned a 270M Model on My Laptop (Full Fine-Tuning, From Scratch)
Part 1 of a 4-part series. Full fine-tuning a tiny Gemma 3 model for intent classification — the generative framing, the loss-masking trick, and why full FT is so learning-rate…
- ·
LoRA: I Trained <1% of a 1.5B Model and Matched a Full Fine-Tune
Part 2 of a 4-part series. How LoRA works (the low-rank trick), a working PEFT config, and three real GPU walls I hit — the FP16 unscale error, an OOM, and a 2-GPU speed mystery.
- ·
QLoRA: Fine-Tuning a 7B Model on a 16GB GPU (It Shrank to 5.4GB in Front of Me)
Part 3 of a 4-part series. QLoRA explained — quantize the frozen base to 4-bit, then LoRA on top. The BitsAndBytesConfig that matters, the memory-footprint moment, and why it's…