Home
Storia in 1 fonti
QLoRA: Fine-Tuning a 7B Model on a 16GB GPU (It Shrank to 5.4GB in Front of Me)
Part 3 of a 4-part series. QLoRA explained — quantize the frozen base to 4-bit, then LoRA on top. The BitsAndBytesConfig that matters, the memory-footprint moment, and why it's about fit, not speed.
Raccontata da
dev.to
Timeline cronologica
- ·
LoRA: I Trained <1% of a 1.5B Model and Matched a Full Fine-Tune
Part 2 of a 4-part series. How LoRA works (the low-rank trick), a working PEFT config, and three real GPU walls I hit — the FP16 unscale error, an OOM, and a 2-GPU speed mystery.
- ·
QLoRA: Fine-Tuning a 7B Model on a 16GB GPU (It Shrank to 5.4GB in Front of Me)
Part 3 of a 4-part series. QLoRA explained — quantize the frozen base to 4-bit, then LoRA on top. The BitsAndBytesConfig that matters, the memory-footprint moment, and why it's…