Tag

#open-weights

Reports tagged open-weights.

Reports

1 report

Aug 2026

08-20 fine-tuning LoRA vs. Full Fine-Tuning in 2026: What Survived the Re-Measurement Two 2024 results anchored how practitioners chose between LoRA and full fine-tuning: "LoRA learns less and forgets less," and the warning that even matched benchmark scores hide structurally different solutions — an "illusion of equivalence." Between 2025 and 2026 both were re-measured, and the answer split by training regime. In supervised fine-tuning the canon survives, but its conditions have been rewritten in terms of adapter capacity, adapter placement, and learning rate. In reinforcement-learning post-training, LoRA now matches full fine-tuning at ranks as low as one — a result the canon never anticipated, resting so far on a lab blog and its reproductions rather than peer review. And three parts of the 2024 answer were never re-tested at all: the effective-rank mechanism offered to explain the gap, the canon's continued-pretraining protocol on current models, and any parity comparison on mixture-of-experts architectures. 27 cited · of 366 sources