Tag
#evaluation
Reports tagged evaluation — page 3 of 3.
Reports
22 reportsAug 2026
08-20 fine-tuning LoRA vs. Full Fine-Tuning in 2026: What Survived the Re-Measurement Two 2024 results anchored how practitioners chose between LoRA and full fine-tuning: "LoRA learns less and forgets less," and the warning that even matched benchmark scores hide structurally different solutions — an "illusion of equivalence." Between 2025 and 2026 both were re-measured, and the answer split by training regime. In supervised fine-tuning the canon survives, but its conditions have been rewritten in terms of adapter capacity, adapter placement, and learning rate. In reinforcement-learning post-training, LoRA now matches full fine-tuning at ranks as low as one — a result the canon never anticipated, resting so far on a lab blog and its reproductions rather than peer review. And three parts of the 2024 answer were never re-tested at all: the effective-rank mechanism offered to explain the gap, the canon's continued-pretraining protocol on current models, and any parity comparison on mixture-of-experts architectures. 08-20 agents The Skill Is Real, the Ritual Is Not Yet Measured Curated skills measurably improve coding agents where they carry procedure the model lacks; the process disciplines the frameworks sell remain unmeasured — and the nearest tests lean the other way.