Tag
#fine-tuning
Reports tagged fine-tuning.
Reports
4 reportsAug 2026
08-25 fine-tuning Can This Checkpoint Still Learn? Why retention and future learnability need separate tests in repeatedly trained neural networks. It frames a three-arm choice: continue, reset training state, or retrain from scratch. 08-23 reasoning How SPADE turns self-generated worlds into a bounded training loop A model can improve without human-written QA pairs or stronger-teacher demonstrations. SPADE shows where the missing supervision moves—and which controls must govern the result. 08-21 distillation How to Full-Train a Distilled Reasoning Model Without Making It Worse A verified teacher answer is an endpoint, not a training plan. The practical unit of reasoning transfer is a full run whose data, objective, student-generated states, and retention gates are all matched to the student. 08-20 fine-tuning LoRA vs. Full Fine-Tuning in 2026: What Survived the Re-Measurement Two 2024 results anchored how practitioners chose between LoRA and full fine-tuning: "LoRA learns less and forgets less," and the warning that even matched benchmark scores hide structurally different solutions — an "illusion of equivalence." Between 2025 and 2026 both were re-measured, and the answer split by training regime. In supervised fine-tuning the canon survives, but its conditions have been rewritten in terms of adapter capacity, adapter placement, and learning rate. In reinforcement-learning post-training, LoRA now matches full fine-tuning at ranks as low as one — a result the canon never anticipated, resting so far on a lab blog and its reproductions rather than peer review. And three parts of the 2024 answer were never re-tested at all: the effective-rank mechanism offered to explain the gap, the canon's continued-pretraining protocol on current models, and any parity comparison on mixture-of-experts architectures.