LoRA vs. Full Fine-Tuning in 2026: What Survived the Re-Measurement

What the 2024 canon actually claimed

The reference point for the whole debate is a 2024 comparison of LoRA and full fine-tuning on programming and mathematics, run in two data regimes: instruction fine-tuning on roughly 100K prompt–response pairs, and continued pretraining on roughly 20B unstructured tokens [1]. Its findings were conditional, not absolute. In standard low-rank settings LoRA substantially underperformed full fine-tuning, and the gap was widest on the hardest domain at the largest data scale: in continued pretraining LoRA lost across every configuration tested. In instruction fine-tuning the gap closed with rank — on HumanEval, rank-256 LoRA reached 0.498 against full fine-tuning's 0.497, though LoRA needed four epochs where full fine-tuning peaked at eight [1]. The compensating result was forgetting: LoRA preserved out-of-domain performance better than full fine-tuning, and better than weight decay or dropout could explain [1]. The proposed mechanism was structural: full fine-tuning's weight updates have an effective rank 10–100 times larger than typical LoRA configurations [1].

The second canon result attacked the notion of equivalence directly. Spectral analysis of the updated weight matrices showed that LoRA introduces new high-ranking singular vectors nearly orthogonal to the pretrained spectrum — "intruder dimensions" — which full fine-tuning does not produce. Causally intervening on those dimensions showed they carry LoRA's forgetting: scaling them down largely restores modeling of the pretraining distribution at minimal cost to the downstream task, and intruders accumulate harmfully under sequential fine-tuning [2]. The message, from October 2024, was that behavioral parity is not solution equivalence.

Out of these two results came the decision rules of the era: when the task is hard or the data large, prefer full fine-tuning; when LoRA falls short, raise the rank. Both rules have since been re-measured, and neither survived intact.

The parity conditions were re-written: capacity, placement, learning rate

The most consequential 2025–26 development is not a refutation of the canon but a re-parameterization of it. A September 2025 study from Thinking Machines Lab replaced the canon's task-difficulty-and-rank framing with three conditions under which LoRA matches full fine-tuning in supervised training: the dataset must fit within the adapter's information capacity; adapters must be placed on all weight matrices, especially the MLP and MoE layers that hold most of the parameters; and LoRA's learning rate must be set roughly 10× the full-fine-tuning optimum [3]. Measured this way — with log-loss scaling curves on instruction and reasoning datasets rather than sampling-based benchmarks — LoRA trains with the same sample efficiency and reaches the same final performance as full fine-tuning across most post-training-scale workloads [3].

Each 2024 condition maps onto this new frame with a revision:

• The data-scale condition is confirmed, with the mechanism changed. LoRA still underperforms when the dataset exceeds adapter capacity, but the failure is a progressive loss of training efficiency, not a hard floor the loss cannot pass [3]. The capacity itself became measurable in July 2026: compression-based analysis finds adapters store roughly a couple of bits per trainable parameter — well short of a full model's budget — so the parity boundary can be estimated from a dataset's information content in advance rather than discovered by failure [4].
• The rank prescription is demoted; placement dominates. Attention-only LoRA underperforms even when its parameter count is matched by raising rank, while all-matrix placement reaches parity at moderate rank — an effect that holds for both a dense model and a sparse mixture-of-experts base [3]. The capacity measurements explain why: the same parameter budget placed in MLP layers holds nearly twice the bits it holds in attention [4]. "Raise the rank" was treating a placement problem as a capacity problem.
• The comparison itself was confounded by learning rate. LoRA's optimal learning rate is consistently about 10× full fine-tuning's, in both supervised and reinforcement learning; comparing at the full-fine-tuning rate manufactures a capability gap [3]. A February 2026 audit of 64 prior LoRA studies found that only one tuned learning rate, batch size, and rank simultaneously, and fewer than 30% tuned the learning rate at all; once learning rates are properly tuned, vanilla LoRA and nine of its variants land within 1–2% of one another [5]. The fair-comparison critique that once protected full fine-tuning from under-tuned LoRA baselines now cuts against the LoRA-variant literature itself, and principled learning-rate scaling rules across ranks — including transfer of tuned rates between LoRA and full fine-tuning — remove the main excuse going forward [6].

One new condition arrived with no 2024 counterpart: in some scenarios, LoRA tolerates large batch sizes worse than full fine-tuning, paying a loss penalty that grows as batch size increases beyond some point and is not mitigated by rank, attributed to the product-of-matrices parametrization — though both methods reach their best loss at smaller batch sizes, so the gap may not matter much in practice [3]. This is a single-team measurement with no independent re-test in the cited literature, and should be treated as provisional.

A register caveat rides this entire section. The backbone of the re-parameterization is an industrial lab's blog post — systematic and reproducible in design, but not peer-reviewed — corroborated by an independent third-party reproduction of the low-data parity result and the ~10× learning-rate ratio [7], and operationalized in Hugging Face's TRL documentation [8]. The canon it revises is peer-cited work: the intruder-dimensions study appears in the NeurIPS 2025 proceedings, while the Biderman study's journal acceptance is author-reported rather than independently confirmed. The revision is the best-supported current account, but it has not passed the same gate as what it replaces.

The regime shift: reinforcement learning changes the answer

The strongest 2025–26 claim is one the 2024 canon never tested, because the RL post-training regime barely existed as a customization path then. Under policy-gradient reinforcement learning, LoRA fully matches full fine-tuning's learning performance even at rank 1 — measured on math-reasoning RL with Llama-3.1-8B on MATH and GSM8K and, at larger scale, with the dense Qwen3-8B-Base on DeepMath, within a study design that varied rank across three orders of magnitude, from 1 to 512 [3].

The proposed mechanism is information-theoretic and gives the result its plausibility beyond the specific experiments. Supervised learning delivers on the order of the number of tokens in bits per episode; policy-gradient RL delivers roughly one bit per episode, through the scalar advantage. In the reported MATH training run — about 10,000 problems at 32 samples each — the entire process needs to absorb around 320,000 bits, while a rank-1 adapter on Llama-3.1-8B already holds about 3 million parameters [3]. RL simply does not write enough information into the weights to need full fine-tuning's capacity. The capacity-measurement work corroborates the asymmetry from an independent angle: secrets that supervised fine-tuning copies into an adapter verbatim, an adapter trained on verifiable rewards never records — RLVR adapters write close to zero distinguishable bits [4]. The mechanism's sharpest prediction has, however, failed its first direct test: a reproduction that swept adapter ranks across reward signals of increasing information density — outcome rewards, process rewards, and dense distillation — found rank-1 LoRA matching full fine-tuning in every regime, with the rank needed to match insensitive to reward density and training stability governed by the reward's structure instead [27]. The parity result replicates; the information-theoretic account of why remains contested.

The result has replications, but all inside one methodological family. An independent GitHub reproduction matched full fine-tuning with rank-1 LoRA on GRPO math training with Qwen3-1.7B [7]. The TRL documentation reproduced the matched reward curve with rank-1 on SmolLM3-3B and codified the recipe: rank 1–32 for RL, rank 256 for post-training-scale SFT, learning rate 10× full fine-tuning, effective batch below 32, at a reported ~67% of full-fine-tuning compute — a figure so far reported only by this cluster [8].

The floor of the parity claim is genuinely contested within the 2025–26 stratum itself. The VERL framework's official documentation — equally a 2026 production RL stack — recommends rank at least 32, citing community-contributed test results in which convergence speed and final performance were almost identical to non-LoRA training at rank 32 for a 0.5B model and rank 128 for a 32B model, with fuller reference results still pending [9]. A 2026 study measures a persisting LoRA-vs-full-fine-tuning gap in RLVR on Qwen-2.5 reasoning benchmarks and spends a partial full-fine-tuning budget to close it [10]. And the SFT-era variant leaderboard does not transfer: PiSSA and MiLoRA, which beat standard LoRA under supervised fine-tuning, can underperform it and destabilize training under RLVR, with orthonormal initialization provably minimizing the LoRA-to-full-fine-tuning gap in that regime [11]. RL-regime parity is real and multiply reproduced; parity at rank 1 specifically is measurement-backed but not uncontested, and no member of the rank-1 cluster is peer-review-attested in the cited literature.

What happened to "forgets less" and the "illusion of equivalence"

The canon's "forgets less" clause had the most interesting fate: its explanation was replaced while the claim itself was never re-measured under controlled conditions.

Within supervised fine-tuning, the finding stands as measured — in 2024, on Llama-2-7B — and no controlled comparison has re-confirmed it since [1]. But two independent 2025 results relocate the cause from the parameterization to the update's statistics. At matched new-task performance, on-policy RL forgets less than SFT, because policy-gradient training is implicitly biased toward KL-minimal solutions — ones that stay close to the base policy — while SFT can converge to distributions arbitrarily far away [12]. A companion study reaches the same place from the data side: RL forgets less than SFT across Llama and Qwen families, and it is the use of on-policy data — not KL regularization or advantage estimation — that underlies the robustness [13]. Read through this lens, LoRA's forgetting advantage looks like one instance of a broader law: small-KL updates forget less, and both bounded-capacity adapters and on-policy RL happen to produce small-KL updates. "Forgets less" changes from a property of LoRA to a property of the update that LoRA tends to induce.

A May 2026 result pushes the re-attribution further and directly against the canon's letter: full fine-tuning run with the same optimizer used in pretraining is reported to forget less than LoRA at equal or better new-task performance in SFT — making optimizer choice a hidden condition of every prior comparison, including the canon's [14]. This is a single unreplicated paper and should not yet carry a decision by itself, but it means even the canon's most durable clause is now conditional on an experimental variable nobody controlled in 2024.

The geometry story, by contrast, was confirmed and strengthened. The intruder-dimension account graduated from observation to predictive law in July 2026: a per-layer critical update strength, computed from the pretrained weight spectrum alone with no fitted parameters, localizes the empirical threshold at which intruders form within a factor of two on 82% of layers (98% with edge evaluations) across dense Transformer, mixture-of-experts, state-space, and encoder-decoder families [15]. Full fine-tuning disperses its update far below every layer's threshold, which resolves the 2024 asymmetry rather than merely restating it; norm-matched interventions show that threshold-crossing layers, not update magnitude, carry the forgetting; and a budget rule derived from the thresholds reduces forgetting by 62% on the most fragile model at no task cost [15]. The "illusion of equivalence" is therefore not just intact — it is now quantitative. Its scope, however, remains supervised: the cited literature contains no measurement of intruder dimensions or solution geometry under RL post-training, so whether rank-1 RL parity also means solution-level equivalence is an open question, not a settled one.

What production practice reveals

Deployed systems are a second kind of evidence: they show where operators with money at stake believe parity holds. The 2026 landscape is not "LoRA versus full fine-tuning" so much as a bifurcation by who owns the model.

Where the customer adapts someone else's model, LoRA has become the product. Open-weight fine-tuning services sell LoRA as the headline offering with full fine-tuning as a priced option beside it [16]; one major provider's tuning service produces a LoRA add-on whose rank defaults to 8 (a power of 2, up to 32) and accepts LoRA adapters tuned on a separate platform and uploaded for inference [17]. Google's managed Gemini tuning on Vertex AI is adapter-based — the tuning API's adapter_size parameter is a LoRA rank knob [18]. The serving side assumes adapter fleets as a given, with multi-tenant LoRA serving now a cloud product and 2026 systems research disaggregating adapter execution from base-model inference — with the caveat that on MoE bases adapter memory scales with expert count, so adapters quietly stop being negligible there [19]. The pattern has reached training itself: multi-tenant RL fine-tuning services train per-tenant LoRA adapters against a shared base model — one production platform runs GRPO with rank-32 per-user adapters swapped between gradient steps [20], and RLVR-as-a-service systems scale across many concurrent LoRA-adapted tasks [21].

Google and OpenAI, meanwhile, are exiting self-serve fine-tuning — one exit complete, the other staged and still under way. Google's own documentation states that since the May 2025 deprecation of Gemini 1.5 Flash-001, no Gemini API or AI Studio model supports fine-tuning, directing users to the enterprise platform [22]. OpenAI's deprecations page records a staged wind-down: from May 7, 2026, organizations that never ran fine-tuning cannot start; from July 2, 2026, organizations without recent fine-tuned-model inference cannot start; and on January 6, 2027, active customers lose the ability to create new fine-tuning jobs, with inference on existing tuned models continuing until each base model is deprecated [23]. The stated rationale — that newer base models follow instructions well enough that fewer use cases require fine-tuning — appears in third-party coverage quoting an OpenAI developer email, not on the vendor's own page [24].

At the frontier, where the trainer owns the model and chases capability, full-parameter training persists — as an engineering feat, not a drifted-into default. A July 2026 systems report details full-parameter post-training of a trillion-parameter-class MoE family, noting that full-parameter training at that scale remains less explored [25]; a frontier open-weight model report from the same month describes RL infrastructure at 2.8T-parameter scale [26].

Read as revealed preference: operators treat LoRA parity as settled for tenant-scale adaptation — including RL-regime services — and treat full fine-tuning as non-negotiable for frontier capability. What no available source measures is the cost of choosing wrong: there is no production A/B or migration postmortem quantifying capability lost to an unnecessary adapter or compute wasted on unnecessary full fine-tuning. The practice evidence shows where the industry has placed its bet, not what the bet costs when it misses.

The dated ledger

Every verdict above rests on one of two evidence strata: the 2023–24 canon (Stratum A) or the 2025–26 re-measurement (Stratum B). The table gathers them, dated, so each claim carries the vintage of the evidence behind it.

What this means in practice

The decision rules that follow are conditional, because the evidence is.

For supervised fine-tuning at tenant scale — instruction tuning, domain adaptation, datasets up to the low hundreds of thousands of examples — use LoRA configured to the current recipe: adapters on all weight matrices including MLP/MoE, learning rate tuned about 10× above the full-fine-tuning optimum, moderate effective batch, and rank chosen against dataset information content rather than task difficulty (a couple of bits per trainable parameter is the working capacity estimate) [3, 4, 8]. Under these conditions the 2024 capability gap does not reproduce.

For very large corpora — continued pretraining, tens of billions of tokens — the canon's condition stands: measured in 2024 on Llama-2-7B, where LoRA underperformed across every configuration tested [1], and consistent with the 2025 capacity account, under which a corpus exceeding the adapter's information capacity is exactly where LoRA should fall behind [3], though never re-run at that scale on current models. Expect LoRA to lose training efficiency past adapter capacity, and budget for full fine-tuning.

For RL post-training, LoRA is a defensible default and the compute implications are real: if parity holds at low rank, seriously customizing a large model through RL costs adapter-training resources, not full-parameter resources, and multi-tenant RL services are already built on that premise [20, 21]. But do not encode rank 1 as policy. The parity floor is the live dispute in the 2026 stratum: TRL's documentation says RL tasks typically require lower LoRA capacity, so ranks of 1–32 can be used [8], while VERL advises a rank of at least 32 and reports near-identical convergence at 32 and 128 [9]; at least one controlled RLVR measurement still finds a gap [10]. If the workload is capability-critical, validate the chosen rank against a high-rank or full-fine-tuning run before committing, and prefer orthonormal-style initialization over SFT-era structured initializations, whose advantages invert under RLVR [11].

For sequential or continual fine-tuning, the geometry evidence argues caution with plain LoRA: intruder dimensions accumulate across rounds and degrade retention [2], and the spectral-threshold rule offers a cheap mitigation if adapters must be stacked [15]. A May 2026 preprint titled "full finetuning with the same optimizer as pretraining forgets less" is worth watching, but this report reaches it only as a bibliography entry in another paper and holds no capture of it [14].

What would change these recommendations: a peer-reviewed replication (or refutation) of rank-1 RL parity; a controlled MoE parity study; an independent batch-size sweep at matched tuned learning rates; a replication of the optimizer-consistency result; and any re-run of the 2024 canon's own protocol on current models — none of which the cited literature yet contains.

Conclusion

Under what conditions does LoRA match full fine-tuning in 2026? In supervised training: when the dataset fits within the adapter's measurable information capacity, adapters sit on all weight matrices, and the learning rate is set about an order of magnitude above the full-fine-tuning optimum — conditions quantified in 2025–26 and consistent with, but sharper than, the 2024 canon. In reinforcement-learning post-training: apparently almost always, down to very low rank, because policy-gradient training writes too little information into the weights to need more capacity — the one genuinely new answer of the re-measurement period, multiply reproduced but not yet peer-reviewed, with its exact rank floor still disputed.

Of the 2024 answer, "learns less" survives only as a capacity statement, and "raise the rank" did not survive at all — placement and learning rate were the real variables. "Forgets less" survives inside SFT while its explanation moved up a level: forgetting tracks the KL-magnitude of the update, which both adapters and on-policy RL keep small. The "illusion of equivalence" survives best of all, now as a predictive per-layer law — but only where it was measured, in supervised training. And three things were never re-tested: the 10–100× effective-rank mechanism, the canon's continued-pretraining protocol on current models, and any parity comparison on MoE architectures. The 2024-vintage decision rule "when in doubt, go full FT" is now wrong for most tenant-scale and RL workloads and right at the frontier and at pretraining scale — which is exactly where the people who still pay for full fine-tuning are running it.

Appendix: Limitations

The central asymmetry of this report is one of register: the 2025–26 stratum that revises the canon is anchored by an industrial lab blog, framework documentation, and preprints, while the 2024 canon it revises is published, peer-cited work. Within that stratum, several load-bearing results are single-source — the batch-size sensitivity condition, the optimizer-consistency counterexample to "forgets less," and the intruder-threshold spectral law are each one team's unreplicated measurement — and the ~67% compute figure for LoRA has no measurement independent of the originating cluster.

The course-project replication swept adapter ranks across three reward densities and reported rank-1 parity in every regime [27]; that capture, however, is archived here only as an unparsed PDF, so it is byte-mangled in our archive and cannot be verified against it verbatim — those figures trace to the extraction layer, which is why this report paraphrases rather than quotes it.

The empirical base is also concentrated: most parity measurements use math- and code-reasoning benchmarks on models in the 1B–70B range, so transfer to other domains and to frontier scale is inferred, not measured. No 2025–26 study re-ran the 2024 canon's own protocol on current models, so canon and current results are compared across different instruments rather than replicated head-to-head. Finally, the production evidence establishes revealed preference but not consequence — no available source quantifies what a wrong LoRA-versus-full-fine-tuning choice costs in deployed capability or wasted compute — and the venue status of some cited work (including the canon's own journal acceptance) is reported by authors rather than independently confirmed here.