# Where an LLM System’s Knowledge Lives—and How to Change It Safely

> An LLM does not have a single knowledge store. A safe change starts with a precise contract, puts mutable information in the least-entangled suitable location, and verifies both the intended effect and everything that must remain unchanged.

Published: 2026-09-02 · Lumisonde (https://lumisonde.com/reports/llm-knowledge-location-and-change/)
Citations JSON (per-claim receipts): https://lumisonde.com/reports/llm-knowledge-location-and-change/citations.json

## The brief

**Question:** How does an LLM system know something, and when knowledge must be added, updated, corrected, or withdrawn, how should we decide where it lives and verify the change?

**Stake:** Teams need to change live policies without leaking them across tenants, erasing valid history, or leaving the assistant to guess when its authority fails.

**Answer:** Treat system knowledge as scoped behavior: use the right meaning for the right audience and time, abstain when authority is missing, and preserve declared invariants. For the tenant policy, select identity and effective time, retrieve the versioned record, consult live account state when needed, place the result in transient context, and override stale context or weight recall. A versioned store, control rules, and sometimes an account service are therefore the least-entangled home for this mutable rule. Accept the change only when matched before-and-after tests cover current and historical answers, abstention, tenant isolation, dependent decisions, and unrelated policies; the evidence establishes no universal architecture or transferable thresholds.

## Introduction

Imagine a fictional company using one LLM assistant for several business tenants. One tenant’s policy says who must approve a travel request. The policy has been corrected with a declared effective time: after that point, a team lead is the approver; before it, the department head was. The assistant must give the new answer for current requests, preserve the old answer for historical questions, keep the rule invisible to other tenants, and stop guessing if the authoritative policy is unavailable.

That apparently small correction is a complete knowledge-management problem. The old rule may be echoed in model weights, carried in a long conversation, stored in an indexed policy document, returned by a policy service, or favored by an instruction that settles conflicts. Replacing one sentence in one place does not establish which version will govern the next answer. Nor does one correct response prove that the change works across paraphrases, time ranges, user roles, or failures.

For a deployed system, “knowing” is therefore a scoped claim about behavior, not a claim that one fact occupies one shelf. Mutable knowledge should live in the least-entangled location that can satisfy its authority, timing, access, rollback, and serving requirements. The change is complete only when the target behavior moves, legitimate consequences move with it, and declared invariants remain within tolerance.

The first step is to state what the assistant must do when a tenant-policy request arrives.

## What would it mean for this system to know the policy?

Before the correction, the team needs a claim it can test. A useful form is:

> This deployed system knows **X**, for purpose **Y**, for audience **Z**, as of time **T**, under conditions **C**, within error tolerance **E**.

For the hypothetical tenant, **X** is not merely “team leads approve travel.” It includes the validity interval: team leads govern current requests after the declared effective time, while department heads remain the right answer for earlier requests. **Y** includes answering questions and routing approvals. **Z** is the authorized tenant audience. **C** includes a functioning policy source and identity checks. **E** must cover wrong answers, unjustified certainty, leakage across tenants, and collateral changes to other policies.

This is intentionally stricter than saying that information is encoded somewhere. Research on LLM memory defines memory broadly as persistent state written during pretraining, fine-tuning, or inference that can later be addressed and stably influence outputs.[1] Yet available information need not govern a response. In one study of nine models answering open-ended questions under knowledge-consistent conditions, produced content reflected both contextual and parametric knowledge—roughly 70% and 30%, respectively.[2] One source can be present while another wins.

As a **proposed engineering test**, treat the scoped knowledge claim as requiring adequacy, elicitation stability, appropriate uncertainty behavior, and preservation outside the target. The assistant should apply the correct policy, survive relevant paraphrases and multi-turn histories, abstain or escalate when authority is missing or conflicting, and avoid degrading unrelated facts, safety, privacy, or service quality. These conditions are a conservative design synthesis, not a universal definition of knowledge. Meaning-preserving prompt changes can alter correctness, while a successful edit can still fail on paraphrases or damage retained knowledge. If the team has measured only one answer, it should report exactly that: the system answered correctly under the tested elicitation.

That scoped test immediately exposes the next problem. To know why the correction might fail, the team must follow every route by which the old or new rule could reach the answer.

## Where can an answer draw its knowledge from?

Follow one request. Tenant identity and request time arrive first; the serving path uses them to select a versioned policy record, consults live account state when the rule requires it, serializes the authorized result into transient context, and applies precedence before the assistant answers or abstains. Model weights, transient context, persistent stores, live tools, and the control layer are the places this path can draw from or conflict with. They are a functional lens, not a universal physical taxonomy: retrieved text becomes transient context before generation, a persistent user memory may live in a database, and one substrate can serve several roles.

### Model weights

Pretraining compresses linguistic competence, task patterns, and factual associations into parameters. This **parametric knowledge** is locally available at inference, adds no retrieval round trip, and can generalize beyond exact stored sentences. Its disadvantages become acute when facts change: it is hard to inspect which training examples support an answer, selectively replace one association, scope it per tenant, or roll it back without affecting nearby behavior.

Fine-tuning and direct model editing are genuine ways to change this layer. Controlled studies have found retrieval-augmented generation more effective than unsupervised fine-tuning for injecting both previously encountered and entirely new facts,[3] with an especially large retrieval advantage on less-popular entities across the models tested.[4] Other work found that fine-tuning examples containing genuinely new facts were learned more slowly and, once learned, increased hallucination against pre-existing knowledge.[5] These findings do not make weight changes useless. They suggest that weights are usually better treated as a home for stable capabilities and reusable behavior than as the first destination for volatile records.

In the policy case, the old approver may still be recalled from weights after the authoritative document changes. That is a conflict to control and test, not a reason to make the weights the policy database. A detachable adapter can isolate domain behavior or a knowledge change from the backbone and simplify enablement or rollback, but it remains a parametric option and still requires tests for leakage, interference, ordering, and composition.

### Transient context

The tenant identity, request time, current conversation, retrieved passages, and other request-scoped inputs meet in **transient context**. It is the fastest place to introduce short-lived facts or instructions: no training is required, and scope can end with the request or session. It is also capacity-limited and sensitive to formatting, position, distraction, and conflicts. A fact in the prompt may be ignored; an untrusted passage may contain instructions; and an open support conversation can preserve the superseded approver after the store has been corrected.

Transient context is therefore the delivery channel for the selected policy, not its durable source of truth. The serving path needs a context refresh or explicit version check before an old turn can govern a new answer. Any fact that must survive beyond the session belongs elsewhere, even though it will eventually be serialized into context for the model to use.

### Persistent stores and memory

With tenant identity and effective time in hand, the serving path can retrieve the matching versioned policy record. Document repositories, vector indexes, structured databases, and explicit cross-session records hold this kind of **persistent non-parametric knowledge**. They are attractive when content must be inspected, versioned, cited, updated independently of model deployment, scoped to users, or rolled back. They can also preserve historical versions, which matters when “Who holds this office?” and “Who held it last year?” must yield different answers.

These benefits are not free. A store needs an ingestion path, schema or chunking policy, index maintenance, access checks, freshness rules, conflict resolution, and observability. Retrieval can return nothing, the wrong version, or a poisoned document. Old and new memories may compete. Deleting a record may require cascading deletion through source data, indexes, caches, replicas, and logs.[1] External storage improves control only to the degree that the surrounding system implements and verifies that control.

For the policy, the store holds both versions with their effective intervals and tenant scope. That makes the intended rule inspectable, but it does not prove that the serving path retrieved the right version or enforced the scope.

### Live tools

If the policy depends on a live account attribute, the serving path next calls an authorized account service. APIs, databases, calculators, search services, and environment calls expose **live or authoritative state** at query time. Inventory, current prices, permissions, account balances, and the status of a transaction usually should not be memorized by the model. The tool should remain authoritative, and the model should transform its result into an answer or action.

Tools shift rather than eliminate the reliability problem. The control layer must authenticate calls, validate arguments and outputs, define timeouts and fallbacks, distinguish observations from instructions, and record which result governed the response. A tool result can be stale, malformed, unauthorized, or simply wrong.

The account lookup is optional, but its failure behavior must be explicit when it is required: the assistant abstains or escalates instead of silently falling back to a plausible rule from weights.

### The control layer

The **control layer** joins those steps. Its instructions, routers, access checks, source-priority rules, and failure handling authenticate the tenant, select the policy by effective time, admit any authorized account result, serialize the selected evidence into context, and decide whether the assistant may answer. Instruction-hierarchy research demonstrates that models can be trained to prioritize higher-privileged instructions over lower-privileged or third-party content,[6] but the hierarchy is a behavior to verify, not a magical barrier.

Content availability answers “Where could the information have come from?” Control answers “Which version was permitted to govern this output?” The corrected store record and authorized tool result must override stale conversational text and weight recall; if the required authority is missing or conflicting, the assistant abstains or escalates. Provenance logs help reconstruct that decision, but a citation or trace alone does not prove causation.

## Why are add, update, correct, and withdraw different contracts?

The policy request combines two changes: it corrects a known wrong answer for current requests and updates a time-varying rule while preserving valid history. Knowledge-editing work formally distinguishes insertion, modification, and erasure,[7] while a deployed system also needs to separate an ordinary update from a correction. For engineering purposes, the distinction can be stated as four operational contracts:

- **Add means presence.** The target was absent or unavailable; after the change, the intended audience should be able to obtain and use it. The main risks are non-delivery, duplication, and conflict with a nearby fact.

- **Update means temporally consistent replacement.** A current value changes, but historical queries may need the old value. The contract therefore needs an effective time, version selection, dependent facts, cache invalidation, and rollback. A new CEO, policy, or product limit is not simply a string substitution.

- **Correct means victory over a known wrong answer.** The previous claim was false. It is not enough for the correction to appear sometimes; the old answer must stop winning across relevant formulations and source conflicts. Edited models have been observed reverting toward pre-edit answers, particularly for popular inherited knowledge.[8]

- **Withdraw means a specified form of absence.** This could mean deleting a record, revoking serving-time access, preventing disclosure, suppressing an output, or making a trained model behave like one that never learned selected data. Those are radically different guarantees.

The policy correction can also change dependent decisions. More generally, changing a person’s office may alter answers about their employer, successor, responsibilities, or the validity interval of related events. RippleEdits showed that editing methods could succeed on the target fact while failing many logically entailed consequences.[9] Testing the target alone therefore leaves the update incomplete.

Withdrawal sits at the boundary of this correction case because its guarantees are harder to establish once the target has entered weights. In the settings studied, approximate unlearning can make a familiar answer harder to elicit without proving that the underlying influence is gone. White-box probes and paraphrase attacks have recovered supposedly deleted facts,[10] while relearning attacks have recovered much of the pre-unlearning performance on related withheld facts.[11] Exact retraining without the data is generally impractical for modern models, and one retrained model’s behavior still cannot certify universal absence across every possible prompt. Withdrawing an external record is more direct and auditable only after every serving copy and derivative is addressed; it cannot remove information already absorbed into weights.

Before implementation, write a **change contract** containing:

1. the target proposition, behavior, or record and its authoritative source;
2. the operation—add, update, correct, or withdraw—and the exact success meaning;
3. effective time, history policy, audience, tenant, region, and access scope;
4. expected dependent changes and explicit invariants;
5. conflict precedence, failure behavior, rollback point, and retention policy;
6. acceptance thresholds for efficacy, uncertainty, collateral effects, latency, and cost.

Without this record, a team cannot distinguish a failed implementation from an underspecified request.

For the running case, the record says this is a correction with a temporal update: the new approver must defeat the known wrong answer for current tenant-scoped requests, the old version must remain available for legitimate historical questions, an unavailable authority must trigger abstention, and unrelated policies must remain invariant. That artifact now determines the placement decision.

## How should you choose where knowledge lives?

The change contract narrows the choice. Tenant scope, version history, a declared effective time, source authority, and rollback point first to a versioned persistent store. The control layer enforces identity and precedence, while a live tool supplies any account state needed to select the rule. The model can still be trained to follow the policy format or call the right tool; the mutable policy text itself does not need to be entangled with that behavior.

There is no universal ranking of locations. The remaining factors—mutability, freshness, authority, reversibility, access, latency, provenance, cost, and behavioral generalization—check whether that case decision transfers to another change. They support the placement decision; they are engineering tools synthesized from bounded studies, not scientific laws.

Start with the least-entangled location that can satisfy every hard requirement:

| Requirement | Default placement | Main verification burden |
|---|---|---|
| Request-specific, short-lived information | Transient context | scope, prompt robustness, expiry |
| Inspectable, versioned, reversible, or tenant-specific content | Persistent store or explicit memory | ingestion, retrieval, access, deletion cascade |
| Live authoritative state | Tool or API | authorization, freshness, schema, fallback |
| Repeatable domain behavior with isolation | Detachable adapter or parametric module | composition, leakage, rollback, regression |
| Broad reusable behavior that must work without retrieval | Backbone weights | generalization, locality, safety, difficult rollback |

Keep the rest as a compact decision record that supports the policy path:

1. **Separate facts from behavior.** Consult the policy clause, private record, price, or current officeholder as a fact. Output style, task procedure, and robust domain behavior may need to be internalized; some apparent architecture problems instead come from prompts, data quality, or workflow.

2. **Mark non-negotiable guarantees.** Query-time authority requires the authoritative tool. Selective revocation or tenant scope keeps durable content outside shared weights and enforces scope through identity, policy, storage, and serving controls; retrieval is useful but is not the only enforcement layer.

3. **Estimate lifecycle and serving costs.** Record change rate, time to effect, query volume, acceptable latency, context cost, training cost, and operational complexity. A universal update-frequency threshold cannot replace this calculation.

4. **Set source precedence before combining layers.** A successful authorized account API overrides model recall; the versioned policy store overrides stale conversational context; and an unavailable authority causes abstention rather than fallback to plausible weights.

5. **Add a hybrid layer only for a named unmet guarantee.** Training stable behavior, retrieving mutable facts, and calling tools for live state is justified only when each source solves a stated requirement. Every source adds conflict and observability surfaces.

6. **Record rejected alternatives.** Explain why context expires too soon, a store adds unacceptable latency, a tool is unavailable offline, or a weight change’s rollback burden is acceptable. This record makes the decision revisable.

Evidence supports the least-entangled default without proving it universally optimal. Retrieval beat unsupervised fine-tuning in several factual-injection settings,[3] especially on long-tail knowledge,[4] yet retrieval can hurt when it introduces unnecessary or poor evidence for facts the model already handles. A reported post-cutoff multi-hop study also found supervised fine-tuning on labeled question-answer pairs outperforming RAG, though its authors noted that task-pattern learning may be mixed with knowledge acquisition.[12] The correct conclusion is conditional: match the architecture to the required guarantee and validate it in the actual regime.

## How can we verify that the change worked?

Replay the same tenant requests before and after the change. The central path is connected: confirm the versioned record and index changed; ask current, historical, paraphrased, and multi-turn questions; present stale conversational text against the new authorized source; remove the authority and require abstention; test another tenant and unrelated policies; then replace or remove the claimed governing source in a sampled audit. The result must show that the intended version wins under the declared conditions and loses where it should not apply.

The supporting checks follow that path rather than creating a separate scorecard:

1. **Contract and baseline.** Before changing anything, freeze representative target cases, held-out paraphrases, dependent cases, conflict cases, and invariants. Record model, prompt, store, index, policy, and tool versions so the after-state has a matched control.

2. **Layer integrity.** Confirm that the correct document version is active, the index contains it, caches expired, any required tool returns the new value, and the policy rule matches the intended audience. For an adapter or weight change, confirm the expected version. This establishes that the write landed.

3. **Direct behavioral efficacy.** Run the deployed generation path. Parametric edits can pass a teacher-forced or internal evaluation yet fail to elicit the new knowledge during autoregressive generation.[13] For this correction, the new current answer must win and the obsolete answer must lose; a withdrawal test would similarly require the forbidden answer to stop winning.

4. **Elicitation robustness.** Ask the current and historical questions through held-out paraphrases, formats, languages, sampling settings, multi-turn histories, indirect questions, and relevant user roles. Measure whether the meaning is right rather than whether one string matches.

5. **Conflict and authority.** Put the old and new versions in different channels, mix trusted and untrusted sources, and make the authority unavailable or malformed. The assistant should follow precedence, abstain, or use the declared fallback. Reverse-edit experiments show that individually successful edits can still leave the pre-edit fact governing answers under conflict.[14]

6. **Ripple and temporal consistency.** Check decisions that should change with the approver rule and historical questions that should retain the earlier answer. Those matched cases show whether the update is coherent beyond its target.

7. **Locality and regression.** Test another tenant, neighboring facts, and broad suites for reasoning, safety, privacy, calibration, latency, and cost. Sequential editing has produced deterioration on general benchmarks and weaker safety even when per-edit metrics looked good.[15] Acceptance must constrain both target success and retained behavior.

Two boundary checks remain brief because the ordinary policy path does not encounter them. 8. **Withdrawal resistance:** match alternate wording and indirect extraction to disclosure suppression, permission-bound tests to access revocation, and privacy attacks, relearning, and white-box probes to any stronger forgetting claim; surface refusal does not certify erasure. 9. **Causal governance:** for high-risk or sampled cases, remove or replace the claimed retrieved passage, tool result, adapter, or edited component and compare the output with a matched control. A citation can show that a source supports a sentence without showing that the source caused it.[16] Human audits of deployed generative search systems have also found substantial gaps in citation support.[17] Because ablation-style attribution is expensive, reserve it for targeted audits.

Finally, roll out progressively. Monitor target failures, old-version leakage, source conflicts, abstention rates, regressions, and service metrics. Keep a reversible deployment boundary and a change record linking the contract, artifact versions, tests, approvals, and rollback. A test suite establishes behavior in sampled conditions; monitoring checks whether the same assumptions survive real traffic.

## Conclusion

An LLM system does not know through one bookshelf in its weights. It knows, operationally, when the deployed combination of weights, transient context, persistent stores, live tools, and control rules can produce and use the intended meaning reliably under declared conditions.

That view changes knowledge management from “inject a fact” into a disciplined sequence. The tenant-policy case began as one wrong answer, but solving it required a scoped knowledge claim, a change contract, a placement decision, a precedence rule, and a system-level comparison of before and after. The same sequence applies to other consequential changes: choose the least-entangled location that meets the contract, define which source wins and how failure is handled, and accept the change only when the target and its legitimate ripples move while declared invariants remain within tolerance.

## Limitations

The source map, four-operation contract, guarantee matrix, least-entangled default, and verification ladder are author-defined frameworks supported by studies in particular retrieval, editing, memory, tool-use, and open-model settings. They are not universal physical taxonomies or proven optimal algorithms. Results from mostly open and mid-sized models, bounded benchmarks, selected attacks, or specific RAG systems do not transfer quantitatively to every frontier or proprietary deployment.

External stores and tools improve inspectability and change control only when access, versioning, provenance, deletion, and observability are correctly implemented. Weight editing and unlearning results show important failure modes, but retraining or a passed attack suite cannot prove universal absence of knowledge. Finally, causal interventions, broad regressions, and adversarial withdrawal tests are costly and design-sensitive; each deployment must set its own risk-based coverage and acceptance thresholds. The open engineering question is how to obtain enough causal assurance for high-impact answers without turning every response into an intervention experiment.

## References

[1] *LLM memory mechanisms survey*, arXiv:2509.18868v1 — https://arxiv.org/html/2509.18868v1
[2] Context-versus-parametric knowledge allocation study, EMNLP 2024 — https://aclanthology.org/2024.emnlp-main.234
[3] Ovadia et al., fine-tuning-versus-retrieval knowledge-injection study, arXiv:2312.05934v3 — http://arxiv.org/abs/2312.05934v3
[4] Soudani et al., study of fine-tuning versus retrieval for less-popular knowledge, arXiv:2403.01432v5 — http://arxiv.org/abs/2403.01432v5
[5] Gekhman et al., study of new knowledge and hallucination under fine-tuning, arXiv:2405.05904v3 — http://arxiv.org/abs/2405.05904v3
[6] Wallace et al., instruction-hierarchy study, arXiv:2404.13208v1 — http://arxiv.org/abs/2404.13208v1
[7] KnowEdit comprehensive knowledge-editing study, arXiv:2401.01286v5 — http://arxiv.org/abs/2401.01286v5
[8] Knowledge-editing pitfalls survey, arXiv:2406.01436v2 — http://arxiv.org/abs/2406.01436v2
[9] RippleEdits evaluation of logically related facts after editing, arXiv:2307.12976v2 — http://arxiv.org/abs/2307.12976v2
[10] Patil et al., study of extraction attacks after deleting sensitive information, arXiv:2309.17410v1 — http://arxiv.org/abs/2309.17410v1
[11] Unlearning removal-versus-hiding study, arXiv:2410.08827v3 — http://arxiv.org/abs/2410.08827v3
[12] Yang et al., post-cutoff multi-hop comparison of fine-tuning and RAG, arXiv:2601.07054v1 — http://arxiv.org/abs/2601.07054v1
[13] EtCon consolidation study of edited knowledge in generation, arXiv:2512.04753v2 — http://arxiv.org/abs/2512.04753v2
[14] Study of conflict and reverse-edit pitfalls in knowledge editing, arXiv:2310.02129v5 — http://arxiv.org/abs/2310.02129v5
[15] Study of general-capability and safety effects after model editing, arXiv:2410.18785v1 — http://arxiv.org/abs/2410.18785v1
[16] ContextCite context-attribution study, arXiv:2409.00729v2 — http://arxiv.org/abs/2409.00729v2
[17] Generative-search verifiability study, arXiv:2304.09848v2 — http://arxiv.org/abs/2304.09848v2

---
Every [N] marker above is verifiable: fetch the citations JSON, match the marker id, and check the extraction (summary and key facts) plus placements against the source URL. Verification guide: https://lumisonde.com/for-agents/