The Memory Was Right. The Decision Was Wrong.

What changes when an event becomes memory?

The problem becomes concrete in one machine managing a project calendar. On Monday it stores, “Supplier review is Tuesday at 10:00.” On Tuesday morning the user says, “Move the supplier review to Thursday at 14:00.” On Thursday the machine must decide whether to send a reminder, and for which time.

The first sentence remains accurate as a record of what the calendar once said. It is also semantically relevant to the reminder task. Yet using it as the current meeting time would be a state error. The second sentence may itself be unsafe to act on if “Thursday” was interpreted in the wrong time zone, if the speaker lacked authority to change the shared calendar, or if the change applied to a different project. Accuracy and relevance describe text-to-past and text-to-task relationships; they do not settle currentness, applicability, scope, authority, or correct use.

This distinction appears after idealized retrieval, not only when search misses. In StateMem, an oracle-retrieval condition supplied all relevant facts, yet one evaluated model still missed 36 of 306 questions; adjudication attributed 16 of those misses to state drift, where the needed fact was present but the assembled state was stale or incomplete.[1] The result is one controlled run with a fixed setup, but it demonstrates the mechanism cleanly: having the fact and computing the present state are different operations.

A benchmark built from true but conflicting memories exposes a neighboring failure. Agents must resolve changes over time, contextual exceptions, and differences in source authority. Models recognized conflicts more often than they chose a calibrated response such as clarification or verification. Adding same-domain distractors — memories topically related but non-diagnostic for the conflict — also imposed a cumulative cost, with aggregate scores declining monotonically as distractor load rose.[2] In a separate controlled transfer study, detailed trajectories from an easier task hurt a related harder task because the old procedure omitted the added cleaning step; abstract procedural lessons transferred more safely.[3]

The reminder therefore depends on a state transition, not a document lookup:

Tuesday 10:00 (observed Monday) → superseded Tuesday morning → Thursday 14:00 (pending scope and time-zone checks) → calendar write confirmed → reminder sent

The smallest executable policy applies the same gate sequence at each arrow:

Every arrow can collapse what happened, what is believed now, and what action succeeded. The first control is whether the incoming event should persist at all.

Which events deserve a durable record?

When the rescheduling message arrives, the machine should first create a candidate record rather than immediately rewriting durable state. The input is the raw event plus its envelope: actor, channel, event time, ingestion time, tenant or user, task or project, and links to the calendar object and prior record. The candidate should carry:

• a stable record ID and type, the original content, and a compact interpretation;
• observed, valid-from, expiry, and review times, plus subject, tenant, task, tool, and environment scope;
• source identity, authority, lineage, verification status, confidence, and unresolved ambiguities;
• links to records it supports, derives from, contradicts, or supersedes, plus sensitivity and deletion class; and
• later helpful, harmful, and neutral outcome counts.

Those fields separate three admission choices. Write when the event is durable, scoped, sufficiently complete, and likely to change a future decision. Buffer when it could matter but a missing referent, time zone, authority check, or outcome prevents safe consolidation. Ignore when it is generic chatter, a duplicate in the same lineage, outside the agent’s mandate, or cheaper to reacquire than to maintain.

An evaluated memory manager uses the same basic action vocabulary—add, search, buffer, and ignore—and bases the choice on value, ambiguity, and completeness.[4] That supports selective admission; the candidate schema and cost weights remain application controls. A deployable gate can compare expected decision benefit with storage, retrieval, verification, privacy, and misapplication costs, but its weights should be tuned on the application’s own decisions.

For the calendar event, “Move it to Thursday” belongs in the buffer if the project or time zone is unclear. A second message that resolves those fields can promote it. Repeated copies from a chat transcript and a meeting summary do not become two independent facts; they attach as descendants of the same user instruction.

How does a new fact become the current state?

Once the rescheduling instruction is admissible, consolidation should run asynchronously within a bounded, local maintenance scope. It writes a new version, marks the Tuesday record as superseded for current scheduling, preserves both raw records and their provenance for historical questions, and recomputes only the derived state that can affect the next action. The reminder, agenda, and availability check are candidates; an unrelated supplier-risk note is not. A recurring verified action sequence may become a candidate procedure, but it should not be promoted from one apparently successful episode.

One long-running memory system supplies a concrete version of this pattern: events enter a current buffer, consolidation preserves source and time metadata, and revisions can explicitly supersede earlier entries. Its forgetting results were much stronger for single-hop than multi-hop updates, a warning that changing one record does not automatically repair every downstream representation.[5] StateMem adds the opposite warning: indiscriminate dependency propagation can also hurt when an apparent dependency is not valid for the current case.[1]

The safe rule is therefore selective invalidation. Maintain a dependency graph, mark reachable derivatives stale, and recompute them only when the edge type and scope justify it. If the system cannot prove that a summary, cache entry, or reminder was repaired, exclude that derivative from action until it is rebuilt. This full propagation discipline remains an engineering safeguard whose best implementation is unsettled.

Consolidation must also preserve disagreement. If the project manager says Thursday but the shared calendar still says Tuesday, the machine should not average the times or let the most repeated version win. It should retain both claims, record their authority and valid times, seek the authoritative live state, and ask for clarification or defer an irreversible action if the conflict remains.

What should reach the model at decision time?

With current state and unresolved conflicts preserved, on Thursday the machine should not search its entire history for “supplier review.” It should begin with an action specification: send-or-withhold reminder; supplier-review event; current project and tenant; local time and time zone; authorized calendar source; required preconditions; and acceptable uncertainty. Scope and time filters run before semantic ranking.

Retrieval then builds the smallest decision-sufficient bundle. Start with the current event record, the latest authoritative instruction, any unresolved conflict, the reminder policy, and evidence that required preconditions hold. Expand only if a conflict cannot be resolved, a required premise lacks support, or a newly found record changes the state. Stop when every necessary premise has current, scoped support; when another retrieval round adds no useful information; or when the context, latency, or tool-call budget is exhausted. If a necessary premise is still unresolved, the correct stopping action is clarification, verification through the live tool, a reversible trial, or abstention—not another unbounded search.

The bundle must count provenance, not copies. A chat message, its generated summary, and a calendar entry produced from that message may look like three agreeing memories while sharing one origin. A constructed correlation benchmark shows how paraphrases and summaries can create a false majority; its provenance-aware method groups descendants rather than allowing repetition to multiply support.[6] Because the comparison does not isolate provenance handling from every semantic and confidence control, the operational rule should be modest: cap the influence of one lineage, expose dissent, and seek support with a different failure mode.

That last condition applies to validators as well as sources. In a self-improving text-to-SQL agent, self-grades overvalued some incorrect episodes, and re-graders with correlated errors did not reliably repair the ranking. Execution-based or otherwise decorrelated checks supported per-memory demotion and improved end-to-end execution in that setting.[7] For the reminder, a live calendar read supplies a check with a different failure mode from a second model paraphrase.

The context presented to the language model should label each item by role—current state, prior state, instruction, policy, conflict, or outcome—and keep the current answer adjacent to the action specification. Historical text should never masquerade as current state simply because it ranked highly.

How should an action change the memory that caused it?

With the decision bundle assembled, the machine immediately revalidates volatile premises: the event still exists, the time is still Thursday at 14:00 in the intended zone, and the recipient list is current. For an action requiring approval, it shows the operator the influential record IDs, source roles, versions, and unresolved conflict flags; for every action, it logs the same fields. That trace makes a later correction attributable to the memory policy rather than to the model in the abstract.

After the tool call, the machine writes an outcome record from the environment: accepted reminder, rejection, corrected time, tool error, user complaint, or no observable signal. It does not treat its own confidence or fluent explanation as success. Verified helpful reuse increments a record’s utility; a corrected or harmful action demotes or quarantines the responsible record and may create a narrow anti-pattern. No signal leaves utility unchanged rather than inventing a reward.

A frozen-model system demonstrates this verifier-gated loop: successful interactions can enter episodic memory, retrieved procedures receive helpful or harmful credit, failures can become local anti-patterns, and low-reward off-scope interactions can be quarantined.[8] The general deployment rule is broader than that experiment: prefer tool results, user corrections, invariant checks, and delayed environment outcomes whose errors are meaningfully different from the agent’s self-assessment.

Forgetting has two meanings in this loop. Soft forgetting changes selection without destroying the record: supersede it, expire it, lower its utility, archive it, or exclude it from ordinary retrieval while retaining audit and historical access. This is the default for the Tuesday meeting time. Hard deletion removes the source and its recoverable derivatives when consent is withdrawn, retention expires, tenancy ends, or security and legal policy require erasure. It must cover indexes, caches, summaries, backups under the applicable retention policy, and derived records that cannot be retained independently. NIST SP 1800-22B treats selective wiping of organizational data and applications, and protection against organizational-data loss, as BYOD controls.[9]

Soft forgetting and hard deletion place different demands on the store.

Which store fits the loop?

Choose the substrate from the lifecycle’s update semantics. Full history is reasonable for a short, bounded episode that fits comfortably in context. Flat retrieval fits stable, independent facts with simple update semantics. Use a compact state table plus a versioned event log for mutable entities such as meetings, and add a temporal or dependency graph only where traversal earns its maintenance cost. Branch-local episodic and procedural stores fit recurring task families that must not contaminate one another. Probabilistic beliefs are warranted only when source reliability can be calibrated and the action policy knows how to use uncertainty. Keep a short working buffer for unresolved candidates. Many deployments need a small combination of these, with one authoritative current-state path.

Once the store matches the update path, transfer depends on how often circumstances recur and how the memory represents them. A controlled continual-learning study found that raw trajectories could cause negative transfer across related tasks while abstract insights were safer.[3] Another evaluation framework requires held-out, compositional, unrelated, and repeated task streams because a memory method can improve familiar cases while losing to a memory-free agent on transfer.[10] Store concrete episodes when circumstances repeat closely; distill procedures only after verified successes across cases; keep exceptions scoped; and prefer no reusable rule when evidence for transfer is sparse.

One evaluated system supports selective admission.[4] Another supports buffered consolidation and supersession.[5] A third couples bounded retrieval with verifier-gated updates.[8] These controls form the policy that the next experiment must test as a whole.

How do we test whether memory helped?

Those architecture choices matter only if they improve the next calendar decision. Recall is a component diagnostic. The primary unit should be a decision with an observable consequence. For the calendar machine, that means whether the correct reminder was sent or withheld, whether constraints were respected, how much correction or regret followed, whether abstention was calibrated, and what latency, tool calls, and context tokens the decision consumed.

Run a paired experiment on the same decision episodes. Hold the model version, system prompt, tools, data snapshot, context and latency budgets, and judge or environment scorer constant. Compare at least four arms: memory off; full history when it fits; the deployed memory policy; and an oracle bundle containing the necessary current premises. Then ablate the write filter, supersession logic, lineage grouping, retrieval stopping rule, and outcome gate one at a time.

The test set should include ordinary cases and protected slices: stale-but-accurate history, ambiguous scope, correlated copies, conflicting authorities, unrelated transfer, changing preconditions, and a clean condition in which the memory is accurate, current, applicable, scoped, and independently supported. Randomize item order and context position. Use multiple seeds where model stochasticity matters, report paired per-item differences with confidence intervals, and fail the release on material regression in a safety-critical slice even if average success rises.

Log the complete path from event ingestion through candidate choice, consolidation, retrieved IDs, filters, final context, action, and outcome. These stage traces distinguish “the right record was absent” from “the right record was present but used incorrectly.” A recent diagnostic protocol proposes paired feature flags, protected slices, stage traces, paired tests, and bootstrap intervals; it is a useful design template, not a substitute for the application’s own environment.[11]

MemoryArena is the closest released starting point for action-facing evaluation: it couples memory to interdependent multi-session tasks and environment feedback, and provides official code and CC-BY-4.0 data.[12] MemoryAgentBench offers MIT-licensed code and data for incremental memory competencies.[13] LongMemEval-V2 provides Apache-2.0 code, data, and an evaluation runner for trajectory-based memory and downstream answers.[14] MemEval provides an Apache-2.0 comparator that standardizes model, embedding, scoring, and token-accounting choices across memory systems.[15] The latter three are chiefly recall or answer-quality harnesses, so add an application environment, consequential actions, and protected decision slices.

Pin every model, prompt, artifact commit, dataset snapshot, tool implementation, scorer, and budget in the experiment manifest. A frontier result without those controls is not a ranking of memory policies; it is a result for one compound stack.

Limitations

The clearest direct findings come from recent preprints, constructed tasks, and often one or a few compound system configurations. As of August 24, 2026, StateMem v1 and CAMA v1 had appeared on August 20, while the diagnostic protocol’s v2 had appeared on August 19. StateMem evaluated Qwen-3.5-9B and DeepSeek-V4-Flash as separate backbones.[1] CAMA evaluated Qwen3.6-27B and DeepSeek-V4-Flash.[6] The diagnostic protocol used GPT-4.1-mini for answers and GPT-4o-mini for open-ended judging.[11] The resulting scores are not a common-stack ranking. The StateMem and diagnostic-protocol results therefore do not provide a shared implementation baseline. Released benchmarks also measure different endpoints, so their headline scores should not be ranked against one another.

No selected study runs an independent, multi-seed, same-harness action experiment that holds every model, prompt, tool, context budget, judge, and version constant while connecting recall errors to environment outcomes. Evidence for provenance handling comes partly from constructed correlations. No result identifies a universally optimal admission or retrieval stopping threshold, a best dependency-propagation policy, or a complete record schema.

Hard deletion draws on adjacent NIST SP 1800-22B BYOD guidance for selective wiping of organizational data and applications and protection against organizational-data loss.[9] End-to-end deletion through an agent’s source records, indexes, caches, summaries, backups, and derivatives remains a prudent safeguard with limited direct validation.

Most importantly, existing failure cases leave at least one decision condition uncontrolled: currentness, applicability, scope, independence, or context use. The clean counterfactual remains open. The available results explain why accurate and relevant memories can hurt; they do not establish that fully current, applicable, scoped, independently supported memories make an agent worse merely by existing.

The next decision is the real memory test

That unresolved counterfactual puts the test at the moment a stored past changes a present action. In the calendar example, success is not recalling both meeting times. It is preserving the old fact as history, recognizing the new fact as current only after resolving its scope, retrieving one independent basis for action, sending the right reminder, and learning from what the environment says happened.

That reframes long-term memory as controlled state use. The unresolved question is now testable: when every known confound is held fixed and a clean memory is inserted into the same decision, does the action improve, remain unchanged, or degrade? Until matched experiments answer it, the safest claim is also the most useful one: memory earns persistence only through better decisions.