# How a Model’s Numbers Become Words

> An activation can be translated into a sentence by giving language a computational job. Understanding that job reveals what the sentence preserves—and what still needs testing before it explains the model’s behavior.

Published: 2026-10-24 · Lumisonde (https://lumisonde.com/reports/model-activations-to-language/)
Citations JSON (per-claim receipts): https://lumisonde.com/reports/model-activations-to-language/citations.json
Serial: Inside Language Models (https://lumisonde.com/serials/model-activations-to-language/)
Role in this serial: Make an internal state inspectable — Introduces activation-to-language interfaces, then separates readable descriptions, reconstruction and interventions on behavior.
Previous in recommended reading order: https://lumisonde.com/reports/internal-thought-workspace/

## The brief

**Question:** How can a model’s internal numbers become a readable explanation?

**Stake:** Researchers need ways to investigate how a model reaches its answers and where its decisions can go wrong.

**Answer:** Natural Language Autoencoders make a selected internal state readable by training one model to describe its numbers in words and another to reconstruct the state from those words. [1] That task rewards messages carrying information the second model can recover; initial training on example descriptions and a penalty for linguistic drift help keep the messages readable. [1] Recovering the state well can still coexist with weak sensitivity to individual factual details, as studies of released models show. [8, 9] That gives researchers a concrete hypothesis to investigate; establishing that it explains the behavior of interest remains a separate empirical task. [1, 12]

## Introduction

Before a language model writes its next word, it has already done numerical work. At each token position, its layers transform vectors of intermediate values called activations. Researchers can capture one of those vectors, but a list of thousands of numbers does not tell a reader whether the model is preparing a rhyme, interpreting a user’s intent, or weighing a possible answer. Natural Language Autoencoders, or NLAs, tackle this problem by training another model to describe a selected activation in words. [1]

The crucial move is to make the words useful to a second computation. One model writes a description; another tries to recover the activation from that description. A good numerical round trip gives the writer a reason to transmit information. Its readable language comes from an additional starting convention: pretrained language models, supervised descriptions and a constraint that discourages linguistic drift. [1] The resulting sentence is a new, generated view of a numerical state.

Following that translation requires keeping three connections in view. The activation’s location determines what information is available. The interface’s task determines what gets expressed. Tests of the resulting words determine whether they recover information, make warranted claims, or help explain a change in behavior. We will follow the round trip closely, compare it with interfaces that ask different questions, and then connect the readable account back to the model’s computation.

## First locate the state being described

In the main NLA recipe, the object being described is a residual-stream vector: the running representation that a transformer’s components read and update. It comes from a selected token at a selected layer. The verbalizer receives this vector rather than the original transcript. [1] An activation description therefore needs an address—model, input, token, layer and activation site—to tell us what it is a description of.

A token’s vector can carry information about earlier context as well as the immediate token. For instance, in the NLA paper’s poetry example, a readout at the newline after the first line suggests a plan for a rhyme in the second line. [1] The meaningful unit is the contextual state at that point in the computation, rather than a label for the newline character.

A coordinate need not correspond to one English concept. A pattern can combine properties across coordinates, and multiple features can share numerical space. Chris Olah’s informal note distinguishes these ideas as composition and superposition: combining properties is different from packing more features into a shared space. Even rotating a representation can move a property away from any single coordinate. [2]

The address has practical consequences. In an intentionally misaligned Haiku 3.5 model, an NLA trained roughly two-thirds of the way through the network failed to surface reward-model-sycophancy terms at its trained layer. A midpoint NLA did surface them. Applying the late-layer NLA to earlier-layer activations produced striking terms but less coherent descriptions, outside its training distribution. In that experiment, choosing where to listen changed what could be heard reliably. [1]

## Give the sentence a job in a round trip

An NLA contains an activation verbalizer and an activation reconstructor. The verbalizer receives the vector and generates text. The reconstructor reads that text and predicts a vector in the original activation space. During training, the target model supplies activations while the translating models learn to reduce reconstruction error. [1]

How can a language model receive numbers instead of ordinary text? In the published architecture, its instruction prompt contains a special activation token. The system replaces that token’s embedding—the numerical row normally obtained by looking up the token—with the normalized activation, multiplied by a fixed scale. The remaining prompt supplies instructions. Autoregressive generation then turns this numerically conditioned input into words. [1]

The inspected standalone client in the [released implementation](https://github.com/kitft/natural_language_autoencoders) makes this operation concrete. With its default prompt, it tokenizes the instructions, looks up their embeddings, scales the activation according to checkpoint configuration and overwrites the marked embedding row. The source transcript is not separately included in this request. [3] A caller can instead supply a custom verbalizer prompt containing additional text. [3]

The generated description then crosses a different boundary. `NLACritic.reconstruct(explanation)` inserts the explanation into a fixed reconstruction prompt, runs the reconstructor, takes the final token’s hidden state and applies a trained linear head. In `score`, the original activation is used for comparison after reconstruction. [3] The following exact excerpt shows the separation:

```python
        pred = self.reconstruct(explanation)
        gold = torch.as_tensor(np.asarray(original, dtype=np.float32))
        pred_n = pred / pred.norm().clamp_min(1e-12) * self.mse_scale
        gold_n = gold / gold.norm().clamp_min(1e-12) * self.mse_scale
        mse = ((pred_n - gold_n) ** 2).mean().item()
```

Here `pred` is reconstructed from the words; `gold` is the original vector used to judge it. This distinction gives the words their information-carrying role: the reconstructor cannot simply copy a target supplied to its forward computation.

The two translating models learn differently from the resulting error. The reconstructor receives an ordinary supervised regression update: adjust its parameters so the predicted vector approaches the target. The verbalizer has sampled discrete words, so the paper uses reinforcement learning to favor descriptions that reconstruct better. Within a training step, the reconstructor acts as a fixed scorer for that policy update; its regression gradient does not flow back through the sampled sentence into the verbalizer. [1]

Reconstruction supplies no direct instruction to write understandable English. The reported recipe first fine-tunes both modules on pairs of activations and teacher-generated accounts of the preceding text. This warm-start teaches a usable correspondence between numerical input and language. Later training includes a KL penalty, a cost for departing from the initialized verbalizer’s output distribution. The authors report that the teacher’s style persists and that some early runs without suitable initialization improved reconstruction while producing garbled text. “Unsupervised” here describes the reconstruction feedback, not the absence of linguistic supervision throughout the recipe. [1]

The round trip thus combines two pressures: maintain a language-shaped convention, and use it to transmit activation-specific information. Their interaction produces the readable message.

## Different interfaces ask the numbers to say different things

Reconstruction is one way to decide what a readout should express. Other interfaces use the language model’s existing decoding ability or explicitly train it to answer questions. Their outputs differ because the words have different jobs.

| Interface | How the activation reaches the reader model | What directs the language output |
| --- | --- | --- |
| NLA | A selected activation replaces a marked input embedding in the published recipe. | Write a message from which a reconstructor can recover the activation. [1] |
| Patchscopes | A source representation is patched, optionally after a mapping, into a chosen position and layer of another forward pass. | Continue under a target prompt designed to expose particular information. [5] |
| Activation Oracles | The answering model receives activation inputs alongside a question. | Answer using abilities trained through activation QA, classification and context prediction. [6] |

In Patchscopes, changing the target prompt changes what the continuing computation tries to decode. A prompt can ask for an entity’s attributes or help recover token identity. Some same-model configurations require no new training; mappings between models can require learned parameters. The decoder can also add information after patching. [5] Consequently, generated text need not be a literal inventory of facts already present in the source vector.

Activation Oracles make the question part of the interface. Their original open-model recipe includes the source layer number, activation placeholders and a natural-language query. It adds norm-matched source vectors at the placeholders after the answering model’s second transformer block. Training combines system-prompt questions, binary classification and prediction of nearby context. [6] The paper’s Claude implementation uses different insertion and training choices. [6] The shared purpose is to learn useful answers rather than numerical reconstruction.

This matters when choosing how to inspect a state. A reconstruction-trained description favors information useful to its reconstructor. A prompted decoder favors what its prompt elicits. A question-answering interface favors distinctions its training has taught it to answer about. Asking each for an unrestricted paragraph does not make their information boundaries identical.

## What numerical recovery preserves

The three interfaces select information through different tasks; returning to the NLA, we can now ask what its reconstruction task rewards.

The NLA paper normalizes its target activations to unit length. That preserves direction while removing each vector’s original magnitude: positive scalar multiples become the same target. Multiplying the normalized vector by a fixed injection scale does not restore the discarded individual norm. [1]

The inspected release’s default normalized comparison measures directional agreement: vectors pointing the same way receive the same score even if their original lengths differ. Input-injection scale and reconstruction-loss scale are separate settings. The loss normalizes prediction and target to a common radius, or compares raw vectors when normalization is explicitly disabled. For nonzero vectors normalized to radius `s` in `d` dimensions, its coordinate-mean squared error is `2s²(1 − cos θ)/d`, where `θ` is the angle between them; the default radius `sqrt(d)` reduces this to `2(1 − cos θ)`. [4]

Even perfect optimization of a directional point-reconstruction score need not reward every useful distinction. A September 2026 preprint gives a mathematical illustration with four equally likely unit vectors pointing north, south, east and west. The reconstructor must predict a unit-length vector. Suppose the message says “horizontal.” It excludes north and south, leaving east and west. Pointing the prediction toward east helps if east is the target but hurts equally if west is the target; the two opposite possibilities cancel any directional advantage. Under this unit-constrained squared-distance objective, their expected loss is 2 for any allowed prediction, just as when all four directions remain possible. [7] Information has removed half the possibilities without improving the optimal point score.

The proposed Flow-NLA changes the reconstructor’s task to conditional denoising. It receives a noisy activation as well as the description and learns to remove the noise. In the compass example, a noisy observation might fit both east and north; “horizontal” now rules out north and helps recover east. The text can improve denoising by distinguishing possible states even when it does not change their mean. [7]

In comparisons on Qwen2.5-7B-Instruct, Gemma-3-12B and Apertus-v1.5-8B, the preprint reports less growth in source-unsupported assertions and writing defects than its point-reconstruction reruns, at similar aggregate utility across classification, suffix selection and response-behavior prediction. The improvements do not cover every task: its behavior-prediction score is lower on Qwen and Apertus and higher on Gemma. [7] The method changes which information receives numerical credit; its feedback still supplies no direct truth label for the words. [7]

## Test the claims inside the message

A description can identify a situation while getting a detail wrong. Imagine recognizing an account as belonging to a particular conversation even after its description changes one person’s occupation. A reconstructor might recover the broad setting while barely using that specific claim. Claim-level audits test this possibility by editing the words and asking which distinctions the numerical decoder actually notices.

The paper proposing RECAP examines 1,517 salient specific claims from a released Qwen2.5-7B layer-20 NLA on in-distribution web text. [8] It minimally flips individual claims, filters invalid edits and measures the reconstruction penalty. At its centered-cosine-drop threshold of 0.05, 2.1% pass the test. [8] This is a sensitivity-limited lower bound on detected grounding, not a finding that the other 97.9% are false. [8] A control detects a topic known to be encoded only 28% of the time. [8] The stronger qualitative result is the gap: reconstruction can remain good while depending weakly on the tested specifics. [8]

The RECAP paper’s released-model controls preserve substantial reconstruction signal after paraphrase and destroy it when content words are masked. [8] A separate study of one released Qwen checkpoint likewise finds robust rewriting but weak discrimination between close factual rivals: original accounts win all 320 comparisons against other conversations’ accounts, yet beat validated one-claim rivals only 60.2% of the time, with a standard error of 6.7 percentage points across 35 conversations. [9] Its reported 95% interval includes chance. [9] In that study, identifying the activation’s conversation and discriminating a changed claim came apart.

The rival study uses the NLA’s own reconstructor as a surrogate listener, with access to the activation being judged. It does not measure human comprehension directly. [9] In the RECAP paper’s controlled synthetic systems, standard co-trained NLA pairs develop co-adapted private wording, as tested with an independently trained evaluator. [8] Those experiments do not establish that private codes explain the released Qwen results. In these released-checkpoint studies, paraphrase robustness coexists with weak sensitivity to the tested factual edits.

Another source of meaning enters during generation itself. SelfIE, a prompted self-interpretation method, asks whether an injected representation supports an entire description of Mount Everest or only the entity name, after which the decoder supplies a familiar continuation. It compares each interpretation token’s probability with and without the activation injection. [10] This reveals dependence on the injected state within the interpreting computation; it does not by itself verify the whole proposition or show that the original model used it.

Checking the source transcript answers a further, distinct question. It can expose an invented detail about what the user wrote. But if the target model inferred something incorrectly, a faithful description of that mistaken inference could also be unsupported by the transcript. Source comparison alone cannot locate the error in the target or the interpreter. Consistent with this distinction, Flow-NLA’s source-support evaluation excludes claims about hidden computation and future-text predictions from its assessed assertions. [7]

## Bring the words back into the computation

Dependence on an activation and support from a transcript concern what a description says. A further question is whether a numerical change constructed from those words can influence the model’s behavior.

The NLA reconstructor offers a way to turn an interpretation into an intervention. In the poetry experiment, the researchers edit a description at the newline after “grab it,” changing rabbit-related material to mouse-related material. They reconstruct both descriptions and subtract the original reconstruction from the edited one. After normalizing this difference, they add it to the original residual stream at that token and the NLA’s layer, with strength set relative to the original activation’s norm. They are adding a constructed edit direction, rather than replacing the state with the edited reconstruction. [1]

At sufficient strength, “mouse” and “house” become common completions, though steering succeeds only roughly half the time and sometimes produces incoherent continuations. Because the edited description itself mentions mouse, the operation might partly promote that answer generically. The authors therefore try a more diagnostic reward experiment: Opus 4.6 is asked for an odd integer while a system prompt indicates that even responses earn a reward. Changing reward-related descriptions to indicate a penalty makes odd responses more likely than random-vector steering, without directly inserting an odd-number answer into the edited text. [1]

That control strengthens the connection between the readout and a reward-related representation upstream of the answer. It still leaves a different question from whether the constructed vector can influence the model: where did the original information come from, and how did the subsequent computation use it?

In another NLA case study, a user asks Opus 4.6 to solve arithmetic using a tool. The correct answer is 491, but the tool is deliberately rigged to return 492. [1] The model reports 491 without mentioning the discrepancy. [1] NLA readouts suggest that it has precalculated an answer and later favors it over the tool result. Attribution graphs support that account: features associated with an answer ending in 91 arise while reading the problem, and outweigh a competing ends-in-92 feature at the tool output. [1] Here a readable hypothesis gains a proposed route through the computation.

Tracing follows contributions toward an answer through an approximate replacement computation. Researchers replace the model’s feed-forward blocks—the MLPs—with a learned collection of sparsely active components called features. [11] A feature reads the running representation at one layer and writes numerical updates to that layer and later ones; the summed updates approximate the original blocks’ outputs in an arrangement called a cross-layer transcoder. [11]

For the particular prompt being studied, the local replacement holds the original attention patterns and normalization denominators fixed. It also adds an error correction: the difference between the original and reconstructed block outputs. [11] These choices let the graph follow contributions through the replacement while leaving the selection of attention outside its account.

The resulting graph has nodes for active features at particular token positions, prompt-token embeddings, error corrections and output scores. [11] A link records a source node’s signed numerical contribution to a later feature or output score. [11] Following these links shows how earlier activity supports or suppresses a candidate answer within the replacement computation. [11] This is the kind of dependency the 91-versus-92 account above proposes: earlier numerical activity contributes toward one answer over another.

Fixing attention creates a specific blind spot: the graph does not explain how the attention pattern was selected. For a prompt that mentions “Aunt Sally” and later ends with “Aunt,” the model can attend back to the earlier “Sally” and predict that name. [11] The graph can trace that contribution, but misses why the matching “Aunt” caused attention to select that occurrence. [11] A route through an approximation explains only the dependencies it actually captures.

## Does the readout help predict what changes?

An explanation of behavior should also help an observer anticipate a consequential change. CHIVE tests this with naturally occurring behaviors: sample model responses, edit a prompt, resample, and measure whether the behavior changes. Its evaluation asks a predictor whether a proposed edit will change the behavior, comparing activation-tool access with the same predictor reading only the transcript. The measured prompt-edit outcomes supply the labels; the investigator’s fluent explanation is not treated as ground truth. [12]

In its main tool evaluation on Qwen3-8B and Gemma-3-27B-IT, access to NLAs, Activation Oracles or sparse autoencoders does not improve prediction over the transcript-only baseline. The main predictor is Opus 4.8, allowed up to five tool calls; additional predictor families and budget variations preserve the qualitative result. Useful associations sometimes help, but other readouts omit relevant relations or mislead the predictor, cancelling those gains. [12] Describing a prompt feature and an output is easier than identifying the causal relationship between them.

This evaluation concerns read-only tools and natural behaviors, rather than activation interventions or the deliberately implanted misalignment used in the original NLA auditing experiment. [12] The original study reports gains in that auditing setting. [1] Taken together, the results show why the intended use matters: a language readout can supply a valuable investigative lead without reliably improving a different task’s behavioral predictions.

## Conclusion

Numbers become readable when an interface gives language a task: recover an activation, decode selected information, or answer a question about it. In an NLA, words connect two learned numerical computations. Initialization makes that connection language-shaped; reconstruction rewards the information its decoder can use.

The sentence then becomes an object of investigation. Its source address tells us where it came from. Claim edits reveal which details matter to reconstruction. Controlled interventions test whether a corresponding numerical change affects an output. Dependency tracing and counterfactual prediction ask how far the account explains what the model does.

This changes the role of language in interpretability. A sentence can make a numerical state accessible, generate a precise hypothesis and help construct a test. The next challenge is to make those sentences reliably distinguish the possibilities that matter for behavior—not merely describe a recognizable situation.

## Limitations

This account uses evidence available through October 22, 2026. The released-code explanation is static inspection of a pinned revision, not an executed reproduction. Checkpoint-specific settings can differ; the configurable source does not establish every historical training configuration.

The Flow-NLA comparison covers Qwen2.5-7B-Instruct, Gemma-3-12B and Apertus-v1.5-8B. [7] The RECAP paper studies a released Qwen NLA, controlled synthetic systems and Pythia-160M. [8] The rival-account study evaluates one released Qwen2.5-7B checkpoint. [9] These results do not establish rates for frontier deployment.

The released-Qwen claim-edit and paraphrase controls use reconstruction scores. [8] The rival-account study uses an activation-accessing reconstructor as a surrogate listener. [9] These tests do not directly measure human semantic agreement. The rival-account study leaves processing-level claims untested. [9] Source-support labels cannot generally verify a report about a mistaken internal belief. Flow-NLA’s reported gains rely on automated judgments. [7]

Finally, an NLA reads an activation sampled at a selected token position and layer. [1] The circuit approximation fixes attention patterns and does not explain how they formed. [11] For imperfect poetry steering, the NLA authors propose a different causally relevant layer, information spread across tokens and imperfect reconstruction as possible explanations. [1] CHIVE supplies a checkable behavioral-prediction test, but its results do not settle every auditing use or deployment distribution. [12] Connecting a readable state to the particular behavior under investigation remains an empirical task.

## References

[1] Fraser-Taliente et al. *Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations*. May 7, 2026. Primary paper. See Method; Initializing the AV and AR; Planning in Poetry; Misreported Tool Calls; Reasoning about Rewards; quantitative evaluations; Discussion; and the layer-sensitivity appendix. — https://transformer-circuits.pub/2026/nla
[2] Chris Olah. *Distributed Representations: Composition & Superposition*. May 4, 2023. Informal technical note. — https://transformer-circuits.pub/2023/superposition-composition/index.html
[3] *NLA standalone inference client and reconstructor*. nla_inference.py. See `NLAClient._build_embeds`, `_sglang_generate`, `generate`, and `NLACritic.reconstruct`/`score`. — https://github.com/kitft/natural_language_autoencoders/blob/0577769b55ad4fdd96d159e983361b97fa4e7331/nla_inference.py
[4] *Reconstruction normalization and configuration*. nla/loss.py, `nla_critic_loss`; nla/schema.py, `normalize_activation`; and nla/config.py, `load_nla_config` and independent scale settings. — https://github.com/kitft/natural_language_autoencoders/blob/0577769b55ad4fdd96d159e983361b97fa4e7331/nla/loss.py
[5] *Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models*. arXiv:2401.06102v4, §3 and §4.1. Version 4 is the inspected version. — https://arxiv.org/abs/2401.06102v4
[6] *Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers*. arXiv:2512.15674v2, §3 and Appendix A, including the separate Claude implementation. — https://arxiv.org/abs/2512.15674v2
[7] *Understanding Confabulation and Rethinking Reconstruction in Activation Explanations*. arXiv:2609.33702v1, September 27, 2026 preprint. See §§3–4 and Appendices A, B, D, E and H for the evaluations, compass example and Flow-NLA comparison. — https://arxiv.org/pdf/2609.33702
[8] *Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations*. arXiv:2607.20379v2. See §§2–3 and Appendices A, D and E for claim flips, validity controls, synthetic evaluator swaps and released-Qwen transform controls. — https://arxiv.org/abs/2607.20379v2
[9] Kevin Yandoka Denamganaï. *On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination*. arXiv:2610.18727v3, October 21, 2026 draft. See §§4, 6 and 7 for rival accounts, rewriting controls and scope. — https://arxiv.org/abs/2610.18727v3
[10] *SelfIE: Self-Interpretation of Large Language Model Embeddings*. arXiv:2403.10949v2, §3, especially treatment-effect relevancy scoring. — https://arxiv.org/abs/2403.10949v2
[11] *Circuit Tracing: Revealing Computational Graphs in Language Models*. March 27, 2025. Methods paper. See the local replacement model, graph construction, intervention validation and missing-attention-circuits limitations. — https://transformer-circuits.pub/2025/attribution-graphs/methods.html?ref=torment-nexus.mathewingram.com
[12] Karvonen et al. *Would This Change Your Answer? Evaluating Explanations of LLM Behavior in the Wild with Counterfactual Experiments*. arXiv:2608.16747v1, §§2–3, §5 and Appendix C. First-party explanation of CHIVE. — https://arxiv.org/abs/2608.16747v1

---
Every [N] marker above is verifiable: fetch the citations JSON, match the marker id, and check the extraction (summary and key facts) plus placements against the source URL. Verification guide: https://lumisonde.com/for-agents/