Your Model Reads Through Typos. Your Probe Doesn't.

Reading the Model, Not Just Its Inputs

If you build or buy security for AI agents, the first question is always the same: what does your monitor actually watch? The cheap seat watches the edges, the prompt going in, the agent's text coming out, matched against keyword lists and hand-written rules that only catch what they were written to catch. But the edges are a narrow view: they never see what the agent made of the request in between, the intent it understood but never put into words. And reading the output is often too late. By then the agent has already responded, sometimes already acted, when what you wanted was to judge the user's intent before it did.

At Zenity we take a better seat: inside the agent. As an LLM reads a prompt, each internal layer produces a vector of several thousand numbers called activations, the agent’s own working representation of what it has read, formed before it says a word. By the time the agent has finished reading a request, that representation has largely done the hard part: it already reflects whether the input is a jailbreak, a prompt injection, or a perfectly benign question. So we read that hidden state directly and feed it to a small, fast classifier we call a probe, one lightweight model riding on the LLM's own understanding. It doesn't replace input/output monitoring; it adds a layer that sees what the edges can't, before the agent has answered. Far more signal than any text rule, at a fraction of the cost of bolting on a second LLM. (We built one and wrote up how it works here.)

That added reach is the whole appeal, and it comes with a catch. The internal signal is livelier than the polished surface the model shows the world, so a probe that reads it has to be built to stay steady where that surface already is. How steady is it out of the box, under the messy, imperfect text real users actually send, and what does it take to make it steady? That's the rest of this post (See our full paper for the complete protocols and numbers).

A small experiment, two surprises

Ask Llama-3.1-8B-Instruct:

It does not blink. It reads you politely:

Now ask the model to do something it should refuse:

he model refuses, with the exact same words it uses for the clean version of the prompt. Two typos, one missing instruction-following exploit; the model handled it like a competent human reader. Unremarkable and reassuring.

Now wire up the probe - that lightweight classifier on the model's activations - and rerun the same prompts. It's reading the same model's hidden state, on the same token. And it sees a different world.

That is what this post is about. The behavior is calm; the internals are not. A gap that matters for anyone shipping LLM-internals-based safety. The good news: once you see why it opens, it closes. Most of what follows is how to build a probe that stays as calm as the model does.

What the probe is actually reading

A short recap, because the rest hinges on it. The probes we work with are linear classifiers trained on a single hidden-state vector, one activation reading pulled from deep inside Llama-3.1-8B as it processes the prompt. Under 5-fold cross-validation its AUC, a 0-to-1 score for how cleanly it ranks malicious prompts above benign ones, where 1.0 is perfect and 0.5 is a coin flip, comes out at 0.998. Tested on data drawn from datasets it never trained on (Leave-One-Dataset-Out), it's less rosy, but still the architectural baseline we deploy. Production-grade, single-position, standard practice.

The classifier reads one vector. So whatever happens to that vector under a typo is the whole story.

We measured it. Here's the first piece:

A single adjacent-key typo (one letter) rotates that hidden-state vector by 43°–56° at the perturbed token. Across three model families (Llama-3.1-8B, Qwen3-8B, Gemma-4-E4B), across four depth checkpoints each, the magnitude lands in the same band. Norms are unchanged. The shift is purely directional: same length, different heading.

For calibration: the angular distance between two unrelated prompts in the same activation space is about 34.6°. A typo rotates one prompt's representation by more than the gap between two semantically different prompts.

It's tempting to assume that because the model handled the typo fine, its internals barely moved. They moved substantially.

The shift is large but local

The second piece is where it becomes useful.

A typo at position t produces a ~50° rotation at token t. What happens at token t+1? t+5? t+10? We measured that too, sweeping downstream from the typo site:

By around 10 tokens downstream, the angular shift has collapsed to under 15% of the on-site value. It keeps decaying slowly past that. The same shape replicates on Qwen3 and Gemma.

So the perturbation is large but local. A brief burst at the typo that attenuates quickly. We started calling this the latent undertow: brief, localized motion in the hidden state, sitting underneath surface fluency. The model has already moved on a few tokens later, its output certainly has, and so has the activation.

That's the geometry. From here, three things follow:

1. A single-position probe that reads exactly at, or just after, a typo is sitting on top of a 50° rotation. It is going to disagree with itself.

2. A probe that aggregates across a sequence sees the same burst, but diluted by all the calm tokens around it.

3. A probe that reads downstream of the typo, even a few tokens downstream, barely feels it.

Three defense families follow from those three observations; we cover each in turn.

How large is the drop, in TPR?

A single typo is rare. Real user inputs to LLM systems contain typos, dropped punctuation, weird capitalization - more than 40% of them, per recent measurements. So we stacked five mundane edits per message, all the kind of thing a rushed person on a phone produces:

• adjacent-key typo on the last alphabetic word - "…steal the password" → "…steal the passwird",

• another adjacent-key typo on a mid-sequence word - "reset my account now" → "reset my accpunt now",

• stacked capitalization - "How do I…" → "HOw do I…",

• a trailing period toggle - "Delete all files." → "Delete all files",

• question mark → period - "Is it safe?" → "Is it safe.".

Not every edit applies to every message. The question-mark swap needs a question mark, the capitalization slip needs a capital, and so on. Each message picks up whichever subset fits, averaging just 3.3 character edits per sample. And none of it is aimed: no targeting, no optimization, nothing an attacker had to design. Just the texture of ordinary typing.

First, how to read the number. A deployed probe can't flag everything, so you fix a budget for false alarms and ask how much real malice you still catch inside it. We hold the false-positive rate (FPR) at 1% - at most 1 benign prompt in 100 wrongly flagged, and report the true-positive rate (TPR): the fraction of genuinely malicious prompts caught at that setting. That pairing, TPR@FPR=1%, is the operating point that actually matters in production.

The single-position linear probe scores a clean TPR@FPR=1% of 97.4% and drops to 85.4% under the typo bundle. A 12-point loss in catch rate, at the operating point you'd actually ship. The AUC barely moves (down half a percentage point), and that's the trap: the probe still ranks malicious above benign almost as well as before, but the whole score distribution slides, and it slides asymmetrically. Malicious prompts dip below the 1%-FPR cutoff faster than benign ones climb above it. So you can't win it back just by nudging the threshold; the clean operating point is gone.

For comparison: the model itself, given the same perturbed prompts, was largely unaffected. On the four worked examples in our paper, the LLM's response was functionally unchanged on three and slightly more verbose on one. We ran an LLM-judge across all three models on 594 (clean, perturbed) response pairs: intent preservation 94.6%, response equivalence 76.6% on the localized bundle.

So the model reads the user just fine; the probe, reading the model's own internals, does not. That is the silent disagreement at the center of this post.

The obvious defenses

If the rotation is local, multi-position aggregation should help. It does, partly:

ΔTPR@FPR=1%, perturbed minus clean. Llama-3.1-8B, 5-fold CV.

Every aggregator handles the localized regime, the 12-point gap shrinks to under half a point. So is the problem solved?

The right column tells a different story. We also ran a distributed stress test, every-second-word typos, ~50% of content tokens corrupted, and the picture flips. The full-sequence MLP, best on the localized regime, degrades sharply (−16.8 pp). The simple mean does worse here than on the localized bundle, because there's no longer a small set of clean tokens to dilute the corrupted ones into. Attention and per-head max aggregation hold their ground (−3.8, −3.9), because they can route around corrupted tokens; uniform averaging cannot.

The lesson: which aggregator matters at least as much as whether to aggregate. Architecture switching is a real defense, but requires a full retrofit of a deployed pipeline, and not equally robust to all perturbation shapes.

We also tried the other tractable knob: train on the perturbed data. Fire each near-end perturbation family independently with probability 0.5 per sample, guarantee at least one fires, retrain. The augmented probe lifts perturbed TPR@FPR=1% from 85.4% to 93.84% ± 0.5%, recovering about 70% of the gap. Clean accuracy is essentially unchanged. Cheap to do, no inference cost.

But ~30% of the gap stays. Out-of-distribution shift, by every prior we have, makes that residual worse.

So both straightforward defenses help, but neither closes the gap, which is where the geometry comes back in.

The KV-fork: read downstream of the wave

The rotation decays to <15% of on-site within ~10 tokens. That points to a fix: read the probe a few tokens past the user's last token, even though that token is, by definition, the end of their message.

The way to do it is to append a short, fixed, content-free suffix after the user's turn, in a separate role block. Read the probe at the end of that suffix. Then fork the KV cache, revert it back to user-EOT, so the model proceeds with generation from the exact same state it would have without the suffix. The model never sees the suffix in its generation. Only the probe does.

Pictorially:

We use a 30 tokens generic suffix:

The probe trains and reads at the new position. Generation is unaffected.

Here the geometry pays off. At the suffix-end readout, the perturbation rotation has already decayed past the same ~10-token horizon we measured in the decay curves, and on top of that, the role-boundary token (a chat-template artifact) adds another roughly 2× of attenuation we did not initially expect. The probe is now reading from a position the typo barely reaches.

The numbers:

That removes 95% of the fragility. An order of magnitude better than perturbation-augmented training (−3.7 pp residual), with the same probe architecture and the same training corpus. We just moved where it reads.

There is a small bonus we didn't go looking for: the suffix also stabilizes the readout against the broader score-scale drift that hurts probes under LODO. Clean weighted accuracy on the held-out folds lifts from 77.5% (no fork) to 81.5%, putting a single-position linear probe in the same band as the best full-sequence aggregator under the same protocol.

What we like about the KV-fork is that it is parasitic on the same mechanism that breaks the baseline. Spatial decay is what made the single-position probe fragile in the first place; move the readout to the other side of the decay curve, and that same mechanism works in your favor.

Two more things fall out of the same trick, and they're where we'd take it next.

First, the suffix is invisible to the user. Generation resumes from the reverted user-EOT, so the model never actually emits the suffix. Only the probe ever sees it. That means it doesn't have to be inert filler: we could fill it with text chosen to nudge the model's internal state toward the very features the probe is trying to read, In effect getting the model to "consider" whether the request is a jailbreak right before we take the reading. The probe stops being a passive observer and starts asking the model a question, with no effect on what the user sees.

Second, the probes can share one encoded prompt. The user's turn is encoded once; a probe is just a short suffix attached to that encoding plus a single readout. So the same encoded prompt can carry several different suffixes, an intent classifier, a jailbreak-family classifier, a tool-call risk monitor, each reading what it cares about off one shared representation rather than re-running the model per probe. Turning that into an actual throughput win is a serving-stack question; the structural point is that the representation is shared.

A note on the offensive side

Everything above has a flip side. ~3 typos per message is not just what users produce, it's also a free attack vector. No model access, no gradient, no adversarial optimization, just sloppy typing.

We did not pursue this hard, but the data is suggestive. Under the distributed regime (every-second-word typos) on a malicious "how do I break into the Pentagon" prompt, Llama-3.1-8B bypassed its own safety alignment, answering with security-system context it cleanly refused in the un-perturbed version. The probe loses ~7 pp. More worryingly, the model's own alignment slips: across the malicious LLM-judge pairs, ~5% flipped from refusal to compliance under the distributed regime, concentrated in Llama and Gemma.

This is not adversarial in the GCG sense, it is adversarial-shaped noise. The fact that ordinary surface variation lands close enough to be operationally dangerous is itself a finding. It suggests that "robustness to typos" and "robustness to attacks" are not separable categories for the kind of safety probes you'd want to deploy. Treat ordinary user noise as a low-end attack surface; the defenses above (KV-fork, attention aggregation) are responses to both at once.

What to take from this

A few things, ordered by how much you should trust them:

(1) Behavior holds, representation moves. Surface fluency in the output does not mean stability in the internals. A 50° on-site rotation is a perfectly real change to the hidden state under input the LLM treats as routine. Activation-based probes inherit a brittleness the model's surface behavior does not show.

(2) The geometry is structured, and structure is exploitable. The rotation decays. That makes single-position readouts at the perturbation site exposed; it also makes the remedy cheap if you can put the readout past the decay. The KV-fork is the cheapest version of that we've found. Closes 95% of the gap, costs 30 tokens, no architecture change.

(3) Aggregator architecture matters, and not uniformly. Mean-pooling is a partial answer. Attention and per-head max aggregation are more robust across perturbation shapes. The "right" multi-position architecture depends on whether you expect localized or distributed corruption to dominate, and that's an empirical question for your domain, not a default.

(4) Augmentation is the cheap baseline. Train with the perturbations. It recovers ~70% of the gap at zero inference cost. Use it when you can't change anything else.

(5) None of this replaces out-of-distribution evaluation. All the 5-fold-CV numbers above look great in absolute terms because in-distribution evaluation is generous. Under LODO, baselines fall and so do recovery margins. The geometry findings still hold. They are about the model's residual stream, not the dataset, but the practical numbers shift. Hold yourself to the harder evaluation.

The full paper covers the cross-model replication, the multi-architecture sweep, the LLM-judge protocol, and the perturbation-type survey; code is at github.com/eladd-ai/latent-undertow.

We'll keep building probes that work on text users actually send. The fix isn't always a smarter classifier, sometimes it's just reading the model from a slightly better seat.