An agent that remembers everything is a secret leak with a good memory

· 10 min read

My agent system keeps everything. Session transcripts, tool calls, and stray observations land in an episodic store on disk, a consolidation cycle mines them for durable facts, and the whole thing syncs to a private git repo so every machine shares one memory.

The persistence is the point. It is also the liability.

The secrets you did not choose#

The easy half of the problem is explicit secrets: values I decided to protect, sitting in an encrypted vault, referenced by name. That half was solved early.

The dangerous class is inline secrets, the ones nobody chose. An API key pasted into a debugging conversation. A connection string riding along in a stack trace. An OAuth token surfaced by a browser tool. The agent sees these all the time, because I see them all the time, and the agent watches me work.

A stateless agent makes this survivable by forgetting. Whatever it saw evaporates with the session. A memory system removes exactly that mercy. The transcript is written to disk, indexed, pushed to a remote, kept. Six months in, one compromised machine or one leaked repo credential means every secret that ever appeared in any conversation, exfiltrated at git-clone speed.

What forced the design was the asymmetry. Scrubbing costs milliseconds at each write. Not scrubbing costs nothing today, and the bill it runs up has no ceiling: every day the system operates, a credential leak exposes more than it would have the day before.

Scrub at the write, not at rest#

Four shapes were on the table.

Trusting the private repo and disk encryption is the default posture, the one you get by doing nothing. It fails on the one axis that matters here: the residual risk only ever grows.

Encrypting the whole episodic layer at rest protects the data and kills the layer. My stack is filesystem-first; the daily log is something I cat, grep, and read in an editor when something looks off. Encryption at rest taxes every one of those reads, slows the consolidation pass, and turns a human-readable record into a blob.

Scrubbing twice, once on write and again before the sync push, sounds like defense in depth. What it actually buys is divergence: the local store holding things the pushed copy does not, two versions of the truth, and no single place to point at and say "this is what the system knows."

So: scrub once, at the one point where data enters persistence. The session-end hook that writes episodic events runs a regex-plus-entropy scrubber, in-process, before anything touches disk. The budget is thirty milliseconds for a typical event, and nothing the scrubber recognizes ever lands in a memory layer in the clear.

The patterns are the ones you would guess. Known prefixes for GitHub, AWS, and LLM-provider keys. Structural matches for three-segment JWTs, user:password@host connection strings, and private key blocks. Emails, always, as PII. And a catch-all at the bottom of the precedence order: any long character run whose Shannon entropy reaches 4.5 bits per character gets flagged as a probable secret even when no named pattern claims it. The code sets the length floor at twenty characters, but the arithmetic sets it at twenty-three: a shorter run cannot hold 4.5 bits per character even with every byte distinct. Most specific match wins.

Redacted, but still shaped#

The scrubber does not replace a secret with [REDACTED]. It replaces it with a typed marker:

<REDACTED:github-token:a3f92>

The kind names the pattern class. The five hex characters are a truncated SHA-256 of the original value, and that truncation is the detail I would defend hardest.

Because the hash is stable, the same secret produces the same marker in every event it appears in. So the consolidation cycle can still see structure. "This token shows up every time that deploy step runs" survives redaction, because co-occurrence survives, and the cycle never sees a value to do it.

Five hex characters is twenty bits, which is nowhere near preimage-resistant. Deliberately so. The marker is a clustering token, not a verification token, and every bit added to it would leak a little more information back out of the thing built to remove it.

The sidecar, and failing closed#

Sometimes I need the original back. A false positive I want to inspect, a key I pasted somewhere and then lost track of. So redaction is not destruction: the originals go to a per-day encrypted sidecar file, keyed by event and marker, using the same key as the vault. The sidecar is committed to the repo, which sounds wrong right up until you notice it is ciphertext. Committed, a new machine can recover originals with the vault key it already needs for everything else, and an attacker with the repo holds ciphertext either way.

Two small decisions in the recovery path do more work than their size suggests. The unredact command prints the recovered value to stderr and keeps stdout for the confirmation line, so pipes and command substitutions, the things that mechanically slurp output into transcripts and sessions, never capture the secret. Recovery is a human verb here; a value that rode stdout back into a session would be sitting on the write path to the very store it was scrubbed from. And the system fails closed: no vault key configured means nowhere to put originals, so episodic writes that would produce redactions are refused outright. A degraded install is loud, not silently lossy.

The question that classes a secret#

Scrubbing covers what the agent sees by accident. What it may touch on purpose is a different question, and getting that one right took me longer than the scrubber did.

The wrong question is "how sensitive is this value?" The question that actually partitions the vault: may an automated flow ever handle this?

Shell-class says yes. The agent interpolates $(vault get <key>) into a command, the value exists in that one command invocation and nowhere else, and the write-time scrubber guarantees it never persists into any memory layer. That guarantee gets exercised for real: in the last thirty days the scrubber caught a vault-held value on its way into the episodic store 57 times. Redaction is not a nice-to-have here. It is the precondition that makes shell-class safe to offer at all.

Strict-class says no, and means it. Human-only cold storage. The CLI read verb refuses strict keys unconditionally. There is no agent-side access path, and that is a decision, not unfinished work.

It was almost otherwise. The original design promised a generic tool: agent names a key and a free-form action, the core decrypts in-process, performs the action, returns the result. The agent never sees the value. On paper it is zero-knowledge, and it demos beautifully.

I rejected it, and wrote the rejection down as permanent rather than deferred. The action string is composed by the agent, and the agent is exactly the party the strict class exists to distrust. Under prompt injection, "perform this action with my production key" becomes an exfiltration primitive by construction. The attacker does not need the value when they can ask the holder of the value to make an authenticated request to a URL they control. A zero-knowledge property that only holds while the agent is uninjected is theater. The tool was not even earning its risk: after seven weeks of daily use, not one real flow had needed it.

What exists instead is three access paths, ordered by how little the agent is trusted:

  1. Shell-class plus redaction. The agent handles the value; persistence is prevented.
  2. Injection at spawn. Secret references in server config resolve to environment variables when an MCP server launches, so the agent orchestrates the server without ever seeing its credentials. The launcher refuses to resolve strict-class references here, on purpose, so a synced, agent-writable config file cannot be edited into an exfiltration path.
  3. In-core flows. One class of operations, hardcoded flow by flow, consumes a vault-held token entirely inside the core binary. The agent triggers the flow and sees only the outcome.

Strict-class sits below all three. Nothing automated touches it, ever.

The honest parts#

The catch-all pattern has a false-positive bill, and I can put numbers on it. Over the last thirty days the scrubber emitted 18,344 markers, and 15,832 of them, 86 percent, came from the catch-all. They cover 7,583 distinct values, against 90 distinct values for every named pattern combined. Nobody has seven and a half thousand real secrets.

When I went to fetch the exact noise share for this post, the health check that is supposed to report it turned out not to exist yet. The decision record promises the check; the code never grew it, which is its own small lesson about the distance between a consequences section and a shipped binary. So I built it before the post shipped: it decrypts the window's sidecars in-process, classifies each value against known non-secret shapes, and lets only counts escape. The verdict on a real week: 64 percent of catch-all markers are identifiably not secrets, and the biggest class by far is long filesystem paths, which clear the entropy bar constantly. I had guessed base64 blobs; I was wrong, and I only know that because the check now exists. The rest stays unclassified on purpose, because a random base64 run is exactly what a real secret looks like, and a classifier that calls it noise has learned to shrug at leaks. The alternative, an entropy bar high enough to never misfire, would be high enough to miss things I care about.

The pattern set is versioned, and old events are not retroactively re-scrubbed. A pattern I add today protects the future, not the past. The guarantee is honestly prospective, and I would rather say so than imply otherwise.

And the sidecar is the only place on disk the captured originals exist. Lose the vault key, lose the recovery path, permanently. That matches the posture the vault already had, and I accept it, but it deserves saying out loud.

What transfers#

The scrubber, the markers, and the sidecar were all in place before the memory layers filled up, and that ordering is the part I would insist on. Retrofit redaction onto a store that already holds six months of transcripts and the past stays exposed no matter what you ship. The time to decide what an agent is allowed to remember is before it starts remembering.