It is 1 a.m. and you rotate your OpenRouter key, because the old one leaked and has to die now. You edit the line in config.yaml, change the copy in .env, and remove a Telegram user from the pairing list who should no longer be able to talk to your agent. You restart nothing — Hermes reloads changes live, and you have relied on that for months. In the morning, one profile is happily using the new key while a follow-profile session keeps firing requests with the old one and filling the log with 401s. The user you revoked? Still getting through.

That is not you forgetting to save. The answer in PR #145651, merged October 27, is a little counter-intuitive: your edit did reach the file, but the process believed the file had not changed. And it believed that because it was watching a fingerprint that does not move when both the read and the write fall inside the same clock tick.

What a cache actually watches

Hermes keeps small caches everywhere. Read auth.json once and remember the parse; read the pairing list once and remember who is approved; read config.yaml once and remember the provider routes. To avoid pointless re-parsing, each cache looks at a file “fingerprint” first. The usual four values:

• mtime — last modification time
• ctime — inode change time
• size — byte count
• inode — the file number

Same fingerprint, reuse the parse. Different fingerprint, re-read. That is correct almost always. It goes wrong when two details line up.

Detail one: same length, in-place overwrite. Turning api_key: sk-old into api_key: sk-new keeps the byte count identical, and writing back in place — through an editor or a script — keeps the inode too. Two of the four values are now guaranteed equal.

Detail two: timestamp resolution. On many kernels (notably CI runners and some older hosts without multigrain timestamps), mtime and ctime come from a fairly coarse clock. If the read and the write land in the same tick, both writes carry exactly the same (mtime_ns, size, ino, ctime_ns). The cache sees an unchanged fingerprint and concludes: nothing to do, keep the old parse.

That is how CI caught it. A test writes config.yaml with port 3, reads it, then rewrites it in place with port 5 — same length, same inode. The assertion failure is almost comically plain:

E  At index 1 diff: 'http://127.0.0.1:3/v1' != 'http://127.0.0.1:5/v1'

Port 3 and port 5 are one character each, so the file is byte-for-byte the same size. The test meant to prove a rotated key reaches follow-profile sessions; it proved the cache was serving the old route instead.

Why a restart “fixes” it and a live reload does not

This is the part that drives people up the wall: the bug only exists while the process is alive.

After a restart every cache starts empty, the first read has to parse, and you see the new value. That is also why the symptom gets misdiagnosed forever: “the config did not apply”, “live reload is broken”, “this profile is stuck”. Every explanation points at the configuration system, while the actual culprit is how the cache key was designed.

The blast radius is wider than one cache. Some keys used the full four-value tuple; some used mtime alone; some used mtime + size. Bypass one and another component may still hand you yesterday’s value.

One helper instead of twenty patches

The fix does not patch each site. It adds a single helper in the repository’s root utils.py:

def file_cache_version(path, st=None, *, always_digest=False):
    ...

It returns two things: the file signature, plus a content digest.

• The signature is still the cheap stat tuple.
• The digest defaults to a “racy window” policy: if the later of mtime and ctime is less than two seconds old (RACY_MTIME_WINDOW_NS), hash the file bytes with blake2b; once the file has settled, the digest becomes an empty string.

In plain terms: a file that was just written is judged by its content, a file that has settled is judged by its stat signature. The only cost is one extra parse for an entry cached while the file was still racy, once it settles. For a cache that is re-checked on every read, that is nothing.

always_digest=True exists for a different shape of caller: a signature that is stored now and compared much later. For those, a digest flipping to an empty string would look like a change, so they hash on every call. The PR names them:

• the served profile’s signature (a false change reconnects the profile’s adapters)
• the parse-failure record (a false change would clear the error while the file is still broken)
• shell-hook consent
• the free-tier inventory stamp
• the onboarding due stamp
• the skills manifest, which is written to disk and compared by the next process

There is one deliberately narrow exception: a worker bound to a frozen config snapshot never hashes config.yaml. The check sits inside the helper, so no caller can make a safe worker open the profile file. Other files — auth.json, .env, pairing lists — keep the digest inside workers.

And it never raises. Given the caller’s earlier stat, a file that has since disappeared or become unreadable gets the error name as its digest, which no readable version can match. The cache invalidates instead of crashing.

Twenty caches, any of which could serve an old secret

The PR then lists every site where a stale read could serve a secret, token, allowlist, route or config, and converts all twenty. A few that map onto everyday use:

Site	What it caches	Key before
hermes_cli/auth.py global auth store	the shared auth.json — your credentials	file_signature
hermes_cli/auth.py Nous login status	whether you are signed in	st_mtime only, plus a 15 s TTL
gateway/pairing.py approved list	the DM allowlist	the four-value tuple
gateway/platforms/webhook.py dynamic routes	per-route HMAC secrets	(mtime_ns, size, ino)
plugins/platforms/feishu/feishu_comment_rules.py	Feishu comment pairing allowlist and rules	st_mtime only
tools/mcp_oauth_manager.py	reloading MCP OAuth tokens	st_mtime_ns only
providers/__init__.py named custom provider	which route a request takes	file_signature
agent/prompt_builder.py skills manifest	the skills snapshot in the prompt	file_signature

Look at what that list touches: credentials, allowlists, routes, and the prompt itself. The two allowlists are the ones that should make you sit up — revoking access is supposed to take effect immediately, not at the next restart.

Concretely, after the fix:

• a rotated api_key (k-old → k-new) reaches follow-profile sessions without a restart;
• a revoked pairing has no access on the very next check;
• a same-length port edit (:3 → :5) changes the route right away;
• a refreshed token, or a same-size config correction, is never shadowed by the previous parse.

What you should do

There is not much to do, but three things are worth knowing.

First, upgrade. A fix like this reaches your install with the next tag; CLI users go through hermes update. If you want the caveats of that path, we covered them in our upgrade guide.

Second, stop using restarts as the way to make config apply. If you built the habit of bouncing the gateway after every config edit because “otherwise it sometimes does not take”, you were probably hitting this. After the update, live reload behaves the way you assumed it did.

Third, verify revocations. If you control who can talk to a bot through the pairing allowlist (the mechanism is described in our profile allowlist write-up), send a message after revoking and confirm it is refused, rather than assuming it worked. Before this fix, “I thought I had revoked that” was a real hole.

It is also worth storing secrets properly. We went through the difference between auth.json and .env, and encrypting with the system keychain, in the secrets and encryption guide.

The takeaway

What makes this bug nasty is not its difficulty. It is that it opens an invisible gap between “I changed it” and “the system is using the new value”. You changed a file; the system trusted a fingerprint; and the fingerprint does not move when both actions land in the same clock tick. The fix follows a good instinct: rather than guessing which parser a client will use, make “these might disagree” detectable — hash the content where you can, hash it every time where you cannot. A cache is supposed to be a performance optimization, not a copy of last week’s world.