Hermes Stops Guessing Your Context Size: Usage Anchoring Aligns Compression Thresholds with Real Numbers

Your session hits turn 40 and everything is fine — then out of nowhere a “context too long, compressing” notice pops up, even though you’re nowhere near the window limit. Or one day the API throws a 413 error saying the request is too large, while the conversation clearly isn’t that long. Both scenarios used to be common in Hermes, and they share one root cause: Hermes was guessing how big your context is — re-estimating the entire transcript turn after turn with heuristics like chars/4 and flat 1500-token images, letting the error compound as history grew. The change merged on August 28 (PR #97206) pulls that root out: context accounting now anchors on provider-reported usage, and the estimation window shrinks from “the whole conversation” to “messages appended since the last response.”

Before: whole-transcript estimation, snowballing error

Every provider response actually carries a precise usage report: usage.prompt_tokens (exactly how many tokens this request sent, including system prompt, tool schemas, and full history) and usage.completion_tokens (how many were generated). The model vendor counts these itself — that’s real ground truth.

But old Hermes barely used it. Every time it needed to check context size, it re-estimated the entire session with heuristics: ASCII chars divided by 4, flat 1500 tokens per image, CJK density rules… The estimate was fine early in a session, but as history grew, the error compounded. Some sessions were over-estimated — compressed before ever hitting the threshold; others under-estimated — the request exceeded the provider limit and got a 413. Issues #89938 and #88960 are typical examples of this “estimate-vs-reality” class.

New mechanism: usage anchoring, error window down to one turn

The core of PR #97206 is one formula:

current context tokens =
  last response's usage.prompt_tokens
  + last response's usage.completion_tokens
  + estimate(ONLY messages appended since that response)

In other words: the real size of the whole conversation comes straight from the provider’s numbers; only the handful of messages added since the last response need estimating. The error window shrinks from “the entire transcript” to “one turn” — and it self-corrects at every response, because the real value is the starting point, never something the estimate has to chase.

The anchor is captured at exactly one site: the usage block right after context_compressor.update_from_response() in the main conversation loop (capture_usage_anchor), updated on every response. Compression thresholds and 413-recovery logic all switch to this anchored number.

Companion fixes: 413 recovery counts bytes

The same wave ships companion fixes (#97197 and friends): 413 recovery used to measure request size in estimated tokens — it now measures actual bytes, because the provider’s 413 judgment is based on HTTP payload size, and no token estimate converts to bytes correctly. The compressor also now actually frees the bytes occupied by historical images during compaction (#97160), so 413 recovery isn’t “barely passing” but “genuinely freeing space.”

What this means for you

  • Fewer surprise compressions: threshold checks align with real numbers, sessions stop being “virtually bloated” and compressed early;
  • Fewer 413s: request-size checks switch from estimates to byte counts, long sessions stop randomly dying with “payload too large”;
  • Honest /context: the numbers you see match the numbers on your provider bill — what you see is real.

Maintainer Teknium put it bluntly: “I’m really getting tired of us estimating tokens. Stop estimating things.” This wave is that sentence, turned into code.

These changes merged on August 28 and currently live on upstream main, not yet in a release tag. They take effect automatically once you hermes update to a build containing them.

Further reading