Why typing gets slower the longer a session runs — and how Hermes Desktop cut the cost by two thirds


It is 3pm and you are scrolling back through a session that has been running since yesterday, looking for the conclusions you reached this morning. You reach the top, drop back into the composer, and start typing — then you stop. Characters arrive one at a time. Deleting a character takes half a second. The caret is a beat behind your fingers. You try another machine; same lag. You assume the model is busy in the background, except the model is sitting there waiting for you to finish the sentence.

The problem was the input box itself. A performance change merged into Hermes Desktop on October 26 (PR #144903) makes the mechanism explicit: the more history is loaded, the slower typing gets, because every keystroke re-rendered the whole transcript. After the fix, the same 300-turn session costs 9.6 ms of main-thread time per key instead of 33.9 ms.

One keystroke, one full re-render

The Desktop chat surface is built on assistant-ui, a component library for chat UIs. The text in your composer does not live quietly inside the composer: it is a value in a shared store, and a surprising number of things subscribe to it —

  • whether the send button is enabled or greyed out;
  • whether a lone ? should surface the help hint;
  • whether the line is a slash command (/model and friends) or an ordinary message;
  • whether the empty placeholder is shown;
  • how hard newlines are laid out.

The subtlety is that checking those subscriptions is not confined to the composer. A store write re-renders the thread’s client resources and re-checks every mounted message’s selectors. Longer transcript, more mounted turns, bigger bill. The comment in the new code puts a number on it: roughly 0.5 ms per keystroke in a short session, up to 19 ms once a long session’s history is on screen — and that is only one part of the cost.

Which explains the counter-intuitive feel of it: typing does not degrade as the session ages so much as it degrades as you scroll back. Scrolling up is what actually mounts the history.

The fix: write to the store only when something visible changes

The change is not a debounce bolted onto the composer. It turns “when must this keystroke reach the store?” into explicit logic. A new module, composer-store-writer.ts, derives a signature from the text with composerTextEdges(text) — exactly the facts the subscribers care about:

text.length === 0                          // empty vs not
trimmed.length > 0                         // has real content
text === '?'                               // a lone question mark
trimmed.length > 0 && !SLASH_COMMAND_RE    // ordinary text, not a slash command
text.trimEnd().includes('\n')              // multi-line layout

If a keystroke leaves that signature untouched, every subscriber would observe the same state, so the write can wait. How long? TYPING_STORE_IDLE_MS = 180 — a 180 ms typing pause and the trailing text lands in the store, where draft persistence and the runtime mirror read it.

Deferring writes needs the edges covered, or you trade a lag for a bug:

  • flush() — land pending text before any read that needs the store current;
  • cancel() — drop pending text on unmount instead of leaving stale state;
  • observed() — adopt a store change made elsewhere (the runtime subscription) as the new baseline, so an old value cannot overwrite a new one.

The sticky-prompt strip (the echoed composer pinned above a long transcript) stopped recomputing styles for every visible turn on each frame as well; it now registers a CSS variable once.

The measurements

The PR ships a reproducible setup: a 300-turn / 1800-message session, scrolled through with 2500 wheel frames so the whole history is mounted (about 42k DOM elements), then 60 typed characters.

Measure Before After
Main-thread time per keystroke, history read up 33.9 ms 9.6 ms
Main-thread time per keystroke, fresh open 12.1 ms 6.0 ms
Store flush per keystroke 18.0 ms 1.3 ms

The author is careful about what those numbers mean, and the honesty is worth repeating: single runs on a shared host, where a second pre-fix run measured 49.3 ms per key. On that 20-core machine input-to-paint stayed near one frame either way — so the difference between 33.9 and 9.6 is mostly felt on slower hardware, where main-thread time is the scarce resource. This is not “everything got three times faster”; it is “your laptop stops paying a per-keystroke tax on history it already rendered”.

What to do with this

The change is on main only right now. v0.22.5 shipped on October 23; this PR merged on October 26, so even the canary tags from that day do not contain it. Three routes:

  1. wait for the next release and hermes update — the least work;
  2. track main if you want it today (the channel switch is described in the update channel guide);
  3. relieve it now by keeping sessions shorter — the composer’s cost scales with mounted history, so starting a new session or tidying session boundaries helps more than any setting.

If your complaint was really “the Desktop starts slowly”, that is a separate problem, covered in the startup latency write-up. This change does nothing for it.

Why this one is worth remembering

We default to blaming the model when an agent feels slow: too many tokens, too much context, a provider queueing requests. This bug is a reminder that the UI layer grows its own performance debt, and that this kind of debt is uniquely invisible — it never appears in a log or a trace, only between your fingers and the screen.

So the next time it feels slow, ask which half is slow. Typing slow while replies arrive normally is a rendering problem. Both slow is when you go look at the model and the network. Splitting that question in two saves most of the debugging.