KrustyPlanet/hermes-agent
6
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

perf(/model): prewarm picker provider-models cache in background (#39847)

Browse source 
* fix: respect disabled auto-compaction on context overflow

Port from anomalyco/opencode#30749.

When compression.enabled is false, NO automatic compaction trigger may
fire. The proactive token-threshold paths (preflight + post-response
should_compress gate) already honoured the setting, but the three
provider-overflow recovery paths in the agent loop — long-context-tier
429, 413 payload-too-large, and context-overflow — called
_compress_context() unconditionally, silently compressing and rotating
the session against the user's explicit choice.

Add a single guard at the top of the overflow-recovery dispatch: when
compression is disabled and the error is one of those three overflow
classes, surface a terminal error (compaction_disabled: True) telling the
user to /compress manually, /new, switch to a larger-context model, or
reduce attachments. Manual /compress (force=True) is unaffected — it never
enters this loop.

Tests: new TestOverflowWithCompactionDisabled (413 + 400 overflow don't
compress when disabled; control case still compresses when enabled).
Existing overflow-recovery tests updated to enable compaction explicitly
(they verify the recovery fires); fixture defaults flipped to True to
match production (compression.enabled defaults to True).

* perf(/model): prewarm picker provider-models cache in background

The no-args /model picker calls list_authenticated_providers(), which
fetches each authenticated provider's live /v1/models list serially. On a
cold or stale (>1h TTL) cache that blocks ~1.5s on the user's critical path
the first time /model is opened in a session.

Warm that exact path off-thread during the idle window right after the CLI
banner is shown: a once-per-process daemon thread runs
list_authenticated_providers() to populate provider_models_cache.json for
every authed provider. By the time the user types /model, the picker hits
the warm disk cache (~136ms vs ~1500ms).

Process-level Event guard (mirrors run_agent's _openrouter_prewarm_done)
ensures at most one thread per process; fully exception-isolated so an
offline/no-creds provider can never affect the session.
This commit is contained in:
Teknium 2026-06-05 06:55:09 -07:00 • committed by GitHub
parent ca1fb32c26
commit 9ca11b35d5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 126 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
cli.py
View file

@ -13094,6 +13094,16 @@ class HermesCLI:
_welcome_color = "#FFF8DC"
self._console_print(f"[{_welcome_color}]{_welcome_text}[/]")
# Warm the /model picker's provider-models cache off-thread during this
# idle window (banner shown, user about to type). The no-args picker
# otherwise blocks ~1-2s on serial /v1/models fetches the first time
# it's opened in a session. Fire-and-forget, guarded once-per-process.
try:
from hermes_cli.model_switch import prewarm_picker_cache_async
prewarm_picker_cache_async()
except Exception:
pass
# Redaction opt-out warning (#17691): ON by default, loud when off.
# The redactor snapshots its state at import time so any toggle now
# won't affect the running process — we just want the operator to