@janrenz's PR #35862 added prompt_caching.enabled=false at init only. But
_anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model
switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers),
which re-enabled markers and re-broke the strict proxy the toggle was meant to fix.
Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False)
on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance
hardening on the cache_ttl read). Add policy-level tests + docs for the toggle.
Follow-up to salvaged PR #35862.
Adds moa.save_traces (default off). When on, every MoA turn that runs the
reference fan-out appends one JSON line to
<hermes_home>/moa-traces/<session_id>.jsonl capturing the TRUE FULL turn:
each reference model's exact input messages (system advisory prompt + full
advisory view, not the truncated display preview) + full output + usage +
per-advisor cost, and the aggregator's exact input (including the injected
reference-context guidance block) + output. Lets MoA runs be audited and
improved offline — what every model saw, said, and cost.
- agent/moa_trace.py: config-gated JSONL writer, profile-aware path via
get_hermes_home(), best-effort (never breaks a turn), moa.trace_dir override.
- agent/moa_loop.py: _RefAccounting now carries full input/output/model/
provider/temperature; create() stashes the full turn on a cache MISS
(once per turn, never on the cache-HIT repeat iterations); non-streaming
aggregator output captured inline, streaming marked + pointed at the
session assistant message. consume_and_save_trace(session_id) flushes it.
- agent/conversation_loop.py: flushes the trace with the live session_id
right after MoA usage consumption. No-op for non-MoA clients.
- hermes_cli/config.py: moa.save_traces + moa.trace_dir defaults.
Traces are a side channel — NOT the messages table, never in replay, safe
to delete. Off by default; only overhead when off is one config read on a
MoA cache-MISS turn.
Tests: full-trace-when-enabled (per-ref input+output+cost, aggregator
input-with-guidance + output), nothing-when-disabled. Live E2E through
run_conversation confirmed the loop wiring writes the file.
MoA ran the reference models before the aggregator but returned only the
aggregator's usage to the loop — _run_reference discarded each advisor
response's .usage entirely. Session accounting (state.db, /insights, cost)
therefore undercounted every MoA turn by the whole reference fan-out, which
is usually the bulk of the spend and scales with advisor count.
- _run_reference normalizes each advisor's usage with ITS OWN resolved
provider/api_mode and prices it at ITS OWN model rate (correct cache-read/
cache-write split), returning a _RefAccounting(usage, cost).
- create() sums advisor usage + cost once per turn (cache MISS only, so a
repeat tool-iteration reusing cached advice does not double-charge) and
exposes it via MoAClient.consume_reference_usage().
- conversation_loop folds advisor tokens into the reported/persisted token
counts and adds advisor cost (priced per-advisor) on top of the
aggregator cost, in both the in-memory session totals and the state.db
per-call delta. Aggregator cost is still priced on aggregator-only usage
so advisor tokens are never repriced at the aggregator rate.
- CanonicalUsage gains __add__ for per-bucket summing.
Tests: advisor usage/cost capture, per-turn sum + consume-clears +
cache-hit no-double-charge, CanonicalUsage.__add__.
_slot_runtime maintained a hand-listed name-preservation set
({nous, anthropic, openai-codex, xai-oauth, bedrock}) that returned bare
provider+model to avoid call_llm collapsing an explicit base_url to the generic
'custom' route. That duplicated _resolve_task_provider_model's
_preserve_provider_with_base_url guard (a provider-catalog capability check)
and had to be extended by hand for every provider with custom auth/signing —
the exact drift that produced the anthropic (#54609) and bedrock (#54912) 429/
empty-response bugs.
Removes the whitelist: _slot_runtime now forwards the resolved base_url/api_key/
api_mode for every slot, and the single chokepoint
(_resolve_task_provider_model -> _preserve_provider_with_base_url) decides
identity preservation. Behavior is unchanged for the five providers — their
provider branches (codex Responses+Cloudflare, xai-oauth, bedrock SigV4,
anthropic OAuth Bearer+anthropic-beta, nous Portal tags) re-resolve their own
credentials by name and ignore a forwarded base_url/api_key, so forwarding is
safe even for bedrock's placeholder 'aws-sdk' key.
Verified via real-import E2E: _slot_runtime -> _resolve_task_provider_model
preserves openai-codex/xai-oauth/bedrock/anthropic/nous (+openrouter control) —
none collapse to custom. Tests updated to assert the pipeline invariant against
the real resolver instead of the removed whitelist's bare-return shape.
#54609 moves anthropic into the _slot_runtime name-preservation set (it must
NOT forward base_url/api_key — OAuth sk-ant-oat* needs the provider branch's
Bearer + anthropic-beta header). The pre-existing parametrized
test_moa_provider_backed_slot_survives_aux_resolution still listed anthropic
asserting the forward path, contradicting the new behavior. anthropic is now
covered by test_slot_runtime_anthropic_oauth_routes_through_provider_branch;
drop it from the forward-path parametrize (minimax-oauth/qwen-oauth remain).
MoA's _slot_runtime() whitelists providers that must keep their provider
identity (so call_llm runs their provider branch) instead of being treated
as a plain custom endpoint via forwarded base_url/api_key. Native anthropic
was missing from this set.
Native anthropic subscription OAuth setup-tokens (sk-ant-oat*) require Bearer
auth plus the 'anthropic-beta: oauth-*' header, which only the anthropic
provider branch adds. Without the whitelist entry, the slot's base_url/api_key
were forwarded and call_llm sent the OAuth token as x-api-key, which Anthropic
rejects with a bare 429 (rate_limit_error with no quota details). This made
anthropic references in MoA presets fail every time.
Add 'anthropic' to the whitelist so native anthropic reference/aggregator
slots route through the provider branch. Extends upstream 9229d0db1 which
added 'nous' for the same reason.
When the summary LLM hits a 429/transient failure, _generate_summary() sets
a cooldown and returns None; compress() inserts a static fallback marker and
returns. Tokens stay above threshold, so should_compress() kept returning
True and every subsequent agent turn re-fired _compress_context() — the CLI
appeared frozen until the cooldown expired.
Add a cooldown guard to should_compress(): return False while
_summary_failure_cooldown_until is in the future. Reuses the existing float;
no new state. Manual /compress (force=True) still clears the cooldown first.
Fixes#11529
Generic provider:custom relays were force-routed to the OpenAI Responses
API whenever the model matched gpt-5*, and a stale persisted
model.api_mode=codex_responses survived /reset and upgrades. Some
OpenAI-compatible relays do not implement Responses semantics, which
surfaced as malformed function_call.name replay errors in gateway sessions.
- runtime_provider: route custom-provider api_mode through
_resolve_plain_custom_api_mode(), which drops a stale codex_responses
unless the URL is direct OpenAI/xAI
- run_agent: _provider_model_requires_responses_api returns False for
custom; direct api.openai.com / api.x.ai URLs still upgrade via
_is_direct_openai_url() / URL detection
- regression coverage for plain relays vs direct OpenAI/xAI URLs
Co-authored-by: HiddenPuppy <HiddenPuppy@users.noreply.github.com>
* fix(agent): drop tool_calls with empty function.name to prevent orphan 400
Salvage of #12807 by @melonboy312 — rebased onto current main (sanitizer
moved to agent_runtime_helpers), scoped to the sanitizer fix, with a
regression test that fails without it.
* fix(agent): repair (not drop) empty-name tool_calls to preserve anti-priming + prevent 400
Dropping empty-name tool_calls in the pre-call sanitizer collided with #47967,
which intentionally keeps an empty-name call paired with a synthesized
'tool name was empty' anti-priming result so weak models self-correct without
a full catalog dump. Dropping the call orphaned that result and stripped the
signal (breaking tests/agent/test_empty_tool_name_loop_dampening.py).
The actual HTTP 400 cause is an ORPHANED function_call_output (adapter drops
the empty-name function_call but keeps its output). Rename the blank name to a
non-empty sentinel instead: the call and its result stay paired, the adapter
no longer drops the function_call, no orphan, no 400 — and the anti-priming
result content the model needs is preserved.
---------
Co-authored-by: Bartok9 <danielrpike9@gmail.com>
background_review hardcoded enabled_toolsets=["memory", "skills"] in the
review fork's whitelist, so a skill-review fork on a profile with
memory_enabled: false still granted the LLM the built-in MEMORY.md read/write
tool — contaminating a profile that opted out of built-in memory. The flag was
already in scope (review_agent._memory_enabled). Include "memory" only when
_memory_enabled or _user_profile_enabled (USER.md also needs the tool).
Layer 1 of #54937 (the path leak) is fixed by this PR's thread-context
propagation: get_memory_dir() is already per-call on main, so once the
bg-review thread inherits the profile override its writes land in the right
profile (verified). This commit closes the remaining whitelist layer.
MoA sessions could not stream: the gateway streaming toggle was a no-op for
provider "moa", so users saw nothing until the entire response finished — minutes
of silence on long turns. The aggregator's reply was always fetched whole.
Root cause was twofold:
1. conversation_loop hard-disabled streaming for provider in {"copilot-acp",
"moa"} (MoA grouped with the ACP client, whose facade isn't a stream).
2. MoAChatCompletions.create() fetched the aggregator response whole via
call_llm(), which had no streaming mode.
For provider "moa", _create_request_openai_client() returns the MoAClient facade
itself, so the existing streaming consumer already calls
MoAChatCompletions.create(stream=True). We reuse that battle-tested consumer
(text-delta delivery, tool_call reassembly, stale-stream detection, non-streaming
fallback) instead of adding a parallel streaming path.
Changes:
- call_llm() gains stream/stream_options. When streaming it returns the raw SDK
stream iterator directly, bypassing _validate_llm_response and the
temperature/max_tokens/payment fallback chain (which assume a complete
response). The caller owns reassembly and fallback.
- MoAChatCompletions.create() runs the references first (unchanged), then when
stream=True returns the aggregator's raw stream, forwarding stream_options and
the consumer's per-request read timeout. stream=False is byte-identical to
before (no stream/stream_options/timeout forwarded).
- conversation_loop streams MoA only when a display/TTS consumer is present;
quiet/subagent/health-check paths keep the complete-response path.
Tests: tests/run_agent/test_moa_streaming.py — create() stream/non-stream
branches, stream_options + timeout forwarding, call_llm raw-stream return vs
validated non-stream. Existing MoA tests unchanged (20 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_resolve_task_provider_model() flattened any explicit base_url to
provider=custom. Correct for bare/custom endpoints, but wrong for
provider-backed routes (anthropic, qwen-oauth, minimax-oauth,
openai-codex, etc.) whose provider branch adds auth refresh, transport,
or request shaping. MoA reference slots resolved through those providers
lost their identity before the aux call, so e.g. a Codex reference hit
chatgpt.com/backend-api/codex without its Cloudflare headers and got
HTML back (surfacing as a spurious rate-limit).
Keep first-class providers intact when paired with a resolved base_url
via _preserve_provider_with_base_url(); bare/custom/auto/unknown and the
direct openai alias still route through custom.
Co-authored-by: Hermes Agent <127238744+teknium1@users.noreply.github.com>
* fix(agent): merge consecutive assistant messages in repair_message_sequence
Strict OpenAI-compatible providers (DeepSeek v4, Moonshot/Kimi) reject a
replayed history where an assistant message carrying tool_calls is
immediately followed by another assistant message instead of its tool
results — HTTP 400 'An assistant message with tool_calls must be
followed by tool messages...'.
repair_message_sequence (the defensive belt run before every API call)
fixed orphan-tool and consecutive-user shapes but never merged
consecutive assistant messages. Adds a Pass 0 that collapses adjacent
assistant turns into one — union of tool_calls, concatenated content,
carried reasoning_content — covering both reported shapes:
- parallel tool calls split across two assistant turns (#29148)
- content-only assistant followed by tool_calls-only assistant (#49147)
A tool result or user turn between two assistants blocks the merge
(distinct, valid rounds). Runs before Pass 1 so the merged union of
tool_call ids is known to the orphan-tool filter.
Closes#29148, #49147.
Co-authored-by: Bartok9 <danielrpike9@gmail.com>
Co-authored-by: woaini30050 <woaini30050@users.noreply.github.com>
Co-authored-by: weidzhou <weidzhou@users.noreply.github.com>
* fix(agent): exempt codex Responses interim turns from assistant merge
The Pass 0 consecutive-assistant merge collapsed codex_responses interim
turns, which legitimately stay separate — each carries its own encrypted
continuation state (codex_reasoning_items / codex_message_items) that
must replay verbatim. Skip the merge when either side is a codex interim
(has codex_reasoning_items / codex_message_items / finish_reason=='incomplete').
Fixes the slice-2 regression in test_run_agent_codex_responses.py
(test_duplicate_detection_distinguishes_different_codex_{reasoning,message_items}).
---------
Co-authored-by: Bartok9 <danielrpike9@gmail.com>
Co-authored-by: woaini30050 <woaini30050@users.noreply.github.com>
Co-authored-by: weidzhou <weidzhou@users.noreply.github.com>
The earlier enterprise base URL change (proxy-ep parsing) gave us URLs
like `api.enterprise.githubcopilot.com`, but ~15 host-matching call
sites still hard-coded `api.githubcopilot.com`. Enterprise users would
therefore drop the `Copilot-Integration-Id: vscode-chat` header at
client-build time, and upstream rejected requests with:
The requested model is not available for integrator "zed"
(or "copilot-language-server") — verify the correct
Copilot-Integration-Id header is being sent.
The header was correct in copilot_default_headers(); it just never
made it into default_headers for non-default hostnames because every
detector compared against the exact string "api.githubcopilot.com".
This commit broadens all those checks to "githubcopilot.com" via
base_url_host_matches (which already does proper subdomain matching),
so api.enterprise.githubcopilot.com, api.business.githubcopilot.com,
etc. all share the same headers, vision routing, max_completion_tokens
selection, and reasoning-effort detection as the default endpoint.
Also adds ".githubcopilot.com" to _URL_TO_PROVIDER so context-window
resolution via models.dev works for enterprise base URLs, and tightens
_is_github_copilot_url to use suffix matching instead of strict equality.
Tests:
- New: enterprise Copilot endpoint preserves Copilot-Integration-Id
- New: enterprise endpoint returns max_completion_tokens (not max_tokens)
- Existing 333 base_url / copilot / aux-client / credential-pool tests pass
Parts 5 of #7731.
_sanitize_api_messages() compared raw tool_call_id strings without
stripping whitespace. When assistant-side IDs and tool-result IDs
diverged due to surrounding whitespace, valid tool results were treated
as orphaned and replaced with [Result unavailable] stub placeholders.
Strip whitespace in _get_tool_call_id_static() (both call_id/id paths,
dict and object) and at the two result_call_id comparison sites in
sanitize_api_messages(). Adds regression tests for preserved-whitespace
results and orphaned-whitespace removal.
Closes#9999
Defense-in-depth on top of _safe_session_filename_component (#5958):
Sink (makes the bad write impossible regardless of entry point):
- run_agent._save_session_log: sanitize session_id before building the
session_{sid}.json snapshot path.
- agent_runtime_helpers.dump_api_request_debug: sanitize before building
the request_dump_{sid}_{ts}.json path.
Boundary (clean 400 instead of a silently-hashed filename):
- api_server rejects path-traversal-shaped X-Hermes-Session-Id on the
session-continuation path and the explicit /api/sessions create path,
reusing gateway.session._is_path_unsafe (mirrors the native gateway's
entry-boundary guard). Also enforces the session-header length cap on
the continuation path.
Tests: traversal session_id stays contained at the write site; sanitizer
always yields a traversal-free segment; the API header rejects
../, absolute, and Windows-traversal IDs with 400.
Google's native Gemini REST endpoint (generativelanguage.googleapis.com,
non-/openai) rejects OpenAI-only stream_options={"include_usage": true},
crashing every streaming chat-completions call with TypeError. Omit it for
that endpoint while keeping it for the Gemini OpenAI-compat shim and all
OpenAI-compatible aggregators (OpenRouter, etc.) so usage accounting is
preserved.
Reuses is_native_gemini_base_url() so the compat shim (.../openai), which
accepts stream_options, is correctly excluded from the omission.
Fixes#14387
Co-authored-by: Hermes Agent <127238744+teknium1@users.noreply.github.com>
When an LLM API call returns HTTP 4xx with an empty parsed SDK `body` ({}),
`_summarize_api_error` fell through to a bare `str(error)`, so users saw only
"HTTP 400" with no provider detail (reported on Windows in #36109). The SDK
leaves `body` empty in this case, but the httpx `response` still carries the
payload in `.text`.
- run_agent.py `_summarize_api_error`: when `body` is empty, fall back to
`response.text` — parse a JSON `error.message`/`message` when present, else
surface the raw (truncated) body. Platform-agnostic diagnostics.
- hermes_cli/oneshot.py: `hermes -z` now runs via `run_conversation` and returns
exit code 2 when the run is failed/partial with no usable final response, so
scripts can detect LLM failures (still 0 when a response — incl. an error
summary as output — is produced).
Tests: new tests/run_agent/test_summarize_api_error.py (empty-body JSON + raw
text, RED/GREEN verified) + oneshot exit-code/`run_conversation` wiring tests.
NOTE: #36109's original root cause (Windows "all providers return empty 400")
is not reproducible on current main (heavy provider-transport churn since
v0.15.1). This change does not claim to fix that root cause — it makes any
empty-body API error LEGIBLE so a future occurrence shows the real provider
message instead of a bare HTTP 400. Relates to #36109 (does not close it).
When a provider's output-layer safety filter (MiniMax "output new_sensitive
(1027)", Azure content_filter, etc.) kills a streaming response after deltas
were already sent, interruptible_streaming_api_call swallows the raw error
into a finish_reason=length partial-stream stub. The conversation loop then
burned 3 continuation retries against the SAME primary — re-hitting the
content-deterministic filter every time — and gave up with "Response remained
truncated after 3 continuation attempts", never consulting fallback_providers.
Builds on @595650661's classifier change (cherry-picked) so error_classifier
recognizes the filter; then:
- chat_completion_helpers: run the swallowed error through error_classifier at
the stub-creation point and stamp _content_filter_terminated on the stub
(single source of truth — no parallel pattern list).
- conversation_loop: read the tag and activate the fallback chain BEFORE
burning any continuation retries; roll partial content back to the last
clean turn and re-issue against the new provider (restart_with_rebuilt_messages).
Plain network stalls are unaffected (only content_policy_blocked is tagged).
Credits #32479 (@sweetcornna) and #33845 (@Tranquil-Flow) which fixed the
same issue via the stub-tag and loop-escalation approaches respectively.
Live E2E confirmed: before, _try_activate_fallback called 0x; after, fallback
fires on the first stub and the fallback provider completes the turn.
The MiniMax output-layer safety filter surfaces the error verbatim as
`output new_sensitive (1027)` (sometimes with additional provider
wrapping like 'Stream stalled mid tool-call: output new_sensitive (1027)').
When the model emits a large tool-call argument block, the upstream
filter trips and the SSE stream is truncated mid-flight, producing
'stream stalled mid tool-call' errors. Until now this case was
misclassified and retried 3x on the same provider, reproducing the same
refusal and burning paid attempts.
Adding `new_sensitive` to `_CONTENT_POLICY_BLOCKED_PATTERNS` routes
it through the existing is_client_error path: skip 3x retry, activate
configured fallback model immediately, surface a clear provider-safety
message to the user.
Refs #32421
The advisory reference view stripped all tool calls and tool results, so
reference models judged a task whose actions and results they never saw — and
references only fired once per user turn, never re-running as the agent's
state advanced through the tool loop.
Two fixes:
- _reference_messages() now PRESERVES the agent's tool calls and tool results,
rendering them inline as text ([called tool: ...] / [tool result: ...]) so a
reference gives an informed judgement on the real current state. Still emits
zero tool-role messages and zero tool_calls arrays (strict providers reject
those), and large tool results are previewed head+tail (4000-char budget).
The required end-on-user shape is met by APPENDING a synthetic advisory user
turn — not by deleting the agent's latest context (which the prior fix did).
- References now re-run on every state change — each new user message AND each
new tool result — instead of once per user turn. The state-sensitive advisory
signature drives the cache: new tool result = miss (re-run), identical-state
re-call = hit (no re-run, no re-emit).
The acting aggregator still receives the full, untrimmed transcript.
* fix(moa): reference advisory view must end with a user turn
MoA reference calls failed with Anthropic models that don't support
assistant prefill (e.g. Claude Opus 4.8): '400 ... must end with a user
message'. The advisory view built by _reference_messages() kept the last
assistant turn's text while dropping the following tool result, leaving a
trailing assistant turn — which Anthropic (and OpenRouter->Anthropic)
interpret as an assistant prefill to continue. References are advisory and
must end on the user turn they answer.
Strip trailing assistant turns from the advisory view (preserving
intervening ones). Update the existing test that encoded the buggy shape
and add a mid-tool-loop regression test.
* feat(moa): give reference models an advisory-role system prompt
Reference models received the bare trimmed conversation with no role
framing, so they assumed they were the acting agent and refused ("I can't
access repositories/URLs from here") or tried to call tools they don't have.
Prepend a dedicated advisory system prompt to every reference call: the
model is an analyst, not the actor — it cannot execute, should not
apologize for lacking tools, and should reason about the presented state to
advise the aggregator/orchestrator on approach, next steps, tool-use
strategy, risks, and anything the acting agent missed. Its output is private
guidance for the aggregator, not a user-facing answer.
The exhaustion-cooldown timing assertions relied on a wall-clock budget
(before + window + 1.0s). On loaded CI runners the activation calls could
exceed the 1s slack, flaking 'Run tests slice 4/8'. Freeze
chat_completion_helpers.time.monotonic so the cooldown math is exact and
load-independent across all four tests.
Three CI flakes hit while landing the credential-pool restore fix; all three
were timing/wall-clock races in the tests, not product bugs (each passes
locally and the assertions are correct):
- test_entire_tree_is_sigkilled_not_just_parent: _terminate_host_pid SIGKILLs
synchronously, but the test's 4s budget after a 1s in-function SIGTERM grace
left almost no slack for the kernel to tear down 3 processes + reparent the
children to zombies under loaded-CI scheduling. Widen the wait to 15s and
make the liveness predicate tolerant of vanished-pid / zombie races. The
assertion never weakens: every tree member must end up dead or zombie.
- test_session_resume_follows_compression_tip: appended messages got
time.time() timestamps (~now) while the test forced session started_at into
the past, so the get_compression_tip MAX(m.timestamp) tiebreaker depended on
wall-clock ordering. Pass explicit, well-separated message timestamps so the
chain resolution is deterministic by construction.
- test_non_retryable_exhaustion_arms_cooldown: asserted the short (5s)
exhaustion cooldown with a tight +1.0s slack, which false-fails when
wall-clock jitter between the 'before' snapshot and the cooldown computation
exceeds a second on a loaded runner. Widen to +30s — still cleanly below the
60s rate-limit window it must distinguish from.
The existing test_anthropic_stream_parser_valueerror_retries_before_delivery
asserted mock_replace.call_count == 1 — i.e. it passed precisely because the
buggy OpenAI rebuild was invoked on the Anthropic path. Repoint it to assert
the corrected close+rebuild-Anthropic behavior (#28161).
interruptible_streaming_api_call() has three connection-pool cleanup
sites that called _replace_primary_openai_client() unconditionally.
For api_mode=anthropic_messages this has two consequences:
1. _replace_primary_openai_client() fails (OPENAI_API_KEY unset on
Anthropic-only configs), so dead connections are never purged.
2. The stale-stream detector's outer-poll site (L1977) is the only
mechanism that can interrupt the worker thread while it blocks in
for event in stream:. Because the Anthropic client is never closed,
the thread stays blocked until the 900 s httpx read-timeout fires,
producing a visible 15-minute hang for Telegram/gateway users on
claude-opus-4-7.
Fix: mirror the existing interrupt-path pattern (L1989-1997) at all
three cleanup sites — if api_mode == "anthropic_messages", call
_anthropic_client.close() + _rebuild_anthropic_client() instead of
_replace_primary_openai_client(). _rebuild_anthropic_client() handles
both direct Anthropic and Bedrock-hosted Claude correctly, unlike the
inline build_anthropic_client() calls in open PR #14430.
PR #14430 (open) covers only the outer stale-detector site (L1977).
PR #23678 (open) covers only the inner retry sites (L1774, L1833).
This PR covers all three sites and uses _rebuild_anthropic_client()
for Bedrock parity.
Fixes#28161
Same bug class as the Anthropic fix (#26293): the OpenAI/aggregator client is
built without max_retries, so the SDK default of 2 applies. The SDK's own 1-2s
backoff ignores Retry-After and retries inside hermes's outer conversation loop,
burning request slots against a rate-limited bucket. Set max_retries=0 at the
single create_openai_client chokepoint (covers init, switch_model, recovery,
restore, request-scoped). auxiliary_client builds its own clients and is not
wrapped by the loop, so it keeps SDK retries.
The Anthropic SDK clients were built without max_retries, so the SDK
default (max_retries=2) retried 429/5xx with its own backoff that ignores
Retry-After — double-retrying inside hermes's outer loop and burning
request slots against a bucket that won't refill for minutes. Set
max_retries=0 on all Anthropic/AnthropicBedrock client constructions so
the outer conversation loop (which already honors Retry-After) owns retry.
Also raise the Retry-After cap in the conversation loop from 120s to 600s.
Anthropic Tier 1 input-token buckets reset in ~171s, so the 120s cap made
hermes retry before the reset window and re-trip the limit.
Refs #26293
A persistent upstream 401 on a single-entry OAuth pool (common for Claude
Max subscribers) made the credential-pool recovery spin forever:
try_refresh_current() re-mints a fresh token and reports success on every
401, so recover_with_credential_pool returned True and the retry loop
continue'd without ever incrementing retry_count or reaching the
auth-failover block. The configured fallback_model never activated and the
agent appeared to hang.
Cap consecutive successful same-entry refreshes (keyed by provider +
pool-entry id) at 2; once exceeded, treat the credential as unrecoverable
and return not-recovered so the loop falls through to
_try_activate_fallback. The 429/billing paths already rotate-or-fall-through
correctly (mark_exhausted_and_rotate returns None on a single entry), so
only the auth-refresh branch needed the cap.
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
When every provider in the fallback chain fails non-retryably back-to-back
(e.g. HTTP 400/402/429 across distinct providers), the within-turn walk is
already bounded — _fallback_index advances monotonically and the loop aborts
when the chain exhausts. The damaging mode is cross-turn: restore_primary_
runtime resets _fallback_index=0 every turn, so a client that re-submits
immediately replays the entire chain, re-marshaling the full (potentially
80k-token) context once per provider every turn with no throttle on the
non-rate-limit path. On constrained hosts this exhausts memory/swap.
Rate-limit/billing failures already arm a 60s cooldown via _rate_limited_until;
the gap was the non-rate-limit case. Now, when the chain exhausts on a non-
rate-limit failure with a non-empty chain, arm a short (5s) cooldown on the
same _rate_limited_until gate (max(), never shrinking an existing window).
The next turn's restore stays gated and does NOT reset the index, so the
chain isn't replayed until the cooldown clears. No new state, no thread sleep,
no false-trip on legitimately long chains (those walk normally within a turn).
Tests: tests/run_agent/test_24996_fallback_exhaustion_cooldown.py
The agent's image-rejection fallback strips images and retries text-only when
a provider rejects image content, which is what lets the gateway drain its
queued messages. The fallback only fires on a hardcoded phrase list, and the
OpenRouter wording — HTTP 404 'No endpoints found that support image input' —
was missing. For OpenRouter-routed non-vision models the fallback never fired,
the retry loop re-sent the same rejected request until exhaustion, and every
subsequent message (including plain text) stayed queued behind the stuck turn.
Add the phrase to _IMAGE_REJECTION_PHRASES (the 404 already passes the 4xx
gate). Add a positive test and a guard test so the sibling OpenRouter
'no endpoints ... data policy / guardrail' 404s do NOT get their images
stripped.
Fixes#21160. Reported by @liu14goal14-ux; PR #21198 by @ygd58.
When a MoA preset is selected, each reference model's answer now renders in the
CLI as a thinking-style block labelled with its source model, BEFORE the
aggregator responds — so the mixture-of-agents process is visible instead of a
silent pause. The aggregator's response (and its tool actions) follow as normal.
Mechanism (shared seam, all surfaces):
- MoAChatCompletions/MoAClient take an optional reference_callback and emit
'moa.reference' (index/count/label/text) per reference, then 'moa.aggregating'
(aggregator label) once. agent_init wires this to the agent's
tool_progress_callback, which every surface already consumes — so the events
reach CLI/TUI/desktop/gateway with no new plumbing.
- CLI _on_tool_progress renders 'moa.reference' as a labelled '┊ ◇ Reference
i/n — <model>' header + a thinking-style preview (reusing _emit_reasoning_
preview), and 'moa.aggregating' as a spinner transition. Display-only; never
touches message history (cache-safe).
Turn-scoped reference cache: the agent loop calls the facade once per tool-loop
iteration, but the advisory message view is identical across iterations within a
turn, so references are now run AND displayed once per user turn (keyed by the
advisory view's signature) instead of re-running/re-spamming on every iteration.
This also cuts reference API cost from O(iterations) back to O(turns).
Verified live via interactive PTY on the opus-gpt preset (gpt-5.5 + opus refs):
reference blocks render once per turn, labelled by model, before the aggregator;
fresh blocks on each new turn; aggregator tool actions still execute.
Follow-up: TUI/desktop rich rendering + gateway batched-summary already receive
the events via tool_progress_callback; their surface-specific renderers are a
separate change.
MoA was calling reference and aggregator models through a bare
call_llm(provider=slot["provider"], model=slot["model"]) with a forced
temperature and a forced max_tokens (the preset's hardcoded 4096). That left
base_url/api_key/api_mode unresolved — so the auxiliary auto-detector guessed
the API surface instead of using the provider's real runtime, and the 4096 cap
truncated long aggregator syntheses.
A MoA slot is just a model selection and must be called the same way any model
is called elsewhere. Each slot is now resolved through resolve_runtime_provider
(the canonical provider→api_mode/base_url/api_key resolver the CLI, gateway, and
delegate_task all use) via a new _slot_runtime() helper, and the resolved
endpoint is passed into call_llm. So a reference/aggregator gets its provider's
actual API surface — MiniMax → anthropic_messages, GPT-5/o-series →
max_completion_tokens, custom endpoints → their base_url — identical to how that
model is handled as the acting model.
MoA also no longer imposes its own output cap: max_tokens defaults to None
(omitted → the model's real maximum) for references and is passed through from
the caller for the aggregator. The preset's hardcoded 4096 is gone. The
max_tokens preset config field is left in place (config/web/desktop unchanged);
it is simply no longer applied as a forced cap.
Tests: slots route through resolve_runtime_provider with resolved base_url/
api_key; resolution errors fall back to bare provider/model; neither call
carries an output cap even when the preset config still contains max_tokens.
When automatic fallback activates a provider that differs from the
primary, try_activate_fallback() cleared the primary's pool (to avoid
cross-provider base_url contamination, #33163) but never loaded the
fallback provider's own pool. The fallback then ran with no pool, so
rate_limit/billing/auth recovery couldn't rotate its credentials.
After clearing a mismatched pool, load_pool(fb_provider) and attach it
when it has credentials, so provider-specific rotation continues to
work on the fallback target.
switch_model() swapped model/provider/base_url/api_key but never
refreshed agent._credential_pool, which stays bound to the original
provider. recover_with_credential_pool() then sees a pool.provider !=
agent.provider mismatch and short-circuits — so a 429/401 on the new
provider gets no rotation and falls through to fallback instead.
Reload load_pool(new_provider) inside switch_model when the provider
changes (or the pool is missing). The reload is inside the protected
swap block and the pool is added to the rollback snapshot, so a failed
client rebuild restores the original pool.
Fixes#16678, #52727.
Follow-up to the #13971 fix: a genuine native Ollama provider reached
through a reverse proxy carries no ollama/:11434 URL signature, so the
restricted detection would miss it. Add provider=="ollama" as an
explicit True case (idea from #14789, @Tranquil-Flow) and cover both it
and the #13971 LiteLLM-proxy-to-zai false-positive with E2E tests.
The _is_ollama_glm_backend() function was too broad: any local endpoint
running a GLM model was treated as Ollama, triggering the stop->length
misreport heuristic introduced in 8011aa3. This caused false truncation
detection on sglang, vLLM, LM Studio, and other non-Ollama servers that
correctly report finish_reason.
When a GLM model on sglang/vLLM returned finish_reason='stop', the agent
mistakenly reclassified it as 'length' if the response didn't end with
a whitelisted punctuation character (ASCII or CJK). This particularly
affected Chinese-language responses and Markdown-formatted text.
Root cause: the is_local_endpoint() fallback assumed any local GLM
endpoint = Ollama. But many non-Ollama servers also run on localhost.
Fix: remove the is_local_endpoint() catch-all. Only detect Ollama via
its distinctive signatures (port 11434, 'ollama' in URL). All other
local servers are assumed to report finish_reason correctly.
This is the correct tradeoff because:
- False negatives (Ollama at custom port, heuristic not triggered) only
mean the user sees a truncated response — same as having no heuristic
- False positives (non-Ollama server, heuristic wrongly triggered) inject
spurious continuation messages into the conversation — strictly worse
Adds two tests:
- sglang GLM response is NOT reclassified as truncated
- Ollama GLM on port 11434 still triggers the heuristic as before
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
Force redact_sensitive_text(force=True) on the browser_type text arg so
recognized credentials (API keys, tokens, JWTs) are masked in tool
progress, previews, callbacks, and return payloads even when the global
security.redact_secrets opt-out is set — a typed credential reaching chat
history is a security boundary, not log hygiene. Normal typed text matches
no pattern and stays fully readable for debuggability.
Tests assert the API-key-shaped secret is masked across every surface and
that normal text passes through unchanged.
Stopping a turn while the model is streaming (stop/esc to redirect) raised
InterruptedError, set final_response to the throwaway "waiting for model
response" sentinel, and persisted messages WITHOUT the assistant text that
was already streamed to the screen. The next turn then had no record of the
half-finished reply, so the model appeared to "forget" what it just said.
Recover the on-screen text from _current_streamed_assistant_text in the
InterruptedError branch and append it as the assistant turn (and surface it
as final_response). The metadata sentinel is kept only when nothing was
streamed yet, preserving the ACP/client suppression behavior.
Completes the partial-stream recovery from 397eae5d9 (which wired the same
_current_streamed_assistant_text salvage into the connection-failure twin
but missed the user-interrupt path). The lossy handler dates to c98ee9852.
