KrustyPlanet/hermes-agent
6
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

fix(caching): include Kimi/Moonshot in OpenRouter prompt cache policy (#25970)

Browse source 
Kimi/Moonshot models on OpenRouter honour the same envelope-layout
cache_control markers as Claude on OpenRouter, but the policy fell
through to (False, False) — serving ~1% cache hits on 64K-token prompts
and re-billing the full prompt every turn. Observed within-turn
progression with cache enabled: 1% -> 67% -> 84% -> 97%.

(cherry picked from commit 3b857a35e, agent/agent_runtime_helpers.py hunk only;
the original PR #26014 branch also carried unrelated .dev-workflow artifacts
which are intentionally not included)
This commit is contained in:
zccyman 2026-07-09 02:08:25 +05:30 • committed by kshitij
parent f556edc10d
commit 750c1310a6
1 changed files with 8 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

9
agent/agent_runtime_helpers.py
View file

@ -1564,6 +1564,13 @@ def anthropic_prompt_cache_policy(
model_lower = eff_model.lower()
provider_lower = eff_provider.lower()
is_claude = "claude" in model_lower
# Kimi / Moonshot family via OpenRouter: same cache_control wire format
# as Claude on OpenRouter (envelope layout). Without this branch
# moonshotai/kimi-k2.6 falls through to (False, False), serving ~1%
# cache hits on 64K-token prompts and re-billing the full prompt on
# every turn. Observed within-turn progression with cache enabled:
# 1% → 67% → 84% → 97% (#25970).
is_kimi = "kimi" in model_lower or "moonshot" in model_lower
is_openrouter = base_url_host_matches(eff_base_url, "openrouter.ai")
# Nous Portal proxies to OpenRouter behind the scenes — identical
# OpenAI-wire envelope cache_control semantics. Treat it as an
@ -1577,7 +1584,7 @@ def anthropic_prompt_cache_policy(
if is_native_anthropic:
return True, True
if (is_openrouter or is_nous_portal) and is_claude:
if (is_openrouter or is_nous_portal) and (is_claude or is_kimi):
return True, False
# Nous Portal Qwen (e.g. qwen3.6-plus) takes the same envelope-layout
# cache_control path as Portal Claude. Portal proxies to OpenRouter