KrustyPlanet/hermes-agent
6
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
hermes-agent/tests
History
Teknium 57625329a2
docs+feat: comprehensive local LLM provider guides and context length warning (#4294) 
* docs: update llama.cpp section with --jinja flag and tool calling guide

The llama.cpp docs were missing the --jinja flag which is required for
tool calling to work. Without it, models output tool calls as raw JSON
text instead of structured API responses, making Hermes unable to
execute them.

Changes:
- Add --jinja and -fa flags to the server startup example
- Replace deprecated env vars (OPENAI_BASE_URL, LLM_MODEL) with
  hermes model interactive setup
- Add caution block explaining the --jinja requirement and symptoms
- List models with native tool calling support
- Add /props endpoint verification tip

* docs+feat: comprehensive local LLM provider guides and context length warning

Docs (providers.md):
- Rewrote Ollama section with context length warning (defaults to 4k on
  <24GB VRAM), three methods to increase it, and verification steps
- Rewrote vLLM section with --max-model-len, tool calling flags
  (--enable-auto-tool-choice, --tool-call-parser), and context guidance
- Rewrote SGLang section with --context-length, --tool-call-parser,
  and warning about 128-token default max output
- Added LM Studio section (port 1234, context length defaults to 2048,
  tool calling since 0.3.6)
- Added llama.cpp context length flag (-c) and GPU offload (-ngl)
- Added Troubleshooting Local Models section covering:
  - Tool calls appearing as text (with per-server fix table)
  - Silent context truncation and diagnosis commands
  - Low detected context at startup
  - Truncated responses
- Replaced all deprecated env vars (OPENAI_BASE_URL, LLM_MODEL) with
  hermes model interactive setup and config.yaml examples
- Added deprecation warning for legacy env vars in General Setup

Code (cli.py):
- Added context length warning in show_banner() when detected context
  is <= 8192 tokens, with server-specific fix hints:
  - Ollama (port 11434): suggests OLLAMA_CONTEXT_LENGTH env var
  - LM Studio (port 1234): suggests model settings adjustment
  - Other servers: suggests config.yaml override

Tests:
- 9 new tests covering warning thresholds, server-specific hints,
  and no-warning cases
2026-03-31 11:42:48 -07:00
..
acp fix(acp): complete session management surface for editor clients (salvage #3501) (#3675) 2026-03-28 23:45:53 -07:00
agent fix: patch _REDACT_ENABLED in test fixture for module-level snapshot 2026-03-31 10:30:48 -07:00
cron fix(cron): resolve human-friendly delivery labels via channel directory (#3860) 2026-03-29 21:24:17 -07:00
fakes fix: streaming tool call parsing, error handling, and fake HA state mutation 2026-03-14 14:27:20 +03:00
gateway feat: support * wildcard in platform allowlists and improve WhatsApp docs 2026-03-31 10:42:03 -07:00
hermes_cli fix(cli): allow empty strings and falsy values in config set 2026-03-31 11:41:12 -07:00
honcho_integration fix(honcho): write config to instance-local path for profile isolation (#4037) 2026-03-30 16:41:19 -07:00
integration refactor: remove mini-swe-agent dependency — inline Docker/Modal backends (#2804) 2026-03-24 07:30:25 -07:00
skills feat(skills): add memento-flashcards optional skill (#3827) 2026-03-29 16:52:52 -07:00
tools fix(browser): skip SSRF check for local backends (Camofox, headless Chromium) (#4292) 2026-03-31 10:40:13 -07:00
__init__.py A bit of restructuring for simplicity and organization 2025-10-01 23:29:25 +00:00
conftest.py fix(approval): show full command in dangerous command approval (#1553) 2026-03-17 02:02:33 -07:00
run_interrupt_test.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_413_compression.py feat: improve context compaction handoff summaries (#1273) 2026-03-14 02:33:31 -07:00
test_860_dedup.py fix: eliminate 3x SQLite message duplication in gateway sessions (#860) 2026-03-10 15:22:44 -07:00
test_1630_context_overflow_loop.py fix: prevent infinite 400 loop on context overflow + block prompt injection via cache files (#1630, #1558) 2026-03-17 01:50:59 -07:00
test_agent_guardrails.py feat: pre-call sanitization and post-call tool guardrails (#1732) 2026-03-17 04:24:27 -07:00
test_agent_loop.py fix: salvage gateway dedup and executor cleanup from PR #993 2026-03-14 11:03:20 -07:00
test_agent_loop_tool_calling.py fix: skip hanging tests + add global test timeout 2026-03-12 01:23:28 -07:00
test_agent_loop_vllm.py test: restore vllm integration coverage and add dict-args regression 2026-03-15 08:02:29 -07:00
test_anthropic_adapter.py fix(auth): use bearer auth for MiniMax Anthropic endpoints (#4028) 2026-03-30 13:19:44 -07:00
test_anthropic_error_handling.py fix(ci): pin acp <0.9 and update retry-exhaust test (#3320) 2026-03-26 19:21:34 -07:00
test_anthropic_oauth_flow.py fix: preflight Anthropic auth and prefer Claude store 2026-03-14 19:38:55 -07:00
test_anthropic_provider_persistence.py fix: preflight Anthropic auth and prefer Claude store 2026-03-14 19:38:55 -07:00
test_api_key_providers.py fix: gate Claude Code credentials behind explicit Hermes config in wizard trigger (#4210) 2026-03-31 02:01:15 -07:00
test_async_httpx_del_neuter.py fix: eliminate 'Event loop is closed' / 'Press ENTER to continue' during idle (#3398) 2026-03-27 09:45:25 -07:00
test_atomic_json_write.py test: cover atomic temp cleanup on interrupts 2026-03-14 22:31:51 -07:00
test_atomic_yaml_write.py test: cover atomic temp cleanup on interrupts 2026-03-14 22:31:51 -07:00
test_auth_codex_provider.py refactor(auth): transition Codex OAuth tokens to Hermes auth store 2026-03-01 19:59:24 -08:00
test_auth_commands.py feat(auth): same-provider credential pools with rotation, custom endpoint support, and interactive CLI (#2647) 2026-03-31 03:10:01 -07:00
test_auth_nous_provider.py Fix nous refresh token rotation failure in case where api key mint/retrieval fails 2026-03-02 17:18:15 +11:00
test_auxiliary_config_bridge.py feat(compression): add summary_base_url + move compression config to YAML-only 2026-03-17 04:46:15 -07:00
test_batch_runner_checkpoint.py fix: sanitize chat payloads and provider precedence 2026-03-13 23:59:12 -07:00
test_cli_approval_ui.py fix(cli): repair dangerous command approval UI 2026-03-14 11:57:44 -07:00
test_cli_background_tui_refresh.py fix(cli): refresh TUI before background task output to prevent status bar overlap (#3048) 2026-03-25 15:00:33 -07:00
test_cli_context_warning.py docs+feat: comprehensive local LLM provider guides and context length warning (#4294) 2026-03-31 11:42:48 -07:00
test_cli_extension_hooks.py refactor(cli): add protected TUI extension hooks for wrapper CLIs 2026-03-21 09:42:07 -07:00
test_cli_init.py feat(cli): configurable busy input mode + fix /queue always working (#3298) 2026-03-26 17:58:40 -07:00
test_cli_interrupt_subagent.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_cli_loading_indicator.py fix(cli): add loading indicators for slow slash commands 2026-03-10 17:31:00 -07:00
test_cli_mcp_config_watch.py fix: auto-reload MCP tools when mcp_servers config changes without restart (#1474) 2026-03-15 19:03:34 -07:00
test_cli_new_session.py fix: complete session reset — missing compressor counters + test 2026-03-20 04:35:17 -07:00
test_cli_plan_command.py fix: save /plan output in workspace (#1381) 2026-03-14 21:28:51 -07:00
test_cli_prefix_matching.py feat: add /tools disable/enable/list slash commands with session reset (#1652) 2026-03-17 02:05:26 -07:00
test_cli_preloaded_skills.py fix: move activated skills line below welcome text 2026-03-23 06:20:19 -07:00
test_cli_provider_resolution.py feat: auto-detect models from server probe in custom endpoint setup (#4218) 2026-03-31 03:29:00 -07:00
test_cli_retry.py test: lock retry replacement semantics 2026-03-14 21:19:22 -07:00
test_cli_secret_capture.py feat: secure skill env setup on load (core #688) 2026-03-13 03:14:04 -07:00
test_cli_skin_integration.py fix(test): add missing voice state attrs to CLI stub in skin tests 2026-03-14 15:00:45 +03:00
test_cli_status_bar.py fix(cli): prevent status bar wrapping into duplicate rows (#3883) 2026-03-29 23:59:07 -07:00
test_cli_tools_command.py fix: resolve 7 failing CI tests (#3936) 2026-03-30 08:10:14 -07:00
test_codex_execution_paths.py fix(tests): provide model name in Codex 401 refresh tests for CI (#4166) 2026-03-30 21:17:09 -07:00
test_codex_models.py fix: add gpt-5.4-mini to Codex fallback catalog (#3855) 2026-03-29 20:10:00 -07:00
test_compression_boundary.py fix(agent): prevent silent tool result loss during context compression (#1993) 2026-03-18 15:22:51 -07:00
test_compression_persistence.py fix: persist compressed context to gateway session after mid-run compression 2026-03-30 18:49:14 -07:00
test_compressor_fallback_update.py feat(providers): add ordered fallback provider chain (salvage #1761) (#3813) 2026-03-29 16:04:53 -07:00
test_config_env_expansion.py feat(config): support ${ENV_VAR} substitution in config.yaml (#2684) 2026-03-23 16:02:06 -07:00
test_context_pressure.py fix: cap context pressure percentage at 100% in display (#3480) 2026-03-27 21:42:09 -07:00
test_context_references.py fix(context): restrict @ references to safe workspace paths (#2601) 2026-03-23 06:40:05 -07:00
test_context_token_tracking.py fix(tests): resolve all consistently failing tests 2026-03-22 05:58:26 -07:00
test_credential_pool.py feat(auth): same-provider credential pools with rotation, custom endpoint support, and interactive CLI (#2647) 2026-03-31 03:10:01 -07:00
test_crossloop_client_cache.py fix(agent): prevent AsyncOpenAI/httpx cross-loop deadlock in gateway mode (#2701) 2026-03-25 17:31:56 -07:00
test_dict_tool_call_args.py test: restore vllm integration coverage and add dict-args regression 2026-03-15 08:02:29 -07:00
test_display.py fix: add upstream guard for non-dict function_args + tests for build_tool_preview 2026-03-09 21:01:40 -07:00
test_evidence_store.py feat: add OSS Security Forensics skill (Skills Hub) (#1482) 2026-03-15 21:59:53 -07:00
test_exit_cleanup_interrupt.py fix: catch KeyboardInterrupt in exit cleanup handlers (#3257) 2026-03-26 14:34:31 -07:00
test_external_credential_detection.py refactor(auth): transition Codex OAuth tokens to Hermes auth store 2026-03-01 19:59:24 -08:00
test_fallback_model.py feat: upgrade MiniMax default to M2.7 + add new OpenRouter models 2026-03-18 02:42:58 -07:00
test_file_permissions.py security: enforce 0600/0700 file permissions on sensitive files (inspired by openclaw) 2026-03-09 02:19:32 -07:00
test_flush_memories_codex.py fix: update all test mocks for call_llm migration 2026-03-11 21:06:54 -07:00
test_hermes_state.py feat(sessions): add --source flag for third-party session isolation (#3255) 2026-03-26 14:35:31 -07:00
test_honcho_client_config.py fix(honcho): auto-enable when API key is present 2026-03-01 03:12:37 -05:00
test_insights.py feat: add route-aware pricing estimates (#1695) 2026-03-17 03:44:44 -07:00
test_interactive_interrupt.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_interrupt_propagation.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_managed_server_tool_support.py test: fix stale CI assumptions in parser and quick-command coverage (#1236) 2026-03-13 21:56:12 -07:00
test_mcp_serve.py feat: add MCP server mode — hermes mcp serve (#3795) 2026-03-29 15:47:19 -07:00
test_minisweagent_path.py chore: remove all remaining mini-swe-agent references 2026-03-24 08:19:23 -07:00
test_model_metadata_local_ctx.py fix: prefer loaded instance context size over max for LM Studio 2026-03-19 21:24:53 +01:00
test_model_provider_persistence.py feat: integrate GitHub Copilot providers across Hermes 2026-03-17 23:40:22 -07:00
test_model_tools.py test: strengthen assertions across 3 more test files (batch 2) 2026-03-05 18:46:30 -08:00
test_model_tools_async_bridge.py fix: use per-thread persistent event loops in worker threads 2026-03-20 15:41:06 -04:00
test_openai_client_lifecycle.py fix: audit fixes — 5 bugs found and resolved 2026-03-16 06:35:46 -07:00
test_packaging_metadata.py chore: prepare Hermes for Homebrew packaging (#4099) 2026-03-30 17:34:43 -07:00
test_percentage_clamp.py fix: cap percentage displays at 100% in stats, gateway, and memory tool (#3599) 2026-03-28 14:55:18 -07:00
test_personality_none.py feat(cli,gateway): add /personality none and custom personality support 2026-03-09 17:31:54 +03:00
test_plugins.py feat: activate plugin lifecycle hooks (pre/post_llm_call, session start/end) (#3542) 2026-03-28 11:14:54 -07:00
test_plugins_cmd.py fix(tests): resolve 10 CI failures across hooks, tiktoken, plugins (#3848) 2026-03-29 20:05:59 -07:00
test_project_metadata.py fix(setup): auto-install matrix-nio during hermes setup (#3873) 2026-03-29 21:53:28 -07:00
test_provider_fallback.py feat(providers): add ordered fallback provider chain (salvage #1761) (#3813) 2026-03-29 16:04:53 -07:00
test_provider_parity.py fix: _allow_private_urls name collision + stale OPENAI_BASE_URL test (#4217) 2026-03-31 03:16:40 -07:00
test_quick_commands.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_real_interrupt_subagent.py fix: thread safety for concurrent subagent delegation (#1672) 2026-03-17 02:53:33 -07:00
test_reasoning_command.py fix: prevent reasoning box from rendering 3x during tool-calling loops (#3405) 2026-03-27 09:57:50 -07:00
test_redirect_stdout_issue.py fix: use session_key instead of chat_id for adapter interrupt lookups 2026-03-12 08:35:45 -07:00
test_resume_display.py feat: display previous messages when resuming a session in CLI 2026-03-08 17:45:45 -07:00
test_run_agent.py fix: strip orphaned think/reasoning tags from user-facing responses 2026-03-31 11:42:44 -07:00
test_run_agent_codex_responses.py fix(codex): handle reasoning-only responses and replay path (#2070) 2026-03-19 10:34:44 -07:00
test_runtime_provider_resolution.py feat(auth): same-provider credential pools with rotation, custom endpoint support, and interactive CLI (#2647) 2026-03-31 03:10:01 -07:00
test_session_reset_fix.py fix(session): clear compressor summary and turn counter on /clear and /new (#3102) 2026-03-25 18:22:21 -07:00
test_setup_model_selection.py feat: add MiniMax M2.7 to hermes model picker and opencode-go (#4208) 2026-03-31 01:54:13 -07:00
test_sql_injection.py fix(security): eliminate SQL string formatting in execute() calls 2026-03-19 15:16:35 +01:00
test_streaming.py fix(test): update streaming test to match PR #3566 behavior change (#3574) 2026-03-28 13:41:23 -07:00
test_surrogate_sanitization.py fix: sanitize surrogate characters from clipboard paste to prevent UnicodeEncodeError (#3624) 2026-03-28 16:53:14 -07:00
test_timezone.py fix: skip stale cron jobs on gateway restart instead of firing immediately 2026-03-16 23:48:14 -07:00
test_tool_call_parsers.py fix(mistral-parser): handle nested JSON in fallback extraction 2026-03-21 09:41:17 -07:00
test_toolset_distributions.py test: add unit tests for 8 modules (batch 2) 2026-02-26 13:54:20 +03:00
test_toolsets.py fix: add missing Platform.SIGNAL to toolset mappings, update test + config docs 2026-03-09 23:27:19 -07:00
test_trajectory_compressor.py fix: URL-based auth for third-party Anthropic endpoints + CI test fixes (#4148) 2026-03-30 20:36:56 -07:00
test_trajectory_compressor_async.py fix: create AsyncOpenAI lazily in trajectory_compressor to avoid closed event loop (#4013) 2026-03-30 13:16:16 -07:00
test_worktree.py fix: harden salvaged worktree include checks 2026-03-14 21:51:27 -07:00
test_worktree_security.py fix: harden salvaged worktree include checks 2026-03-14 21:51:27 -07:00