c73e74386b
Adds Vertex AI as a first-class provider for Gemini models via Vertex's OpenAI-compatible endpoint. Vertex authenticates with short-lived OAuth2 access tokens (service-account JSON or ADC), not a static API key — the missing piece behind the recurring requests (#13484, #12639, #56259). - agent/vertex_adapter.py: OAuth2 token minting + refresh-on-expiry (5-min margin), ADC->service-account fallback, global vs regional endpoint URLs. Config precedence: env var > config.yaml > default. - plugins/model-providers/vertex/: provider profile (auth_type=vertex), reuses Gemini's extra_body.google.thinking_config translation. - runtime_provider: vertex short-circuit BEFORE the credential pool so a credentials-file path is never mistaken for a static API key; mints a fresh token + computes base_url per resolve. - run_agent + conversation_loop: _try_refresh_vertex_client_credentials() re-mints the token and rebuilds the client on a mid-session 401, so a long-lived gateway agent survives token expiry (~1h). - auxiliary_client: vertex auth_type branch for side-LLM tasks. - config.yaml: vertex.project_id / vertex.region (non-secret, bridged to env); credential path stays in .env (VERTEX_CREDENTIALS_PATH). - setup wizard + model picker: dedicated _model_flow_vertex; curated google/gemini-* model list; --provider choices. - pricing/metadata: Vertex prices off the gemini docs snapshot; endpoint host auto-maps to the vertex provider (no probe spam). - lazy_deps + pyproject [vertex] extra: google-auth, opt-in only. - docs: guides/google-vertex.md + providers page; tests for adapter + runtime resolution. Salvages and modernizes #8427 by @slawt onto current main: rewired from the legacy PROVIDER_REGISTRY path to the provider-profile architecture, moved non-secret config out of .env into config.yaml, and added the per-turn 401 token-refresh the original lacked.
80 lines
4 KiB
Python
80 lines
4 KiB
Python
"""Per-attempt recovery bookkeeping for the conversation turn loop.
|
|
|
|
The inner retry loop in ``run_conversation`` (``while retry_count <
|
|
max_retries``) makes several distinct recovery attempts on a single model API
|
|
call: a credential-pool 429 retry, a per-provider OAuth refresh (codex,
|
|
anthropic, nous, copilot), a long-context compression restart, a length-
|
|
continuation restart, and a handful of format-recovery branches (thinking-
|
|
signature stripping, multimodal-tool-content stripping, llama.cpp grammar
|
|
fallback, image shrink, invalid-encrypted-content, 1M-beta header).
|
|
|
|
Each of those branches is guarded by a one-shot boolean so it fires at most
|
|
once per attempt. They used to be ~16 bare ``*_attempted`` / ``has_retried_*``
|
|
/ ``restart_with_*`` locals declared inline before the loop and threaded
|
|
through its 2,400-line body. ``TurnRetryState`` collapses them into one object
|
|
the loop mutates in place (``state.codex_auth_retry_attempted = True``), giving
|
|
the recovery bookkeeping a single named, testable home.
|
|
|
|
Loop-control variables (``retry_count``, ``max_retries``,
|
|
``max_compression_attempts``) intentionally stay as plain locals — they are the
|
|
``while`` mechanics, not recovery bookkeeping, and putting them on the object
|
|
would add indirection without clarifying anything.
|
|
|
|
This module is dependency-free so it can be unit-tested in isolation and
|
|
imported by the turn loop without an import cycle.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from dataclasses import dataclass, fields
|
|
|
|
|
|
@dataclass
|
|
class TurnRetryState:
|
|
"""One-shot recovery guards + restart signals for a single API-call attempt.
|
|
|
|
A fresh instance is created for each iteration of the outer turn loop
|
|
(once per ``api_call_count``). Each guard fires its recovery branch at most
|
|
once; the ``restart_with_*`` signals are read by the loop after the attempt
|
|
to decide whether to rebuild the request and retry.
|
|
"""
|
|
|
|
# ── Per-provider OAuth / credential refresh guards ───────────────────
|
|
codex_auth_retry_attempted: bool = False
|
|
anthropic_auth_retry_attempted: bool = False
|
|
nous_auth_retry_attempted: bool = False
|
|
nous_paid_entitlement_refresh_attempted: bool = False
|
|
copilot_auth_retry_attempted: bool = False
|
|
vertex_auth_retry_attempted: bool = False
|
|
|
|
# ── Format / payload recovery guards ─────────────────────────────────
|
|
thinking_sig_retry_attempted: bool = False
|
|
invalid_encrypted_content_retry_attempted: bool = False
|
|
image_shrink_retry_attempted: bool = False
|
|
multimodal_tool_content_retry_attempted: bool = False
|
|
oauth_1m_beta_retry_attempted: bool = False
|
|
llama_cpp_grammar_retry_attempted: bool = False
|
|
|
|
# ── Transport / rate-limit recovery ──────────────────────────────────
|
|
primary_recovery_attempted: bool = False
|
|
has_retried_429: bool = False
|
|
|
|
# ── Auth-failure provider failover ───────────────────────────────────
|
|
# Set once we've escalated a persistent 401/403 (after the per-provider
|
|
# credential-refresh attempt above failed) to the fallback chain, so we
|
|
# don't loop on the same auth failover within one attempt.
|
|
auth_failover_attempted: bool = False
|
|
|
|
# ── Restart signals (read by the outer loop after the attempt) ───────
|
|
restart_with_compressed_messages: bool = False
|
|
restart_with_length_continuation: bool = False
|
|
# Set when a content-filter stream stall (e.g. MiniMax "new_sensitive")
|
|
# has been escalated to the fallback chain: the partial-stream content
|
|
# was rolled back off ``messages`` and the loop should re-issue the API
|
|
# call against the newly-activated provider (#32421).
|
|
restart_with_rebuilt_messages: bool = False
|
|
|
|
def __iter__(self):
|
|
# Convenience for debugging / tests: iterate (name, value) pairs.
|
|
for f in fields(self):
|
|
yield f.name, getattr(self, f.name)
|