mirror of
https://github.com/router-for-me/CLIProxyAPI.git
synced 2026-06-22 21:40:46 +08:00
* feat(plugin): add ModelRouter before auth with single-slot routing targets ## Motivation Plugins that need to change execution based on the **original inbound request** (protocol format, raw body, headers, query, stream flag, metadata, etc.) often resorted to virtual/trampoline models or routing inside interceptors. This commit adds **ModelRouter**: a pluggable layer **before** model-to-provider resolution and AuthManager credential selection, so plugins can declare who executes a request without spoofing the client model name. This is a **new capability**, not a bugfix on the existing chain. With no ModelRouter plugins loaded, behavior matches upstream. ## Pipeline placement - `execute`, `stream`, and `count` (and image paths via AuthManager) call `applyModelRouter()` before building `coreexecutor.Request`. - Routing runs **before** the request interceptor (before auth), so routers see the client’s original context. After a plugin executor is chosen, the existing **after-auth interceptor → response/stream interceptor** chain still applies. - Internal `ExecuteModel` / `ExecuteModelStream` (host callbacks) support `SkipRouterPluginID` so nested calls do not re-enter the same router. ## Routing API (single slot, mutually exclusive) `ModelRouteResponse` uses **one target slot** to avoid ambiguity when both `TargetExecutorPluginID` and `TargetProvider` were set and the host ignored one: | Field | Meaning | |-------|---------| | `Handled` | `false`: this router declines; try the next router or default path | | `TargetKind` | `self` \| `executor` \| `provider` (pick one) | | `Target` | `self`/`executor`: plugin ID; `provider`: built-in provider key | | `TargetModel` | Optional on `provider` only; empty keeps client `RequestedModel` | | `Reason` | Optional diagnostic text | - **self**: the router plugin’s own executor (`Target` normalized to the router’s plugin ID). - **executor**: another plugin’s executor; host pre-checks with `executorPluginReady()` (executor declared and provider identifier resolvable) to avoid handled routes that 500 at execution. - **provider**: skip registry model resolution; fixed built-in AuthManager path; optional `TargetModel` for execution model only—**does not** change outward requested-model metadata. Routers run in **descending plugin priority** (tie-break: ascending plugin ID). Panic, error, invalid target, or unavailable executor/provider → log and **fall through to the next router**; if none handle, use the original provider+auth flow. ## Context exposed to routers `ModelRouteRequest` includes: - `SourceFormat`, `RequestedModel`, `Stream` - `Headers`, `Query`, `Body` (defensive copies) - `Metadata` (best-effort read-only context snapshot) - `AvailableProviders`: built-in provider keys with at least one **non-disabled** auth (`AuthManager.AvailableProviders()`). **Does not** reflect per-model cooldown or transient unavailability—treat as an optimistic snapshot. Adds `AuthManager.HasProviderAuth()` and `AvailableProviders()`, excluding `Disabled` and `StatusDisabled` auths consistently with credential selection. ## Host and RPC - Go plugins: `pluginapi.ModelRouter` + `RouteModel()`. - RPC plugins: `pluginabi.MethodModelRoute` (`model.route`), capability flag `model_router`. - `pluginhost.Host` implements `RouteModel` / `RouteModelExcept`; handlers use `SetModelRouterHost` or a `PluginHost` type assertion; **direct executor** paths use `ExecutePluginExecutor*` / `CountPluginExecutor`. - No bundled example ModelRouter plugin; capability is active only when a third-party plugin declares `model_router` and loads. ## Plugin RPC schema (policy A, upstream-aligned) - `pluginabi.SchemaVersion` stays **1**: capability additions (`model_router`, `model.route`) do not bump the number; increment only on breaking RPC JSON changes. - Host sends `schema_version` at register; reject only if the plugin declares a **higher** version than the host. - No unpublished “ModelRouter requires schema ≥ 3” gate (v3 single-slot API was never public). - Existing plugins and examples without `model_router` (`schema_version: 1`) need no changes. - RPC ModelRouter: `schema_version: 1` + `model_router: true` + implement `model.route`. ## Path consistency within this commit - Provider routes reuse image-only model checks (e.g. `gpt-image-2`) on the normalized model, same as the default AuthManager path. - `count` aligned with execute/stream: `SkipRouterPluginID`, query/headers injection, interceptor skip semantics. - Handlers: `modelRoutersEnabled` treats hosts without `HasModelRouters` as disabled (same as before ModelRouter existed); `pluginhost.Host` implements the detector. - API docs: `ModelRouter` explicitly includes built-in **provider** targets (in addition to plugin executors and the router’s own executor). ## Testing go test ./internal/pluginhost ./sdk/api/handlers ./sdk/pluginapi ./sdk/pluginabi ./sdk/cliproxy/auth go build -o test-output ./cmd/server && rm test-output go test ./... * fix(handlers): address ModelRouter review feedback - Use modelExecutionQuery for plugin executor and AuthManager paths so inbound URL query matches router/header behavior - Guard queryFromContext when gin Request.URL is nil - Read plugin executor stream chunks via nextStreamChunk to exit on cancel - Drop redundant clonePluginMetadata on capability record meta Tests cover query propagation, stream cancel, and nil URL safety. * feat(plugin): add Claude web search router example Add a Claude Code web_search ModelRouter example that can route matching Claude requests through Antigravity, Codex, xAI, or Tavily. The plugin includes executor orchestration, backend fallback/penalty handling, Tavily API key support, Claude-compatible response assembly, stream forwarding, and focused unit coverage for detection, fallback routing, model resolution, penalties, stream forwarding, and Tavily behavior. Verification: go test -count=1 ./... in examples/plugin/claude-web-search-router/go; go build -buildmode=c-shared for the plugin; go build ./cmd/server; live local CPA curl coverage for plugin load, four explicit routes, fallback, and Codex spark routing. * fix(pluginhost): validate executor routes before fallback * fix(pluginhost): skip oauth-only executor routes
Claude Code Web Search Router (ModelRouter example)
This plugin demonstrates ModelRouter on Claude Code built-in web_search requests (see temp/1.json in the repo root for a captured request/response).
What it detects
- Inbound protocol
claude/anthropic tools[]withtypeweb_search_20250305orweb_search_20260209- Optional Claude Code heuristics: system text like “web search tool use”, or user text
Perform a web search for the query: …
Routes (route config)
| Value | Behavior |
|---|---|
fallback (default) |
Plugin executor runs antigravity → codex → xai → tavily (built-ins via host.model.*, Tavily in-plugin). On 429/503/502, tries the next backend in the same request. Backends that fail often are deprioritized on later requests (in-memory penalty; no extra config). |
antigravity_google / codex_web_search / xai_web_search / tavily |
Same orchestration for that backend’s chain member(s): execution retry + penalty apply when multiple backends are eligible. |
default_provider |
default_provider + optional default_provider_model via built-in AuthManager (not orchestrated). |
Routing for fallback requires at least one runnable backend (providers in AvailableProviders where needed, resolvable antigravity model, or tavily_api_keys). |
xAI web search notes (aligned with upstream docs)
- Model: xAI documents
grok-4.3for server-sideweb_search. This example setsTargetModeltogrok-4.3whenxai_modelis empty (do not forwardclaude-sonnet-4-6to xAI). - Request shape: Responses API
input+tools[]with"type": "web_search". Optionalfilters.allowed_domains/filters.excluded_domains(max 5 each, mutually exclusive). - Claude mapping today:
internal/translator/codex/claudecopies Claudeallowed_domains→filters.allowed_domains. Claudeblocked_domainsis not mapped toexcluded_domainsyet. - Executor:
xai_executornormalizes tools (drops unsupportedexternal_web_accessif present) and posts to/responses. - Response: Citations / server tool metadata come back through OpenAI Responses SSE and are converted toward Claude
server_tool_use/web_search_tool_resultwhere the response translator supports it.
Configuration
Plugin config lives under plugins.configs.claude-web-search-router (key must match the plugin name). Load the shared library via plugins.path.
Recommended: fallback chain (default)
Tries antigravity → codex → xai → tavily; configure tavily_api_keys so the last step can succeed when built-in providers are missing or unavailable.
plugins:
path:
- /absolute/path/to/examples/plugin/bin/claude-web-search-router-go.dylib
configs:
claude-web-search-router:
enabled: true
priority: 20
route: fallback
antigravity_model: "" # empty: registry lookup, then first supports_web_search
codex_model: "gpt-5.4-mini"
xai_model: "grok-4.3"
tavily_api_keys:
- "tvly-xxxxxxxx"
# - "tvly-yyyyyyyy" # optional: round-robin
require_web_search_only: true
Omit route to use the same default (fallback).
Minimal fallback (Tavily as last resort only)
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: fallback
tavily_api_keys:
- "tvly-xxxxxxxx"
require_web_search_only: true
Single backend (no fallback)
Antigravity only:
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: antigravity_google
antigravity_model: "gemini-3.1-flash-lite"
require_web_search_only: true
Codex only:
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: codex_web_search
codex_model: "gpt-5.4-mini"
require_web_search_only: true
xAI only:
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: xai_web_search
xai_model: "grok-4.3"
require_web_search_only: true
Tavily only (plugin executor):
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: tavily
tavily_api_keys:
- "tvly-xxxxxxxx"
require_web_search_only: true
Built-in provider via default_provider:
plugins:
configs:
claude-web-search-router:
enabled: true
priority: 20
route: default_provider
default_provider: claude
default_provider_model: ""
require_web_search_only: true
Disable or relax detection
plugins:
configs:
claude-web-search-router:
enabled: false # plugin declines; host may use default Claude path
# Or keep enabled but allow mixed tool lists:
claude-web-search-router:
enabled: true
route: fallback
require_web_search_only: false
Config field reference
| Field | Description |
|---|---|
enabled |
false → Handled: false for all web_search matches |
priority |
Host plugin order for ModelRouter (higher runs earlier; see main repo plugins docs) |
route |
fallback (default), antigravity_google, codex_web_search, xai_web_search, tavily, default_provider |
antigravity_model |
Antigravity execution model; never the client Claude model name |
codex_model |
Codex model; empty → gpt-5.4-mini |
xai_model |
xAI model; empty → grok-4.3 |
default_provider / default_provider_model |
Used when route=default_provider |
tavily_api_keys |
Required for route=tavily or fallback last step |
require_web_search_only |
true matches Claude Code–style exclusive web_search tools |
Build
make -C examples/plugin bin/claude-web-search-router-go.dylib
Use .so on Linux and .dll on Windows. Point plugins.path at the built artifact.