mirror of
https://github.com/router-for-me/CLIProxyAPI.git
synced 2026-06-23 06:25:51 +08:00
87132e54d7
* feat(plugin): add ModelRouter before auth with single-slot routing targets ## Motivation Plugins that need to change execution based on the **original inbound request** (protocol format, raw body, headers, query, stream flag, metadata, etc.) often resorted to virtual/trampoline models or routing inside interceptors. This commit adds **ModelRouter**: a pluggable layer **before** model-to-provider resolution and AuthManager credential selection, so plugins can declare who executes a request without spoofing the client model name. This is a **new capability**, not a bugfix on the existing chain. With no ModelRouter plugins loaded, behavior matches upstream. ## Pipeline placement - `execute`, `stream`, and `count` (and image paths via AuthManager) call `applyModelRouter()` before building `coreexecutor.Request`. - Routing runs **before** the request interceptor (before auth), so routers see the client’s original context. After a plugin executor is chosen, the existing **after-auth interceptor → response/stream interceptor** chain still applies. - Internal `ExecuteModel` / `ExecuteModelStream` (host callbacks) support `SkipRouterPluginID` so nested calls do not re-enter the same router. ## Routing API (single slot, mutually exclusive) `ModelRouteResponse` uses **one target slot** to avoid ambiguity when both `TargetExecutorPluginID` and `TargetProvider` were set and the host ignored one: | Field | Meaning | |-------|---------| | `Handled` | `false`: this router declines; try the next router or default path | | `TargetKind` | `self` \| `executor` \| `provider` (pick one) | | `Target` | `self`/`executor`: plugin ID; `provider`: built-in provider key | | `TargetModel` | Optional on `provider` only; empty keeps client `RequestedModel` | | `Reason` | Optional diagnostic text | - **self**: the router plugin’s own executor (`Target` normalized to the router’s plugin ID). - **executor**: another plugin’s executor; host pre-checks with `executorPluginReady()` (executor declared and provider identifier resolvable) to avoid handled routes that 500 at execution. - **provider**: skip registry model resolution; fixed built-in AuthManager path; optional `TargetModel` for execution model only—**does not** change outward requested-model metadata. Routers run in **descending plugin priority** (tie-break: ascending plugin ID). Panic, error, invalid target, or unavailable executor/provider → log and **fall through to the next router**; if none handle, use the original provider+auth flow. ## Context exposed to routers `ModelRouteRequest` includes: - `SourceFormat`, `RequestedModel`, `Stream` - `Headers`, `Query`, `Body` (defensive copies) - `Metadata` (best-effort read-only context snapshot) - `AvailableProviders`: built-in provider keys with at least one **non-disabled** auth (`AuthManager.AvailableProviders()`). **Does not** reflect per-model cooldown or transient unavailability—treat as an optimistic snapshot. Adds `AuthManager.HasProviderAuth()` and `AvailableProviders()`, excluding `Disabled` and `StatusDisabled` auths consistently with credential selection. ## Host and RPC - Go plugins: `pluginapi.ModelRouter` + `RouteModel()`. - RPC plugins: `pluginabi.MethodModelRoute` (`model.route`), capability flag `model_router`. - `pluginhost.Host` implements `RouteModel` / `RouteModelExcept`; handlers use `SetModelRouterHost` or a `PluginHost` type assertion; **direct executor** paths use `ExecutePluginExecutor*` / `CountPluginExecutor`. - No bundled example ModelRouter plugin; capability is active only when a third-party plugin declares `model_router` and loads. ## Plugin RPC schema (policy A, upstream-aligned) - `pluginabi.SchemaVersion` stays **1**: capability additions (`model_router`, `model.route`) do not bump the number; increment only on breaking RPC JSON changes. - Host sends `schema_version` at register; reject only if the plugin declares a **higher** version than the host. - No unpublished “ModelRouter requires schema ≥ 3” gate (v3 single-slot API was never public). - Existing plugins and examples without `model_router` (`schema_version: 1`) need no changes. - RPC ModelRouter: `schema_version: 1` + `model_router: true` + implement `model.route`. ## Path consistency within this commit - Provider routes reuse image-only model checks (e.g. `gpt-image-2`) on the normalized model, same as the default AuthManager path. - `count` aligned with execute/stream: `SkipRouterPluginID`, query/headers injection, interceptor skip semantics. - Handlers: `modelRoutersEnabled` treats hosts without `HasModelRouters` as disabled (same as before ModelRouter existed); `pluginhost.Host` implements the detector. - API docs: `ModelRouter` explicitly includes built-in **provider** targets (in addition to plugin executors and the router’s own executor). ## Testing go test ./internal/pluginhost ./sdk/api/handlers ./sdk/pluginapi ./sdk/pluginabi ./sdk/cliproxy/auth go build -o test-output ./cmd/server && rm test-output go test ./... * fix(handlers): address ModelRouter review feedback - Use modelExecutionQuery for plugin executor and AuthManager paths so inbound URL query matches router/header behavior - Guard queryFromContext when gin Request.URL is nil - Read plugin executor stream chunks via nextStreamChunk to exit on cancel - Drop redundant clonePluginMetadata on capability record meta Tests cover query propagation, stream cancel, and nil URL safety. * feat(plugin): add Claude web search router example Add a Claude Code web_search ModelRouter example that can route matching Claude requests through Antigravity, Codex, xAI, or Tavily. The plugin includes executor orchestration, backend fallback/penalty handling, Tavily API key support, Claude-compatible response assembly, stream forwarding, and focused unit coverage for detection, fallback routing, model resolution, penalties, stream forwarding, and Tavily behavior. Verification: go test -count=1 ./... in examples/plugin/claude-web-search-router/go; go build -buildmode=c-shared for the plugin; go build ./cmd/server; live local CPA curl coverage for plugin load, four explicit routes, fallback, and Codex spark routing. * fix(pluginhost): validate executor routes before fallback * fix(pluginhost): skip oauth-only executor routes
160 lines
6.3 KiB
Go
160 lines
6.3 KiB
Go
package pluginhost
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
|
|
"github.com/router-for-me/CLIProxyAPI/v7/sdk/pluginapi"
|
|
)
|
|
|
|
type rpcLifecycleRequest struct {
|
|
ConfigYAML []byte `json:"config_yaml"`
|
|
SchemaVersion uint32 `json:"schema_version"`
|
|
}
|
|
|
|
type rpcRegistration struct {
|
|
SchemaVersion uint32 `json:"schema_version"`
|
|
Metadata pluginapi.Metadata `json:"metadata"`
|
|
Capabilities rpcCapabilities `json:"capabilities"`
|
|
}
|
|
|
|
type rpcCapabilities struct {
|
|
ModelRegistrar bool `json:"model_registrar"`
|
|
ModelProvider bool `json:"model_provider"`
|
|
AuthProvider bool `json:"auth_provider"`
|
|
FrontendAuthProvider bool `json:"frontend_auth_provider"`
|
|
FrontendAuthProviderExclusive bool `json:"frontend_auth_provider_exclusive"`
|
|
Scheduler bool `json:"scheduler"`
|
|
ModelRouter bool `json:"model_router"`
|
|
Executor bool `json:"executor"`
|
|
ExecutorModelScope pluginapi.ExecutorModelScope `json:"executor_model_scope"`
|
|
ExecutorInputFormats []string `json:"executor_input_formats,omitempty"`
|
|
ExecutorOutputFormats []string `json:"executor_output_formats,omitempty"`
|
|
RequestTranslator bool `json:"request_translator"`
|
|
RequestNormalizer bool `json:"request_normalizer"`
|
|
RequestInterceptor bool `json:"request_interceptor"`
|
|
ResponseTranslator bool `json:"response_translator"`
|
|
ResponseBeforeTranslator bool `json:"response_before_translator"`
|
|
ResponseAfterTranslator bool `json:"response_after_translator"`
|
|
ResponseInterceptor bool `json:"response_interceptor"`
|
|
StreamChunkInterceptor bool `json:"response_stream_interceptor"`
|
|
ThinkingApplier bool `json:"thinking_applier"`
|
|
UsagePlugin bool `json:"usage_plugin"`
|
|
CommandLinePlugin bool `json:"command_line_plugin"`
|
|
ManagementAPI bool `json:"management_api"`
|
|
}
|
|
|
|
type rpcIdentifierResponse struct {
|
|
Identifier string `json:"identifier"`
|
|
}
|
|
|
|
type rpcExecutorStreamResponse struct {
|
|
Headers http.Header `json:"headers,omitempty"`
|
|
Chunks []pluginapi.ExecutorStreamChunk `json:"chunks,omitempty"`
|
|
}
|
|
|
|
type rpcAuthLoginStartRequest struct {
|
|
pluginapi.AuthLoginStartRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcAuthLoginPollRequest struct {
|
|
pluginapi.AuthLoginPollRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcAuthRefreshRequest struct {
|
|
pluginapi.AuthRefreshRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcAuthModelRequest struct {
|
|
pluginapi.AuthModelRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcExecutorRequest struct {
|
|
pluginapi.ExecutorRequest
|
|
StreamID string `json:"stream_id,omitempty"`
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcExecutorHTTPRequest struct {
|
|
pluginapi.ExecutorHTTPRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcRequestInterceptRequest struct {
|
|
pluginapi.RequestInterceptRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcModelRouteRequest struct {
|
|
pluginapi.ModelRouteRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcResponseInterceptRequest struct {
|
|
pluginapi.ResponseInterceptRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcStreamChunkInterceptRequest struct {
|
|
pluginapi.StreamChunkInterceptRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcThinkingApplyRequest struct {
|
|
pluginapi.ThinkingApplyRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcManagementRequest struct {
|
|
pluginapi.ManagementRequest
|
|
HostCallbackID string `json:"host_callback_id,omitempty"`
|
|
}
|
|
|
|
type rpcManagementRegistrationResponse struct {
|
|
Routes []pluginapi.ManagementRoute `json:"routes,omitempty"`
|
|
Resources []pluginapi.ResourceRoute `json:"resources,omitempty"`
|
|
}
|
|
|
|
type rpcEmptyResponse struct{}
|
|
|
|
func rpcCapabilitiesFromPlugin(plugin pluginapi.Plugin) rpcCapabilities {
|
|
caps := plugin.Capabilities
|
|
return rpcCapabilities{
|
|
ModelRegistrar: caps.ModelRegistrar != nil,
|
|
ModelProvider: caps.ModelProvider != nil,
|
|
AuthProvider: caps.AuthProvider != nil,
|
|
FrontendAuthProvider: caps.FrontendAuthProvider != nil,
|
|
FrontendAuthProviderExclusive: caps.FrontendAuthProvider != nil && caps.FrontendAuthProviderExclusive,
|
|
Scheduler: caps.Scheduler != nil,
|
|
ModelRouter: caps.ModelRouter != nil,
|
|
Executor: caps.Executor != nil,
|
|
ExecutorModelScope: normalizedExecutorModelScope(caps),
|
|
ExecutorInputFormats: append([]string(nil), caps.ExecutorInputFormats...),
|
|
ExecutorOutputFormats: append([]string(nil), caps.ExecutorOutputFormats...),
|
|
RequestTranslator: caps.RequestTranslator != nil,
|
|
RequestNormalizer: caps.RequestNormalizer != nil,
|
|
RequestInterceptor: caps.RequestInterceptor != nil,
|
|
ResponseTranslator: caps.ResponseTranslator != nil,
|
|
ResponseBeforeTranslator: caps.ResponseBeforeTranslator != nil,
|
|
ResponseAfterTranslator: caps.ResponseAfterTranslator != nil,
|
|
ResponseInterceptor: caps.ResponseInterceptor != nil,
|
|
StreamChunkInterceptor: caps.StreamChunkInterceptor != nil,
|
|
ThinkingApplier: caps.ThinkingApplier != nil,
|
|
UsagePlugin: caps.UsagePlugin != nil,
|
|
CommandLinePlugin: caps.CommandLinePlugin != nil,
|
|
ManagementAPI: caps.ManagementAPI != nil,
|
|
}
|
|
}
|
|
|
|
func marshalRPCResult(v any) ([]byte, error) {
|
|
result, errMarshal := json.Marshal(v)
|
|
if errMarshal != nil {
|
|
return nil, errMarshal
|
|
}
|
|
return marshalRPCEnvelope(json.RawMessage(result))
|
|
}
|