- add shared `EnsureResponsesUsageDetails` helper to patch `usage` objects with:
- `output_tokens_details.reasoning_tokens = 0`
- `input_tokens_details.cached_tokens = 0`
- for both plain JSON and SSE `data:` frames, including multi-line frames
- apply the helper to OpenAI Response format outputs in non-stream and stream paths across executors/plugins so translated payloads consistently include required usage details
- update websocket/completion payload builders to emit default `usage` detail fields for prewarm/finish responses
Closes: #4985
- Propagate pending terminal errors when image/response data streams close, emitting `error`/`failed` SSE events (or HTTP error responses when no stream started) instead of dropping them.
- Sanitize and normalize streamed terminal errors before writing, and reuse normalized errors for cancellation.
- Improve SSE parsing/frame handling for multiline and cross-chunk payloads, prioritize payload/event-level failures, and avoid mutating emitted frame buffers by cloning chunks before queueing.
Closes: #4904
- Added a plugin refresh-compat executor wrapper that forwards normal OpenAI-compat execution paths while routing `Refresh` to plugin `AuthProvider`/Home refresh logic.
- Updated refresh lookup to use the effective executor key from auth metadata so namespaced compatibility providers can resolve their refresh executors correctly.
- Changed OpenAI-compat registration to wrap built-in executors with the plugin-refresh wrapper when a matching plugin auth provider exists, while preserving bare executors otherwise.
- Made `OpenAICompatExecutor.Refresh` fail fast for OAuth-style credentials (with refresh tokens) instead of silently returning unchanged auth.
Closes: #4719
- Add `is-compat` support to OpenAI compatibility model config, capabilities, hashing, and example config.
- Propagate `IsCompat` through API-key model resolution and switch OpenAI-compat executor translation to compatibility-aware routing.
- Keep Claude assistant thinking content in compatibility mode while keeping default behavior unchanged when `is-compat` is disabled.
Closes: #4776
- Track terminal `[DONE]` state in translator and ignore all subsequent SSE payloads
- Stop OpenAI-compatible stream processing once `[DONE]` is received, only emit synthetic done when missing
- Add tests covering translator and executor behavior to prevent post-DONE trailing chunks from being forwarded
Closes: #4783
- Add OpenAI-compat executor-time normalization for `tool` message content, converting non-string tool results to plain text and replacing image parts with a clear unsupported marker for models configured with text-only `input-modalities`.
- Apply normalization in both regular and streaming OpenAI-compat execution paths before prompt-cache processing.
- Update config example to document `[text]` `input-modalities` for upstreams that reject multimodal tool-result content.
Closes: #4737
- Introduced a new `support-prompt-cache-key` configuration option to enable optional prompt cache key derivation for supported models.
- Updated relevant API handlers and internal logic to handle the new configuration.
- Enhanced executor workflows to apply prompt cache keys conditionally during request processing.
- Added tests to validate prompt cache key behavior across various scenarios and providers.
- Updated configuration examples and documentation for the new option.
Closes: #4700
- Replaced instances of `thinking.ApplyThinking` with `helps.ApplyRequestThinking` across all executors for consistency.
- Updated `applyGeminiInteractionsThinking` to accept `cliproxyexecutor.Request` and `Options`.
- Centralized logic for request thinking application to `helps` package for improved maintainability.
Closes: #4618
- Updated Kimi and Gemini executors to use `TranslateRequestWithCodexMultiAgentV2` for improved multi-agent v2 optimization.
- Enhanced configuration and API server to support `CodexOptimizeMultiAgentV2`.
- Added codex multi-agent V2 optimizations in `config`, `executor`, and `watcher` modules.
- Introduced `ClaudeInputTokenState` to track and estimate input token usage for Claude message_start events.
- Added `TranslateStreamWithClaudeInputTokens` for token-aware stream translation.
- Updated executors (XAI, Kimi, Gemini) to leverage the new logic.
- Included robust test cases for token counting, JSON validation, and concurrent tokenization scenarios.
- Introduced `SetStringIfDifferent`, `SetBoolIfDifferent`, and similar helpers in `payload_helpers` to avoid unnecessary writes during JSON modifications.
- Simplified message patching in `kimi_executor` for reasoning and tool call adjustments.
- Updated all executors (Gemini, Codex, XAI, Antigravity, etc.) to use the new helpers, enhancing readability and potentially reducing overhead.
- Removed obsolete `filterKimiEmptyAssistantMessages` function in `kimi_executor`.
- Introduced `StreamUsageBuffer` to encapsulate stream usage observation and publishing logic.
- Updated OpenAI-compatible executors and helpers to use `StreamUsageBuffer` for consistent stream usage tracking.
- Added unit tests to validate `StreamUsageBuffer` behavior, ensuring proper handling of usage details.
Closes: #4053
- Added an example plugin `host-model-callback` in Go to summarize host model callbacks.
- Implemented `cliproxy_plugin_init`, `cliproxyPluginCall`, and other plugin functions for callback handling.
- Introduced API handlers for `ModelExecution` and `ModelExecutionStream` with support for both streaming and non-streaming requests.
- Included unit tests (`model_execution_test.go`) to validate execution logic and streaming responses.
- Replaced `NewUsageReporter` with `NewExecutorUsageReporter` to include executor type in usage records.
- Updated all executors to use the new reporter implementation.
- Extended `UsageReporter` to track and publish executor type.
- Added tests to validate proper executor type recording and handling.
- Enhanced RedisQueue plugin and payload schema with executor type support.
- Introduced Time-To-First-Token (TTFT) measurement and reporting across major executors.
- Added TTFT calculation to `UsageReporter`, including support for HTTP clients and WebSocket communication.
- Updated tests to validate TTFT tracking in streamed and non-streamed scenarios.
- Ensured integration with `usage` plugin and augmented usage records with TTFT data.
- Introduced `SetTranslatedReasoningEffort` method in `UsageReporter` to capture and log reasoning efforts from translated payloads.
- Updated executors to incorporate the new reporting functionality for handling reasoning efforts across various providers.
- Enhanced logging for thinking level extraction with new helper function `ExtractTranslatedReasoningEffort`.
- Introduced OpenAI-compatible image model support in the API, enabling integration through image generation and editing endpoints.
- Added registry type for OpenAIImageModelType to classify and validate compatibility.
- Implemented request handling for OpenAI-compatible image models, including JSON and multipart formats.
- Enhanced executor methods to support OpenAI-compatible image streaming and non-streaming requests.
- Included tests to validate model registration, streaming behavior, and multipart payload formatting.
- Introduced `resolvePayloadRulePaths` function to dynamically resolve rule paths supporting array queries and complex logic.
- Updated payload processing logic (`apply defaults`, `overrides`, `filters`) to handle resolved paths for better flexibility.
- Added helper functions for path parsing, query matching, and logical resolution to improve modularity and reusability.
- Introduced payload condition match logic, including `match`, `not-match`, `exist`, and `not-exist` rules in `PayloadConfig`.
- Enhanced `payloadModelRulesMatch` function to support conditional checks at various levels.
- Added helper methods for evaluating JSON path conditions and values.
- Updated tests to validate new conditional rules against different payload scenarios.
- Added `count` attribute to `homeAuthCount` requests to improve home message batching.
- Enabled usage statistics for home mode by default and added config-level enforcement.
- Adjusted failure logging to include detailed metadata in `UsageReporter`.
- Updated multiple executors to pass error details to `PublishFailure` for better debugging.
- Enhanced unit tests to validate `count` behavior and usage statistics enforcement across components.
- Updated all references from v6 to v7 for `github.com/router-for-me/CLIProxyAPI`.
- Ensured consistency in imports within core libraries, tests, and integration tests.
- Added missing tests for new features in Redis Protocol integration.
- Added trimming for stream input lines to prevent processing of unnecessary whitespace.
- Improved handling of unsupported prefixes and malformed JSON responses, ensuring errors are recorded and propagated appropriately.
Fixed: #2690
- Moved `ApplyThinking` logic earlier in `openai_compat_executor` to align with configuration application sequence.
- Added test to verify payload override precedence over Thinking suffix configuration.
All streaming executors use bare channel sends (out <- chunk) inside goroutines
that process upstream SSE responses. When the downstream consumer disconnects
(client timeout, network drop, etc.), these sends block indefinitely, causing
the goroutine and all associated resources (HTTP response body, scanner buffers,
translation state) to leak permanently.
Over time, leaked goroutines accumulate monotonically, leading to RSS growth
from ~30MB to 3.7GB+ and eventual OOM kills on resource-constrained VPS hosts.
Fix: Replace all bare 'out <- ...' sends with:
select {
case out <- ...:
case <-ctx.Done():
return
}
This ensures goroutines terminate promptly when the request context is canceled,
allowing GC to reclaim all associated resources.
Affected executors (9 files, 36+ send sites):
- antigravity_executor.go (5 sites)
- gemini_cli_executor.go (6 sites)
- gemini_vertex_executor.go (6 sites)
- aistudio_executor.go (4 sites)
- gemini_executor.go (3 sites)
- openai_compat_executor.go (3 sites)
- claude_executor.go (4 sites)
- codex_executor.go (2 sites)
- kimi_executor.go (3 sites)
- Introduced `DisableImageGenerationMode` with support for `false`, `true`, and `chat` values.
- Updated payload handling to preserve `image_generation` on images endpoints when `chat` mode is enabled.
- Modified OpenAI image handlers (`ImagesGenerations`, `ImagesEdits`) to respect tri-state logic.
- Added unit tests for `DisableImageGenerationMode` behavior and endpoint-specific handling.
- Enhanced configuration diff logging to support `DisableImageGenerationMode`.
- Introduced a `Disabled` flag to OpenAI compatibility configurations.
- Updated routing, auth selection, and API handling logic to respect the `Disabled` state.
- Extended relevant APIs, YAML configurations, and data structures to include the `Disabled` field.
- Adjusted all relevant loops and filters to skip disabled providers.
Closes: #3060#3059#2977
- Replaced all instances of `bytes.Clone` with direct references to enhance efficiency.
- Simplified payload handling across executors and translators by eliminating unnecessary data duplication.
Update ApplyThinking signature to accept fromFormat and toFormat parameters
instead of a single provider string. This enables:
- Proper level-to-budget conversion when source is level-based (openai/codex)
and target is budget-based (gemini/claude)
- Strict budget range validation when source and target formats match
- Level clamping to nearest supported level for cross-format requests
- Format alias resolution in SDK translator registry for codex/openai-response
Also adds ErrBudgetOutOfRange error code and improves iflow config extraction
to fall back to openai format when iflow-specific config is not present.
Refactored `applyPayloadConfig` to `applyPayloadConfigWithRoot`, adding support for default rule validation against the original payload when available. Updated all executors to use `applyPayloadConfigWithRoot` and incorporate an optional original request payload for translations.
Expose thinking/effort normalization helpers from the executor package
so conversion tests use production code and stay aligned with runtime
validation behavior.
When using OpenAI-compatible providers with model aliases (e.g., glm-4.6-zai -> glm-4.6),
the alias resolution was correctly applied but then immediately overwritten by
ResolveOriginalModel, causing 'Unknown Model' errors from upstream APIs.
This fix skips the ResolveOriginalModel override when a model alias has already
been resolved, ensuring the correct model name is sent to the upstream provider.
Co-authored-by: Amp <amp@ampcode.com>
