- Updated `ensureCacheControl` to inject cache_control into the last non-deferred tool instead of the last tool.
- Adjusted `injectToolsCacheControl` to ignore tools marked with `defer_loading` for prompt caching.
- Added unit tests to validate behavior for deferred tools, existing cache_control, and mixed tool arrays.
Closes: #4561
- Updated Kimi and Gemini executors to use `TranslateRequestWithCodexMultiAgentV2` for improved multi-agent v2 optimization.
- Enhanced configuration and API server to support `CodexOptimizeMultiAgentV2`.
- Added codex multi-agent V2 optimizations in `config`, `executor`, and `watcher` modules.
- Introduced `ClaudeInputTokenState` to track and estimate input token usage for Claude message_start events.
- Added `TranslateStreamWithClaudeInputTokens` for token-aware stream translation.
- Updated executors (XAI, Kimi, Gemini) to leverage the new logic.
- Included robust test cases for token counting, JSON validation, and concurrent tokenization scenarios.
- Refactored token counting logic into a reusable `countXAIInputTokens` function for better readability and maintainability.
- Replaced direct encoder usage with structured input handling, including JSON parsing and segment collection.
- Switched tokenizer from `Cl100kBase` to `O200kBase` to support larger contexts.
Closes: #4492
Preserve Codex and XAI upstream message-too-big closes across the Responses WebSocket boundary while treating them as request-scoped errors. Prevent CPA reconnect and credential cooldown/fallback, scope close state to the active connection, retain terminal errors under backpressure, and ensure downstream disconnect cleanup cannot wait behind a blocked writer.
- Added `GetGJSONBytesNoCopy` in `internal/util` for safe, no-copy JSON parse operations leveraging `unsafe`.
- Replaced `gjson.GetBytes` with the new helper in key payload processing paths (`kimi_executor`, `vertex_payload_helpers`, etc.) to improve performance.
- Added unit tests for behavior validation, including edge cases with empty input.
- Introduced `SetStringIfDifferent`, `SetBoolIfDifferent`, and similar helpers in `payload_helpers` to avoid unnecessary writes during JSON modifications.
- Simplified message patching in `kimi_executor` for reasoning and tool call adjustments.
- Updated all executors (Gemini, Codex, XAI, Antigravity, etc.) to use the new helpers, enhancing readability and potentially reducing overhead.
- Removed obsolete `filterKimiEmptyAssistantMessages` function in `kimi_executor`.
- Updated `SanitizeCodexInputItemIDs` to remove encrypted reasoning items with IDs exceeding the Codex length limit.
- Refactored logic to deterministically shorten other overlong input item IDs, ensuring compliance with the Codex format.
- Added new tests to verify dropping behavior for encrypted reasoning and shortening of lengthy IDs without encrypted content.
- Added `promoteXAIAdditionalTools` to normalize and migrate `additional_tools` to the top-level `tools` array in XAI request bodies.
- Updated test cases to validate the promotion logic and ensure unsupported `additional_tools` items are excluded from input payloads.
Closes: #4434
- Updated Kimi executor to delegate non-streaming and streaming Claude requests to the Claude executor when `SourceFormat` is `claude`.
- Injected `base_url` for Claude API compatibility during delegation.
- Updated Kimi executor to route all Claude requests through the OpenAI-compatible chat completions API.
- Removed legacy delegation to Claude executor for non-streaming and streaming execution.
- Enhanced translator integration for request normalization and Kimi model stripping.
- Updated tests to validate chat completions behavior, including upstream request logging, payload transformation, and response handling.
- Deleted Kimi K3[1M] variant and associated metadata.
Claude Code sends the configured model name (e.g. kimi-k3[1m]) to
CLIProxyAPI. The Kimi executor forwarded this internal ID to Kimi's
https://api.kimi.com/coding endpoint. Kimi treats kimi-k3 and
kimi-k3[1m] as 256K-context models, while only bare k3 supports the
full 1M context window.
Normalize the upstream model by stripping the CLIProxyAPI kimi- prefix
and any Claude Code [1m] suffix while preserving a trailing thinking
suffix (e.g. (1024)), so that requests for kimi-k3, kimi-k3[1m],
kimi-k3(1024), and kimi-k3[1m](1024) are all sent to Kimi as k3 or
k3(1024). This applies to:
- Execute (Claude and OpenAI branches)
- ExecuteStream (Claude and OpenAI branches)
- CountTokens
Validation (using the stored Kimi token):
- GET /v1/models shows k3 with context_length 1048576.
- Direct /v1/messages calls with ~270k input tokens:
- model=k3 succeeds.
- model=k3[1m], kimi-k3, kimi-k3[1m] fail with:
'Your request exceeded model token limit: 262144 (requested: 270014)'.
Add unit tests for normalizeKimiUpstreamModel.
Closes#4418
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Added an `incomingHeaders` argument to `applyClaudeHeaders` to enable external header injection for flexibility.
- Updated Kimi executor to delegate header support to Claude executor, facilitating upstream request logging and beta header forwarding.
- Introduced tests for delegated configuration, header handling, and upstream request/response logging to improve coverage.
- Introduced `normalizeCodexWebsocketParallelToolCalls` and `normalizeCodexParallelToolCalls` to enforce consistent `parallel_tool_calls` handling.
- Updated WebSocket and HTTP executor logic to support headers during parallel tool normalization.
- Refactored redundant logic by consolidating normalization routines for improved clarity and maintainability.
- Introduced `normalizeXAIObjectRootUnionBranchTypes` for ensuring object-only branches in root-level unions.
- Updated XAI executor logic to handle additional schema adjustments, ensuring compatibility with stricter validations.
- Extended test coverage to validate schema normalization, including both functional and custom tool scenarios.
Closes: #4397
- Introduced agent-specific cache key isolation to ensure distinct prompt caching across agents.
- Improved reasoning replay handling for Claude models, including cumulative tool turn restoration and validation.
- Enhanced HTTP and WebSocket session consistency during execution scope and cache replay synchronization.
- Added new unit tests and extended coverage for prompt caching, agent isolation, and replay cache functionality.
Closes: #4352
- Introduced per-connection active channel management for Codex and XAI websocket sessions.
- Enhanced session behavior to support clearing and reassigning active channels tied to specific connections.
- Added support for detecting and detaching stale connections upon session target changes.
- Refactored websocket message handling logic to ensure connection-specific channel consistency.
- Extended test coverage for websocket session behaviors, including reconnections, target switching, and pinned auth transitions.
- Implemented deterministic shortening of overlong Codex input item IDs to meet 64-character limit.
- Added `SanitizeCodexInputItemIDs` helper for efficient ID shortening in request bodies.
- Updated Codex executor and websocket implementations to utilize the new helper.
- Introduced unit tests to validate ID length restrictions, avoidance of collisions, and deterministic behavior.
Closes: #4401
- Switched from `xaiChatBaseURL` to `xaiCompactBaseURL` for compact requests to avoid 404 errors from CLI chat-proxy.
- Updated headers to use standard API headers for compact endpoints.
- Added `xaiCompactBaseURL` helper function for dedicated compact request base URL resolution.
- Adjusted comments to clarify handling of compact and websocket transports.
Closes: #4376
- Updated `normalizeClaudeSamplingForUpstream` to delete `top_p` from payloads.
- Adjusted test to validate the removal of both `temperature` and `top_p`.
Closes: #4383
- Updated `X-Msh-Version` and `User-Agent` headers in both `kimi.go` and `kimi_executor.go` to dynamically include the current version from `buildinfo`.
- Replaced hardcoded values with `buildinfo.Version` to ensure consistency with the build version.
- Enhanced response handling to support `response.incomplete` events across translator components and executors.
- Refactored `resultErrorFromError` to centralize error conversion and consolidate repeated logic for constructing custom errors.
- Updated handling of `max_output_tokens` and similar terminal reasons in OpenAI and Gemini translator workflows.
- Introduced `IsRequestScoped` and `IsRequestScopedError` to differentiate between request-scoped and non-request-scoped errors.
- Added comprehensive test cases for incomplete responses, terminal failures, and error propagation in both streaming and non-streaming conditions.
Closes: #3055
- Refactored `xaiFunctionParametersNeedSimplification` to identify schemas xAI cannot process, including those with non-object types in unions.
- Limited parameter simplifications to problematic schemas instead of specific tools or namespaces.
- Clarified and updated debug messages to reflect schema rejection and hang prevention.
Closes: #4343
Propagate client generate metadata through usage records while keeping
legacy callers that omit the field enabled by default. Preserve an
existing context generate=false when metadata does not set the value,
and normalize omission to true only at publishing boundaries.
- Introduced `deferredRequestBodyCapture` for spooling and retaining large error-only request bodies for improved logging.
- Enhanced `RequestLoggingMiddleware` to capture request body based on size and error criteria.
- Implemented decompression and truncation for large request bodies with Zstd encoding.
- Added support for deferred API request logging and context tracking.
- Included extensive test coverage for error-specific body capture and serialization behavior.
* fix(usage): preserve request service tier metadata
* fix(usage): collapse request tier metadata
* fix(usage): emit only service_tier and response_service_tier
Collapse request tier to service_tier for usage records and queue payloads. Keep RequestServiceTier as a deprecated input-only alias for older plugin callers.
Normalize and drop tool_choice entries that reference tools removed by
normalizeXAITools before injecting native x_search, then allow x_search in
allowed_tools without duplicates so Grok can select the injected tool.
- Introduced `xai-api-key` to configuration for supporting xAI provider integration.
- Added API key operations: creation, updates, deletion, and sanitization.
- Implemented backend enhancements for credential-specific routing, response filtering with model aliasing, and WebSocket handling for xAI services.
- Included extensive test coverage to validate configuration parsing, CRUD operations, and the new functionalities.
- Updated related modules to handle `xai-api-key`, including the watcher, management handlers, and configuration sanitizers.
- Added `clientDeclaredTools` to `xaiInternalXSearchResponseFilter` for improved filtering precision.
- Implemented `collectXAIClientDeclaredToolKeys` to register client-declared tools with effective normalization to function calls.
- Prevented leakage of internal traces (e.g., `xs_call-*`) while preserving client-declared tools with overlapping names.
- Introduced multiple test cases to ensure filter correctness and preserve expected WebSocket response outputs.
- Enhanced completed output handling in response filtering.
Closes: #4282 / PR #4284.
* fix(executor): strip orphan reasoning ids when store is disabled
Codex backend requires store=false and treats reasoning items with an
id but no usable encrypted_content as store lookups, which 404 with
"Item with id not found". Drop those orphan ids after removing invalid
encrypted_content (and when encrypted_content is already missing),
while preserving ids when store=true.
* fix(executor): skip id delete when reasoning item has no id
Avoid sjson.Delete on encrypted_content drops that never carried an id,
matching the orphan-id path's Exists() guard.
Previously each dropped encrypted_content re-ran sjson.DeleteBytes on
the whole body/array per item, causing O(n^2) work on inputs with many
reasoning items.
Now the array is walked once, editing only offending items in place
and lazily building a replacement "input" array via strings.Builder.
The builder stays nil (no allocation) when nothing needs sanitizing,
and the array is spliced back with a single SetRawBytes call.
