Ensure all Gemini and Gemini Vertex execution and token count pathways invoke SanitizeGeminiRequestThoughtSignatures before dispatching upstream requests. This prevents raw non-Gemini (e.g. Claude CAIS) thought signatures from leaking to upstream Gemini endpoints while preserving valid native protobuf signatures and setting appropriate validator bypass sentinels for function calls.
Co-authored-by: W ARELIK <warelik@WARELIK-MB.local>
The xAI executor only switched on response.completed, so any turn that
ended with the spec-correct response.incomplete terminal event fell out
of the loop. Non-streaming requests were reported to the client as a 408
("stream disconnected before response.completed") even though the
upstream request succeeded, and because 408 is retryable it burned
credential rotations against a healthy pool. Streaming requests forwarded
the terminal event without patching the collected output items or
publishing usage.
Accept response.incomplete alongside response.completed in both paths,
mirroring the Codex executor, and keep the reasoning replay cache gated
on response.completed since a truncated turn has no replayable state.
* feat(codex): add opt-in stream bootstrap buffering
The upstream smuggles capacity rejections into an HTTP 200 stream. The
handshake events arrive normally and only a later event carries
{"error":{"type":"service_unavailable_error","code":
"server_is_overloaded"}}. By then the executor has already handed the
first chunk downstream, the response is committed, and the conductor can
no longer retry on another credential, so the request fails even though
other credentials were available.
When codex.stream-bootstrap-buffering is enabled the executor holds back
the handshake events until it can tell whether the stream carries real
output or a rejection. An overload rejection then fails the attempt
before any chunk is delivered, letting the conductor retry on another
credential; every other terminal failure is flushed in order and
delivered in-stream exactly as before.
Detection uses an event-type allow-list rather than a fixed count. On the
websocket transport codex.rate_limits and codex.response.metadata arrive
before response.created, making the first generated event the fifth
frame, so a small counter would release the stream before the rejection
is visible. Buffering is bounded and hitting the bound degrades to the
original unbuffered behaviour.
Two details are load-bearing. The error must be returned synchronously:
delivering it as the first stream chunk makes ExecuteStream downgrade it
into a committed 200 and the status is lost. And the websocket path must
not signal an upstream disconnect for a rejection it intends to retry,
because the downstream handler closes the client connection on that
signal and the retry would have nowhere to deliver.
The 503 status is produced only on this path rather than in the shared
codexTerminalFailureStatus mapping, so disabling the feature restores the
previous behaviour exactly, including cooldown classification and
retry-after parsing.
Defaults to false: response headers are withheld until generation
starts, which can trip client or reverse-proxy read timeouts.
* test(codex): pin bootstrap overload failover through the conductor
Executor-level tests cannot show what the client finally receives. These
exercise ExecuteStream end to end to pin three properties that are easy
to regress:
- consecutive overloaded credentials are skipped until one serves the
request, and retries are capped by max-retry-credentials rather than
multiplying with request-retry
- exhausting the pool surfaces the upstream status instead of a
committed 200 stream
- with buffering disabled the rejection stays an in-stream error on a
committed stream, which is the behaviour the feature must preserve
The third case also documents why the executor returns its error
synchronously: an error arriving as the first stream chunk is wrapped and
downgraded into a committed 200, silently losing the status.
- Preserve API key entries with empty `api_key` when `base_url` is configured, and extend config dedupe/ID logic to include base URL, proxy, prefix, and headers so identities are stable.
- Update config/auth handling so `auth_kind=apikey` is treated as config API-key auth even without an `api_key` field, enabling base_url-only credential records.
- Ensure Gemini/Codex/XAI request path clears `Authorization`/provider auth headers when token is empty to avoid leaking unrelated auth state.
- Add bidirectional remapping for nested tool_references inside tool_search_tool_result across non-stream, SSE stream, and multi-turn message history.
- Add advisor_ and agent_toolset_ to IsClaudeServerToolType to prevent schema stripping and MCP aliasing on native Anthropic server tools.
- Wrap MCP alias restoration errors in claudeMCPAliasRestoreError with IsRequestScoped() bool to avoid cooling down healthy OAuth credentials.
- Add unit tests for tool_search_tool_result remapping, error variants, server tool recognition, and error scoping.
* fix(executor): prepend empty user turn for model-first requests targeting Gemini/Antigravity (#4959)
When forwarding sliced conversation histories or tool calls across OpenAI Responses,
OpenAI Chat Completions, Claude Messages, and native Gemini, native Gemini and Antigravity
Gemini endpoints require that conversation contents begin with a user turn.
Normalize leading turns at the executor boundary rather than the translator layer:
- Prepend an empty user turn ({"role":"user","parts":[{"text":""}]}) for Gemini, Gemini Vertex,
AI Studio, and Antigravity Gemini generation and CountTokens requests if the first turn is 'model'.
- Keep Antigravity Claude requests untouched to avoid adapter 400 errors.
- Ensure normalization runs after payload rules so payload index overrides target the original turns.
- Use no-copy GJSON inspection to keep overhead zero on valid user-first requests.
* fix(executor): inject Antigravity leading user after reasoning replay (#4959)
Replay can insert a model functionCall at contents[0] for sliced
tool-result history. Run the empty-user prepend on the final
requestPayload, after sanitize and prepareAntigravityGeminiReasoningReplayPayload.
- Propagate request headers into custom-header resolution for OpenAI/Gemini/XAI/Codex execution and websocket flows.
- Resolve auth `header:` values like `$ABC` from incoming request headers at request time and omit headers when no value is available.
- Add documentation for the dynamic custom-header behavior in `config.example.yaml`.
Closes: #5053
- Extract the credential-identity block shared by the streaming and
non-streaming Claude paths into applyClaudeCLIIdentity, so the identity
seed choice (API key versus stable Kimi auth identity) cannot drift
between the two paths
- Move stripDefaultKimiClaudeCodeAttribution next to the other attribution
and CCH helpers in claude_signing.go; it is only called from the Claude
executor paths and never from the Kimi executor itself
- Reattach the addConfigHeadersToAttrs doc comment to its function in the
watcher synthesizer helpers
Claude Code 2.1.220 through 2.1.234 emit the cch attribution only for
firstParty on api.anthropic.com and for vertex; every other backend sends
the billing header unsigned. CPA had dropped its endpoint check, so an
opted-in API key signed a per-request hash on any gateway and could bust
that gateway's prompt cache.
- Restore the endpoint gate in claudeCCHSigningEnabled: a real Claude OAuth
credential still signs on every upstream, because a downstream Claude Code
pointed at CPA cannot produce that value itself, while a claude-code-cli
API key signs only on api.anthropic.com or Vertex
- Drop the unused origin parameter from Claude fingerprint policy resolution
and restore the original resolveClaudeWirePolicy signature; the wire profile
follows the credential and only CCH follows the origin
- Add config.NormalizeClaudeFingerprintProfile / ValidateClaudeFingerprintProfile
as the single source of truth for fingerprint-profile values
- Reject unknown fingerprint-profile values in the Management API, and warn
once per distinct value at request time instead of on every resolution,
which previously logged about four warnings per request for one typo
- Preserve unrecognized values through config sanitization so rewriting a
config file never discards operator input
- Update config.example.yaml and tests for the origin-scoped CCH behavior
* feat(config): add fingerprint-profile to Claude keys and auth JSON
- Add FingerprintProfile to ClaudeKey configuration struct and normalizer
- Track fingerprint-profile in config diff
- Map fingerprint-profile / fingerprint_profile to auth attributes in file and config synthesizers
- Support fingerprint-profile in Management API PatchClaudeKey and normalization
- Add Claude billing attribution string manipulation utilities in internal/util
- Document fingerprint-profile options in config.example.yaml
* feat(claude): add fingerprint policy and request-local CLI identity
- Centralize Claude fingerprint policy resolution in claude_fingerprint_policy.go
- Support stable Claude CLI identity synthesis (UUIDv5 account_uuid and SHA-256 device_id)
seeded from API keys or stable OAuth IDs, keeping access tokens isolated
- Warn on unrecognized fingerprint-profile values
* feat(claude): apply CLI fingerprint to Messages and keep API keys caller-owned
- Wire centralized fingerprint policy into Claude and Kimi executors
- Keep first-party Anthropic API keys and delegated providers caller-owned by default
- Apply Claude Code CLI wire profile (betas, metadata, diagnostics, MCP aliases)
when fingerprint-profile=claude-code-cli is configured
- Strictly align CCH signing with native Claude Code 2.1.220: only first-party
api.anthropic.com and Vertex sign dynamic CCH; third-party gateways and Kimi
receive billing header without cch= to avoid prompt cache busting
- Respect caller-owned count_tokens bodies by default while aligning CLI shape on opt-in
- Fall back to CLIProxyAPI/<version> User-Agent when caller sends no UA in caller-owned mode
- Scope custom operator header overrides accurately in caller-owned mode
- Add comprehensive test coverage for policy resolution, gateway opt-in, Kimi, and token counting
- add shared `EnsureResponsesUsageDetails` helper to patch `usage` objects with:
- `output_tokens_details.reasoning_tokens = 0`
- `input_tokens_details.cached_tokens = 0`
- for both plain JSON and SSE `data:` frames, including multi-line frames
- apply the helper to OpenAI Response format outputs in non-stream and stream paths across executors/plugins so translated payloads consistently include required usage details
- update websocket/completion payload builders to emit default `usage` detail fields for prewarm/finish responses
Closes: #4985
injectClaudeCodeCurrentDate inserted the reminder at index 0 of the first user message. Anthropic requires the message after an assistant tool_use turn to lead with its tool_result blocks, so the request was rejected with 400. Advance the insert index past leading tool_result blocks, matching the existing guard in prependClaudeSystemRemindersToFirstUserMessage. Every other content shape keeps the current index-0 placement.
The word-based virtual server spans only ~2048^2 names, and plausible
real MCP server names such as file_system or web_search are valid BIP-39
word pairs. When a caller's own server matched the derived one, its tools
stopped passing through and entered alias recovery instead: they were
silently restored to an unrelated proxied tool, or failed the request
with a 500 when no semantic suffix matched.
Record untouched caller MCP tool names as identity entries in the reverse
map, skip those entries when collecting virtual servers and recovery
candidates, and forward them unchanged on an exact hit. Recording is
skipped when nothing was aliased, so an untouched request still keeps an
empty reverse map and a no-op restore path.
Also warn instead of silently forwarding an original name when the alias
space is exhausted, report an empty embedded wordlist, trace the
semantic-suffix fallback because it guesses rather than fails, and build
both alias entry points through one shared constructor so the exhaustion
tests cannot drift away from the production path.
Refs #4916
Replace high-entropy Base32 alias IDs with request-local BIP-39 English
words so weaker models are less likely to drift tool names. Keep a
two-word virtual server plus one-word tool ID, linearly probe wordlist
space on collision without self-overlap, and fall through to unambiguous
longest semantic-suffix recovery after a successful but wrong parse.
Fixes#4916
- Add `defer reporter.EnsurePublished(ctx)` to Gemini, Gemini Vertex, and AI Studio streaming goroutines so stream reporting is always finalized on exit.
- Update Gemini usage parsing to reject all-zero `usageMetadata` frames, preventing placeholder usage events from being accepted.
Closes: #4964
- Normalize repeated `mcp__<server>__` alias prefixes during reverse remapping to resolve stacked aliases.
- Add a semantic-suffix fallback when parsing fails, allowing unambiguous recovery from malformed tool IDs.
Closes: #4916
- Handle reversed tool names starting with `mcp__<server>__<server>__` by resolving them against the exact alias map before regular matching.
- Prevent failed reverse remapping for malformed/duplicated MCP alias patterns by short-circuiting to the direct alias match path.
Closes: #4916
- Propagate pending terminal errors when image/response data streams close, emitting `error`/`failed` SSE events (or HTTP error responses when no stream started) instead of dropping them.
- Sanitize and normalize streamed terminal errors before writing, and reuse normalized errors for cancellation.
- Improve SSE parsing/frame handling for multiline and cross-chunk payloads, prioritize payload/event-level failures, and avoid mutating emitted frame buffers by cloning chunks before queueing.
Closes: #4904
- Introduce a shared `isUsableKimiReasoning` helper and constant for the placeholder string.
- Prevent tool-call normalization from propagating or reusing placeholder/empty reasoning content.
- Preserve and reuse the latest valid assistant reasoning when normalizing `reasoning_content` for tool messages.
Closes: #4899
- Track normalized ID state (`occupied`/`preserved`) during preprocessing to avoid remapping valid existing IDs.
- Resolve collisions by appending deterministic hash-based suffixes until a free ID is found.
- Share occupancy tracking across shortening and remapping so sanitized IDs remain stable and idempotent.
Closes: #4891
A caller can put a {"role":"system"} turn inside messages. Models older than
the role=system turn reject it outright, verified against api.anthropic.com on
both /v1/messages and /v1/messages/count_tokens:
400 role 'system' is not supported on this model
claude-haiku-4-5, claude-sonnet-4-5, claude-sonnet-4-6 and claude-opus-4-6
answer that way, while claude-sonnet-5 and claude-opus-5 accept the turn.
claudeLegacySystemReminderModels already enumerates that boundary, which is why
claudeCodeCLIBetas withholds mid-conversation-system-2026-04-07 for those
models, but caller-provided turns were forwarded unchanged and always spent an
upstream call on a guaranteed rejection.
The native client does not produce the pairing either: it gates the turn on the
model. In 314 captured native requests the turn appears only on
claude-opus-5 and claude-sonnet-5, and on none of the 43 requests addressed to
a model in that set. That is an observation about the captures rather than a
proof about the upstream, so it only corroborates the measured rejection.
Validate the finished body, and only inside the evidence that produced the
rule:
- The check runs after the body is finalized and before
http.NewRequestWithContext. Payload rules can rewrite model and messages long
after translation, so an earlier check would not describe what is sent.
- Only Anthropic's first-party origin is covered, matching the reasoning
shouldUseClaudeUpstreamTokenCount already applies to count_tokens. A
third-party gateway may map these model IDs onto something that accepts the
turn and therefore decides for itself.
- A confirmed native caller keeps the passthrough. It gates the turn itself, so
its body is forwarded untouched and the upstream error reaches it unchanged.
- Unknown and future model IDs stay optimistic and are forwarded, matching how
checkSystemInstructions treats them.
- rebuild_mid_system_message still folds caller turns into the system slot; the
final check therefore preserves the explicit escape hatch.
Cloaking adds one extra ordering case: it can place a caller's top-level system
prompt into a role=system turn for a modern model before a payload rule changes
the model to legacy. Track only the exact contiguous turns that CPA inserted,
using both their position and the corresponding message-count increase as
provenance. After payload rules settle the model, replay those turns through the
existing legacy <system-reminder> path. Pre-existing caller turns, even if they
have identical content, remain caller-owned and are still rejected. If payload
rules also rewrite the tracked messages, reconciliation fails closed and final
validation returns 400 rather than guessing provenance.
The 400 is request-scoped like claudeCallerSystemBlockError: the invalid
body/model pairing is independent of first-party credential health, so no
credential is cooled or retried.
Translated requests never carry the pairing, because every non-Claude source
format hoists system content into the top-level system field. Tests pin that for
OpenAI, Gemini, Responses and Interactions, and separately drive Execute,
ExecuteStream and CountTokens through an injected first-party transport.
Anthropic rejects the injected clear_thinking_20251015 strategy unless
thinking is enabled or adaptive:
`clear_thinking_20251015` strategy requires `thinking` to be enabled or
adaptive
Both sides of the automatic injection only special-cased the literal
{"type":"disabled"}, so an absent thinking field slipped through
gjson's empty string and the request was sent with a strategy the API
refuses. Two reachable paths produced it:
- a cloaked caller that never sent thinking at all;
- a caller whose thinking was enabled at injection time and then removed
by disableThinkingIfToolChoiceForced, which runs between injection and
reconciliation.
Both now test the accepted values instead of excluding the disabled one,
via a shared claudeThinkingAcceptsClearThinking helper, so reconciliation
also withdraws an object CPA injected itself once thinking disappears.
Caller-owned and payload-rule-owned objects keep their existing
precedence and are never withdrawn.
Verified against api.anthropic.com: context_management with thinking
absent or disabled returns 400, with thinking enabled returns 200. The
existing expectation that forced tool choice retains the automatic object
pinned the rejected shape and is inverted accordingly.
Claude Code issues measured Haiku helper requests that intentionally omit the
claude-code beta and, for the minimal shape, the system field. Treat those
profiles as confirmed native clients so CPA does not cloak or rewrite them.
Accept the native metadata builder optional parent_session_id, require
platform headers only for presence and software baseline rather than exact
equality, derive timeout and version from the same header defaults the emitter
uses, and keep discriminating signals strict: exact beta allowlist, body
shape, lowercase hex CCH, and session binding. Preserve helper transport
headers, omit synthetic stream false on markerless helpers, and gate the
billing and CCH fallback on system presence so a later pipeline-attached system
prompt cannot go upstream unsigned while the measured no-system wire stays
intact.
Cloaked Responses/Chat/Gemini→Claude traffic was skipping ensureCacheControl
once the first-user cloak marker existed, freezing multi-turn cache hits.
Always ensure section-independent system and rolling-message breakpoints for
non-native callers, keep confirmed Claude Code placement intact, and leave
tools unstamped when a usable system prompt already covers the prefix.
Align the wire shape with the Claude Code 2.1.220/2.1.221/2.1.227 cache-control
constructor: the default is {"type":"ephemeral"} with no ttl. A 1h body ttl is
applied only after placement, only for OAuth credentials, and only onto blocks
that already carry cache_control, so it stays strictly paired with
extended-cache-ttl-2025-04-11. Explicit caller-owned 5m choices survive.
Match the native rolling selector (skip thinking-like assistant tails; final
system string special case), fall back to a tools breakpoint when system is
absent or blank, and leave empty/whitespace string systems unconverted so they
cannot double-stamp tools plus a useless system host.
Review and live-test follow-ups to the shared upstream transport.
Bound the cache with an LRU that closes idle connections on eviction, so
rotating a credential's proxy or supplying a per-request base transport can
no longer leak pools.
Stop deriving a pool scope from Auth.Label: it is documented as an optional
human readable label for logging and carries no uniqueness guarantee, so two
OAuth identities sharing a label would share one TCP/TLS pool. Prefer a
refresh-token digest, which stays stable across access-token rotation and is
available to refresh requests that run before any access token exists.
Replace a typed-nil *http.Transport taken from the request context. It passes
the interface nil check, so leaving it in place made http.Client fall back to
http.DefaultTransport, which advertises h2 over ALPN and breaks the
HTTP/1.1-only fingerprint.
Only widen pool limits: treat MaxIdleConns == 0 and IdleConnTimeout == 0 as
unlimited, and leave a negative MaxIdleConnsPerHost alone because that is how
an operator disables pooling.
Size the cache for large deployments. An unused entry costs under 1 KB and no
goroutines, whereas evicting a live pool forces a fresh TCP + TLS handshake,
so capacity is not the lever for bounding memory.
Execute, executeClaudeNonStream and ExecuteStream all assign the validated
original payload to the request and then translate both values. Since both
translations saw the same bytes, every Antigravity request paid for a second
full scan of the client payload. On a captured 24MB tool-history request that
second pass cost roughly 0.9s of CPU and 651MB of allocations.
Translate once when both inputs share a backing array and hand the caller an
independent duplicate, because later stages edit the working copy in place.
Payloads that genuinely differ still get two translations.
resetAntigravityCreditsRetryState assigned a fresh sync.Map to each
package-level credits variable. Credits hint refreshes run on background
goroutines that outlive the request, so they can still be writing those
maps when a test's cleanup runs. Replacing the variable is an
unsynchronized write to the same memory the goroutine is mutating, which
made `go test -race ./internal/runtime/executor` fail with two data races
in TestAntigravityExecute_NoCreditsWithoutConductorFlag.
Empty each map in place with sync.Map.Clear instead. Clear is safe against
concurrent users, and the reset semantics are unchanged.
The races were pre-existing and reproduce identically on b921b5d0; only
test code changes here. Production code never assigned these variables, so
it was never affected.
before go test -race ./internal/runtime/executor FAIL, 2 data races
after go test -race ./internal/runtime/executor ok, 0 data races
The replay request index exists solely so that the next item in a
sequential apply loop observes the mutated payload. Both loops rebuilt it
after every successful mutation, including after the final item, whose
rebuild no successor ever reads.
insertAntigravityReasoningReplayItemsWithSchemas is worse than that: its
index parameter is a local pointer, so a rebuild is discarded the moment
the function returns. applyAntigravityReasoningReplayItems only ever
passes a single eligible item, which made every one of those rebuilds
dead work, and the caller then rebuilt the same payload again.
Rebuild only when a successor still has to read the index. Behavior is
unchanged: 800 differential rows over 400 randomized multi-turn payloads
(570 of them mutating) hash-match the previous implementation byte for
byte, under both nil and populated tool schemas.
BenchmarkApplyAntigravityReasoningReplayItems/indexed (1 MiB, 32 turns):
before ~302 ms/op 66309 allocs/op
after ~269 ms/op 62153 allocs/op
A real 24 MiB production payload (182 contents, 96 ledger items) shows
the effect the synthetic benchmark understates:
before 6723 ms/op 223381 allocs/op
after 4035 ms/op 144789 allocs/op
The indexing change left two parallel implementations of the same replay
matching logic: the filter path used the request index while the merge and
insert paths still rescanned the payload. Filter and merge must agree on which
part a ledger item targets, so the duplication was a latent source of signatures
being replayed onto the wrong part.
Move the write path onto the index. Every lookup in the merge path already ran
before the first mutation, so one index describes the whole call; insert rebuilds
it after each mutation to keep sequential semantics. Collapse the thought
signature locator into thoughtSignaturePartIndex, shared by the eligibility check
and the write path.
Delete the superseded payload-scanning implementations along with functions that
had no production callers left: antigravityNeedsSignatureReplayForExistingFunctionCall,
antigravityRequestHasMatchingFunctionResponse, antigravityPayloadHasFunctionCallID,
filterAntigravityReasoningReplayItemsForRequest, insertAntigravityReasoningReplayItems,
mergeAntigravityFunctionCallPartReplay and antigravitySetReplayItemContextHash.
Freeze the pre-index implementations into a dedicated oracle test file. The
differential tests previously called the production functions they were meant to
check, so consolidating the logic would have silently turned them into
self-comparisons.
Malformed non-array parts now fail closed consistently. gjson's Result.Array()
yields a one-element slice for a non-array non-null value, so the old fallback
scans could match a functionCall inside a parts object while the primary ID
lookup could not. The end state was already identical because such a payload
cannot be written to; the behavior is pinned by a test.
Also cover the positional fallback for pre-targetHash cache entries, which had
no test at all, and add a write-path benchmark.
Document that the replay request index is request-scoped, retains no-copy GJSON
results aliasing the payload, and is not safe for concurrent use, and that the
context fingerprints snapshot a running hash lazily in content order.
Replace the fingerprint byte counter with a boolean since it only distinguishes
an empty fingerprint from a hashed one.
Guard reasoningReplayItemsFromRequest on validContents so a valid but empty
contents array keeps returning an empty non-nil slice, exactly as it did before
the index was introduced, and lock that invariant with a test.
Cache request contents, function locations, and incremental context fingerprints so large replay histories are scanned once. Rebuild the request index only after mutations and reuse the precomputed response context hash in replay accumulators.
- Create a fresh uTLS/HTTP-2 connection per request instead of reusing cached connections.
- Pass request context through dialing and TLS handshake to make `RoundTrip` cancelable.
- Wrap response bodies so the underlying HTTP/2 connection is closed when the body is closed, and clean up promptly on request/response failures.
Closes: #4878
