Commit Graph

210 Commits

Author SHA1 Message Date
Luis Pater
cd5af08e31 Merge pull request #5928 from Viggo95/feat/codex-response-model-observability 2026-09-18 21:05:35 +08:00
Luis Pater
cc545cbf90 fix(openai): align tool call messages and preserve ordering on ambiguous outputs
- Add `AlignOpenAIToolCallMessages` to reorder tool results immediately after the matching assistant tool calls while preserving original content and numeric precision.
- Prevent deferred message reordering and call ID guessing when tool outputs are incomplete, duplicate, or missing IDs.
- Normalize translated requests after applying summary configuration in Codex multi-agent execution.

Closes: #5925
2026-09-18 12:32:21 +08:00
Viggo95
25f40d8cf8 feat(codex): record upstream response model and warn on silent model substitution
Codex upstreams can silently serve a different model than the one requested
(HTTP 200, with response.model naming the substitute). The proxy kept no record
of it: nothing logged, nothing reported, only the pass-through response body.

- Add Record.ResponseModel to sdk/cliproxy/usage, aligned with the existing
  ResponseServiceTier field, and emit it from the redis usage queue as the
  optional response_model payload field alongside response_service_tier. Only
  the record for the requested model carries it: additional-model records
  (image generation tool usage) describe a side model the upstream response
  never refers to, and would otherwise look like a substitution downstream.
- Add internal/runtime/executor/helps/response_model.go with
  extractCodexResponseModelEvent (SSE frames and raw JSON, restricted to the events
  that embed the authoritative response object, rejecting non-string and
  oversized upstream model names) and IsCodexModelSubstituted (both sides
  trimmed, lower-cased and stripped of thinking suffixes, dated aliases such as
  gpt-5.6-terra-2026-05-13 accepted in either direction).
- UsageReporter records the served model on the event path and emits the WARN
  when the attempt publishes its usage record, so no logging work happens
  before the first event is forwarded. Repeats are throttled per
  (auth id, requested model, served model) with a 10 minute window, because on
  an affected credential every request is substituted and an unthrottled
  warning would mirror the whole request volume into the logs. The credential
  is labelled auth_index=<index> only: codex credential file names embed the
  account e-mail, which must not be written to the logs at request rate.

Coverage, by entry point. The served model is observed on the HTTP streaming
path (both the bootstrap-buffered handshake and the streaming goroutine), the
HTTP non-streaming Execute loop, the websocket streaming and non-streaming
paths, and the two /responses-shaped image entry points. The remaining codex
entry points cannot report it and are therefore left alone: executeCompact
(/responses/compact answers with a compaction object that has no event type and
no response.model), the two direct image endpoints (/images/generations and
/images/edits answer in the Images API shape and stream image_generation.*
events), and CountTokens (counts locally with tiktoken, never reaching an
upstream).

TokenAccountingSchemaVersion is not bumped: it versions the token accounting
contract (token breakdown semantics), and this change only adds an optional
non-token field that leaves existing consumers and all token math untouched.

Note: response_model ships with the usage record and is the counting source;
the WARN is a throttled alerting signal and must not be used to count
substitutions.

Tests: table-driven unit tests for both helpers over real model ids, reporter
tests covering the published record, the single throttled warning, the absence
of account identifiers in it, concurrent observation and publishing under
-race, the throttle window and its entry bound, an executor-level guard for the
observeCodexTokenEvent wiring and the per-model records, plus a redisqueue
payload assertion for response_model. gofmt, go vet, go test -race on the
touched packages and go test ./... are clean.
2026-09-18 12:31:52 +08:00
Luis Pater
1cce932573 fix(claude): skip retry-after header on overage-only rejections
- Skip parsing the `Retry-After` header when a rate limit rejection is overage-only to prevent global credential cooldowns.
- Allow exponential backoff to handle model recovery while keeping shared subscription windows available for other models.

Closes: #5920
2026-09-18 09:15:48 +08:00
Luis Pater
44eaef0009 feat(claude): support model-level cooling and scope overage rate limits
- Add `claude.model-level-cooling` configuration to scope rate limit cooldowns to the requested model.
- Treat overage-only and spend cap rejections as model-scoped when shared subscription windows remain healthy.
- Propagate model-level cooling settings into streaming, token counting, and direct execution error classifiers.

Closes: #5915
2026-09-18 01:55:36 +08:00
Luis Pater
b6fe4f20c4 fix(devin): handle orphaned tool results and normalize function result payloads
- Match tool results against pending tool calls and downgrade unmatched results to user messages.
- Prevent downgraded orphaned tool results from consuming images intended for user turns.
- Unwrap protocol wrapper envelopes and extract structured text parts while preserving arbitrary business JSON.
- Provide a placeholder for empty or whitespace-only tool results.

Closes: #5911
2026-09-18 01:34:38 +08:00
Luis Pater
9e10db53ad fix(devin): aggregate tool calls by id and track cache write tokens
- Track and aggregate tool calls by call ID instead of slot index in streaming and buffered execution.
- Support raw arguments from invalid JSON fields for custom tool calls.
- Parse usage field 4 as cache write tokens instead of adding to prompt tokens.
- Align client metadata with the default client name and drop deprecated tag 28.

Closes: #5910
2026-09-18 01:04:43 +08:00
Luis Pater
8c664b2fed Merge pull request #5896 from router-for-me/translator
feat(translator): preserve model metadata in requests
2026-09-17 13:52:35 +08:00
Luis Pater
ad088a8795 fix(devin): filter automation update tools and sanitize tool descriptions
- Filter out `automation_update` tools and sanitize tool descriptions in Devin wire requests and logs.
- Strip additional Codex prompt directives from system messages.
- Support `children` field fallback when collecting namespace tools.
2026-09-17 13:51:26 +08:00
hkfires
a9e92b8145 feat(translator): preserve model metadata in requests 2026-09-17 13:10:47 +08:00
Luis Pater
c4982e846e fix(executor): strip relayed tool result images for text-only models
- Replace tool image placeholders with omission markers for compatibility with text-only upstream models.
- Strip synthetic image relay notices and image parts from user messages.

Closes: #5884
2026-09-17 09:57:13 +08:00
Luis Pater
4613cfd44e Merge pull request #5870 from avabbbb/fix/devin-high-demand-rate-limit
fix(devin): classify high-demand errors as rate limits
2026-09-17 03:59:25 +08:00
bekkilove
d44901f91c fix(devin): classify high-demand errors as rate limits
Devin upstream occasionally encodes transient capacity failures using
Connect code permission_denied with message containing 'high demand'.
CPA's ParseDevinTrailerError currently maps every permission_denied to
HTTP 403. The auth cooldown manager interprets 403 as a 30-minute model
permission cooldown, keeping a recovered model locally unavailable.

This fix narrowly reclassifies the observed high-demand variant as
HTTP 429 (Too Many Requests), so it enters the quota/retry cooldown
path instead of the long permission denial path. Genuine
permission_denied errors (model access denied, plan entitlement denied,
etc.) remain HTTP 403.

Tests: 4 new cases in TestParseDevinTrailerError covering transient
high-demand (429), genuine permission error (403), resource_exhausted
unchanged (429), and case-insensitive matching. All existing tests
pass with no regressions.
2026-09-16 19:24:08 +08:00
bekkilove
dea4ce8aeb fix(devin): support swe-1-6-slow model variant 2026-09-16 16:49:49 +08:00
sususu
4c331bb953 fix(devin): unwrap repeated field 28 groups, merge partial field 7 usage, and harden APICall escaping 2026-09-14 09:32:05 +08:00
sususu
b4749cb204 fix(devin): parse field 8 header submessages, accumulate field 4 prompt tokens, and add field 28 usage fallback 2026-09-14 09:19:51 +08:00
sususu
30b2ac8996 refactor(devin): deduplicate auth credentials extraction, filter sparse tool calls, and optimize model lookup 2026-09-13 23:34:43 +08:00
sususu
fb2c1c1afa fix(devin): transport reuse, interleaved stream steps, strings.Builder panic, and updater URL 2026-09-13 23:34:43 +08:00
sususu
926450e87b fix(devin): dynamic catalog-driven chat_model_uid resolution and effort clamping 2026-09-13 23:34:43 +08:00
sususu
1604cb0334 fix(devin): normalize upstream internal errors to 502 Bad Gateway 2026-09-13 23:34:43 +08:00
sususu
98b106f0e8 fix(devin): store transient quota metrics strictly in Quota.Signals and keep Metadata static 2026-09-13 23:34:43 +08:00
sususu
6a239f5715 fix(devin): propagate stream chunk errors, fix truncated protobuf infinite loop, respect ctx roundtripper, and limit auth response body reads 2026-09-13 23:34:43 +08:00
sususu
86de823daa perf(devin): cache Devin HTTP transports per proxy URL to reuse connection pools 2026-09-13 23:34:43 +08:00
sususu
6c7d2d57f7 perf(devin): cache sensitive word regex matcher, preallocate request bytes and frame decompression buffer 2026-09-13 23:34:43 +08:00
sususu
5d0c77cf3f fix(devin): enforce Connect-RPC EOS trailer invariant, validate frame flag, and bind OAuth callback to ctx 2026-09-13 23:34:43 +08:00
sususu
a5ea971f35 fix(devin): sort streaming tool call stop events and normalize prompt CRLF 2026-09-13 23:34:43 +08:00
sususu
f1f5506c0b fix(devin): align wire protocol, harden streaming, and resolve multi-turn tool/signature parity
- Wire parity: align Connect-RPC Sentry-Trace, User-Agent suppression, float32 double pattern, and dynamic 732-char hex device fingerprint
- Session ordinal & cache: implement process-scoped Field 15.2 with bounded LRU (5000 entries) and Field 15.4=14 user boundary; prioritize stable session_id over previous_interaction_id to preserve prompt caching
- Streaming robustness: unblock hung TCP reads on client cancellation via context watcher; accurately propagate stream read errors and trailer errors instead of swallowing truncated frames
- Thought signature & reasoning: emit raw delta signatures directly in active thought steps; eliminate redundant tail base64 re-encoding; ensure 1:1 assistant signature and thinking alignment across multi-turn history
- Tool call de-multiplexing: route parallel tool calls by tc.Index in both streaming step events and non-streaming aggregations
- Security & transport: escape OAuth callback error HTML against reflected XSS, enforce strict state validation, and isolate Devin HTTP transport with tr.Clone()
2026-09-13 23:34:43 +08:00
sususu
bf06746d42 feat(devin): map model aliases and swe-1-7/haiku/sonnet/gpt-4-1 UIDs 2026-09-13 23:34:43 +08:00
sususu
2683ec201d feat(cmd): add fetch_devin_models CLI tool for dynamic model catalog extraction 2026-09-13 23:34:43 +08:00
sususu
469aa3678f feat(devin): parse protobuf timestamp and harden partial failure logging 2026-09-13 23:34:43 +08:00
sususu
16cb6c0b02 feat(devin): add symmetric decoded upstream response in request log 2026-09-13 23:34:43 +08:00
sususu
0c2351bb89 feat(devin): support none thinking level for glm-5-2 2026-09-13 23:34:43 +08:00
sususu
308e5ad3b1 feat(devin): add deepseek-v4-flash and deepseek-v4-1-flash models 2026-09-13 23:34:43 +08:00
sususu
2caab7dbf9 feat(devin): enhance request-log with intermediate interactions and decoded upstream body 2026-09-13 23:34:43 +08:00
sususu
ea2f29feec feat(devin): add devin/gemini-3-8-flash and devin/grok-4-6 model definitions and signature recognition
- Register static model definitions for devin/gemini-3-8-flash (1M context, Google) and devin/grok-4-6 (500k context, xAI).
- Support gemini38Efforts (low/medium/high) and grok46Efforts (low/medium/high/xhigh) in ResolveDevinChatModelUID, supporting both colon and parenthesis suffix parsing.
- Recognize Gemini Tink thought signatures (AY-prefix / 0x01 Tink header) in detectSignatureType and parseSignatureBytes.
- Add unit tests for both models in registry and devin_models.
2026-09-13 23:34:43 +08:00
sususu
5b8e3821b1 fix(devin): strip system prompt lines matching configured sensitive words to evade unicode normalization bypass 2026-09-13 23:34:43 +08:00
sususu
c0b76c2d09 refactor(devin): keep sensitive words strictly external in config.yaml without hardcoding 2026-09-13 23:34:43 +08:00
sususu
c0b86059c4 fix(devin): sanitize claude subagent identity and emoji directives to prevent content policy 403 2026-09-13 23:34:43 +08:00
sususu
f5247e496f fix(devin): restrict sensitive word obfuscation strictly to system prompt only 2026-09-13 23:34:43 +08:00
sususu
02fd1bde78 feat(devin): restrict glm-5-2 to free tier, remove static swe-1-7-lightning, and harden cloak 2026-09-13 23:34:43 +08:00
sususu
eed249072d feat(devin): prefix all Devin model IDs with devin/ namespace 2026-09-13 23:34:43 +08:00
sususu
cbe800aa28 feat(devin): bind upstream session_id and cascade_id to CPA canonical session 2026-09-13 23:34:43 +08:00
sususu
f94752762b feat(devin): add Devin/Cognition provider integration and CLI OAuth
Implement the full Devin/Cognition Connect-RPC provider support across all CPA endpoints (/v1/chat/completions, /v1/messages, /v1/responses), complete with binary protobuf wire framing, streaming tools/arguments delta handling, thinking/reasoning replay, and CLI OAuth authentication.

Key highlights:
- Wire Protocol & Streaming:
  * Implemented Connect-RPC uncompressed 5-byte framing (0x00 + 4-byte length + protobuf) for ApiServerService/GetChatMessage.
  * Implemented Devin protobuf encoder/decoder in internal/runtime/executor/helps/devin_wire.go, including ClientMetadata, prompts, tools, completion_config, and multimodal image handling (Prompt Field #10).
  * Stream frame consumption via interactions protocol, correctly mapping arguments_delta and tracking multiple sequential tool calls (currentToolCallActive).
  * Streaming thought summary and sealed.v1 signature deltas targeting the thinking step.

- Model Registration & Thinking Clamping:
  * Registered static fallback models in model_definitions.go (swe-2, claude-fable-5-1, gpt-6-astra, swe-1-7-lightning, glm-5-2, glm-5-3).
  * Configured ThinkingSupport with discrete levels per model family.
  * Implemented CPA-standard nearest-neighbor clamping for thinking levels (minimal/low -> medium, xhigh -> max for swe-2).
  * Mapped thinking effort to Devin upstream model UID (e.g. swe-2-medium, swe-2-high, swe-2-max).

- Sensitive Words & System Prompt Sanitization:
  * Added devin.sensitive-words configuration in internal/config/config_types.go and config.go, matching Antigravity conventions.
  * Supported zero-width space (\u200b) obfuscation in prompts, tools, and system instructions via SensitiveWordMatcher.
  * Stripped Claude Code billing headers (x-anthropic-billing-header:) and CLI identity signatures from system instructions and tool descriptions to avoid upstream content filter rejections.

- Signature Compatibility:
  * Added SignatureProviderSWE = "swe" recognizing sealed.v1.* reasoning signatures in internal/signature/provider_compatibility.go.
  * Propagated reasoning.encrypted_content on Responses API and thinking.signature on Messages API.

- Authentication:
  * Implemented Devin PKCE OAuth flow with loopback callback server and headless manual token/code paste (--no-browser).
  * Registered Devin authenticator in SDK and CLI (-devin-login flag).
  * Integrated with management OAuth session endpoints and credentials manager.
2026-09-13 23:34:43 +08:00
Luis Pater
94d6eb535e docs(config): document payload filter examples for codex tools
- Add example payload filter rules in `config.example.yaml` for stripping tools from both flat and nested `additional_tools` Codex request payloads.

Closes: #5792
2026-09-13 22:12:28 +08:00
Luis Pater
f702bc1ac2 feat(codex): preserve native fidelity for responses-lite requests
- Detect native responses-lite requests via headers and client metadata.
- Skip instructions normalization and synthetic session cloaking for native requests.
- Preserve upstream completion output during websocket response forwarding.

Closes: #5780
2026-09-13 15:05:03 +08:00
Luis Pater
377c315fd7 fix(claude): anchor billing fingerprint to initial turn for cloaked cache stability
- Derive billing fingerprint message text from the first user message to prevent prompt cache invalidation across turns.
- Centralize continuity tag resolution to maintain stable prompt IDs and request references across requests.

Closes: #5730
2026-09-11 16:59:57 +08:00
Luis Pater
4edf9d1dd6 fix(thinking): extract configuration_update reasoning effort in codex usage reporting
- Inspect `input` items for the latest `configuration_update` reasoning effort before falling back to top-level configuration.
- Route `codex`, `xai`, and `openai-response` providers through usage-specific thinking config extraction.
- Support `none`, `auto`, and explicit reasoning level modes from in-turn updates.

Closes: #5714
2026-09-11 12:15:57 +08:00
Luis Pater
c8f723e0fb feat(usage): propagate upstream base_url across usage records and plugin auth
- Add `BaseURL` field to usage records and host auth file entries.
- Extract `base_url` from auth attributes and metadata during usage reporting.
- Propagate `base_url` through plugin usage adapters and runtime auth callbacks.

Closes: #5693
2026-09-11 00:25:17 +08:00
Luis Pater
6a73f39627 fix(claude): preserve 1h cache ttl and beta header for subagent requests
- Retain `cache_control` blocks with 1h TTL and `extended-cache-ttl` beta header when explicitly requested by subagents.
- Detect 1h TTL configuration from request payloads and incoming Anthropic-Beta headers.
- Ensure `extended-cache-ttl` beta header is preserved or injected when 1h TTL is present.

Closes: #5629
2026-09-10 10:58:34 +08:00
Luis Pater
3bf787fc1d feat(auth): propagate canonical session id for custom header expansion
- Initialize `util.SessionIDResolver` to resolve session IDs from request contexts, metadata, and headers.
- Ensure canonical session metadata is injected into execution options across execution flows.
- Propagate session context in executors to support `$CPA-SESSION-ID` expansion in custom headers.
- Sync cleared or updated session identities back to execution contexts during conductor execution.

Closes: #5690
2026-09-10 01:43:37 +08:00