46 Commits

Author SHA1 Message Date
sususu98
18883a57b4 fix(usage,auth): propagate request-scoped error rule classification to usage fail.code 2026-11-10 14:17:12 +08:00
hkfires
216f2947fc feat(home): report reused dispatch turns in usage
Add attempt-scoped HomeSelection metadata carrying the reused and
additional flags, scheduling route model, and credential policy
through usage records to the Redis queue. Mark retained executions
and Codex duplex successors as reused so Home can compensate later
turns after completion without redispatching or reacquiring a
concurrency slot.

Mark split additional-model records as additional so they never add
scheduling charges, while preserving independent per-response
execution IDs for deduplication. Document the deferred accounting
and add coverage across conductor, duplex, reporter, and queue.
2026-11-09 22:31:47 +08:00
Dylan
d1e5cc607a fix(codex): reject unsupported hosted image requests 2026-11-06 18:38:00 +08:00
Luis Pater
f2e255f73e fix(usage): clear consumed queue slot in usage manager dispatcher
- Zero the dequeued queueItem slot in Manager.run before reslicing the queue so the backing array does not retain references to consumed request contexts and usage records while idle.
- Add a weak-pointer garbage collection regression test verifying that consumed request context payloads and record reference fields are reclaimed when the manager is idle.

Closes: #7267
2026-11-05 16:22:02 +08:00
sususu98
6d7e6a3bb9 feat(usage): extract cached image/audio and interactions modality tokens 2026-11-03 22:44:16 +08:00
sususu
bd5ec5310a feat(usage): capture structured Gemini, Vertex, AI Studio, and Antigravity usage 2026-10-29 14:44:33 +08:00
sususu
24840fddf8 feat(usage): capture structured xAI usage across chat, responses, and media 2026-10-29 14:40:08 +08:00
sususu
89e472f325 feat(usage): capture structured Anthropic Messages upstream usage details 2026-10-29 14:39:43 +08:00
Luis Pater
b74422180e fix(auth,usage): snapshot request state to avoid reading reused gin.Context (#7056) 2026-10-27 21:22:46 +08:00
Luis Pater
aa4130e64c fix(gemini): keep truncated HTTP 200 streams out of credential cooldown and failover
- Tag locally synthesized missing-finishReason failures as stream-truncated in the Gemini and Vertex executors, and carry that classification through the auth manager so it skips credential cooldown and stops cross-credential replay while the public 502 and partial usage are preserved.
- Carry the truncation classification through usage reporting and the Home queue payload so Home-side schedulers also skip the spurious cooldown, and add regression coverage for both the manager and usage paths.

Closes: #7005
2026-10-26 21:07:08 +08:00
Luis Pater
e5321f535c fix(auth,pluginapi,usage): share per-selection ID between Scheduler.Pick and UsageRecord
- generate a unique selection_id for each handled Scheduler.Pick decision and freeze it into the execution context and UsageReporter
- propagate SelectionID through core usage records and pluginapi.UsageRecord while clearing stale IDs on unhandled, rejected, Home, and credits fallback paths

Closes: #6988
2026-10-26 03:25:42 +08:00
Luis Pater
07b5a7a1ca feat(usage): report effective outbound service tier after payload rules
- Capture the post-translation, post-payload-rule service tier on UsageReporter and propagate EffectiveServiceTier through SDK usage records, plugin usage adapters, and Redis usage queue payloads.
- Ensure executors record translated request metadata after final payload configuration rules so server-side service_tier overrides are accurately reported alongside client and response tiers.

Closes: #6873
2026-10-22 21:05:27 +08:00
Luis Pater
37fe2608ee feat(usage): propagate access provider and native key metadata to usage records
- Inject API key, access provider, and native key flags into the request context during authentication
- Add `IsNativeKey` and `AccessProvider` fields to usage records, plugin API types, and redis queue payloads
- Provide context helper functions and automatic fallback resolution in the usage manager and reporter

Closes: #6509
2026-10-10 22:16:00 +08:00
Luis Pater
31f4cfab3f feat: migrate CLIProxyAPI to version 8 2026-09-27 14:42:26 +08:00
Luis Pater
2fe9932bb9 feat(usage): track execution request ID and trace ID across usage records
- Add `RequestID` (unique execution UUID) and `TraceID` (inbound request ID) fields to usage records and plugin API
- Generate unique UUIDs per model attempt in `UsageReporter` while linking to parent trace context
- Expose `execution_id` and `trace_id` in Redis queue usage payloads while preserving legacy `request_id`
2026-09-23 19:31:39 +08:00
Viggo95
25f40d8cf8 feat(codex): record upstream response model and warn on silent model substitution
Codex upstreams can silently serve a different model than the one requested
(HTTP 200, with response.model naming the substitute). The proxy kept no record
of it: nothing logged, nothing reported, only the pass-through response body.

- Add Record.ResponseModel to sdk/cliproxy/usage, aligned with the existing
  ResponseServiceTier field, and emit it from the redis usage queue as the
  optional response_model payload field alongside response_service_tier. Only
  the record for the requested model carries it: additional-model records
  (image generation tool usage) describe a side model the upstream response
  never refers to, and would otherwise look like a substitution downstream.
- Add internal/runtime/executor/helps/response_model.go with
  extractCodexResponseModelEvent (SSE frames and raw JSON, restricted to the events
  that embed the authoritative response object, rejecting non-string and
  oversized upstream model names) and IsCodexModelSubstituted (both sides
  trimmed, lower-cased and stripped of thinking suffixes, dated aliases such as
  gpt-5.6-terra-2026-05-13 accepted in either direction).
- UsageReporter records the served model on the event path and emits the WARN
  when the attempt publishes its usage record, so no logging work happens
  before the first event is forwarded. Repeats are throttled per
  (auth id, requested model, served model) with a 10 minute window, because on
  an affected credential every request is substituted and an unthrottled
  warning would mirror the whole request volume into the logs. The credential
  is labelled auth_index=<index> only: codex credential file names embed the
  account e-mail, which must not be written to the logs at request rate.

Coverage, by entry point. The served model is observed on the HTTP streaming
path (both the bootstrap-buffered handshake and the streaming goroutine), the
HTTP non-streaming Execute loop, the websocket streaming and non-streaming
paths, and the two /responses-shaped image entry points. The remaining codex
entry points cannot report it and are therefore left alone: executeCompact
(/responses/compact answers with a compaction object that has no event type and
no response.model), the two direct image endpoints (/images/generations and
/images/edits answer in the Images API shape and stream image_generation.*
events), and CountTokens (counts locally with tiktoken, never reaching an
upstream).

TokenAccountingSchemaVersion is not bumped: it versions the token accounting
contract (token breakdown semantics), and this change only adds an optional
non-token field that leaves existing consumers and all token math untouched.

Note: response_model ships with the usage record and is the counting source;
the WARN is a throttled alerting signal and must not be used to count
substitutions.

Tests: table-driven unit tests for both helpers over real model ids, reporter
tests covering the published record, the single throttled warning, the absence
of account identifiers in it, concurrent observation and publishing under
-race, the throttle window and its entry bound, an executor-level guard for the
observeCodexTokenEvent wiring and the per-model records, plus a redisqueue
payload assertion for response_model. gofmt, go vet, go test -race on the
touched packages and go test ./... are clean.
2026-09-18 12:31:52 +08:00
Luis Pater
c8f723e0fb feat(usage): propagate upstream base_url across usage records and plugin auth
- Add `BaseURL` field to usage records and host auth file entries.
- Extract `base_url` from auth attributes and metadata during usage reporting.
- Propagate `base_url` through plugin usage adapters and runtime auth callbacks.

Closes: #5693
2026-09-11 00:25:17 +08:00
sususu
580df36423 feat(usage): propagate session and parent session hierarchy to usage reporting queue
- Reuse coresession.ExtractSessionInfo across HTTP headers and request payloads to unify canonical session prefix namespaces with the scheduler.
- Extract hierarchical session identities in two phases: initial extraction from request headers on entry, and authoritative deep extraction once request payloads and metadata are available.
- Support Claude Code multi-level subagents (X-Claude-Code-Agent-Id, metadata.agent_id) and Codex thread fork lineages.
- Propagate SessionID and ParentSessionID across ClientRequestMetadata, UsageReporter, and coreusage.Record without root_session_id.
- Include session_id and parent_session_id in queuedUsageDetail for Home LPushUsage forwarding and Redis consumption with self-loop guards.
- Add comprehensive test coverage for canonical headers, body extraction, ghost parent elimination, and self-referential loop guards.
2026-09-06 10:45:32 +08:00
Luis Pater
9721d9939e feat(usage): track streaming execution state in usage records
- Add `Stream` field to usage records, context helpers, and Redis queue payloads.
- Propagate streaming mode across execution handlers, conductors, and usage reporters.

Closes: #5361
2026-08-31 20:29:19 +08:00
Supra4E8C
1e38a3a544 fix: retry Home OAuth requests after unauthorized 2026-08-03 21:39:48 +08:00
hkfires
a63da8ae76 Revert "Merge pull request #4687 from router-for-me/fix/home-401-refresh-recovery"
This reverts commit 4a31513673, reversing
changes made to 7d00936acc.
2026-07-31 16:35:33 +08:00
Supra4E8C
4db8e12029 fix: recover Home OAuth credentials after 401 2026-07-31 01:34:11 +08:00
Dylan
416a080174 fix(usage): add normalized token accounting v2 2026-07-23 21:27:03 +08:00
Luis Pater
768b4c49fd feat(usage): track generate flag with backward-compatible defaults
Propagate client generate metadata through usage records while keeping
legacy callers that omit the field enabled by default. Preserve an
existing context generate=false when metadata does not set the value,
and normalize omission to true only at publishing boundaries.
2026-07-15 21:30:09 +08:00
sususu98
03d58c4402 fix(usage): collapse OpenAI service tier metadata (#4275)
* fix(usage): preserve request service tier metadata

* fix(usage): collapse request tier metadata

* fix(usage): emit only service_tier and response_service_tier

Collapse request tier to service_tier for usage records and queue payloads. Keep RequestServiceTier as a deprecated input-only alias for older plugin callers.
2026-07-14 14:19:13 +08:00
Supra4E8C
dc4be1674a feat(usage): report request and response service tiers 2026-07-11 00:49:30 +08:00
Luis Pater
d625caddd9 feat(pluginhost): add capabilities for command-line flag handling and plugin execution
- Implemented command-line flag registration and execution for plugins with priority-based conflict resolution.
- Enabled plugin-owned command-line flag execution and persistence of plugin-auth data.
- Added new `Host` methods to support command-line capabilities, including flag normalization, validation, and execution state management.
- Introduced unit tests to ensure coverage for command-line plugin functionality, including auth data persistence.
- Updated configs to normalize plugins during initialization.
2026-06-06 18:35:17 +08:00
Luis Pater
959067edfb feat(usage): introduce executor type tracking in usage reporting
- Replaced `NewUsageReporter` with `NewExecutorUsageReporter` to include executor type in usage records.
- Updated all executors to use the new reporter implementation.
- Extended `UsageReporter` to track and publish executor type.
- Added tests to validate proper executor type recording and handling.
- Enhanced RedisQueue plugin and payload schema with executor type support.
2026-06-02 00:43:16 +08:00
Luis Pater
71c185f614 feat(usage): add service tier tracking and defaults in usage reporting
- Introduced `service_tier` metadata key to capture client-requested service tiers.
- Updated usage records, context propagation, and plugins to include service tier data.
- Added default handling logic for cases where `service_tier` is absent.
- Implemented tests for `service_tier` extraction, defaults, and updates across components.
2026-05-28 22:15:54 +08:00
Luis Pater
94c1b25146 feat(executor): add TTFT tracking and reporting for enhanced performance metrics
- Introduced Time-To-First-Token (TTFT) measurement and reporting across major executors.
- Added TTFT calculation to `UsageReporter`, including support for HTTP clients and WebSocket communication.
- Updated tests to validate TTFT tracking in streamed and non-streamed scenarios.
- Ensured integration with `usage` plugin and augmented usage records with TTFT data.
2026-05-28 02:59:24 +08:00
Luis Pater
11f0f906bd feat(logging): add SetTranslatedReasoningEffort to track reasoning levels in usage reporting
- Introduced `SetTranslatedReasoningEffort` method in `UsageReporter` to capture and log reasoning efforts from translated payloads.
- Updated executors to incorporate the new reporting functionality for handling reasoning efforts across various providers.
- Enhanced logging for thinking level extraction with new helper function `ExtractTranslatedReasoningEffort`.
2026-05-28 02:19:45 +08:00
yavon007
0de0ad0d36 Add reasoning effort to usage events 2026-05-19 22:10:48 +08:00
Luis Pater
ad98c9549a feat(runtime): track upstream response headers in logging and usage reporting
- Added APIs to store, retrieve, and clone upstream response headers in context for detailed logging.
- Updated `RecordAPIResponseMetadata`, `RecordAPIWebsocketHandshake`, and related methods to capture response headers.
- Extended `UsageReporter` to include response headers in published usage records.
- Enhanced payload tests to validate response headers' integrity and persistence.
- Refactored `usage.Record` to support optional `ResponseHeaders` field.
2026-05-19 01:29:23 +08:00
Luis Pater
bd8c05a830 feat(usage): add support for detailed token breakdown in usage tracking
- Introduced `CacheReadTokens` and `CacheCreationTokens` to enhance token breakdown.
- Refactored `parseClaudeUsageNode` for cleaner and reusable logic.
- Adjusted helpers and updated token calculations to align with the new fields.
2026-05-12 11:59:07 +08:00
Luis Pater
66c3dae06b feat(home): implement count for home auth dispatch requests and enable usage statistics
- Added `count` attribute to `homeAuthCount` requests to improve home message batching.
- Enabled usage statistics for home mode by default and added config-level enforcement.
- Adjusted failure logging to include detailed metadata in `UsageReporter`.
- Updated multiple executors to pass error details to `PublishFailure` for better debugging.
- Enhanced unit tests to validate `count` behavior and usage statistics enforcement across components.
2026-05-10 01:30:43 +08:00
Luis Pater
ba5d8ca733 feat(usage): add support for requested model alias handling
- Introduced methods for setting and retrieving model aliases in execution and usage contexts.
- Enhanced `UsageReporter` and related structures to include client-requested aliases.
- Updated tests to validate alias propagation and ensure correct usage reporting.
- Adjusted metadata handling in CLIProxyAPI executors to address alias integration.
2026-05-05 01:47:53 +08:00
Luis Pater
28d78273e4 feat(api): implement protocol multiplexer and Redis queue for usage integration
- Added `protocol_multiplexer.go`, enabling support for both HTTP and Redis protocols on a single listener.
- Introduced `redis_queue_protocol.go` to handle Redis-compatible RESP commands for queue management.
- Integrated `redisqueue` package, supporting in-memory queuing with expiration pruning.
- Updated server initialization to manage a shared listener and multiplex connections.
- Adjusted `Handler` to adopt `AuthenticateManagementKey` for modular key validation, supporting both HTTP and Redis flows.
2026-04-25 18:52:24 +08:00
clcc2019
c1bf298216 refactor: streamline usage reporting by consolidating record publishing logic
- Introduced a new method `buildRecord` in `usageReporter` to encapsulate record creation, improving code readability and maintainability.
- Added latency tracking to usage records, ensuring accurate reporting of request latencies.
- Updated tests to validate the inclusion of latency in usage records and ensure proper functionality of the new reporting structure.
2026-03-20 19:44:26 +08:00
Luis Pater
3ce0d76aa4 feat(usage): add import/export functionality for usage statistics and enhance deduplication logic 2025-12-26 11:49:51 +08:00
Luis Pater
cc3cf09c00 **feat(auth): add AuthIndex for diagnostics and ensure usage recording** 2025-11-19 22:02:40 +08:00
Luis Pater
20985d1a10 Refactor executor error handling and usage reporting
- Updated the Execute methods in various executors (GeminiCLIExecutor, GeminiExecutor, IFlowExecutor, OpenAICompatExecutor, QwenExecutor) to return a response and error as named return values for improved clarity.
- Enhanced error handling by deferring failure tracking in usage reporters, ensuring that failures are reported correctly.
- Improved response body handling by ensuring proper closure and error logging for HTTP responses across all executors.
- Added failure tracking and reporting in the usage reporter to capture unsuccessful requests.
- Updated the usage logging structure to include a 'Failed' field for better tracking of request outcomes.
- Adjusted the logic in the RequestStatistics and Record methods to accommodate the new failure tracking mechanism.
2025-10-21 11:22:24 +08:00
Luis Pater
32a8102d71 feat(usage): add support for tracking request source in usage records
- Introduced `Source` field to usage-related structs for better origin tracking.
- Updated `newUsageReporter` to resolve and populate the `Source` attribute.
- Implemented `resolveUsageSource` to determine source from auth metadata or API key.
2025-10-14 02:11:43 +08:00
Luis Pater
f5dc380b63 rebuild branch 2025-09-25 10:32:48 +08:00
Luis Pater
3f69254f43 remove all 2025-09-25 10:31:02 +08:00
Luis Pater
582677d067 refactor(usage): replace channel-based queue with mutex-protected slice
- Switched to a slice-based queue with mutex and condition variable for better control over queuing and dispatching.
- Removed fixed buffer size to handle dynamic queuing.
- Enhanced shutdown logic to safely close the queue and wake up waiting goroutines.
2025-09-24 03:59:26 +08:00
Luis Pater
3ade03f3b3 feat(usage): implement usage tracking infrastructure across executors
- Added `LoggerPlugin` to log usage metrics for observability.
- Introduced a new `Manager` to handle usage record queuing and plugin registration.
- Integrated new usage reporter and detailed metrics parsing into executors, covering providers like OpenAI, Codex, Claude, and Gemini.
- Improved token usage breakdown across streaming and non-streaming responses.
2025-09-24 03:49:09 +08:00