Commit Graph

45 Commits

Author SHA1 Message Date
Luis Pater
9721d9939e feat(usage): track streaming execution state in usage records
- Add `Stream` field to usage records, context helpers, and Redis queue payloads.
- Propagate streaming mode across execution handlers, conductors, and usage reporters.

Closes: #5361
2026-08-31 20:29:19 +08:00
hkfires
6a489fa84d fix(auth): prefer errors from upstream attempts
Track when executor calls cross an upstream transport boundary and use that
signal to keep model/provider errors from being replaced by later local
preparation, selection, or internal failures.

Mark HTTP, websocket, relay, and usage-tracked transports as upstream
attempts, while avoiding marks for local validation, logging, missing
sessions, and successful websocket handshakes before request send.

Parse relative auth expiry metadata and adjust Antigravity refresh timing.
2026-08-29 12:50:46 +08:00
hkfires
9a2201c36a fix(auth): forward Home unauthorized upstream errors
Stop refreshing Home-owned OAuth credentials after upstream 401s.
Preserve marked upstream response bodies for direct responses, usage
records, request logs, and websocket handshake failures.
2026-08-29 12:50:44 +08:00
sususu98
4b2beb3da1 feat(executor): measure effective TTFT with protocol-aware token classification (#5313)
* feat(executor): measure effective TTFT with protocol-aware token classification

- Introduce protocol-aware streaming token classification under internal/runtime/executor/helps
- Implement responses_ttft_helpers.go for OpenAI Responses / Codex WebSocket & SSE streaming
- Filter out container metadata, rate limits, handshake headers, and tool execution outputs
- Implement dual-track firstPacketDuration fallback for abnormal / zero-token stream terminations
- Add TrackHTTPClientRoundTripOnly to bypass false TTFT triggers on first response body reads
- Optimize UsageReporter.ObserveTokenEvent hot path with fast-path RLock short-circuiting (0 B/op)
- Align terminal event completions across SSE, WebSocket streaming, and non-streaming executions
- Add EnsurePublished metric recording fallback for successful terminal turns lacking usage blocks
- Add stub helpers with comprehensive specifications for Chat, Claude, and Gemini protocols

* fix(executor): normalize SSE response.done and terminate incomplete WS streams

* fix(usage): capture first-packet fallback on initial body reads with TrackHTTPClientRoundTripOnly
2026-08-29 01:22:21 +08:00
sususu
0f69f09a70 fix(logging): exclude HTTP 499 and client cancellations from forced error logs 2026-08-19 18:58:47 +08:00
Luis Pater
7cf92793c1 fix(gemini): skip zero-token usage placeholders and always finalize stream reporters
- Add `defer reporter.EnsurePublished(ctx)` to Gemini, Gemini Vertex, and AI Studio streaming goroutines so stream reporting is always finalized on exit.
- Update Gemini usage parsing to reject all-zero `usageMetadata` frames, preventing placeholder usage events from being accepted.

Closes: #4964
2026-08-14 22:38:56 +08:00
Luis Pater
4b3cc55cdc fix(cliproxy): centralize client error status mapping and apply context cancellation/deadline HTTP codes
Closes: #4601
2026-08-08 04:53:34 +08:00
sususu
8cf1d46f06 fix(usage): account for Claude thinking tokens 2026-08-03 22:30:00 +08:00
Supra4E8C
1df21b14bf fix(usage): update token fingerprint after refresh 2026-08-03 21:39:48 +08:00
Supra4E8C
1e38a3a544 fix: retry Home OAuth requests after unauthorized 2026-08-03 21:39:48 +08:00
hkfires
a63da8ae76 Revert "Merge pull request #4687 from router-for-me/fix/home-401-refresh-recovery"
This reverts commit 4a31513673, reversing
changes made to 7d00936acc.
2026-07-31 16:35:33 +08:00
Supra4E8C
4db8e12029 fix: recover Home OAuth credentials after 401 2026-07-31 01:34:11 +08:00
Luis Pater
42f36b94e0 fix(usage): harden canonical token normalization 2026-07-24 01:18:18 +08:00
Dylan
fe8a616aa3 fix(usage): classify partial token accounting correctly 2026-07-24 00:31:32 +08:00
Dylan
416a080174 fix(usage): add normalized token accounting v2 2026-07-23 21:27:03 +08:00
Luis Pater
768b4c49fd feat(usage): track generate flag with backward-compatible defaults
Propagate client generate metadata through usage records while keeping
legacy callers that omit the field enabled by default. Preserve an
existing context generate=false when metadata does not set the value,
and normalize omission to true only at publishing boundaries.
2026-07-15 21:30:09 +08:00
sususu98
03d58c4402 fix(usage): collapse OpenAI service tier metadata (#4275)
* fix(usage): preserve request service tier metadata

* fix(usage): collapse request tier metadata

* fix(usage): emit only service_tier and response_service_tier

Collapse request tier to service_tier for usage records and queue payloads. Keep RequestServiceTier as a deprecated input-only alias for older plugin callers.
2026-07-14 14:19:13 +08:00
Luis Pater
2075f77c8e feat(executor): enhance cache_tokens handling and extend parsing logic with alias normalization
- Added support for `cache_read_tokens` and `cache_creation_tokens` aliases across `prompt_tokens_details`, `input_tokens_details`, and other usage details.
- Updated token parsing logic to handle multiple aliases and improve fallback scenarios.
- Extended test coverage for alias normalization, including `cache_write_tokens`, `cacheCreationTokens`, and `cachedContentTokenCount`.
- Improved total token calculation by incorporating conditional read token addition logic.
2026-07-12 03:22:41 +08:00
Supra4E8C
bc812e5f74 perf(usage): avoid parsing irrelevant stream chunks 2026-07-11 02:42:42 +08:00
Supra4E8C
ea20742e77 fix(usage): retain response tier without usage 2026-07-11 01:37:39 +08:00
Supra4E8C
dc4be1674a feat(usage): report request and response service tiers 2026-07-11 00:49:30 +08:00
Luis Pater
b4c594050e chore(models): update default client version and user agent, revise GPT-5.5 configurations
- Bumped the `defaultClientVersion` and `defaultCodexUserAgent` to `0.144.0`.
- Replaced GPT-5.6 Sol with GPT-5.5 in model registry, adjusting metadata including context window length and reasoning levels.
- Updated `usage_helpers` to include `cache_write_tokens` logic for input token details.
2026-07-10 01:54:37 +08:00
Luis Pater
dea4787970 refactor(executor): centralize OpenAI stream usage handling with StreamUsageBuffer
- Introduced `StreamUsageBuffer` to encapsulate stream usage observation and publishing logic.
- Updated OpenAI-compatible executors and helpers to use `StreamUsageBuffer` for consistent stream usage tracking.
- Added unit tests to validate `StreamUsageBuffer` behavior, ensuring proper handling of usage details.

Closes: #4053
2026-07-08 04:01:22 +08:00
Luis Pater
8b9c4da245 feat(interactions): add support for Google Interactions
- Introduced API handlers and executor logic for Google Interactions
- Added request and response transformations for OpenAI and Claude Interactions.
- Integrated Gemini API with Interactions support.
- Updated tests to validate Interactions request parsing and error handling.
- Refactored translator logic for Interactions data flows.
2026-07-07 02:39:13 +08:00
Luis Pater
70053beadb feat(auth): refactor credential kind detection and add dynamic source classification
- Introduced `AuthKind` and `AuthSourceKind` methods for improved credential type and source classification.
- Replaced legacy fallback logic with normalized attribute-based handling in `AuthKind`.
- Consolidated metadata inspection for both API key and OAuth attributes.
- Updated calls to replace direct `AccountInfo` usage with `AuthKind` and `AuthSourceKind`.
- Enhanced unit tests to validate explicit and fallback credential resolution scenarios.
2026-06-23 23:21:33 +08:00
Luis Pater
78ba8ba731 chore: remove Gemini CLI-related translator packages and logic
- Deleted `geminicli` provider and related `Apply` logic.
- Removed all translator packages specific to Gemini CLI (Claude, Codex integrations).
- Purged associated test files for Gemini CLI translation.
- Removed `GeminiAuthenticator` and all associated authentication logic (OAuth flows, token handling, refresh logic).
- Deleted internal/executor Gemini OAuth support, including bearer token handling and runtime API logic.
- Purged all tests, configs, and command-line flags specific to Gemini OAuth flows.
- Updated documentation and aliases to reflect Gemini removal.
- Renamed `parseRetryDelay` to `ParseRetryDelay` and `deleteJSONField` to `DeleteJSONField`.
- Updated references in `antigravity_executor` and tests to use the new `helps` package.
- Adjusted import paths and test cases to ensure compatibility with the new location.
- Updated README files to reflect changes in the retry logic references.
- Updated `.github/ISSUE_TEMPLATE/bug_report.md` to remove deprecated Gemini CLI mention.
2026-06-18 13:33:10 +08:00
Luis Pater
959067edfb feat(usage): introduce executor type tracking in usage reporting
- Replaced `NewUsageReporter` with `NewExecutorUsageReporter` to include executor type in usage records.
- Updated all executors to use the new reporter implementation.
- Extended `UsageReporter` to track and publish executor type.
- Added tests to validate proper executor type recording and handling.
- Enhanced RedisQueue plugin and payload schema with executor type support.
2026-06-02 00:43:16 +08:00
Luis Pater
71c185f614 feat(usage): add service tier tracking and defaults in usage reporting
- Introduced `service_tier` metadata key to capture client-requested service tiers.
- Updated usage records, context propagation, and plugins to include service tier data.
- Added default handling logic for cases where `service_tier` is absent.
- Implemented tests for `service_tier` extraction, defaults, and updates across components.
2026-05-28 22:15:54 +08:00
Luis Pater
65e760aa1a feat(usage): include cache tokens in total token calculation and add tests
- Updated `TotalTokens` calculation to account for `CacheReadTokens` and `CacheCreationTokens`.
- Added tests to validate accurate token aggregation and fallback behavior for `CachedTokens`.
2026-05-28 21:34:54 +08:00
Luis Pater
94c1b25146 feat(executor): add TTFT tracking and reporting for enhanced performance metrics
- Introduced Time-To-First-Token (TTFT) measurement and reporting across major executors.
- Added TTFT calculation to `UsageReporter`, including support for HTTP clients and WebSocket communication.
- Updated tests to validate TTFT tracking in streamed and non-streamed scenarios.
- Ensured integration with `usage` plugin and augmented usage records with TTFT data.
2026-05-28 02:59:24 +08:00
Luis Pater
11f0f906bd feat(logging): add SetTranslatedReasoningEffort to track reasoning levels in usage reporting
- Introduced `SetTranslatedReasoningEffort` method in `UsageReporter` to capture and log reasoning efforts from translated payloads.
- Updated executors to incorporate the new reporting functionality for handling reasoning efforts across various providers.
- Enhanced logging for thinking level extraction with new helper function `ExtractTranslatedReasoningEffort`.
2026-05-28 02:19:45 +08:00
yavon007
0de0ad0d36 Add reasoning effort to usage events 2026-05-19 22:10:48 +08:00
Luis Pater
ad98c9549a feat(runtime): track upstream response headers in logging and usage reporting
- Added APIs to store, retrieve, and clone upstream response headers in context for detailed logging.
- Updated `RecordAPIResponseMetadata`, `RecordAPIWebsocketHandshake`, and related methods to capture response headers.
- Extended `UsageReporter` to include response headers in published usage records.
- Enhanced payload tests to validate response headers' integrity and persistence.
- Refactored `usage.Record` to support optional `ResponseHeaders` field.
2026-05-19 01:29:23 +08:00
Luis Pater
bd8c05a830 feat(usage): add support for detailed token breakdown in usage tracking
- Introduced `CacheReadTokens` and `CacheCreationTokens` to enhance token breakdown.
- Refactored `parseClaudeUsageNode` for cleaner and reusable logic.
- Adjusted helpers and updated token calculations to align with the new fields.
2026-05-12 11:59:07 +08:00
Luis Pater
a44e5eb1ab Merge branch 'v7' into dev 2026-05-10 02:33:42 +08:00
Luis Pater
66c3dae06b feat(home): implement count for home auth dispatch requests and enable usage statistics
- Added `count` attribute to `homeAuthCount` requests to improve home message batching.
- Enabled usage statistics for home mode by default and added config-level enforcement.
- Adjusted failure logging to include detailed metadata in `UsageReporter`.
- Updated multiple executors to pass error details to `PublishFailure` for better debugging.
- Enhanced unit tests to validate `count` behavior and usage statistics enforcement across components.
2026-05-10 01:30:43 +08:00
Luis Pater
e50cabac4b chore: upgrade CLIProxyAPI dependency to v7 across the project
- Updated all references from v6 to v7 for `github.com/router-for-me/CLIProxyAPI`.
- Ensured consistency in imports within core libraries, tests, and integration tests.
- Added missing tests for new features in Redis Protocol integration.
2026-05-08 11:46:46 +08:00
mochenya
99dfbaef61 fix(executor): ignore null OpenAI stream usage chunks
- Added validation so OpenAI-style usage parsing only accepts object payloads with token fields.
- Prevented streaming usage:null chunks from publishing zero-token records before the final usage chunk arrives.
- Reused the shared OpenAI-style parser for stream usage to support both chat completions and responses token field names.
- Added tests covering null usage chunks and input/output token usage fields in streaming responses.
2026-05-05 12:31:33 +08:00
Luis Pater
ba5d8ca733 feat(usage): add support for requested model alias handling
- Introduced methods for setting and retrieving model aliases in execution and usage contexts.
- Enhanced `UsageReporter` and related structures to include client-requested aliases.
- Updated tests to validate alias propagation and ensure correct usage reporting.
- Adjusted metadata handling in CLIProxyAPI executors to address alias integration.
2026-05-05 01:47:53 +08:00
sususu98
4982512da2 fix: parse gemini cli usage metadata variants 2026-04-29 13:10:53 +08:00
Luis Pater
04a336f7df fix(usage_helpers): skip zero-token usage in additional model records
- Added `buildAdditionalModelRecord` to filter out zero-token usage details.
- Introduced `hasNonZeroTokenUsage` helper function for token usage validation.
- Updated tests to cover scenarios for zero and non-zero token usage.
2026-04-27 10:56:22 +08:00
Luis Pater
c7b28ba058 feat(executor): add support for Codex image generation tool usage tracking
- Introduced `publishCodexImageToolUsage` to report image generation tool metrics.
- Updated executor logic to handle image generation tool events and defaults.
- Added parsing logic for `image_gen` tool usage details in `helps/usage_helpers.go`.
- Updated `UsageReporter` for additional model-specific usage publishing.
- Refactored usage detail normalizations.

Closes: #3063
2026-04-26 22:19:03 +08:00
Luis Pater
28d78273e4 feat(api): implement protocol multiplexer and Redis queue for usage integration
- Added `protocol_multiplexer.go`, enabling support for both HTTP and Redis protocols on a single listener.
- Introduced `redis_queue_protocol.go` to handle Redis-compatible RESP commands for queue management.
- Integrated `redisqueue` package, supporting in-memory queuing with expiration pruning.
- Updated server initialization to manage a shared listener and multiplex connections.
- Adjusted `Handler` to adopt `AuthenticateManagementKey` for modular key validation, supporting both HTTP and Redis flows.
2026-04-25 18:52:24 +08:00
Michael
8b9dbe10f0 fix: record zero usage 2026-04-06 20:19:42 +08:00
Luis Pater
d2c7e4e96a refactor(runtime): move executor utilities to helps package and update references 2026-04-01 03:08:20 +08:00