mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-02 23:56:24 +08:00
staging
39 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
firat.sertgoz
|
fddf56be7a | docs(engine): clarify ENGINE_V2 opt-in startup (#2694) | ||
|
Henry Park
|
a56fec7ebc | perf: fix multi-tenant inference latency (per-conversation locking + workspace indexing) (#2127) | ||
|
Coffee
|
d0096dfc38 |
feat: NEAR AI MCP server (#2009)
Co-authored-by: Robert Yan <46699230+think-in-universe@users.noreply.github.com> |
||
|
Illia Polosukhin
|
5c35b58ff1 |
feat(auth): direct OAuth/social login with Google, GitHub, Apple, and NEAR wallet (#1798)
* feat(auth): add direct OAuth/social login with Google and GitHub (#1771) Add optional OAuth authentication so users can sign in directly via Google or GitHub without requiring admin-created tokens or a reverse-proxy SSO setup. On successful OAuth, the system creates or links a user via the existing UserStore, issues an API token, and sets it as an HttpOnly cookie — reusing the existing DbAuthenticator for subsequent requests. Key changes: - user_identities table (PostgreSQL V15 + libSQL migration) for linking external provider accounts to internal users - IdentityStore trait with dual-backend implementations - OAuthProvider trait with Google (OIDC id_token) and GitHub (API-based) provider implementations - In-memory CSRF + PKCE state store with TTL and capacity bounds - Cookie-based session extraction in auth middleware - User resolution: existing identity → email linking → new account creation - All behind OAUTH_ENABLED=true flag; existing auth paths unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add email domain restrictions for OAuth and OIDC login (#1771) Add configurable email domain restrictions so admins can limit OAuth and OIDC login to specific organizations: - OAUTH_ALLOWED_DOMAINS: comma-separated list of allowed email domains, applied to all OAuth providers and OIDC (e.g., company.com,partner.org) - GOOGLE_ALLOWED_HD: restrict Google login to a specific Workspace domain via the `hd` authorization parameter + server-side validation - Domain check enforced in both the OAuth callback handler and the OIDC JWT middleware path (extracts email claim from validated JWT) - Add setup documentation in .env.example with step-by-step instructions for configuring Google and GitHub OAuth credentials Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR review — security hardening and cleanup (#1798) Fixes from Gemini and Copilot review: Security (critical/high): - Validate `aud` claim in Google id_token to prevent token substitution - Sanitize `redirect_after` to relative paths only (prevent open redirects) Correctness (medium): - Remove orphaned token: replace `create_user_with_identity_and_token` with `create_user_with_identity` — single token created in callback handler - Logout now revokes the API token (not just clears cookie) - Session tokens expire after 30 days (matching cookie lifetime) - Store decoded Google claims in raw_profile (not JWT string) - Propagate GitHub email fetch errors instead of swallowing - Fix `list_identities_for_user` to propagate row iteration errors Cleanup (low): - Extract `SESSION_COOKIE_NAME` constant - Add Secure flag to logout cookie clearing - Add single-quote escaping in error_page HTML - Fix garbled unicode in auth.rs comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add Apple Sign In provider (#1807) Add Apple Sign In as an OAuth provider alongside Google and GitHub. Apple-specific handling: - JWT client_secret generation (ES256-signed, team_id/key_id/private_key) - response_mode=form_post — Apple POSTs the callback instead of GET - POST callback route added alongside existing GET route - User name extracted from Apple's `user` form field (sent only on first authorization) and merged into the profile - id_token decoded with aud + issuer validation - email_verified handles both boolean and string "true"/"false" formats Configuration: - APPLE_CLIENT_ID, APPLE_TEAM_ID, APPLE_KEY_ID - APPLE_PRIVATE_KEY_PATH (file) or APPLE_PRIVATE_KEY_PEM (inline) - Setup instructions added to .env.example Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add NEAR wallet login via NEP-413 signature verification (#1807) Add NEAR wallet authentication as a fourth login method alongside Google, GitHub, and Apple. Unlike OAuth, NEAR uses a challenge-response flow with Ed25519 signature verification. Backend: - GET /auth/near/challenge — generate a random nonce (32 bytes hex) - POST /auth/near/verify — verify Ed25519 signature + NEAR RPC access key check, then issue session token via existing user resolution pipeline - NearNonceStore: in-memory nonce store with 5-min TTL and replay protection - Supports both base58 (NEAR standard) and hex key/signature encoding - New dependency: bs58 0.5 for base58 decoding Frontend: - Login screen discovers enabled providers via GET /auth/providers - Shows social login buttons (Google, GitHub, Apple, NEAR) dynamically - NEAR button loads @hot-labs/near-connect via ESM CDN import - Wallet connection → signMessage → POST to /auth/near/verify → session - OAuth cookie-based sessions auto-detected on page load (existing flow) Configuration: - NEAR_AUTH_ENABLED=true - NEAR_AUTH_NETWORK=mainnet|testnet (defaults to mainnet) - NEAR_AUTH_RPC_URL (auto-detected from network) Closes #1807 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address second round of PR review comments (#1798) - Fix early return in with_oauth() that skipped NEAR setup and OIDC domain restrictions when no OAuth redirect providers were configured - Add active-status check before linking identity by verified email (prevents linking to suspended/deactivated accounts) - Remove inline onclick handlers from login buttons (CSP compliance) - Export SESSION_COOKIE_NAME from auth.rs, reuse in handlers and middleware - NEAR challenge returns structured message ("Sign in to IronClaw\nNonce: {nonce}") that both client and server use for signature verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address human reviewer security findings (#1798) Six fixes from serrrfirat's review: 1. Domain check now requires email_verified=true before trusting the email for domain restriction — prevents unverified emails from bypassing access control (e.g., GitHub unverified fallback) 2. First-user bootstrap race documented — concurrent first logins may both see has_any_users()=false, but the second gets member role. Acceptable tradeoff; unique constraint prevents identity duplication. 3. NEP-413 payload mismatch fixed — server now builds the exact borsh-serialized NEP-413 payload (tag + message + nonce + recipient) that the wallet signs, instead of raw message bytes 4. Token extraction priority fixed — explicit ?token= query param now takes precedence over session cookie, preventing SSE/WS user mismatch when a browser has both a cookie and a query-param token 5. NEAR domain suffix check hardened — requires exact match or dotted subdomain boundary (alice.company.near passes, evilcompany.near does not) 6. NEAR network surfaced to frontend — /auth/providers response includes near_network field, frontend wallet connector uses it instead of hardcoded mainnet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): improve login page UX when OAuth providers are enabled When OAUTH_ENABLED=true with providers configured, the login screen now shows social login buttons (Google, GitHub, Apple, NEAR) as the primary action. The token input is collapsed behind a clickable "or use a token" divider for API users. Without OAuth: unchanged — token input is the only option. With OAuth: social buttons first, token input expandable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): hide token form until providers are discovered The token input was visible by default, causing it to flash before OAuth buttons appeared. Now: - Token form starts hidden (display:none) - /auth/providers fetch determines what to show - With providers: social buttons shown, token form behind "or use a token" - Without providers (or fetch fails): token form shown as fallback - After OAuth redirect: cookie-based autoAuth skips login screen entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): replace Connected indicator with user avatar + account menu Replace the "Connected" status indicator in the header with a user avatar button that shows connection status via an overlay dot. Clicking the avatar opens a dropdown with: - Display name, email, and role - Connection status (green/red dot + text) with gateway stats - Sign out button (calls POST /auth/logout, clears session, reloads) Avatar source: - OAuth logins: profile photo from Google/GitHub/Apple (avatar_url) - Token logins: initials from display_name (colored circle) Backend: profile_get_handler now queries user_identities for avatar_url from linked OAuth accounts. Frontend: social buttons are primary when OAuth is enabled, token input collapsed behind "or use a token" divider. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): remove Connected section from dropdown, fix avatar loading - Remove the connection status section from the user dropdown (was redundant with the avatar dot) - Add update_identity_profile() to IdentityStore — updates display_name and avatar_url on re-login so avatars load for accounts created before the avatar field was wired - Call update_identity_profile() in resolve_user() when an existing identity is found (re-login path) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): restore gateway stats in dropdown, add avatar debug logging - Bring back gateway stats section in user dropdown (without the word "Connected" — just the server stats like uptime, model, channels) - Add debug tracing to Google provider (logs picture claim from id_token) and profile handler (logs identity count + avatar_url) to diagnose why avatar isn't loading for Google OAuth accounts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix Google avatar not loading, restore gateway stats - Add referrerpolicy="no-referrer" to avatar img — Google's lh3.googleusercontent.com returns 403 when Referer header is sent from a different origin - Add crossorigin="anonymous" for CORS - Use display:block explicitly instead of empty string - Add onerror fallback to initials if image fails to load - Restore gateway stats section in dropdown (without "Connected" text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix squeezed avatar image in header Add min-width/min-height and flex-shrink:0 to both the avatar button and the img element so they don't get compressed by the tab-bar flex layout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): position avatar img and initials absolutely inside button Both children were competing for flex space, causing 0px width. Now both are position:absolute inside the 32px button, layered on top of each other. The JS toggles display:block/none to show the right one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): rewrite avatar loading — CSS src selector + onload/onerror Previous approach: inline style display:none toggled by JS. Failed because display:none prevented image fetch in some browsers, and position:absolute elements competed for z-index. New approach: - No inline style on img — CSS hides it via .user-avatar-img (display:none) - CSS .user-avatar-img[src] shows it (display:block, z-index:1) - JS sets src, onload hides initials, onerror removes src as fallback - Initials always rendered first as the base layer - Removed crossorigin="anonymous" which can cause CORS failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): prefetch avatar with new Image() before showing Use a throwaway Image() to prefetch the avatar URL. Only when onload fires, set src on the real <img> and unhide it. This avoids all CSS display/src selector issues — the real img element only gets a src after the image is confirmed loadable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): set avatar src directly, set referrerPolicy in JS The new Image() prefetch was failing because the programmatic Image object didn't have referrerPolicy set. Simplify: set referrerPolicy and src directly on the real <img> element, unhide it immediately, and use onload to hide initials. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): explicit display:block on avatar img, removeAttribute hidden - Add display:block to .user-avatar-img CSS (img elements default to inline which can cause rendering issues with position:absolute) - Bump z-index to 2 to ensure img renders above initials - Use removeAttribute('hidden') instead of hidden=false - Use style.display='none' on initials instead of hidden attribute Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): swap img/initials DOM order so img paints on top Put <img> after <span> in DOM order. With both position:absolute, later elements paint on top. Combined with z-index:2 on img vs z-index:0 on initials, the avatar photo should now reliably cover the initials circle when loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): add OAuth avatar domains to Content-Security-Policy The CSP had img-src 'self' data: which blocked Google and GitHub avatar images from loading. Added: - img-src: *.googleusercontent.com, avatars.githubusercontent.com - script-src: esm.sh (for near-connect dynamic import) - connect-src: esm.sh, *.near.org (for NEAR RPC) - form-action: Google, GitHub, Apple OAuth endpoints This was the root cause of avatar images not rendering despite correct src URLs — the browser silently blocked them via CSP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): show welcome card for new OAuth users with no threads New OAuth users have no assistant thread yet, so switchToAssistant() was never called, and loadHistory() never ran to show the welcome card. Now explicitly show the welcome card when there's no current thread and no assistant thread (brand-new user). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting for new OAuth users on workspace creation New OAuth users saw an empty chat because the bootstrap greeting was only persisted when the agent loop processed the first message (take_bootstrap_pending check in agent_loop.rs:1314). But OAuth users land on the web UI without sending any message. Fix: WorkspacePool now checks take_bootstrap_pending() after seed_if_empty() and persists the GREETING.md content into the assistant conversation immediately. This runs in a background task so it doesn't block the workspace creation. The greeting is in the DB before the frontend loads threads/history. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting synchronously, not in background Move greeting persistence from tokio::spawn to the same await chain as seed_if_empty() so the greeting is guaranteed to be in the DB before the workspace is returned to the caller. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): seed bootstrap greeting in chat_threads_handler for new users The WorkspacePool approach didn't work because the workspace pool is only accessed by memory handlers — chat_threads_handler runs first when a new user loads the page. Move the greeting seed to chat_threads_handler: after get_or_create_assistant_conversation, check if the conversation has zero messages and inject the GREETING.md content. This guarantees the greeting is in the DB before the thread list is returned to the frontend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(agent): consolidate bootstrap greeting into chat_threads_handler Remove three redundant greeting insertion paths from agent_loop.rs: 1. Single-user startup (Agent::run bootstrap_thread_id) 2. Single-user SSE broadcast after startup 3. Multi-tenant message handler (take_bootstrap_pending on first msg) Also remove the dead WorkspacePool greeting code in server.rs. The single source of truth is now chat_threads_handler: when the assistant conversation is created with zero messages, GREETING.md is inserted. This works for all auth modes (token, OAuth, OIDC) and both single-user and multi-tenant deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: downgrade workspace seed log from info to debug Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian's blocking review items 1. Add Google iss validation — set_issuer(&["https://accounts.google.com"]) to match Apple's issuer check. Prevents cross-provider id_token acceptance. 2. Convert oauth_rate_limiter from global RateLimiter to per-IP PerUserRateLimiter(20, 60). Extracts client IP from X-Forwarded-For header. One user retrying no longer locks out all OAuth for everyone. Also: - sanitize_redirect now rejects backslash (/\) open redirect vector - All auth handlers extract headers for per-IP rate limiting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): stop inserting greeting on every page load The previous check used list_conversations_with_preview with limit=1 and defaulted to is_empty=true when the assistant thread wasn't in the result (unwrap_or(true)). This caused the greeting to be inserted on every chat_threads_handler call. Fix: use list_conversation_messages_paginated(assistant_id, None, 1) to directly check if the assistant conversation has any messages. Only insert the greeting when the message list is truly empty. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for bootstrap greeting and cookie auth Five tests covering the greeting behavior and OAuth session auth: 1. test_greeting_inserted_once_for_new_user — verifies greeting appears exactly once and is not duplicated on second page load 2. test_greeting_not_duplicated_on_rapid_calls — 5 concurrent /api/chat/threads requests produce exactly 1 greeting 3. test_each_user_gets_own_greeting — multi-user: Alice and Bob each get their own assistant thread with separate greetings 4. test_cookie_auth_works_for_threads — cookie-based auth (ironclaw_session=token) works for protected endpoints 5. test_existing_conversation_no_greeting — pre-populated conversations are not overwritten with the greeting These tests would have caught the unwrap_or(true) bug that caused greeting re-insertion on every page load. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix near-connect CDN URL (package is v0.x, not v1) The @hot-labs/near-connect package is version 0.11.1 — there is no v1 release. The @1 version specifier returned 404 from esm.sh. Changed to @0.11 which resolves correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): support base64 encoding for NEAR wallet signatures NEAR wallets (e.g. HOT) may return signatures and public keys in base64 format, not just base58/hex. Added base64 standard and URL-safe decoding to decode_multiformat(), which is used by both decode_near_public_key and decode_near_signature. Also: - Added debug logging to near_verify_handler to trace credential formats - Updated CSP img-src to allow wallet logos (raw.githubusercontent.com, jsdelivr.net, near.org, pages.near.org) - Added CSP frame-src for near-connect wallet sandboxes (iframes) - Added blob: to img-src for inline wallet icons Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): widen CSP connect-src and img-src for NEAR wallet resources near-connect fetches wallet manifests from raw.githubusercontent.com and cdn.jsdelivr.net, and wallet logos from app.hot-labs.org. These were blocked by the restrictive connect-src and img-src policies. - connect-src: added raw.githubusercontent.com, *.jsdelivr.net, *.cloudflare.com - img-src: added *.hot-labs.org Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): try both NEP-413 and raw message for NEAR signature verification Different NEAR wallets may sign the full NEP-413 borsh payload or just the raw message string. Try NEP-413 first, fall back to raw message bytes. This makes verification work with HOT wallet and other wallets that may not implement the full NEP-413 serialization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): fix NEP-413 field order and try both payload layouts The NEP-413 borsh payload field order was wrong. Our implementation had tag → message → nonce → recipient → callback_url, but the NEAR docs (docs.near.org/web3-apps/backend-login) show tag → message → recipient → nonce. Now tries both field orderings (v1 and v2), plus SHA256 variants, plus raw message bytes — covering all known wallet implementations. Tests updated: verify_near_signature tested with raw message, NEP-413 v2, and wrong-key rejection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP for wallet ecosystem — allow all HTTPS for connect/img/frame The NEAR wallet ecosystem spans dozens of domains (intear.tech, hot-labs.org, meteorwallet.app, herewallet.app, etc.) that change as new wallets are added. Whitelisting each one is a losing game. Relax CSP to allow all HTTPS for: - connect-src: wallet manifests, wallet JS modules, RPC endpoints - img-src: wallet logos from various CDNs - frame-src: wallet sandbox iframes script-src remains restricted to specific CDNs (jsdelivr, cloudflare, esm.sh) — this is the security-critical directive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow unsafe-inline scripts for NEAR wallet sandbox iframes NEAR wallet sandboxes (MeteorWallet, etc.) use inline scripts inside their iframe sandboxes. The CSP script-src blocked these, preventing wallets from loading. Added 'unsafe-inline' to script-src. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP style-src and font-src for wallet iframes NEAR wallet sandboxes load fonts from rsms.me, cdnfonts.com, and embed data: font URIs. Relaxed style-src and font-src to allow all HTTPS sources and data: URIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 4 — 14 fixes serrrfirat (high/medium): 1. decode_multiformat ambiguity: replaced with context-aware decoders. NEAR pubkeys enforce ed25519: prefix + base58 (unambiguous). Signatures try base64 first (most wallets), then base58. 2. UTF-8 slicing panic: use safe_truncate() with char_indices() 3. NEAR pubkey → RPC format: re-encode decoded bytes as ed25519:{base58} for the canonical format expected by view_access_key 4. GitHub redirect_uri: now included in token exchange form body Copilot (medium/low): 5. near_network stored explicitly in GatewayState (not inferred from URL) 6. NEAR verify sets HttpOnly session cookie (consistent with OAuth flow) 7. Reuse reqwest::Client via LazyLock (no per-request allocation) 8. OAuth module doc updated to list all 4 providers 9. Rate limiter comment fixed (was stale "10 requests") 10. Profile identity error logged at warn (not silently swallowed) 11. Config doc updated for Apple/NEAR requirements 12. Test file doc comment updated to match actual coverage 13. RPC status check before JSON parse 14. GitHub token exchange includes redirect_uri Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR feedback on cookie auth and NEAR sessions * fix(auth): address code review — tighten CSP, fix races, improve security - Tighten CSP: narrow connect-src/img-src/frame-src to specific origins instead of blanket `https:` (prevents data exfiltration) - Fix greeting race: add atomic add_conversation_message_if_empty using INSERT...WHERE NOT EXISTS (both PostgreSQL and libSQL) - Case-insensitive email matching: use LOWER() in identity lookups and normalize emails to lowercase on storage - Add X-Real-IP fallback for rate limit key when X-Forwarded-For missing - Add OAuthError::SignatureVerification variant (was misusing ProfileFetch) - Fix dead branch in with_oauth (has_near check inside !has_near block) - Fix _user → user in logout_handler (variable is actually used) - Add partial index WHERE email IS NOT NULL to libSQL (match PostgreSQL) - Downgrade noisy tracing::debug to trace in profile handler - Add i18n for "Sign out" button (en + zh-CN) - Remove duplicate test_session_cookie_auth_passes test - Update E2E bootstrap tests to match new DB-based greeting architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove garbled unicode character in section comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 5 — admin race, 303 redirect, OIDC email_verified - Atomic first-user admin: create_user_with_identity now promotes to admin inside the DB transaction with UPDATE...WHERE COUNT(*)=1, eliminating the TOCTOU race where two concurrent first logins both get admin role (both PostgreSQL and libSQL) - Apple callback redirect: use 303 See Other instead of 307 Temporary so POST form_post callbacks are converted to GET on redirect - OIDC domain restriction: now requires email_verified=true before checking domain allowlist, preventing unverified emails from bypassing the restriction - Postgres add_conversation_message_if_empty: call touch_conversation after insert to match libSQL behavior and keep last_activity current - Greeting seeding: log errors instead of silently discarding with let _ Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): advisory lock for admin election, skip empty query tokens - Postgres first-user admin: add pg_advisory_xact_lock before the COUNT(*)=1 promotion to serialize concurrent transactions under READ COMMITTED isolation (prevents two admins on concurrent signup) - Empty ?token= query parameter no longer overrides a valid session cookie — trimmed empty tokens return None from query_token() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian security review — redirect, sweep, NEAR sigs Blocking issues from security review: 1. redirect_after hardened: strict URL-safe char allowlist in sanitize_redirect (blocks /%09/ and encoded separators), plus re-validation before use in handle_callback (defense in depth) 2. Sweep tasks shutdown-aware: OAuth state store and NEAR nonce store sweep loops now select on a watch channel and exit when the sender is dropped (stored in GatewayState.oauth_sweep_shutdown) 3. NEAR signature verification tightened: removed raw-message-bytes and SHA256-of-raw fallbacks that lacked nonce binding (replay risk). Only NEP-413 structured payloads (v1 + v2) are accepted. Added test_verify_near_signature_rejects_raw_message regression test. Non-blocking: 4. NEAR RPC client timeout: set 10s timeout on the static reqwest client to prevent indefinite hangs on slow RPC endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): percent-decode redirect_after before validation is_safe_redirect now percent-decodes the URL and re-validates against the // and /\ guards, preventing smuggling via %2f%2f or %5c. Added 5 regression tests covering normal paths, protocol-relative, absolute URLs, encoded smuggling, and sanitize_redirect filtering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): case-insensitive get_user_by_email, require OIDC iss/aud claims - get_user_by_email now uses LOWER() in both PostgreSQL and libSQL, matching the case-insensitive identity lookup. This ensures admin-created users with different email casing are correctly linked during OAuth account resolution. - OIDC validation now adds iss/aud to required_spec_claims when configured, rejecting JWTs that omit these claims entirely (not just mismatches). Updated two tests from assert-passes to assert-rejects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): normalize UserRecord.email to lowercase, add aria-label to avatar - UserRecord.email now lowercased on create (matching identity records), preventing case-mismatched duplicates against the UNIQUE constraint - Avatar button: added aria-label with i18n (en + zh-CN) for screen readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
rajulbhatnagar
|
d12b8bd7ec |
feat: Add ACP (Agent Client Protocol) job mode for delegating to any compatible coding agent (#1600)
* feat: add ACP (Agent Client Protocol) job mode for delegating to any compatible coding agent Add a third container job mode (`JobMode::Acp`) that spawns any ACP-compliant agent (Goose, Codex, Gemini CLI, Cline, Copilot, etc.) as a subprocess inside a Docker container and communicates via the standard ACP protocol (JSON-RPC over stdio). **Bridge runtime** (`src/worker/acp_bridge.rs`): - Spawns agent subprocess, performs ACP handshake (initialize → session → prompt) - Translates ACP SessionNotification events to IronClaw's JobEventPayload stream - Auto-approves permissions (Docker container is the security boundary) - Supports follow-up prompts from the orchestrator - Detects agent process exit via oneshot channel to prevent infinite polling **User configuration** (mirrors MCP server pattern): - `ironclaw acp add/list/remove/toggle/test` CLI commands - DB-backed persistence with `~/.ironclaw/acp-agents.json` disk fallback - Per-agent `enabled` flag + global `ACP_ENABLED` toggle - `ironclaw acp test` spawns agent, verifies ACP handshake, reports capabilities **System integration**: - `ExtensionKind::AcpAgent` in extension manager (12 match arms) - `agent_name` parameter in CreateJobTool resolves agent from AcpAgentsFile - Mode stored as `"acp:<agent_name>"` for restart support - Doctor validation, status display, boot screen, app startup logging - Web UI: extension install mapping, job restart, follow-up prompt support Closes #1506 * test: add comprehensive ACP test coverage (22 new tests) Bridge: ToolCall, ToolCallUpdate, thought-image, max_turn_requests, session_id propagation, text_from_content_block, multibyte truncation. Config: AcpModeConfig defaults, settings resolution, env overrides. Job tool: schema includes "acp" mode + agent_name, mode="acp" requires agent_name parameter, JobMode::Acp as_str/display. Job manager: JobMode::Acp as_str/display, acp_memory_limit_mb default. CLI: parse_env_var valid/invalid/equals-in-value, command variants. * refactor: make IronClawAcpClient reusable for CLI test command Extract AcpEventSink trait so the same Client implementation (permission auto-approval, event translation) is shared between the container bridge (posts to orchestrator HTTP API) and the CLI test command (prints to stdout). Also extracts ironclaw_init_request() to avoid duplicating the ACP handshake parameters between bridge and test command. * fix(sandbox): use host.docker.internal on all platforms for orchestrator URL The orchestrator host was hardcoded to 172.17.0.1 on Linux, which is only correct for the default Docker bridge network. Environments with custom bridge IPs break container-to-host connectivity. Since all containers already set extra_hosts with host-gateway, using host.docker.internal works on all platforms and network configurations. * fix ACP PR review feedback * fix ACP DB error fallback * fix clippy after staging merge --------- Co-authored-by: Rajul Bhatnagar <brajul@amazon.com> Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
Illia Polosukhin
|
5435b38eca |
feat(workspace): metadata-driven indexing/hygiene, document versioning, and patch (#1723)
* feat(workspace): metadata-driven indexing/hygiene, document versioning, and patch support Foundation for the extensible frontend system. Workspace documents now support metadata flags (skip_indexing, skip_versioning, hygiene config) via folder-level .config documents and per-file overrides, replacing hardcoded hygiene targets and indexing behavior. Key changes: - DocumentMetadata type with resolution chain (doc → folder .config → defaults) - Document versioning: auto-saves previous content on write/append/patch - Workspace patch: search-and-replace editing via memory_write tool - Hygiene rewrite: discovers cleanup targets from .config metadata instead of hardcoded daily/ and conversations/ directories - memory_read gains version/list_versions params - memory_write gains metadata/old_string/new_string/replace_all params - V14 migration adds memory_document_versions table (both PG + libSQL) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback — transaction safety, patch mode, formatting - Wrap libSQL save_version in a transaction to prevent race condition where concurrent writers could allocate the same version number - Make content optional in memory_write when in patch mode (old_string present) — LLM no longer forced to provide unused content param - Improve metadata update error handling with explicit match arms - Run cargo fmt across all files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings — write-path performance, version pruning, descriptions 1. Resolve metadata once per write: write(), append(), and patch() now call resolve_metadata() once and pass the result to both maybe_save_version() and reindex_document_with_metadata(), cutting redundant DB queries from 3-5 per write down to 1 resolution. 2. Optimize version hash check: replaced get_latest_version_number() + get_version() (2 queries) with list_versions(id, 1) (1 query) for the duplicate-hash check in maybe_save_version(). 3. Wire up version_keep_count: hygiene passes now prune old versions for documents in cleaned directories, enforcing the configured version_keep_count (default: 50). Removes the TODO comment. 4. Fix misleading tool description: patch mode works with any target including 'memory', not just custom paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: wire remaining unwired components — changed_by, layer versioning 1. changed_by now populated: all write paths pass self.user_id as the changed_by field in version records instead of None, so version history shows who made each change. 2. Layer write/append versioned: write_to_layer() and append_to_layer() now auto-version and use metadata-optimized reindexing, matching the standard write()/append() paths. 3. append_memory versioned: MEMORY.md appends now auto-version with metadata-driven skip and shared metadata resolution. 4. Remove unused reindex_document wrapper: all callers now use reindex_document_with_metadata directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive coverage for versioning, metadata, patch, and hygiene 26 new tests covering critical and high-priority gaps: document.rs (7 unit tests): - is_config_path edge cases (foo.config, empty string, .config/bar) - content_sha256 with empty string (known SHA-256 constant) - content_sha256 with unicode (multi-byte UTF-8) - DocumentMetadata merge: null overlay, nested hygiene replaced wholesale, both empty, non-object base memory.rs (2 schema tests): - memory_write schema includes patch/metadata params, content not required - memory_read schema includes version/list_versions params hygiene.rs (5 integration tests): - No .config docs → no cleanup happens - .config with hygiene disabled → directory skipped - Multiple dirs with different retention (fast=0, slow=9999) - Documents newer than retention not deleted - Version pruning during hygiene (keep_count=2, verify pruned) workspace/mod.rs (14 integration tests): - write creates version with correct hash and changed_by - Identical writes deduplicated (hash check) - Append versions pre-append content - Patch: single replacement, replace_all, not-found error, creates version - Patch with unicode characters - Patch with empty replacement string - resolve_metadata: no config (defaults), inherits from folder .config, document overrides .config, nearest ancestor wins - skip_versioning via .config prevents version creation [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address zmanian review — PG transaction safety, identity protection, perf Must-fix: 1. PostgreSQL save_version now uses a transaction with SELECT FOR UPDATE to prevent concurrent writers from allocating the same version number, matching the libSQL implementation. 2. Restore identity document protection in hygiene cleanup_directory(). MEMORY.md, SOUL.md, IDENTITY.md, etc. are now protected from deletion regardless of which directory they appear in, via is_identity_document() case-insensitive check. This restores the safety net that was removed when migrating from hardcoded to metadata-driven hygiene. Should-fix: 3. resolve_metadata() now uses find_config_documents (single query) + in-memory nearest-ancestor lookup, instead of O(depth) serial DB queries walking up the directory tree. 4. memory_write validates that at least one mode is provided (content for write/append, or old_string+new_string for patch) with a clear error message upfront, instead of relying on downstream empty checks. 5. Fixed misleading GIN index comment in V15 migration. 9. Added "Fail-open: versioning failures must not block writes" comments to all `let _ = self.maybe_save_version(...)` call sites. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt * fix: address Copilot review — DoS prevention, no-op skip, duplicate hygiene Security: - Reject empty old_string in both workspace.patch() and memory_write tool to prevent pathological .matches("") behavior (DoS vector) Correctness: - Remove duplicate hygiene spawn in multi-user heartbeat — was running both via untracked tokio::spawn AND inside the JoinSet, causing double work and immediate skip via global AtomicBool guard - Disallow layer param in patch mode — patch always targets the default workspace scope; combining with layer could silently patch the wrong document - Restore trim-based whitespace rejection for non-patch content validation (was broken when refactoring required fields) Performance: - Short-circuit write() when content is identical to current content, skipping versioning, update, and reindex entirely - Normalize path once at start of resolve_metadata instead of only for config lookup (prevents missed document metadata on unnormalized paths) Cleanup: - Remove duplicate tests/workspace_versioning_integration.rs (same tests already exist in workspace/mod.rs versioning_tests module) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: eliminate flaky hygiene tests caused by global AtomicBool contention All hygiene tests that used run_if_due() were flaky when running concurrently because they competed for the global RUNNING AtomicBool guard. Rewrote them to test the underlying components directly: - metadata_driven_cleanup_discovers_directories: now uses find_config_documents() + cleanup_directory() directly - multiple_directories_with_different_retention: now uses cleanup_directory() per directory directly - cleanup_respects_cadence: rewritten as a sync unit test that validates state file + timestamp logic without touching the global guard Verified stable across 3 consecutive runs (3793 tests, 0 failures). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata ordering, PG locking, hygiene safety 1. Metadata applied BEFORE write/patch (#10-11,15): metadata param is now set via get_or_create + update_metadata before the write/patch call, so skip_indexing/skip_versioning take effect for the same operation instead of only subsequent ones. 2. Layer write doc ID (#13-14): metadata no longer re-reads after write since it's applied upfront. Removes the stale-scope risk. 3. Version param overflow (#16): validates version is 1..i32::MAX before casting, returns InvalidParameters on out-of-range. 4. Hygiene protection list (#18): added HYGIENE_PROTECTED_PATHS that includes MEMORY.md, HEARTBEAT.md, README.md (missing from IDENTITY_PATHS). cleanup_directory now uses is_protected_document() which checks both lists with case-insensitive matching. 5. PG FOR UPDATE on empty table (#22-24): now locks the parent memory_documents row (SELECT 1 FROM memory_documents WHERE id=$1 FOR UPDATE) before computing MAX(version), which works even when no version rows exist yet. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata merge, retention guard, migration ordering 1. **Metadata merge in memory_write tool**: incoming metadata is now merged with existing document metadata via `DocumentMetadata::merge()` instead of full replacement, so setting `{hygiene: {enabled: true}}` no longer silently drops a previously-set `skip_versioning: true`. 2. **Minimum retention_days**: `HygieneMetadata.retention_days` is now clamped to a minimum of 1 day during deserialization, preventing an LLM from writing `retention_days: 0` and causing mass-deletion on the next hygiene pass. 3. **Migration version ordering**: renumbered document_versions migration to come after staging's already-deployed migrations (PG: V15→V16, libSQL: 15→17). Documented the convention that new migrations must always be numbered after the highest version on staging/main. 4. **Duplicate doc comment**: removed duplicated line on `reindex_document_with_metadata`. 5. **HygieneSettings**: added `version_keep_count` field to persist the setting through the DB-first config resolution chain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip metadata pre-apply when layer is specified, clean up stale comment 1. When a layer is specified, skip the metadata pre-apply via get_or_create — it operates on the primary scope and would create a ghost document there while the actual content write targets the layer's scope. 2. Removed stale "See review comments #10-11,15" reference; the surrounding comment already explains the rationale. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use BEGIN IMMEDIATE for libSQL save_version to serialize writers The default DEFERRED transaction only acquires a write lock at the first write statement (INSERT), not at the SELECT. Two concurrent writers could both read the same MAX(version) before either inserts, causing a UNIQUE violation. BEGIN IMMEDIATE acquires the write lock upfront, matching the existing pattern in conversations.rs. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: document trust boundary on metadata/versioning WorkspaceStore methods These methods accept bare document UUIDs without user_id checks at the DB layer. The Workspace struct (the only caller) always obtains UUIDs through user-scoped queries first. Document this trust boundary explicitly on the trait so future implementors/callers know not to pass unverified UUIDs from external input. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address new Copilot review comments — ghost doc, param validation, overflow 1. Skip metadata pre-apply in patch mode to avoid creating a ghost empty document via get_or_create when the document doesn't exist, which would change a "not found" error into "old_string not found". 2. Validate list_versions and version as mutually exclusive in memory_read to avoid ambiguous behavior (list_versions silently won). 3. Clamp version_keep_count to i32::MAX before casting to prevent overflow on extreme config values. 4. Mark daily_retention_days and conversation_retention_days as deprecated in HygieneSettings — retention is now per-folder via .config metadata. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: apply metadata in patch mode via read(), reorder libSQL migrations 1. Metadata is no longer silently ignored in patch mode — uses workspace.read() (which won't create ghost docs) instead of skipping entirely, so skip_versioning/skip_indexing flags take effect for patches on existing documents. 2. Reorder INCREMENTAL_MIGRATIONS to strictly ascending version order (16 before 17) to match iteration order in run_incremental(). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove duplicate is_patch_mode, add TODO comments for known limitations - Remove duplicate `is_patch_mode` binding in memory_write (was computed at line 280 and again at line 368). - Document multi-scope hygiene edge case: workspace.list() includes secondary scopes but workspace.delete() is primary-only, causing silent no-ops for cross-scope entries. - Document O(n) reads in version pruning as acceptable for typical directory sizes. - Add TODO on WorkspaceError::SearchFailed catch-all for future cleanup into more specific variants. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reindex on no-op writes for metadata changes, use read_primary in patch 1. write() no longer fully short-circuits when content is unchanged — it still resolves metadata and reindexes so that metadata-driven flags (e.g. skip_indexing toggled via memory_write's metadata param) take effect immediately even without a content change. 2. Patch-mode metadata pre-apply now uses workspace.read_primary() instead of workspace.read() to ensure we target the same scope that patch() operates on, preventing cross-scope metadata mutation in multi-scope mode. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Artem
|
8638895879 |
feat(gemini_oauth): full Gemini CLI OAuth integration with Cloud Code API (#1356)
* feat: integrate Gemini CLI OAuth with Cloud Code API
- Add gemini_oauth.rs: full OAuth flow with PKCE, token refresh,
and Cloud Code project discovery (loadCodeAssist + onboardUser)
- Route preview/gemini-3 models through cloudcode-pa.googleapis.com
with proper project ID injection in request payload
- Trigger OAuth login during onboarding wizard (not first chat message)
- Support manual redirect URL paste as fallback (tokio::select race)
- Parse 429 rate-limit errors with retry_after from Google response
- Add static model list: gemini-1.5/2.0/2.5/3.0/3.1 variants
- Add GeminiOauthConfig with default credentials path (~/.gemini/)
* feat(gemini): implement function calling, generationConfig, and update models
- Implement function calling support (functionDeclarations, functionResponse)
- Add functionCall SSE parsing and empty stream retry support
- Add generationConfig (temperature, maxOutputTokens)
- Add thinkingConfig for Gemini 3 and thinking models
- Add toolConfig (functionCallingConfig.mode)
- Fix .expect() panics with .ok_or_else()
- Restrict oauth credentials file permissions to 0600
- Update docs and FEATURE_PARITY.md
- Update wizard to current Gemini 3.1 and 2.5 models
* fix: address code review issues in gemini-cli OAuth integration
- Add cache_read_input_tokens/cache_creation_input_tokens fields (value 0)
- Implement manual Debug for OAuthCredential to redact tokens
- Fix hardcoded /tmp: use GeminiOauthConfig::default_credentials_path()
- Replace emoji output with plain text markers
- Propagate Client::builder() errors instead of silent fallback
- Use tokio::fs for all file I/O in CredentialManager (was std::fs)
- Use if let Some(ref pid) to avoid consuming credential.project_id
- Extract uses_cloud_code_api() helper; route by major version (gemini-2+)
- Concatenate multiple system messages into systemInstruction
- Include functionCall parts in assistant message conversion
- Add 401 retry loop with allow_retry flag for auth failures
- Remove biased from tokio::select! in OAuth callback handler
- Remove hardcoded context_length 1M; vary by model family
- Change GOOG_API_CLIENT from Node.js spoof to gl-rust/1.0.0
- Implement list_models() with static model list
- Move create_gemini_oauth_provider() before test module (clippy)
- Fix 9 additional clippy warnings (collapsible_if, map_or, needless_borrow)
- Run cargo fmt
* Add dedicated regression tests for Gemini OAuth fixes
* style: fix formatting in Gemini OAuth regression tests
* feat(gemini-oauth): implement code review v3 refinements
- Add force_refresh() for 401 retry (bypass timestamp check)
- Standardize Gemini model list across docs, wizard, and provider
- Restore gemini-3 check for thinkingConfig
- Redact sensitive tokens in GoogleTokenRefreshResponse Debug output
- Use dynamic version for GOOG_API_CLIENT
- Improve model_metadata() context length heuristics
- Use strip_prefix("data:") for safer SSE parsing
- Skip re-auth in wizard if keeping existing provider
* feat(gemini_oauth): full Cloud Code API integration with project discovery
- Register gemini_oauth as a dedicated backend in config/llm.rs (skip
registry fallback, preserve backend name, suppress unknown-backend warning)
- Fix app.rs credential guard to exclude backends with dedicated configs
(gemini_oauth, bedrock) from the provider.is_none() check
- Auto-discover Cloud Code project_id via loadCodeAssist when credentials
lack it (e.g. created by the original Gemini CLI)
- Persist discovered project_id to credentials file for subsequent runs
- Add safety settings (BLOCK_NONE), gated behind GEMINI_SAFETY_BLOCK_NONE env
- Add thinkingConfig: budget-based for Gemini 2.5, level-based for Gemini 3.x
(without includeThoughts to avoid empty responses from reasoning.rs stripping)
- Add thought signature injection for Gemini 3.x preview APIs
- Add history curation to filter invalid model outputs before re-sending
- Add extended generationConfig env vars (topP, topK, seed, penalties,
responseMimeType, responseJsonSchema, cachedContent)
- Add custom headers support via GEMINI_CLI_CUSTOM_HEADERS
- Add API key auth mode (GEMINI_API_KEY + GEMINI_API_KEY_AUTH_MECHANISM)
- Add SSE metadata extraction (modelVersion, credits, promptFeedback,
groundingMetadata, citationMetadata, cachedContentTokenCount)
- Add countTokens API support
- Add new models to wizard (gemini-3.1-pro-preview-customtools,
gemini-3-pro-preview, gemini-3.1-flash-lite-preview)
- Update docs/LLM_PROVIDERS.md with new models and routing rules
- Rewrite regression tests with comprehensive coverage (23 unit tests pass)
* fix: CI violations — add safety comment on expect, fix fmt
- Add '// safety: hardcoded literal' to regex .expect() to satisfy
the no-panic-in-prod CI check
- Fix cargo fmt whitespace in collapsible if-let chain
* fix: address PR review feedback from gemini-code-assist
- Fix parse_custom_headers to preserve commas in values by splitting
only on commas followed by a header-name:colon pattern (manual scan
instead of simple split(','))
- Use matches! macro for backend exclusion check in app.rs
- Merge SSE metadata extraction into single pass (was iterating twice)
- Replace fragile substring-based context_length with explicit match
on known Gemini model IDs via gemini_context_length()
- Add missing models to regression test (8 models, not 5)
* fix: address Copilot PR review feedback
- Fix empty text part for assistant messages with tool calls
(curate_contents could drop entire model turn)
- Propagate cache_read/creation_input_tokens in complete_with_tools
- Log warning on save_credential failure instead of silently ignoring
- Fix doc comment to mention underscore in header name pattern
- Handle gemini-oauth (hyphen variant) in setup wizard display
- Fix docs: thinkingConfig uses thinkingBudget/thinkingLevel, not
includeThoughts
* fix: add missing allow_always field after staging merge
* fix(gemini_oauth): align header parser doc with implementation [skip-regression-check]
Update parse_custom_headers doc comments to include underscore in the
header-name character class, matching the actual implementation.
Also fix formatting from merge.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(gemini_oauth): curate_contents per-part filtering and dead code removal
Fix curate_contents to filter invalid parts individually instead of
dropping entire model turn sequences. Previously a single empty text
part would discard all consecutive model turns including valid
functionCall parts, breaking the tool-call flow.
Also remove unused MID_STREAM_* constants.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(gemini_oauth): rustfmt formatting [skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(llm): support smart routing cheap model for gemini_oauth backend
Add explicit gemini_oauth handling in create_cheap_provider_for_backend()
to create a GeminiOauthProvider with the cheap model swapped in. Without
this, setting LLM_CHEAP_MODEL with gemini_oauth backend would fail with
a confusing "no registry provider config available" error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add Gemini OAuth env vars to .env.example [skip-regression-check]
Document GEMINI_MODEL, GEMINI_CREDENTIALS_PATH, GEMINI_API_KEY, and
all extended generation config env vars in the example config file.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Illia Polosukhin
|
6232609080 |
feat(llm): add GitHub Copilot as LLM provider (#1512)
* Add github copilot as LLM provider.
* Fix Copilot in Openclaw
* security: harden Copilot OAuth token handling
C1: Use secrecy::SecretString for oauth_token and cached session token
in CopilotTokenManager/CachedCopilotToken. Expose only at HTTP
header injection point via .expose_secret().
C2: Document risks of hardcoded VS Code OAuth client ID and editor
identity headers (ToS, rotation, staleness). Remove the unreliable
paste-token setup path (setup_github_copilot_manual_token).
C3: Fix TOCTOU race in get_token() — re-check token validity after
acquiring write lock so concurrent callers don't all perform
redundant token exchanges.
I1: Remove dead empty else {} block in get_token().
I2: Map 401 responses to LlmError::AuthFailed instead of RequestFailed
so retry/circuit-breaker logic handles auth failures correctly.
I3: Replace prepare_github_copilot_setup() with call to existing
set_llm_backend_preserving_model() helper to avoid logic drift.
I4: Add unit tests for CopilotTokenManager (caching, invalidation,
expiry/buffer behavior), poll response parsing (all OAuth device
flow states), and DeviceCodeResponse/CopilotTokenResponse deserialization.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address review feedback and code improvements (takeover #1202)
- Fix ContentPart::Text being silently dropped in convert_messages
- Replace custom truncate_for_error with crate::util::floor_char_boundary
- Fix CLAUDE.md: accurately describe dedicated provider (not "OpenAI-compatible path")
- Fix "Github" -> "GitHub" capitalization in READMEs
- Add manual token paste option to setup wizard (not just device login)
- Fix missing extension_manager field in EngineContext (merge fixup)
- cargo fmt applied
Co-Authored-By: fallenwood <fallenwood@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review feedback for GitHub Copilot provider
- Plumb request_timeout_secs into GithubCopilotProvider (was hardcoded 120s)
- Forward stop_sequences to Copilot API via OpenAI `stop` field
- Skip empty text part in multimodal message conversion
- Improve paste-token wizard hint with specific file path guidance
Co-Authored-By: fallenwood <fallenwood@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: 401 retry, retryable token exchange errors, shared retry-after parsing
- Retry once inline on 401 after token invalidation (was returning
AuthFailed immediately, guaranteeing user-visible failure)
- Map token exchange failures to RequestFailed (retryable) instead of
AuthFailed (non-retryable by RetryProvider)
- Use shared crate::llm::retry::parse_retry_after for HTTP-date support
and safe 60s default
- Improve paste-token wizard hint: mention `gh auth token` as primary source
Co-Authored-By: fallenwood <fallenwood@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: 401 retry error mapping, retry status logging, token whitespace safety
- Map 401 retry get_token() failure to RequestFailed (retryable),
consistent with initial token acquisition path
- Log retry response status before returning AuthFailed
- Trim oauth_token in exchange_copilot_token to prevent header panics
from whitespace in env vars
Co-Authored-By: fallenwood <fallenwood@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Fallenwood <fallenwood.y@outlook.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: fallenwood <fallenwood@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Illia Polosukhin
|
3da9810e87 |
feat(llm): Add OpenAI Codex (ChatGPT subscription) as LLM provider (#1461)
* feat(llm): add OpenAI Codex backend config and OAuth session manager Add OpenAiCodex as a new LLM backend variant with config for auth endpoint, API base URL, client ID, and session persistence path. The session manager implements OpenAI's device code auth flow (headless-friendly, no browser required on the server) with automatic token refresh, following the same persistence pattern as the existing NEAR AI session manager. Closes #742 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(llm): add Responses API client and token-refreshing decorator Native Responses API client for chatgpt.com/backend-api/codex/responses, the endpoint that works with ChatGPT subscription tokens. Handles SSE streaming, text completions, and tool call round-trips. Token-refreshing decorator wraps the provider to pre-emptively refresh OAuth tokens before API calls and retry once on auth failures. Reports zero cost since billing is through subscription. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(llm): wire OpenAI Codex into provider factory, CLI, and setup wizard Connect the new provider to the LLM factory, add openai_codex to the CLI --backend flag, and add it as an option in the onboarding wizard. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(llm): address PR #744 review feedback (20 items) Review fixes for the OpenAI Codex provider PR: - Remove dead `generate_pkce()` code (device flow gets PKCE from server) - Fix `refresh_tokens()` to use `.form()` instead of `.json()` per OAuth spec - Inline codex dispatch into `build_provider_chain()` (single async function, no separate `assemble_provider_chain()` helper — matches main's pattern) - Remove Clone from `OpenAiCodexSession`, restrict fields to `pub(crate)` - Propagate HTTP client builder error instead of silent fallback - Redact device code response body from debug log - Change `set_model()` in TokenRefreshingProvider to delegate to inner - Replace hardcoded `/tmp/` test path with `tempfile::tempdir()` - Accept `request_timeout_secs` from config instead of hardcoded 300s - Parse `Retry-After` header on 429 responses (matches nearai_chat.rs pattern) - Reuse `normalize_schema_strict()` for Codex tool definitions - Add warning log for dropped image attachments - Add doc comments on `list_models()` and `include` field - Add `OPENAI_CODEX_API_URL` to `.env.example` - Fix codex error message in `create_llm_provider()` for clarity - Revert unrelated `.worktrees` addition to `.gitignore` - Update `src/llm/CLAUDE.md` with Codex provider docs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address review feedback and harden OpenAI Codex provider (takeover #744) Security: - Add SSRF validation (validate_base_url) on OPENAI_CODEX_AUTH_URL and OPENAI_CODEX_API_URL, matching the pattern used by all other base URL configs (regression test for #1103 included) Correctness: - Add missing cache_write_multiplier() and cache_read_discount() trait delegation in TokenRefreshingProvider - Cap device-code polling backoff at 60s to prevent unbounded interval growth on repeated 429 responses - Default expires_in to 3600s when server returns 0, preventing immediately-expired sessions - Fix pre-existing SseEvent::JobResult missing fallback_deliverable field in job_monitor.rs tests Cleanup: - Extract duplicated make_test_jwt() and test_codex_config() into shared codex_test_helpers module Co-Authored-By: Sanjeev-S <Sanjeev-S@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback on OpenAI Codex provider (#1461) - Login command now resolves OPENAI_CODEX_* env overrides even when LLM_BACKEND isn't set to openai_codex (Copilot review) - Setup wizard "Keep current provider?" for codex no longer re-triggers device code login — mirrors Bedrock's keep-and-return pattern (Copilot) - Revert provider init log from info back to debug (Copilot) - Add warning log when token expires_in=0, before defaulting to 3600s (Gemini review) Co-Authored-By: Sanjeev-S <Sanjeev-S@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Sanjeev Suresh <Sanjeev-S@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
806d402876 |
feat: chat onboarding and routine advisor (#927)
* feat: port NPA psychographic profiling system into IronClaw
Port the complete psychographic profiling system from NPA into IronClaw,
including enriched profile schema, conversational onboarding, profile
evolution, and three-tier prompt augmentation.
Personal onboarding moved from wizard Step 9 to first assistant
interaction per maintainer feedback — the First Contact system prompt
block now instructs the LLM to conduct a natural onboarding conversation
that builds the psychographic profile via memory_write.
Changes:
- Enrich profile.rs with 5 new structs, 9-dimension analysis framework,
custom deserializers for backward compatibility, and rendering methods
- Add conversational onboarding engine with one-step-removed questioning
technique, personality framework, and confidence-scored profile generation
- Add profile evolution with confidence gating, analysis metadata tracking,
and weekly update routine
- Replace thin interaction style injection with three-tier system gated on
confidence > 0.6 and profile recency
- Replace wizard Step 9 with First Contact system prompt block that drives
conversational onboarding during the user's first interaction
- Add autonomy progression to SOUL.md seed and personality framework to
AGENTS.md seed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: replace chat-based onboarding with bootstrap greeting and workspace seeds
Remove the interactive onboarding_chat.rs engine in favor of a simpler
bootstrap flow: fresh workspaces get a proactive LLM greeting that
naturally profiles the user. Identity files are now seeded from
src/workspace/seeds/ instead of being hardcoded. Also removes the
identity-file write protection (seeds are now managed), adds routine
advisor integration, and includes an e2e trace for bootstrap greeting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(safety): sanitize identity file writes via Sanitizer to prevent prompt injection
Identity files (SOUL.md, AGENTS.md, USER.md, IDENTITY.md) are injected into
every system prompt. Rather than hard-blocking writes (which broke onboarding),
scan content through the existing Sanitizer and reject writes with High/Critical
severity injection patterns. Medium/Low warnings are logged but allowed.
Also clarifies AGENTS.md identity file roles (USER.md = user info, IDENTITY.md =
agent identity) and adds IDENTITY.md setup as an explicit bootstrap step.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update profile_onboarding_completed comment to reflect current wiring
The field is now actively used by the agent loop to suppress BOOTSTRAP.md
injection — remove the stale "not yet wired" TODO.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(setup): use env_or_override for NEARAI_API_KEY in model fetch config
When the user authenticates via NEAR AI Cloud API key (option 4),
api_key_login() stores the key via set_runtime_env(). But
build_nearai_model_fetch_config() was using std::env::var() which
doesn't check the runtime overlay — so model listing fell back to
session-token auth and re-triggered the interactive NEAR AI
authentication menu.
Switch to env_or_override() which checks both real env vars and the
runtime overlay.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(agent): correct channel/user_id in bootstrap greeting persist call
persist_assistant_response was called with channel="default",
user_id="system" but the assistant thread was created via
get_or_create_assistant_conversation("default", "gateway") which owns
the conversation as user_id="default", channel="gateway". The mismatch
caused ensure_writable_conversation to reject the write with:
WARN Rejected write for unavailable thread id user=system channel=default
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(web): remove all inline event handlers for CSP compliance
The Content-Security-Policy header (added in
|
||
|
Octopus
|
2d0b195321 |
feat: upgrade MiniMax default model to M2.7 (#1357)
* feat: upgrade MiniMax default model to M2.7 - Add MiniMax-M2.7 and MiniMax-M2.7-highspeed to model list - Set MiniMax-M2.7 as default model - Keep all previous models as alternatives - Update related tests * fix: use canonical model name in test per review Use MiniMax-M2.7-highspeed (canonical casing) in the reasoning models test for consistency with the documentation and provider configuration. [skip-regression-check] |
||
|
ZeroTrust
|
1b59eb6b39 |
feat: Reuse Codex CLI OAuth tokens for ChatGPT backend LLM calls (#693)
* feat: add Codex auth.json token reuse for LLM authentication When LLM_USE_CODEX_AUTH=true, IronClaw reads the Codex CLI's auth.json (default ~/.codex/auth.json) and extracts the API key or OAuth access token. This lets IronClaw piggyback on a Codex login without implementing its own OAuth flow. New env vars: - LLM_USE_CODEX_AUTH: enable Codex auth fallback (default: false) - CODEX_AUTH_PATH: override path to auth.json * fix: handle ChatGPT auth mode correctly Switch base_url to chatgpt.com/backend-api/codex when auth.json contains ChatGPT OAuth tokens. The access_token is a JWT that only works against the private ChatGPT backend, not the public OpenAI API. Refactored codex_auth.rs to return CodexCredentials (token + is_chatgpt_mode) instead of just a string key. * fix: Codex auth takes highest priority over secrets store When LLM_USE_CODEX_AUTH=true, Codex credentials are now loaded before checking env vars or the secrets store overlay. Previously the secrets store key (injected during onboarding) would shadow the Codex token. * feat: Responses API provider for ChatGPT backend - New CodexChatGptProvider speaks the Responses API protocol - Auto-detects model from /models endpoint (gpt-4o -> gpt-5.2-codex) - Adds store=false (required by ChatGPT backend) - Error handling with timeout for HTTP 400 responses - Message format translation: Chat Completions -> Responses API - SSE response parsing for text, tool calls, and usage stats - 7 unit tests for message conversion and SSE parsing * fix: SSE parser uses item_id instead of call_id for tool call deltas The Responses API sends function_call_arguments.delta events with item_id (e.g. fc_...) not call_id (e.g. call_...). The parser now keys pending tool calls by item_id from output_item.added and tracks call_id separately for result matching. * fix: strip empty string values from tool call arguments gpt-5.2-codex fills optional tool parameters with empty strings (e.g. timestamp: ""), which IronClaw's tool validation rejects. Strip them before passing to tool execution. * fix: prevent apiKey mode fallback to ChatGPT token When auth_mode is explicitly 'apiKey' but the key is missing/empty, do not fall through to check for a ChatGPT access_token. This prevents returning credentials with is_chatgpt_mode: true and routing to the wrong LLM provider. * refactor: reuse single reqwest::Client across model discovery and LLM calls Create Client once in with_auto_model, pass &Client to fetch_default_model, and move it into the provider struct. Eliminates the redundant Client::new() that wasted a connection pool. * fix: bump client_version to 1.0.0 to unlock gpt-5.3-codex and gpt-5.4 The /models endpoint gates newer models behind client_version. Version 0.1.0 only returns up to gpt-5.2-codex, while 1.0.0+ also returns gpt-5.3-codex and gpt-5.4. * feat: user-configured LLM_MODEL takes priority over auto-detection Fetch the full model list from /models endpoint. If LLM_MODEL is set, validate it against the supported list and warn with available models if not found. If LLM_MODEL is not set, auto-detect the highest-priority model. Also bumps client_version to 1.0.0 to unlock gpt-5.3/5.4. * fix: add 10s timeout to model discovery HTTP request Prevents startup from blocking indefinitely if chatgpt.com is slow or unreachable. Uses reqwest per-request timeout. * docs: add private API warning for ChatGPT backend endpoint The chatgpt.com/backend-api/codex endpoint is private and undocumented. Add warning in module docs and a runtime log on first use to inform users of potential ToS implications. * feat: implement OAuth 401 token refresh for Codex ChatGPT provider On HTTP 401, if a refresh_token is available, the provider now automatically refreshes the access token via auth.openai.com/oauth/token (same protocol as Codex CLI) and retries the request once. Refreshed tokens are persisted back to auth.json. Changes: - codex_auth: read refresh_token, add refresh_access_token() and persist_refreshed_tokens() - codex_chatgpt: RwLock for api_key, 401 detection + retry in send_request, send_http_request helper - config/llm: thread refresh_token/auth_path through RegistryProviderConfig - llm/mod: pass refresh params to with_auto_model * refactor: lazy model detection via OnceCell, remove block_in_place Model is no longer resolved during provider construction. Instead, resolve_model() uses tokio::sync::OnceCell to lazily fetch from /models on the first LLM call. This eliminates the block_in_place + block_on workaround in create_codex_chatgpt_from_registry. - with_auto_model (async) -> with_lazy_model (sync constructor) - resolve_model() added with OnceCell-based lazy init - build_request_body takes model as parameter - model_name() returns resolved or configured_model as fallback * feat: support multimodal content (images) in Codex ChatGPT provider message_to_input_items now checks content_parts for user messages. ContentPart::Text maps to input_text and ContentPart::ImageUrl maps to input_image, matching the Responses API format used by Codex CLI. Falls back to plain text when content_parts is empty. Also updates client_version to 0.111.0 for /models endpoint. Adds test: test_message_conversion_user_with_image * refactor: move codex_auth module from src/ to src/llm/ codex_auth is only used by the LLM layer (codex_chatgpt provider and config/llm). Moving it under src/llm/ reflects its actual scope. - Remove pub mod codex_auth from lib.rs - Add pub mod codex_auth to llm/mod.rs - Update imports: super::codex_auth, crate::llm::codex_auth * Fix codex provider style issues * Use SecretString throughout codex auth refresh flow * Use SecretString for codex access tokens * Reuse provider client for codex token refresh * Stream Codex SSE responses incrementally * Fix Windows clippy and SQLite test linkage * Trigger checks after regression skip label * Tighten codex auth module handling |
||
|
Ethan Clarke
|
863702a87a |
feat: add MiniMax as a built-in LLM provider (#940)
Add MiniMax to the provider registry with OpenAI-compatible protocol. Available models: - MiniMax-M2.5 (default) - 204,800 token context window - MiniMax-M2.5-highspeed - same performance, faster inference Configuration: LLM_BACKEND=minimax MINIMAX_API_KEY=<your-key> Supports both global (api.minimax.io) and China mainland (api.minimaxi.com) endpoints via MINIMAX_BASE_URL env var. Co-authored-by: PR Bot <pr-bot@minimaxi.com> |
||
|
Zaki Manian
|
195ff44b1a | fix(security): migrate webhook auth to HMAC-SHA256 signature header (#970) | ||
|
Zaki Manian
|
8bbb43da52 |
fix(security): require explicit SANDBOX_ALLOW_FULL_ACCESS to enable FullAccess policy (#967)
* fix(security): require explicit SANDBOX_ALLOW_FULL_ACCESS to enable FullAccess policy FullAccess policy bypasses Docker entirely and runs commands via sh -c directly on the host. Previously, setting SANDBOX_POLICY=full_access alone was sufficient to enable this, which could be triggered accidentally or via prompt injection if tool approval is bypassed. This adds a double opt-in guard: - New SANDBOX_ALLOW_FULL_ACCESS=true env var must ALSO be set for FullAccess to take effect. Without it, the policy is downgraded to WorkspaceWrite with a tracing::error! log. - At execution time, every FullAccess command emits a tracing::warn! with the command and working directory for audit visibility. - The FullAccess variant now documents its blast radius (host shell, unrestricted filesystem/network/environment). - SandboxConfig and SandboxModeConfig gain an allow_full_access field, wired through from_env() and the builder. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(sandbox): address review feedback on FullAccess double opt-in - Add doc comment on builder .policy() warning that FullAccess requires .allow_full_access(true) or execution will return SandboxError::Config - Sanitize audit log: log only binary name instead of full command to prevent secret leakage; add [FullAccess] prefix for grep-ability - Add test_builder_full_access_without_allow_returns_error test covering the builder path without explicit allow_full_access(true) - Fix doc comment mismatch: config.rs and SandboxPolicy::FullAccess docs said "will downgrade to WorkspaceWrite" but runtime returns SandboxError::Config -- aligned docs with actual behavior Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: merge duplicate mod tests; add allow_full_access to struct initializers After upstream merge, src/config/sandbox.rs had two issues: - Duplicate mod tests block (upstream's original tests at line 271 + our new FullAccess guard tests at line 478) caused E0428 compile error - Upstream test struct literals for SandboxModeConfig were missing the new allow_full_access field (E0063) Fixes: merge the two mod tests into one; add allow_full_access: false to the sandbox_mode_config_custom_values and sandbox_mode_to_sandbox_config test struct initializers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Gabe Hamilton <gabe@near.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
83950d11a4 |
fix: job token budget, iteration cap → Failed, web cancel stops worker (#788)
* feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix: add job token budget, change iteration cap to Failed, fix web cancel (#698) Jobs could enter infinite retry loops because: (1) no token budget was enforced, (2) iteration cap marked jobs as Stuck (allowing self-repair to restart them), and (3) the web UI cancel button only updated the DB without stopping the running worker. - Add `max_tokens_per_job` config (settings.json + AGENT_MAX_TOKENS_PER_JOB env var, default 0 = unlimited) with per-job metadata override - Track token usage after respond_with_tools() and fail the job on budget exceeded - Change iteration cap and persistent rate limiting from mark_stuck to mark_failed, preventing self-repair restart loops - Fix web cancel handler to call scheduler.stop() which updates in-memory state AND aborts the worker task, falling back to DB-only update Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — always persist cancel to DB, simplify token check - Cancel handler now always persists Cancelled to DB regardless of whether scheduler.stop() ran, fixing the edge case where stop() returns Ok(()) for jobs not in the scheduler map - Collapse nested ifs per clippy (let-chains) - Add NOTE comment about select_tools() not exposing TokenUsage [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: rustfmt formatting in wizard.rs (pre-existing) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Zaki Manian
|
200aed16cd |
feat: configurable LLM request timeout via LLM_REQUEST_TIMEOUT_SECS (#615) (#630)
Add LLM_REQUEST_TIMEOUT_SECS env var (default: 120) to configure the HTTP request timeout for LLM API calls. Primarily useful for local models (Ollama, vLLM, LM Studio) that need more time for prompt evaluation on consumer hardware. The timeout is applied to the NearAI provider's HTTP client. Other providers (Anthropic, OpenAI) use rig-core's default client. - Add request_timeout_secs field to LlmConfig - Thread timeout through create_llm_provider -> NearAiChatProvider - Add NearAiChatProvider::new_with_timeout constructor - Add .env.example documentation - 2 regression tests for default and custom timeout values Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
firat.sertgoz
|
11c5e25422 |
feat(setup): Anthropic OAuth onboarding with setup-token support (#384)
* feat(setup): add Anthropic OAuth and Codex OAuth onboarding flows Add OAuth token authentication as an alternative to API keys during onboarding for both Anthropic (via `claude login`) and OpenAI/Codex (via `~/.codex/auth.json`). Key changes: - New `AnthropicOAuthProvider` using `Authorization: Bearer` header (rig-core hardcodes `x-api-key` which rejects OAuth tokens) - Wizard auth method selector: "Direct API Key" vs "OAuth Token" for both Anthropic and OpenAI providers - Codex token extraction from `$CODEX_HOME/auth.json` / `~/.codex/auth.json` - Claude Code sandbox sub-step in Docker setup (checks for credentials) - Secret injection mappings for `ANTHROPIC_OAUTH_TOKEN` and `CODEX_OAUTH_TOKEN` - `CODEX_OAUTH_TOKEN` falls back to `OPENAI_API_KEY` (same Bearer auth) Supersedes #143 which had a broken auth flow (OAuth token sent as x-api-key → 401). Credit to @bigguybobby for the original approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: persist OAuth tokens in bootstrap .env and re-extract at startup OAuth tokens stored only in the secrets DB were invisible to Config::from_env() which runs before the DB connects (chicken-and-egg). Two fixes: 1. write_bootstrap_env() now persists ANTHROPIC_OAUTH_TOKEN and CODEX_OAUTH_TOKEN to ~/.ironclaw/.env (same pattern as NEARAI_API_KEY) 2. main.rs re-extracts a fresh token from the OS credential store (macOS Keychain / ~/.claude/.credentials.json) before config resolution, handling token expiry (8-12h) gracefully Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: persist all LLM credentials in bootstrap .env, not just NEAR AI All providers had the same chicken-and-egg issue: API keys stored in the secrets DB were invisible to Config::from_env() which runs before DB connects. Only NEARAI_API_KEY was written to bootstrap .env. Now write_bootstrap_env() persists all credential env vars: NEARAI_API_KEY, ANTHROPIC_API_KEY, ANTHROPIC_OAUTH_TOKEN, OPENAI_API_KEY, CODEX_OAUTH_TOKEN, LLM_API_KEY, TINFOIL_API_KEY. Also: setup_api_key_provider() now sets the env var during the wizard session so write_bootstrap_env() can pick it up. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address security review findings for OAuth onboarding - Extract "oauth-placeholder" to named OAUTH_PLACEHOLDER constant shared across config and wizard to prevent silent drift - Document plaintext credential tradeoff in write_bootstrap_env (API keys stored with 0o600 permissions, recommend full-disk encryption) - Add blocking "Press Enter" wait in Anthropic OAuth retry flow so user has time to run `claude login` in another terminal - Add escape hatch from manual OAuth paste back to API key flow (empty input switches to setup_api_key_provider) - Fix Retry-After header: parse u64 seconds into Duration before passing to LlmError::RateLimited - Make config::llm module pub(crate) for constant visibility - Use .bearer_auth() instead of manual format!("Bearer {}") - Remove response body from debug log (may contain PII) - Update Anthropic API version to 2024-10-22 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * security: remove plaintext credentials from bootstrap .env Credentials (API keys, OAuth tokens) were being written in plaintext to ~/.ironclaw/.env to work around a chicken-and-egg problem: Config::from_env() runs before the encrypted secrets DB is connected. Instead of storing secrets on disk, LlmConfig::resolve() now defers gracefully when credentials are missing — it returns None for the provider config instead of hard-erroring with MissingRequired. After the DB connects, AppBuilder::build_all() loads secrets from encrypted storage via inject_llm_keys_from_secrets() and re-resolves the config. For Anthropic OAuth tokens (which expire in 8-12h), the secret injection step also tries the OS credential store (macOS Keychain / Linux credentials.json) for a fresh token, overriding the potentially stale copy in the DB. Changes: - LlmConfig::resolve(): OpenAI, Anthropic, OpenAI-compatible, and Tinfoil all return None instead of MissingRequired when credentials are absent - write_bootstrap_env(): no longer writes any credential env vars - inject_llm_keys_from_secrets(): refreshes Anthropic OAuth from OS credential store before overlay is finalized - main.rs: removed OAuth re-extraction hack (no longer needed) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: load OS credential store tokens even without secrets DB The OAuth token extraction from macOS Keychain / Linux credentials files was only running inside inject_llm_keys_from_secrets(), which requires the encrypted secrets DB. When no master key is configured, init_secrets() returned early — skipping both DB secret loading AND OS credential store extraction, leaving the Anthropic OAuth token unavailable. Split into two paths: - inject_llm_keys_from_secrets(): loads from encrypted DB + OS stores - inject_os_credentials(): loads from OS stores only (no DB needed) init_secrets() now calls inject_os_credentials() and re-resolves config even in the no-master-key early-return path, so `claude login` tokens are always available regardless of secrets DB state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add anthropic-beta header required for OAuth authentication Anthropic's api.anthropic.com requires the `anthropic-beta: oauth-2025-04-20` header to accept OAuth Bearer tokens. Without it, the API returns 401 "OAuth authentication is currently not supported." Also reverts API version to 2023-06-01 since the OAuth beta flag does not support the 2024-10-22 version (returns 400 "not a valid version"). This was the same bug that caused PR #143's 401 errors — the beta header was missing entirely. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Anthropic and OpenAI model resolution respects selected_model The Anthropic and OpenAI config resolution ignored settings.selected_model entirely, only checking the provider-specific env var (ANTHROPIC_MODEL, OPENAI_MODEL) and falling back to a hardcoded default. This meant the model chosen during onboarding wizard was silently overridden. Now follows the same pattern as NearAI and OpenAI-compatible: env var > settings.selected_model > hardcoded default. Also deduplicated the Anthropic config construction (two identical branches for API key vs OAuth now share model/base_url resolution). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add provider resolution tests for all LLM backends Covers deferred resolution (no credentials → None instead of error), credential presence, model selection fallback chain, and OAuth token routing for Anthropic, OpenAI, Tinfoil, Ollama, and NearAI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle nested tokens.access_token format in Codex auth.json Codex CLI stores OAuth tokens in a nested format under tokens.access_token (ChatGPT OAuth flow), not at the top level. Also adds ENV_MUTEX to Codex token tests for thread safety. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: remove Codex OAuth onboarding (incompatible with OpenAI API) Codex CLI OAuth tokens use a different endpoint (chatgpt.com/backend-api/codex) and the Responses API wire format, not api.openai.com with Chat Completions. The tokens lack the model.request scope needed for the platform API, so they can't be used as drop-in OPENAI_API_KEY replacements. Removes: extract_codex_oauth_token(), wizard Codex OAuth flow, CODEX_OAUTH_TOKEN env var support, and related tests. OpenAI onboarding now uses direct API key only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix formatting for CI (cargo fmt) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address Gemini review feedback - Use ? operator for ANTHROPIC_MODEL/BASE_URL env resolution instead of .ok().flatten() to propagate ConfigErrors consistently - Skip Tool messages without tool_call_id with a warning instead of using unwrap_or_default() which would send empty string to Anthropic - Extract credential check into closure to reduce duplication in Claude Code sandbox setup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(review): address PR review feedback for OAuth onboarding - Gate ANTHROPIC_OAUTH_TOKEN resolution to Anthropic provider only (was needlessly checked for all registry providers) - Add 3 regression tests for OAuth config resolution: - oauth_token sets placeholder api_key - real api_key takes priority over oauth - non-Anthropic providers don't pick up oauth_token - Validate OAuth token prefix (sk-ant-oat) in wizard to catch accidentally pasted API keys - Improve error body read handling in AnthropicOAuthProvider (was silently swallowing read errors with unwrap_or_default) - Remove extra blank line in write_bootstrap_env - Remove stale blank line in RegistryProviderConfig doc comment [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR #384 review comments Blocker: - Replace OnceLock<HashMap> with LazyLock<Mutex<HashMap>> for INJECTED_VARS so both inject_os_credentials() and inject_llm_keys_from_secrets() merge data instead of the second caller silently dropping its entries. High: - Add 401 retry with OS credential store re-extraction in AnthropicOAuthProvider, recovering from expired OAuth tokens (~8-12h) without manual intervention. - Fix comment in app.rs: ~/.codex/auth.json → ~/.claude/.credentials.json. Medium: - Remove unsafe { std::env::set_var } from wizard; use thread-safe inject_single_var() overlay instead (safe on multi-threaded Tokio). - Add post-init validation in AppBuilder: fail early with clear error when LLM_BACKEND is set but no credentials were resolved after secret injection. - Add sk-ant-oat prefix validation in parse_oauth_access_token(). - Only route to AnthropicOAuthProvider when api_key is missing or equals OAUTH_PLACEHOLDER (API key takes priority over OAuth token). - Teach fetch_anthropic_models() to use Bearer auth when only OAuth token is available (model listing no longer fails for OAuth-only users). Low: - Use optional_env() in wizard credential checks to read from injected overlay, not just raw env vars. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: cargo fmt Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com> |
||
|
Illia Polosukhin
|
424a0366a9 |
feat: enable Anthropic prompt caching via automatic cache_control injection (#660)
* feat(llm): add Anthropic prompt caching and cache token tracking - Inject cache_control via additional_params for Claude models in rig_adapter - Add cache_read_input_tokens and cache_creation_input_tokens to CompletionResponse and ToolCompletionResponse - Extract cached_input_tokens from rig-core unified Usage - Add is_anthropic_model() detection helper with provider prefix support - Log prompt cache hits at debug level (consistent with response_cache) - Add 7 unit tests for cache injection and model detection - Update all mock providers and test fixtures with new fields * feat(cost): apply 90% cache discount to prompt-cached tokens in CostGuard - Add cache_read_input_tokens to TokenUsage so cache counts flow from CompletionResponse through the reasoning layer to the dispatcher - Update CostGuard::record_llm_call() to accept cache_read_input_tokens: cached tokens are billed at 10% of the normal input rate - Thread cache_read_input_tokens from dispatcher into CostGuard - Add test_cache_discount_reduces_cost verifying exact savings match 90% of input cost for fully-cached requests - Update all existing test callers with zero-cache parameter * refactor(cache): scope cache_control to Anthropic backend and validate model support - Replace model-name-based is_anthropic_model() with explicit enable_prompt_cache flag on RigAdapter, set only for the direct Anthropic backend via with_prompt_cache(true) - Add supports_prompt_cache() to validate model names per Anthropic docs: only Claude 3+ models support caching; claude-2 and claude-instant are excluded to prevent 400 errors - Warn when caching is enabled but model does not support it - Replace is_anthropic_model tests with flag-based and model validation tests * fix(cache): validate model at construction and propagate cache metrics through proxy - Move supports_prompt_cache() check into with_prompt_cache() so unsupported models are detected once at construction, not per request - Add cache_read_input_tokens and cache_creation_input_tokens to ProxyCompletionResponse and ProxyToolCompletionResponse with serde(default) for backward compatibility - Pass cache metrics through orchestrator proxy instead of zeroing - Use claude-opus-4-6 in cache discount test to match Anthropic semantics * feat(llm): add configurable cache retention with write surcharge - Add CacheRetention enum (none/short/long) to AnthropicDirectConfig - Parse ANTHROPIC_CACHE_RETENTION env var (default: short) - Inject TTL-aware cache_control (short=5m ephemeral, long=1h) - Extract cache_creation_input_tokens from raw Anthropic response - Add cache_write_multiplier() to LlmProvider trait (1.25x short, 2.0x long) - Pipe dynamic write multiplier through dispatcher to CostGuard - Add TokenUsage.cache_creation_input_tokens field - Add tests for Long TTL injection, 5m and 1h write surcharges - Document ANTHROPIC_CACHE_RETENTION in .env.example * docs: fix stale cache_retention field comment * fix: resolve CI failures after upstream merge - Add missing cost_per_token arg to cache test callsites - Apply cargo fmt to long lines in tests and tracing macros * fix: address Copilot review feedback - Use saturating_add for cache token sum to prevent u32 overflow - Tighten supports_prompt_cache to explicitly match claude-3+/claude-4+ and named families (claude-sonnet/claude-opus/claude-haiku) * fix: adapt prompt caching to registry architecture and add missing cache fields - Resolve merge conflicts: adapt CacheRetention and cache injection to the declarative provider registry (RegistryProviderConfig replaces AnthropicDirectConfig) - Parse ANTHROPIC_CACHE_RETENTION env var in create_anthropic_from_registry() - Use Anthropic automatic caching via top-level cache_control in additional_params (rig-core #[serde(flatten)] places it at request root) - Add cache_read/creation_input_tokens fields to all mock LlmProviders added on main after PR #291 branched (response_cache, dispatcher, provider_chaos, trace_llm) - Suppress clippy::too_many_arguments on record_llm_call and build_rig_request - Add regression tests for cache injection (short/long/none) and cache_write_multiplier values Co-Authored-By: Canvinus <44225021+Canvinus@users.noreply.github.com> * fix: delegate cache_write_multiplier through provider wrappers and make cache_read_discount configurable The 6 decorator providers (Retry, CircuitBreaker, Failover, SmartRouting, CachedProvider, RecordingLlm) did not delegate cache_write_multiplier() to their inner provider, causing it to always return 1.0 instead of the actual 1.25x/2.0x from RigAdapter. This fix adds delegation for both cache_write_multiplier() and the new cache_read_discount() method. Also makes the cache read discount per-provider instead of hardcoding Anthropic's 90% discount (÷10). OpenAI uses 50% (÷2), so the discount is now returned by each provider via the LlmProvider trait. Addresses review feedback on PR #660. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: cargo fmt Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add CacheRetention FromStr/Display unit tests Tests cover primary values, aliases (off/disabled/5m/ephemeral/1h), case-insensitivity, invalid input error, and Display round-trip. Addresses Copilot review feedback on PR #660. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Andrey <canvi@2bb.dev> Co-authored-by: Andrey Gruzdev <44225021+Canvinus@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nick Pismenkov
|
d195222124 |
feat: Wire memory hygiene retention policy into heartbeat loop (#629)
* feat: Wire memory hygiene retention policy into heartbeat loop * review fix * linter fix * fix tests |
||
|
Nick Pismenkov
|
9ae04f14e3 |
feat: restart (#531)
* feat: restart * review fixes * add IRONCLAW_IN_DOCKER env variable * review fixes * fix tests * set default value as false |
||
|
ibhagwan
|
b0b3a50fa3 |
feat(channels): add native Signal channel via signal-cli HTTP daemon (#271)
* feat(channels): add native Signal channel via signal-cli HTTP daemon Implement a native Rust Signal channel that connects to a running signal-cli daemon's HTTP endpoint, enabling Signal messaging without WASM overhead. Architecture: - SSE listener at /api/v1/events for receiving messages with automatic reconnection and exponential backoff - JSON-RPC client at /api/v1/rpc for sending messages and typing indicators - Reply target tracking via Arc<RwLock<HashMap>> to route responses back to the correct DM or group conversation Features: - User allowlisting supporting E.164 phone numbers, bare UUIDs, and uuid:-prefixed identifiers (matching OpenClaw's format) - Group allowlisting with wildcard (*) support - Configurable story and attachment-only message filtering - Health check via signal-cli /api/v1/check - Broadcast support to all tracked reply targets Configuration via environment variables: - SIGNAL_HTTP_URL, SIGNAL_ACCOUNT (required) - SIGNAL_ALLOWED_USERS, SIGNAL_ALLOWED_GROUPS - SIGNAL_IGNORE_ATTACHMENTS (default: false) - SIGNAL_IGNORE_STORIES (default: true) Includes unit tests covering allowlist logic, envelope parsing, recipient targeting, SSE deserialization, and edge cases. * refactor(signal): remove expect|unwrap calls - Change SignalChannel::new to return Result<Self, ChannelError> - Replace .expect() on reqwest client build with proper error handling - Replace .expect() on NonZeroUsize with compile-time const using unsafe new_unchecked - Propagate errors through test helpers to avoid unwraps in tests * fix(signal): prevent OOM from chunked response without Content-Length Use bytes_stream() to check response size during download rather than buffering entire body first. This closes the OOM vector where a malicious signal-cli daemon could send unbounded chunked data. * fix(signal): align is_e164 minimum digits with setup wizard Both now require 7-15 digits after '+', preventing environment variable bypass of the stricter onboarding validation. * refactor(signal): extract from_parts constructor Extract SignalChannel::from_parts() used by both new() and sse_listener() to ensure consistent object construction. * chore: remove redundant unused var * refactor(signal): rename allowed_users to allow_from and add dm_policy/group_policy - Rename allowed_users -> allow_from for consistency with other channels - Rename allowed_groups -> allow_from_groups - Add dm_policy field: 'open', 'allowlist', or 'pairing' (default: 'pairing') - Add group_policy field: 'allowlist', 'open', or 'disabled' (default: 'allowlist') - Add group_allow_from field that inherits from allow_from if empty - Implement dm_policy and group_policy logic in message processing - Add environment variable resolution: SIGNAL_ALLOW_FROM, SIGNAL_ALLOW_FROM_GROUPS, SIGNAL_DM_POLICY, SIGNAL_GROUP_POLICY, SIGNAL_GROUP_ALLOW_FROM - Add setup wizard prompts for new policy options - Note: full pairing flow (PairingStore integration) marked as pending for future PR * feat(signal): implement DM pairing workflow for unapproved senders - Add PairingStore integration to check approved senders - Handle pairing requests for unknown senders with dm_policy=pairing - Send pairing reply message with approval instructions - Update FEATURE_PARITY.md to reflect DM pairing support * chore(ci): fix clippy warnings |
||
|
DevBroco
|
7f68207f1e |
Feat/completion (#240)
* feat: add OpenRouter usage examples * feat: add HTPS headers * feat: add shell completion generation via clap_complete * feat: add shell completion generation via clap_complete * feat: add shell completion generation via clap_complete * Refactor completion: use clap_complete::Shell directly, improve tests, remove tracing duplication, fix .env.example and Cargo.toml * fix: rename init_cli_logging to init_cli_tracing (sync with main) --------- Co-authored-by: BroccoliFin <mikhailsadovoy@MacBook-Air-MacMike.local> Co-authored-by: firat.sertgoz <f@nuff.tech> |
||
|
Nitanshu Lokhande
|
3124ab2b7f |
docs: add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) (#193)
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers - Expand .env.example with Together AI and Fireworks AI example configs - Add "Alternative LLM Providers" section to README with quickstart snippet Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: firat.sertgoz <f@nuff.tech> |
||
|
alexthebuildr
|
493e4578d0 |
feat: support custom HTTP headers for OpenAI-compatible provider (#269)
Add LLM_EXTRA_HEADERS env var (format: Key:Value,Key2:Value2) to inject custom HTTP headers into every request to OpenAI-compatible endpoints. This enables OpenRouter attribution headers (HTTP-Referer, X-Title) and other service-specific headers without code changes. Closes #179 Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> |
||
|
Illia Polosukhin
|
448383cfb0 |
refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably - Filter out `type: "reasoning"` output items from NEAR AI Responses API parsing so chain-of-thought never reaches the UI (nearai.rs) - Rewrite clean_response with regex-based tag stripping that is code-aware (preserves tags inside fenced blocks and inline backticks), supports 9+ tag names (think, thought, reasoning, reflection, etc.), handles <final> extraction, pipe-delimited tags, and case/whitespace tolerance (reasoning.rs) - Add Reasoning::complete() helper so all non-agentic LLM call sites (summarize, suggest, heartbeat, compaction) get automatic response cleaning; thread SafetyLayer through to those callers - Change persist_turn from fire-and-forget tokio::spawn to awaited async so both user and assistant messages are written before returning, preventing data loss on shutdown/restart - Pass input_count through seed_response_chain so response chaining delta calculation is accurate after thread hydration on restart - Make NearAiResponse.usage optional and preserve response_id in alt response path for chaining continuity - Persist session token to DB during onboarding wizard so runtime loads it without legacy-key fallback; suppress spurious warning on fresh installs - Fix dev tool double-registration when builder already registers them - Load dotenv/ironclaw env for doctor and status subcommands - Reduce startup log noise (demote info→debug for skills, remove redundant info lines) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Nudge to not loop over tools continuesly * refactor: remove Responses API, consolidate NEAR AI to Chat Completions only The Responses API provider (nearai.rs, 1278 lines) added significant complexity (response chaining state machine, delta message calculation, previous_response_id persistence) for marginal benefit. This consolidates to the Chat Completions API only, upgrading NearAiChatProvider with dual auth (session token + API key) and 401 retry for session token renewal. - Delete src/llm/nearai.rs (Responses API provider) - Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models - Remove response_id from CompletionResponse and ToolCompletionResponse - Remove seed_response_chain/get_response_chain_id from LlmProvider trait - Remove response chain persistence from agent (thread_ops, session) - Remove NearAiApiMode enum and NEARAI_API_MODE config - Clean up all wrapper providers (retry, circuit_breaker, failover, cache) - Update documentation (CLAUDE.md, .env.example) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: runtime log level control via gateway UI and URL parameter Add server-side log level switching using tracing_subscriber::reload::Layer so the EnvFilter can be swapped at runtime without restarting. Expose via GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs toolbar, and a ?log_level=debug URL parameter for one-click activation. Also applies cargo fmt to pre-existing files (llm/, tests/). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
5725a62c83 |
fix: onboarding errors reset flow and remote server auth (#185, #186) (#248)
* fix: incremental settings persistence and remote server auth (#185, #186) Persist settings after each wizard step so failures don't lose prior progress. Load existing settings on re-run to recover from partial onboarding. Add manual token paste option for remote/headless servers where browser OAuth is unreachable, and support IRONCLAW_OAUTH_CALLBACK_URL for custom callback URLs. Color prompt output (green/red/blue prefixes). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace session token paste with API key entry, address PR review Replace option 4 in NEAR AI auth menu from session token paste to NEAR AI Cloud API key entry (cloud.near.ai). Also address all PR review feedback: restrict .env file permissions to 0o600, mask API key input with secret_input, fix libsql loaded flag in try_load_existing_settings, add ENV_MUTEX to oauth_defaults tests, and add NEARAI_API_KEY to secrets injection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: deduplicate keys in upsert_bootstrap_var When the .env file contains duplicate keys (e.g. from manual editing), only write the replacement once and skip subsequent duplicates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: NEARAI_SESSION_TOKEN env var takes precedence over file-based tokens Hosting providers inject session tokens via env var and expect them to be used directly. Previously the env var was only picked up when no session file existed and was treated as a legacy migration. Now the env var always wins, without persisting to disk. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: distinguish NEAR AI Chat and NEAR AI Cloud providers Split documentation into two clearly named modes: - NEAR AI Chat: Responses API at private.near.ai, session token auth - NEAR AI Cloud: Chat Completions API at cloud-api.near.ai, API key auth Update default base URLs so each mode points to its correct endpoint. Update .env.example, deploy/env.example, CLAUDE.md, setup spec, and code comments across config/llm.rs, nearai.rs, nearai_chat.rs, mod.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: wizard recovery ordering — load DB before persist, fresh choices win Previously, persist_after_step() ran after Step 1 but before try_load_existing_settings(), bulk-upserting defaults that clobbered prior settings. Additionally, merge_from gave stale DB values precedence over fresh Step 1 choices. Fix: snapshot Step 1 settings, load DB, then re-apply the snapshot. This ensures prior progress (steps 2-7) is recovered while fresh Step 1 choices override stale DB values. Add two tests verifying wizard recovery merge ordering. Addresses PR review comments from Copilot on wizard.rs:150, wizard.rs:1607, and wizard.rs:1626. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix rustfmt formatting in config/llm.rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: collapse nested if per clippy collapsible_if lint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use print_success for API key confirmation, fix menu spacing - Use print_success() for colored output consistency in api_key_login - Fix box-drawing alignment: options 1-2 had an extra trailing space Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nitanshu Lokhande
|
fa64df05ff |
feat: wire memory hygiene into the heartbeat loop (#195)
* feat: wire memory hygiene into heartbeat loop (#166) * refactor: address PR review comments for hygiene wiring * style: fix fmt import ordering and clippy too_many_arguments warning * fix: update heartbeat integration test to pass HygieneConfig argument HeartbeatRunner::new() now requires a HygieneConfig as its second argument after the hygiene wiring refactor. Pass the default config in the integration test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> |
||
|
DevBroco
|
05cb01816b |
feat: add OpenRouter usage examples (#189)
Co-authored-by: BroccoliFin <mikhailsadovoy@MacBook-Air-MacMike.local> |
||
|
Illia Polosukhin
|
cfb579a4bb |
feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows (#57)
* feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows Adds JobEventsTool and JobPromptTool so the main agent can read container event logs and send follow-up prompts to running Claude Code sessions. A background JobMonitor forwards container assistant messages into the agent loop via a new inject channel on ChannelManager. CreateJobTool now accepts a project_dir parameter for mounting existing cloned repos into containers, and spawns the monitor automatically for async jobs. Also: Dockerfile bumped to Rust 1.88 (rig-core needs let chains), GITHUB_TOKEN forwarded into containers for gh CLI auth, and truncate() fixed for multi-byte char boundary panics. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Add ownership checks to JobEventsTool and JobPromptTool via ContextManager to prevent users from accessing other users' jobs (IDOR) - Combine Dockerfile gh CLI install into single apt-get layer - Handle truncate() edge case when max falls inside first multi-byte char - Log actual count of registered job management tools - Document fire-and-forget job monitor lifecycle - Add tests for ownership rejection and schema validation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Replace hardcoded GITHUB_TOKEN with on-demand credential delivery Containers now fetch credentials via authenticated GET /worker/{id}/credentials endpoint instead of receiving them baked into env vars at creation time. Secrets are decrypted from SecretsStore on demand, scoped per-job via CredentialGrant, and revoked automatically when the job completes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address sandbox audit findings (CONNECT tunnel, readonly_rootfs, type consolidation) - Implement real CONNECT tunnel with bidirectional TCP piping via hyper upgrade - Fix readonly_rootfs to apply for both ReadOnly and WorkspaceWrite policies - Consolidate duplicate CredentialMapping/CredentialLocation into secrets::types - Share reqwest::Client across proxy requests instead of per-request allocation - Store Docker connection and reuse across executions - Remove .unwrap() from proxy response builders with safe fallbacks - Add output truncation to direct (non-container) execution (64KB limit) - Delete dead src/tools/sandbox.rs (ToolSandbox never used) - Fix connect_docker error message to list all attempted socket paths - Update proxy credential injection to handle all CredentialLocation variants - Use glob-based host_patterns matching for credential lookup in proxy policy Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Dockerfile: install curl+ca-certificates before fetching GitHub CLI GPG key - JobEventsTool/JobPromptTool: reject missing context (prevents IDOR bypass) - parse_credentials: validate env var names against denylist and pattern - resolve_project_dir: require explicit paths to exist before validation - Credential serving: lower log level from info to debug Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address orchestrator audit findings (constant-time auth, error handling, tests) - auth: constant-time token comparison via subtle::ConstantTimeEq - auth: replace hand-rolled hex_encode with std::fmt::Write fold - api: report_status now updates ContainerHandle (was a no-op) - api: log complete_job errors instead of silently discarding - job_manager: log Docker cleanup errors in stop_job/complete_job - job_manager: extract validate_bind_mount_path with proper error on missing home_dir and mandatory base dir creation before canonicalize - job_manager: cache Docker connection across operations - error: remove dead OrchestratorError::AuthFailed and ContainerTimeout - Add 13 new tests (prompt queue, credentials, events, status, paths) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use floor_char_boundary in sandbox manager truncate to prevent multi-byte panics String::truncate() panics when the index falls mid-way through a multi-byte UTF-8 character. Use the same floor_char_boundary utility already used in worker/runtime.rs and tools/builtin/shell.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: default base_url to private.near.ai for Responses API mode Session tokens only authenticate against private.near.ai, not cloud-api.near.ai. The default base_url now matches the api_mode: - Responses (session token): https://private.near.ai - ChatCompletions (API key): https://cloud-api.near.ai This broke when the multi-provider merge introduced cloud-api.near.ai as the unconditional default. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use private.near.ai as default base URL for all API modes private.near.ai now supports both Responses and ChatCompletions endpoints, so there is no reason to route through cloud-api.near.ai. This also fixes session token auth which only works against private.near.ai. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: harden libSQL concurrency, fix Claude Code Docker auth and permissions Three fixes for the sandbox/Claude Code pipeline: 1. SQLite "database is locked": set WAL journal mode in migrations and PRAGMA busy_timeout=5000 on every connection across LibSqlBackend, LibSqlSecretsStore, and LibSqlWasmToolStore (~83 async call sites). 2. Claude Code container auth: extract OAuth token from macOS Keychain (or Linux ~/.claude/.credentials.json) at startup and inject via CLAUDE_CODE_OAUTH_TOKEN env var. Removes the broken bind-mount approach that failed on uid mismatch. 3. Claude Code tool permissions: wire CLAUDE_CODE_ALLOWED_TOOLS env var through to the worker binary (was hardcoded to empty vec), and expand defaults to include all standard tools (Read, Write, Edit, Glob, Grep, NotebookEdit, Bash, Task, WebFetch, WebSearch). Also adds --verbose flag to claude CLI (required with stream-json + -p), failover provider model switching, and nearai models endpoint fix. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: stream event parsing, job ID prefix resolution, session renewal in list_models Three fixes for the Docker/gateway pipeline: 1. Claude Code stream event parsing (claude_bridge.rs): Rewrite ClaudeStreamEvent to match actual NDJSON format where content blocks are nested under message.content[], not at the top level. Add handler for "user" events (tool_result blocks) and emit result text as a "message" event so reviews appear in gateway activity view. 2. Job ID prefix resolution (job.rs): Add resolve_job_id() that accepts short hex prefixes (like git short SHAs) in addition to full UUIDs. The LLM sees truncated IDs in job monitor messages like "[Job f2854dd8]" and can now use them directly with job_status/cancel/events/prompt tools. 3. Session renewal in list_models (nearai.rs): list_models() now retries with OAuth renewal on 401, matching send_request()'s existing behavior. Previously it returned SessionExpired immediately, causing the setup wizard to fall back to defaults instead of prompting re-authentication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: /model command now lists available models Previously /model with no args only showed the current model name. Now it fetches and displays all available models from the provider, marking the active one, so users can see what's available before switching with /model <name>. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR #57 review findings (set_var UB, tunnel timeout, restart creds) - Replace unsafe `std::env::set_var` in worker runtime and Claude bridge with `Command::envs()` injection via a new `extra_env` field on `JobContext`, avoiding undefined behavior in the multi-threaded tokio runtime. - Add 30-minute timeout to CONNECT tunnel `copy_bidirectional` in the sandbox proxy to prevent stuck connections from leaking spawned tasks. - Persist credential grants (as JSON in the description column) on `SandboxJobRecord` so `jobs_restart_handler` can restore them instead of passing `vec![]`, which caused restarted containers to lose access to their original secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address second round of PR #57 review comments - Normalize host_patterns to lowercase in proxy policy matching - Push LIMIT into SQL for list_job_events (Database trait + both backends) - Remove unused was_explicit binding in job tool - Return 500 instead of 200 in make_response fallback path - Update copy_auth_from_mount docstring for env-var default - Use entry.file_type() instead of is_dir() to avoid following symlinks Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address third round of PR #57 review comments - Restore glob patterns in default_claude_code_allowed_tools (Bash -> Bash(*)) - Add tracing::warn for credential grant serialize/deserialize failures - Wrap extra_env in Arc<HashMap> to avoid deep cloning per tool call - Document unsupported credential locations (AuthorizationBasic, UrlPath) - Document TOCTOU window in validate_bind_mount_path - Expand doc comments on JobEventsTool and JobPromptTool Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address fourth round of PR #57 review comments - Document CONNECT tunnel task lifecycle (timeout is the cleanup mechanism) - Remove secret names from error-level credential logs to prevent leaking - Expand DANGEROUS_ENV_VARS denylist with language runtime hijack vectors Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address fifth round of PR #57 review comments - Promote job monitor startup log to info level for observability - Require minimum 4-char prefix in resolve_job_id to limit enumeration - Cap credential grants at 20 per job to bound column storage - Clamp job events limit to 1..1000 to prevent memory abuse Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing closing brace for SkillsConfig impl block The merge resolution dropped the closing `}` for `impl SkillsConfig`, causing a compilation error in CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
DevBroco
|
436dda0f2f |
docs: add .env.example examples for Ollama and OpenAI-compatible (#110)
* docs: add .env.example examples for Ollama and OpenAI-compatible * docs: update .env.example with commented examples --------- Co-authored-by: BroccoliFin <mikhailsadovoy@MacBook-Air-MacMike.local> |
||
|
Elliot Braem
|
202665a55c |
Fixes build, adds missing sse event and correct command (#11)
* add missing type * prune * readme * minor * update to .ironclaw |
||
|
Illia Polosukhin
|
f3c85f57fc |
Add proactivity features: memory CLI, session pruning, self-repair notifications, slash commands, status diagnostics, context warnings
Closes the proactivity gap with six features: - Memory CLI (`ironclaw memory search/read/write/tree/status`) for direct workspace access without starting the full agent - Session pruning background task that evicts idle sessions (configurable TTL, default 7 days) - Self-repair notifications broadcast recovery results through channel manager instead of silent logging - `/heartbeat`, `/summarize`, `/suggest` slash commands for manual heartbeat trigger, thread summarization, and next-step suggestions - `ironclaw status` diagnostics command checking DB, session, secrets, embeddings, WASM tools, channels, heartbeat, and MCP servers - Context pressure warning that notifies users before auto-compaction fires Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
235f6aae18 |
Add heartbeat integration, planning phase, and auto-repair
- Add HeartbeatConfig for proactive periodic execution with channel notifications - Add use_planning option to Worker for ActionPlan generation before tool execution - Implement tool failure tracking in database (V3 migration) - Add auto-repair via Builder for broken WASM tools in self_repair.rs - Record tool failures in Worker for self-repair tracking - Update .env.example with new configuration options Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
2df4a4f5f0 | Login flow | ||
|
Illia Polosukhin
|
aea3f47f8b | Implementing WASM runtime | ||
|
Illia Polosukhin
|
3718cfa767 |
Simplify workspace to path-based storage, remove legacy code
- Consolidate all migrations into V1__initial.sql - Replace DocType enum with flexible path-based file storage - Add list_workspace_files SQL function for directory listing - Update memory tools for path-based API (memory_read, memory_write, memory_search, memory_list) - Remove unused OpenAI/Anthropic providers (NEAR AI only) - Simplify config to remove multi-provider support - Update CLAUDE.md documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
f29892b3fb |
Add NEAR AI chat-api as default LLM provider
Adds NearAiProvider that uses the NEAR AI unified API at api.near.ai/v1/responses with session token authentication. This provides access to multiple models (OpenAI, Anthropic, etc.) through a single endpoint with user auth and usage tracking. - Add src/llm/nearai.rs with complete provider implementation - Add NearAiConfig to config.rs with session_token, model, base_url - Add NearAi variant to LlmProvider enum (accepts nearai/near-ai/near_ai) - Change default provider from OpenAi to NearAi - Update .env.example with NEAR AI configuration - Update CLAUDE.md documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
8c38566378 | Initial implementation of the agent framework |