mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-02 23:56:24 +08:00
staging
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Pierre LE GUEN
|
0892f56af9 |
fix(wasm): remove stale 10M fuel limit from settings DB (#2851)
* fix(wasm): remove stale 10M fuel limit from settings DB Databases that persisted `wasm.default_fuel_limit = 10000000` before the code default was bumped to 500M (limits.rs, config/wasm.rs) still read the old value at startup because DB settings take priority over code defaults. This caused WASM tools like google_slides to fail with "Fuel exhausted: execution exceeded 10000000 fuel units" even though the code default is 500M. Add migration V25 (both PostgreSQL and libSQL) that deletes the stale setting row when its value is <= 10M, so the 500M code default takes effect. Users who intentionally set a custom limit above 10M are unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: trigger fresh run with skip-regression-check label [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(wasm): extract JSONB scalar before cast, narrow to exact match (#2851) Address review feedback: - PostgreSQL: use (value#>>'{}')::BIGINT to extract JSONB scalar as text before casting, preventing runtime errors on JSONB columns - libSQL: use json_extract(value, '$') for equivalent JSON extraction - Narrow predicate from <= to = 10000000 to avoid deleting intentionally lowered custom fuel limits Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: serrrfirat <f@nuff.tech> |
||
|
Zaki Manian
|
c835fe99d3 |
fix(ci): make tests resilient to sandboxed/offline environments (#2257)
* fix(ci): make tests resilient to sandboxed/offline environments Tests failed in CI because DNS resolution is unavailable in the sandbox, the process runs as root, and an HTTP proxy intercepts outbound traffic. Core fix: add `dns_probe_available()` to `config::helpers` — a cached, 2-second-timeout probe that detects whether external DNS works. When DNS is unavailable, `validate_base_url_with_policy()` skips the IP resolution/SSRF check while still enforcing syntactic URL validation. Additional fixes: - webhook_server: bind non-local IP (192.0.2.1) instead of privileged port 1, which succeeds as root - tunnel/custom: use closed localhost port instead of TEST-NET-1 IP that the proxy intercepts - mcp/auth: use IP literal instead of hostname requiring DNS - wasm/http_security: add .no_proxy() so pinned resolution test works behind egress proxy - wasm/runtime: remove `enabled = true` from cache TOML config, which was removed as a valid field in Wasmtime 43 https://claude.ai/code/session_01PUK8B5x6dKTG3bxWeSWdaH * fix(deny): add RUSTSEC-2026-0097 ignore, remove stale wasmtime advisories The rand 0.8.5 unsoundness advisory requires the `log` feature which is not enabled on our dep. Wasmtime 43 patches the 4 previously-ignored advisories so those ignores are removed. https://claude.ai/code/session_01LR9WsjkTuMNA6xkGS4TgMt * fix(security): use time-limited DNS probe cache and resolve target hostname Replace OnceLock-based permanent DNS probe cache with a Mutex-guarded cache that expires after 5 minutes, preventing transient DNS unavailability at startup from permanently disabling SSRF validation. Additionally, try resolving the actual target hostname before falling back to the generic probe. This avoids false negatives in firewalled environments where the generic probe target (previously dns.google) may be blocked but the actual target is reachable. Addresses review feedback from serrrfirat on PR #2257. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt and fix clippy collapsible_if after rebase on staging https://claude.ai/code/session_01T4ysh3bVb44UustMmJcQgQ --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
firat.sertgoz
|
532fc61d25 |
feat: admin management panel — web UI for users and usage monitoring (#1963)
* feat(web): add admin management panel * fix(web): address admin panel review findings * fix(web): address remaining admin review feedback * refactor(web): type admin api responses * fix(db): aggregate admin usage summary in sql * Add audit logging for admin privileged state-changes Add structured tracing (warn-level) to suspend, activate, delete, and update handlers so that privileged admin actions are recorded with the acting admin's user_id, the action performed, and the target user. Addresses security assessment item #1 from PR review. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix PairingStore::new() call in test after staging merge Use PairingStore::new_noop() since the test doesn't need a real DB. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(admin): address PR #1963 review feedback - Fix total_jobs semantics: query agent_jobs directly instead of counting via LEFT JOIN on llm_calls (which missed jobs without LLM calls). Fixed in both libSQL and PostgreSQL backends. - Fix showConfirmModal XSS: escape message parameter internally instead of relying on callers to sanitize. - Add explicit ::numeric cast to PG COALESCE(SUM(cost), 0) to prevent integer type inference. - Use info! instead of warn! for successful admin audit events (update, suspend, activate, delete) — warn implies anomaly. - Add missing index on llm_calls.created_at for both PG (V21 migration) and libSQL (incremental migration 21). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address remaining admin panel review follow-ups * fix: address review comments — query consolidation, CSP, docs, security notes - Collapse 4 redundant llm_calls subqueries into single subquery (libsql + pg) - Add WARNING to V21 migration about table lock risk with CONCURRENTLY note - Add performance doc comments on admin_usage_summary full-table scan - Add CSP and noindex meta tags to admin.html - Add JSDoc for showConfirmModal documenting auto-escaping - Add sessionStorage threat model security comment - Add serde(flatten) collision risk doc on AdminUserDetailResponse - Add TODO(#1968) for inline styles migration to CSS custom properties - Add PG parity test stub for admin_usage_summary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: renumber migration from V21/23 to V24 to avoid conflicts with staging Staging added V21 (backfill_conversation_source_channel), V22 (sandbox_restart_params), and V23 (list_workspace_files_escape_like). Renumber our llm_calls_created_at_index migration to V24 in both PG and libSQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback — CSP, logging, validation, dispatch-exempt, tests - Remove 'unsafe-inline' from script-src CSP; move CSP to HTTP response header - Change audit tracing::info! to tracing::debug! (TUI corruption) - Add dispatch-exempt annotation on usage_summary_handler - Add server-side input validation on users_create_handler (name length, email, role) - Rename detailRowHtml to detailRowRawHtml with XSS safety comment - Add real PG integration test for admin_usage_summary with non-zero data Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): skip test-only directories in no-panics check Files under `src/**/tests/*.rs` are Rust test sub-modules included behind `#[cfg(test)]` — they are never compiled in production builds. The no-panics checker was flagging `.unwrap()` and `assert!()` in helper functions at module level in these files as production code. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(admin): scope cost aggregates to 30d, drop external fonts, flatten detail response Addresses review feedback on #1963: - Scope all llm_calls aggregates to the 30d `since` window so the admin dashboard query is served by `idx_llm_calls_created_at` rather than a full table scan. Drops the unused all-time `total_cost` subquery from both libsql and postgres backends. - Self-contain the admin SPA — remove `fonts.googleapis.com` / `fonts.gstatic.com` link tags from admin.html and tighten the admin CSP to fully same-origin. Typography degrades to the system-font fallback already listed in `font-family`. - Fold `metadata` into `AdminUserInfo` (optional, skip-if-none) and remove the `#[serde(flatten)]` wrapper, eliminating the documented collision risk. - Add regression test asserting `since` actually bounds the LLM aggregates (future `since` should yield zero LLM counts without affecting non-windowed counts). --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com> |
||
|
firat.sertgoz
|
9399fcccc3 |
fix(auth) first-pass Gmail OAuth auth prompt in chat (#2038)
* Fix first-pass OAuth auth prompts in chat * Fix single auth prompt selection per turn * Persist auth prompts across approval pauses * Fix dispatcher clippy regressions * fix(auth): sanitize retry prompts for invalid tokens * fix(auth): address review feedback for PR #2038 - Validate auth_url/setup_url schemes: only https:// allowed, rejecting javascript:, file://, and other dangerous schemes (security) - Emit OAuth auth prompt alongside approval card so users see the connect button without waiting for approval to resolve - Refactor handle_auth_intercept to accept ParsedAuthData directly instead of synthesizing fake JSON (brittleness fix) - Add PendingAuthPrompt::new() constructor with non-empty extension_name validation - Add regression tests for URL sanitization and constructor validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix cargo fmt formatting in dispatcher tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> |
||
|
Illia Polosukhin
|
980d60ea45 |
[codex] Stabilize auth readiness and gate flows (#2050)
* Unify extension readiness and refresh dynamic tool leases * Fix v2 OAuth refresh and scope legacy credential fallback * Stabilize auth readiness and gate flows * Tighten auth token submission and OAuth fallback * Expose tool registry database handle * Handle expired runtime credentials in auth preflight * Fix E2E regressions on extension lifecycle branch * Normalize OAuth auth descriptors and flow launchers * Address review feedback on gate routing and latent actions * Apply formatter cleanup in tests * Address auth API review follow-ups * Generalize Google auth fallback and bundle alias metadata * Skip MCP OAuth when Authorization header is configured * Re-emit pending approval gates on follow-up * Open OAuth auth links in a new tab * Move shared OAuth runtime into auth module * Fix CI lint failures after staging merge * Unify OAuth resume and user greeting lifecycle * Ignore E2E virtualenv * Repair staging-merge build break in extension lifecycle paths The previous merge of staging into extension-lifecycle (commit |
||
|
Illia Polosukhin
|
6895cdad9e |
fix(db): repair V6 migration checksum and guard against re-modification (#1328) (#2101)
* fix(db): repair V6 migration checksum and guard against re-modification (#1328) PR #1151 modified the already-released migrations/V6__routines.sql in place, causing refinery's checksum validation to abort startup on every existing PostgreSQL deployment upgrading to v0.19.0. Revert V6 to its v0.18.0 content (V13 already applies the schema change incrementally and is idempotent for fresh installs that received the modified V6). Add a runtime checksum realignment step that rewrites refinery_schema_history rows whose stored checksum disagrees with the embedded SQL — this handles both populations of databases in the wild (pre-#1151 originals and post-#1151 fresh installs). Add migrations/checksums.lock pinning every migration's SipHasher13 checksum and a `released_migrations_are_immutable` cargo test that fails if any migration is modified or added without a matching lockfile entry. A second hard-coded sentinel test pins V6's literal v0.18.0 checksum so the guard cannot be defeated by editing both the migration and the lockfile in the same commit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): address review feedback on migration_fixup (#2101) - Drop hard-coded `public.` schema qualifier from the existence probe so PostgreSQL resolves `refinery_schema_history` via the active search_path, matching how refinery itself locates the table and how the subsequent UPDATE statement is written. Without this, deployments using a non-default schema would silently skip the realignment. - Use `IS DISTINCT FROM` instead of `<>` so a corrupted row with a NULL checksum is repaired rather than silently skipped. - Add `explanation` field to `KnownDivergence` and use it in the realignment warning so future entries are not coupled to the V6/#1328 wording. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): narrowly whitelist V6 known-bad checksum (#2101) Per @serrrfirat's review, the previous IS DISTINCT FROM-based realignment would silently rewrite *any* non-canonical V6 checksum, masking unrelated corruption or manual tampering instead of narrowly exempting the one historical released mismatch. Add a `known_bad_checksums: &'static [u64]` field to `KnownDivergence` listing the exact historical bad value(s), and rewrite only rows whose stored checksum is in that whitelist via `WHERE checksum = ANY($4)`. Anything else is left alone so refinery still aborts startup loudly. The single known-bad V6 value (`11230857244097235596`) is the SipHasher13 of `git show 878a67cd:migrations/V6__routines.sql` (the post-#1151 content) and is pinned by a new sentinel test `v6_known_bad_checksum_matches_post_1151_content` so the whitelist cannot drift or be silently widened. Also adds an ignored bootstrap helper `compute_checksum_for_external_file` for computing checksums of external SQL files when adding future entries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): assert in release + add postgres integration test (#2101) Address two follow-up review comments from @serrrfirat: 1. The defensive `debug_assert!` guarding against the canonical checksum being listed in `known_bad_checksums` is stripped in release builds, so the safety net was absent in production. `KNOWN_DIVERGENCES` has at most a handful of entries — switch to `assert!` so the guard runs in release too. Cost is one constant- time slice lookup per startup. 2. The `realign_diverged_checksums` SQL path was never exercised against a real database. Refactor into a thin pub wrapper plus an injectable `realign_diverged_checksums_with` inner helper, and add a `#[cfg(feature = "integration")]` test that: - skips gracefully if no DATABASE_URL is reachable - creates `refinery_schema_history` if missing - seeds a synthetic V99999 row with a deliberately-wrong checksum - calls the realignment with a custom divergence list (no collision with real V6 rows in shared CI databases) - asserts the row now holds the canonical checksum - re-runs the realignment and asserts a no-op Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): replace assert! with returned error to satisfy no-panics check (#2101) The previous commit changed `debug_assert!` → `assert!` to keep the canonical-in-known-bad-list guard active in release builds, but this trips the project's "No panics in production code" CI check (the regex matches `assert!` outside test attributes). Replace with an early `return Err(DatabaseError::Migration(...))` so the guard still runs in release builds — startup refuses to proceed with a misconfigured `KNOWN_DIVERGENCES` table — without using a panicking macro. This is also more idiomatic for a function that already returns `Result`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): address review follow-ups on migration_fixup (#2101) - parse_lockfile() now panics on duplicate migration keys instead of silently overwriting earlier entries — a stray duplicate could mask the actual pinned checksum and weaken the immutability guard (Copilot review). - Add `rejects_canonical_in_known_bad_checksums` integration test exercising the defensive Err path that refuses startup when a KnownDivergence has its canonical checksum listed in its own known_bad_checksums list (serrrfirat review). - Document why `tracing::warn!` is intentional in the realignment fix-up despite CLAUDE.md's warning about info!/warn! corrupting the TUI: this code runs at startup before any channel/REPL/TUI is initialized, so terminal-rendering interference is impossible. If the call site ever moves later in startup, downgrade to debug! or pre-buffer (illblackdragon review). - Cross-reference comments in src/history/store.rs and src/setup/wizard.rs pointing each other out so future changes to the migration fix-up call site stay in sync (illblackdragon review). - Sort migrations/checksums.lock by parsed migration version (V1, V2, ..., V10, V11, ...) instead of lex order (V10 before V2). The resulting file reads in numeric order which makes review diffs easier to scan (illblackdragon review). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): consolidate migration entry points + advisory lock + no leaks (#2101) Address three Medium-severity findings from @serrrfirat's review: 1. **Duplicate call sites** — extract `run_postgres_migrations_with_fixup(client)` in `crate::db::migration_fixup` that bundles fix-up + refinery into a single function. Both `Store::run_migrations` and `SetupWizard::run_migrations_postgres` now call it. Eliminates the class of bug where a future entry point could forget the fix-up. The previous comment-based coupling was an interim measure. 2. **Concurrent startup race** — the new helper acquires `pg_advisory_lock(1328)` (issue number, easy to grep in `pg_locks`) before realignment and releases it after refinery returns, on every exit path including errors. Serializes concurrent migration runs across replicas — also hardens the pre-existing refinery race that has always existed for multi-replica starts. Uses session-level advisory lock (not `pg_advisory_xact_lock`) because refinery's `run_async` opens its own internal transactions. 3. **`Box::leak` in tests** — refactor `KnownDivergence` to be lifetime-generic (`KnownDivergence<'a>`). Production `KNOWN_DIVERGENCES` is `&[KnownDivergence<'static>]` — no external API change. Both integration tests now use stack-allocated `&[u64]` slices, no `Box::leak`. Removes the leak-sanitizer false positive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Henry Park
|
3004583b2a |
feat(ownership): centralized ownership model with typed identities, DB-backed pairing, and OwnershipCache (#1898)
* feat(ownership): add OwnerId, Identity, UserRole, can_act_on types Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): private OwnerId field, ResourceScope serde derives, fix doc comment Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(tenant): replace SystemScope::db() escape hatch with typed workspace_for_user(), fix stale variable names - Add SystemScope::workspace_for_user() that wraps Workspace::new_with_db - Remove SystemScope::db() which exposed the raw Arc<dyn Database> - Update 3 callers (routine_engine.rs x2, heartbeat.rs x1) to use the new method - Fix stale comment: "admin context" -> "system context" in SystemScope - Rename `admin` bindings to `system` in agent_loop.rs for clarity Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(tenant): rename stale admin binding to system_store in heartbeat.rs * refactor(tenant): TenantScope/TenantCtx carry Identity, add with_identity() constructor and bridge new() - TenantScope: replace `user_id: String` field with `identity: Identity`; add `with_identity()` preferred constructor; keep `new(user_id, db)` as Member-role bridge; add `identity()` accessor; all internal method bodies use `identity.owner_id.as_str()` in place of `&self.user_id` - TenantCtx: replace `user_id: String` field with `identity: Identity`; update constructor signature; add `identity()` accessor; `user_id()` delegates to `identity.owner_id.as_str()`; cost/rate methods updated accordingly - agent_loop: split `tenant_ctx(&str)` into bridge + new `tenant_ctx_with_identity(Identity)` which holds the full body; bridge delegates to avoid duplication Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(db): add V16 tool scope, V17 channel_identities, V18 pairing_requests migrations - PostgreSQL: V16__tool_scope.sql adds scope column to wasm_tools/dynamic_tools - PostgreSQL: V17__channel_identities.sql creates channel identity resolution table - PostgreSQL: V18__pairing_requests.sql creates pairing request table replacing file-based store - libSQL SCHEMA: adds scope column to wasm_tools/dynamic_tools, channel_identities, pairing_requests tables - libSQL INCREMENTAL_MIGRATIONS: versions 17-19 for existing databases - IDEMPOTENT_ADD_COLUMN_MIGRATIONS: handles fresh-install/upgrade dual path for scope columns - Runner updated to check ALL idempotent columns per version before skipping SQL - Test: test_ownership_model_tables_created verifies all new tables/columns exist after migrations Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(db): use correct RFC3339 timestamp default in libSQL, document version sequence offset Replace datetime('now') with strftime('%Y-%m-%dT%H:%M:%fZ', 'now') in the channel_identities and pairing_requests table definitions (both in SCHEMA and INCREMENTAL_MIGRATIONS) to match the project-standard RFC 3339 timestamp format with millisecond precision. Also add a comment clarifying that libSQL incremental migration version numbers are independent from PostgreSQL VN migration numbers. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(ownership): bootstrap_ownership(), migrate_default_owner, V19 FK migration, replace hardcoded 'default' user IDs - Add V19__ownership_fk.sql (programmatic-only, not in auto-migration sweep) - Add `migrate_default_owner` to Database trait + both PgBackend and LibSqlBackend - Add `get_or_create_user` default method to UserStore trait - Add `bootstrap_ownership()` to app.rs, called in init_database() after connect_with_handles - Replace hardcoded "default" owner_id in cli/config.rs, cli/mcp.rs, cli/mod.rs, orchestrator/mod.rs - Add TODO(ownership) comments in llm/session.rs and tools/mcp/client.rs for deferred constructors Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): atomic get_or_create_user, transactional migrate_default_owner, V19 FK inline constant, fix remaining 'default' user IDs - Delete migrations/V19__ownership_fk.sql so refinery no longer auto-applies FK constraints before bootstrap_ownership runs; add OWNERSHIP_FK_SQL constant with TODO for future programmatic application - Remove racy SELECT+INSERT default in UserStore::get_or_create_user; both PostgreSQL (ON CONFLICT DO NOTHING) and libSQL (INSERT OR IGNORE) now use atomic upserts - Wrap migrate_default_owner in explicit transactions on both backends for atomicity - Make bootstrap_ownership failure fatal (propagate error instead of warn-and-continue) - Fix mcp auth/test --user: change from default_value="default" to Option<String> resolved from configured owner_id - Replace hardcoded "default" user IDs in channels/wasm/setup.rs with config.owner_id - Replace "default" sentinel in OrchestratorState test helper with "<unset>" to make the test-only nature explicit Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): remove default user_id from create_job(), change sentinel strings to <unset> - Gate ContextManager::create_job() behind #[cfg(test)]; production code must use create_job_for_user() with an explicit user_id to prevent DB rows with user_id = 'default' being silently created on the production write path. - Change the placeholder user_id in McpClient::new(), new_with_name(), and new_with_config() from "default" to "<unset>" so accidental secrets/settings lookups surface immediately rather than silently touching the wrong DB partition. - Same sentinel change for SessionManager::new() and new_async() in session.rs; these are overwritten by attach_store() at startup with the real owner_id. - Update tests that asserted the old "default" sentinel to expect "<unset>", and switch test_list_jobs_tool / test_job_status_tool to create_job_for_user("default") to keep ownership alignment with JobContext::default(). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(db): add ChannelPairingStore sub-trait with resolve_channel_identity, upsert/approve pairing, PostgreSQL + libSQL implementations Adds PairingRequestRecord, ChannelPairingStore trait (5 methods), and generate_pairing_code() to src/db/mod.rs; implements for PgBackend in postgres.rs and LibSqlBackend in libsql/pairing.rs; wires ChannelPairingStore into the Database supertrait bound; all 6 libSQL unit tests pass. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(db): atomic libSQL approve_pairing with BEGIN IMMEDIATE, add case-insensitive/expired/double-approve tests Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(ownership): add OwnershipCache for zero-DB-read identity resolution on warm path Converts src/ownership.rs to src/ownership/ module directory and adds src/ownership/cache.rs with a write-through in-process cache mapping (channel, external_id) -> Identity. Wired as Arc<OwnershipCache> on AppComponents for Task 8 pairing integration. All 7 cache unit tests pass. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(e2e): add ownership model E2E tests and extend pairing tests for DB-backed store Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): remove unused asyncio import, add fallback assertion in test_pairing_response_structure Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(tenant): unit tests for TenantScope::with_identity and AdminScope construction Adds 5 focused unit tests verifying TenantScope::with_identity stores the full Identity (owner_id + role), TenantScope::new creates a Member-role identity, and AdminScope::new returns Some for Admin and None for Member. Uses LibSqlBackend::new_memory() as the test DB stub. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): recover from RwLock poison instead of expect() in OwnershipCache Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(ownership): integration tests for bootstrap, tenant isolation, and ChannelPairingStore Adds tests/ownership_integration.rs covering migrate_default_owner idempotency, TenantScope per-user setting isolation (including Admin role bypass check), and the full ChannelPairingStore lifecycle (upsert, approve, remove, multi-channel isolation). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(test): remove duplicate pairing tests and flaky random-code assertion from integration suite Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(pairing): rewrite PairingStore to DB-backed async with OwnershipCache Replaces the file-based pairing store (~/.ironclaw/*-pairing.json, *-allowFrom.json) with a DB-backed async implementation that delegates to ChannelPairingStore and writes through to OwnershipCache on reads. - PairingStore::new(db, cache) uses the DB; new_noop() for test/no-DB - resolve_identity() cache-first lookup via OwnershipCache - approve(code, owner_id) removes channel arg (DB looks up by code) - All WASM host functions updated: pairing_upsert_request uses block_in_place, pairing-is-allowed renamed to pairing-resolve-identity returning Option<String>, pairing-read-allow-from deprecated (returns empty list) - Signal channel receives PairingStore via new(config, db) constructor - Web gateway pairing handlers read from state.store (DB) directly - extensions.rs derive_activation_status drops PairingStore dependency; derives status from extension.active and owner_binding flag instead - All test call sites updated to use new_noop() Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pairing): add missing pairing_store field to all GatewayState initializers, fix disk-full post-edit compile Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(channels): remove owner_id from IncomingMessage, user_id is the canonical resolved OwnerId `owner_id` on `IncomingMessage` was always a duplicate of `user_id` — both fields held the same value at every call site. Remove the field and `with_owner_id()` builder, update the four WASM-wrapper and HTTP test assertions to use `user_id`, and drop the redundant struct literal field in the routine_engine test helper. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(channels): remove stale owner_id param from make_message test helper Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(e2e): add browser/Playwright tests for ownership model — auth screen, chat UI, owner login Adds five Playwright-based browser tests to the ownership model E2E suite verifying the web UI experience: authenticated owner sees chat input, unauthenticated browser sees auth screen, owner can send a message and receive a response, settings tab renders without errors, and basic page structure is correct after login. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(settings): migrate channel credentials from plaintext settings to encrypted secrets store Moves nearai.session_token from the plaintext DB settings table to the AES-256-GCM encrypted secrets store (key: nearai_session_token). - SessionManager gains an `attach_secrets()` method that wires in the secrets store; `save_session` writes to it when available and `load_session_from_secrets` is called preferentially over settings - `migrate_session_credential()` runs idempotently on each startup in `init_secrets()`, reading the JSON session from settings, writing it to secrets, then deleting the plaintext copy - Wizard's `persist_session_to_db` now writes to secrets first, falling back to plaintext settings only when secrets store is unavailable - Plaintext settings path is preserved as fallback for installs without a secrets store (no master key configured) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(settings): settings fallback only when no secrets store, verify decryption before deleting plaintext Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): ROLLBACK in libSQL migrate_default_owner, shared OwnershipCache across channels, add dynamic_tools to migration, fix doc comment - libSQL migrate_default_owner: wrap UPDATE loop in async closure + match to emit ROLLBACK on any mid-transaction failure (mirroring approve_pairing pattern) - Both backends: add dynamic_tools to the migrate_default_owner table list so agent-built tools are migrated on first pairing - setup_wasm_channels: accept Arc<OwnershipCache> parameter instead of allocating a fresh cache, share the AppComponents cache - SignalChannel:🆕 accept Arc<OwnershipCache> parameter and pass it to PairingStore instead of allocating a new cache - PairingStore: fix module-level and struct-level doc comments to accurately describe lazy cache population after approve() Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(web): use can_act_on for authorization in job/routine handlers instead of raw string comparisons Replace 12 raw `user_id != user.user_id` / `user_id == user.user_id` string comparisons in jobs.rs and 4 in routines.rs with calls through the canonical `can_act_on` function from `crate::ownership`, which is the spec-mandated authorization mechanism. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: include remaining modified files in ownership model branch * fix: add pairing_store field to test GatewayState initializers, update PairingStore API calls in integration tests Add missing `pairing_store: None` to all GatewayState struct initializers in test files. Migrate old file-based PairingStore API calls (PairingStore::new(), PairingStore::with_base_dir()) to the new DB-backed API (PairingStore::new_noop()). Rewrite pairing_integration.rs to use LibSqlBackend with the new async DB-backed PairingStore API. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: cargo fmt * fix(pairing): truly no-op PairingStore noop mode, ensure owner user in CLI, fix signal safety comments - PairingStore::upsert_request now returns a dummy record in noop mode instead of erroring, and approve silently succeeds (matching the doc promise of "writes are silently discarded"). - PairingStore::approve now accepts a channel parameter, matching the updated DB trait signature and propagated to all call sites (CLI, web server, tests). - CLI run_pairing_command ensures the owner user row exists before approval to satisfy the FK constraint on channel_identities.owner_id. - Signal channel block_in_place safety comments corrected from "WASM channel callbacks" to "Signal channel message processing". Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pairing): thread channel through approve_pairing, add created flag, retry on code collision, remove redundant indexes Addresses PR review comments: - approve_pairing validates code belongs to the given channel - PairingRequestRecord.created replaces timing heuristic - upsert retries on UNIQUE violation (up to 3 attempts) - redundant indexes removed (UNIQUE creates implicit index) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): migrate api_tokens, serialize PG approvals, propagate resolved owner_id Addresses PR review P1/P2 regressions: - api_tokens included in migrate_default_owner (both backends) - PostgreSQL approve_pairing uses FOR UPDATE to prevent concurrent approvals - Signal resolve_sender_identity returns owner_id, set as IncomingMessage.user_id with raw phone number preserved as sender_id for reply routing - Feishu uses resolved owner_id from pairing_resolve_identity in emitted message - PairingStore noop mode logs warning when pairing admission is impossible [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pr-review): sanitize DB errors in pairing handlers, fix doc comments, add TODO for derive_activation_status - Pairing list/approve handlers no longer leak DB error details to clients - NotFound errors return user-friendly 'Invalid or expired pairing code' message - Module doc in pairing/store.rs corrected (remove -> evict, no insert method) - wit_compat.rs stub comment corrected to match actual Val shape - TODO added for derive_activation_status has_paired approximation * fix(pr-review): propagate libSQL query errors in approve_pairing, round-trip validate session credential migration, fix test doc comment - libSQL approve_pairing: .ok().flatten() replaced with .map_err() to propagate DB errors - migrate_session_credential: round-trip compares decrypted secret against plaintext before deleting - ownership_integration.rs: doc comment corrected to match actual test coverage * fix(pairing): store meta, wrap upserts in transactions, case-insensitive role/channel, log Signal DB errors, use auth role in handlers - Store meta JSONB/TEXT column in pairing_requests (PG migration V18, libSQL schema + incremental migration 19) - Wrap upsert_pairing_request in transactions (PG: client.transaction(), libSQL: BEGIN IMMEDIATE/COMMIT/ROLLBACK) - Case-insensitive role parsing: eq_ignore_ascii_case("admin") in both backends - Case-insensitive channel matching in approve_pairing: LOWER(channel) = LOWER($2) - Log DB errors in Signal resolve_sender_identity instead of silently discarding - Use auth role from UserIdentity in web handlers (jobs.rs, routines.rs) via identity_from_auth helper - Fix variable shadowing: rename `let channel` to `let req_channel` in libsql approve_pairing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(security): add auth to pairing list, cache eviction on deactivate, runtime assert in Signal, remove default fallback, warn on noop pairing codes Addresses zmanian's review: - #1: pairing_list_handler requires AuthenticatedUser - #2: OwnershipCache.evict_user() evicts all entries for a user on suspension - #3: debug_assert! for multi-thread runtime in Signal block_in_place - #9: Noop PairingStore warns when generating unredeemable codes - #10: cli/mcp.rs default fallback replaced with <unset> * fix(pairing): consistent LOWER() channel matching in resolve_channel_identity, fix wizard doc comment, fix E2E test assertion for ActionResponse convention * fix(pairing): apply LOWER() consistently across all ChannelPairingStore queries (upsert, list_pending, remove) All channel matching now uses LOWER() in both PostgreSQL and libSQL backends: - upsert_pairing_request: WHERE LOWER(channel) = LOWER($1) - list_pending_pairings: WHERE LOWER(channel) = LOWER($1) - remove_channel_identity: WHERE LOWER(channel) = LOWER($1) Previously only resolve_channel_identity and approve_pairing used LOWER(), causing inconsistent matching when channel names differed by case. * fix(pairing): unify code challenge flow and harden web pairing * test: harden pairing review follow-ups * fix: guard wasm pairing callbacks by runtime flavor * fix(pairing): normalize channel keys and serialize pg upserts * chore(web): clean up ownership review follow-ups * Preserve WASM pairing allowlist compatibility --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
d789a5d270 |
fix(db): swap V16/V17 to match production PG (document_versions before user_identities) (#1931)
Production PostgreSQL already has V15=conversation_source_channel and V16=document_versions applied. user_identities must be V17. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
3974163e00 |
fix(db): keep V15=conversation_source_channel to match production PG (#1928)
Production PostgreSQL already has V15__conversation_source_channel applied. Renumber to match: V15=conversation_source_channel, V16=user_identities, V17=document_versions. Update libSQL incremental migrations and idempotent list to match. The V15 repair still handles databases that mis-recorded V15 as "document_versions". Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
a68358086a |
fix(db): resolve V15 migration numbering conflict (#1923)
* fix(db): resolve V15 migration numbering conflict between user_identities and conversation_source_channel A merge conflict left two PostgreSQL migrations at V15. This renumbers them (V15=user_identities, V16=conversation_source_channel, V17=document_versions) to match the libSQL incremental ordering. Adds user_identities, document_versions, and source_channel to the libSQL base schema so fresh databases get all tables. Includes a one-time repair for existing databases where V15 was mis-recorded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix rustfmt formatting in repair_misnumbered_v15 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): address PR review — proper error handling and tighter repair condition - Replace .ok().flatten() with explicit error propagation via .map_err()? so DB errors during V15 repair are surfaced, not silently swallowed - Tighten repair condition from `!= "user_identities"` to `== "document_versions"` to only fix the specific known-bad case from the merge conflict Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
5c35b58ff1 |
feat(auth): direct OAuth/social login with Google, GitHub, Apple, and NEAR wallet (#1798)
* feat(auth): add direct OAuth/social login with Google and GitHub (#1771) Add optional OAuth authentication so users can sign in directly via Google or GitHub without requiring admin-created tokens or a reverse-proxy SSO setup. On successful OAuth, the system creates or links a user via the existing UserStore, issues an API token, and sets it as an HttpOnly cookie — reusing the existing DbAuthenticator for subsequent requests. Key changes: - user_identities table (PostgreSQL V15 + libSQL migration) for linking external provider accounts to internal users - IdentityStore trait with dual-backend implementations - OAuthProvider trait with Google (OIDC id_token) and GitHub (API-based) provider implementations - In-memory CSRF + PKCE state store with TTL and capacity bounds - Cookie-based session extraction in auth middleware - User resolution: existing identity → email linking → new account creation - All behind OAUTH_ENABLED=true flag; existing auth paths unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add email domain restrictions for OAuth and OIDC login (#1771) Add configurable email domain restrictions so admins can limit OAuth and OIDC login to specific organizations: - OAUTH_ALLOWED_DOMAINS: comma-separated list of allowed email domains, applied to all OAuth providers and OIDC (e.g., company.com,partner.org) - GOOGLE_ALLOWED_HD: restrict Google login to a specific Workspace domain via the `hd` authorization parameter + server-side validation - Domain check enforced in both the OAuth callback handler and the OIDC JWT middleware path (extracts email claim from validated JWT) - Add setup documentation in .env.example with step-by-step instructions for configuring Google and GitHub OAuth credentials Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR review — security hardening and cleanup (#1798) Fixes from Gemini and Copilot review: Security (critical/high): - Validate `aud` claim in Google id_token to prevent token substitution - Sanitize `redirect_after` to relative paths only (prevent open redirects) Correctness (medium): - Remove orphaned token: replace `create_user_with_identity_and_token` with `create_user_with_identity` — single token created in callback handler - Logout now revokes the API token (not just clears cookie) - Session tokens expire after 30 days (matching cookie lifetime) - Store decoded Google claims in raw_profile (not JWT string) - Propagate GitHub email fetch errors instead of swallowing - Fix `list_identities_for_user` to propagate row iteration errors Cleanup (low): - Extract `SESSION_COOKIE_NAME` constant - Add Secure flag to logout cookie clearing - Add single-quote escaping in error_page HTML - Fix garbled unicode in auth.rs comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add Apple Sign In provider (#1807) Add Apple Sign In as an OAuth provider alongside Google and GitHub. Apple-specific handling: - JWT client_secret generation (ES256-signed, team_id/key_id/private_key) - response_mode=form_post — Apple POSTs the callback instead of GET - POST callback route added alongside existing GET route - User name extracted from Apple's `user` form field (sent only on first authorization) and merged into the profile - id_token decoded with aud + issuer validation - email_verified handles both boolean and string "true"/"false" formats Configuration: - APPLE_CLIENT_ID, APPLE_TEAM_ID, APPLE_KEY_ID - APPLE_PRIVATE_KEY_PATH (file) or APPLE_PRIVATE_KEY_PEM (inline) - Setup instructions added to .env.example Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add NEAR wallet login via NEP-413 signature verification (#1807) Add NEAR wallet authentication as a fourth login method alongside Google, GitHub, and Apple. Unlike OAuth, NEAR uses a challenge-response flow with Ed25519 signature verification. Backend: - GET /auth/near/challenge — generate a random nonce (32 bytes hex) - POST /auth/near/verify — verify Ed25519 signature + NEAR RPC access key check, then issue session token via existing user resolution pipeline - NearNonceStore: in-memory nonce store with 5-min TTL and replay protection - Supports both base58 (NEAR standard) and hex key/signature encoding - New dependency: bs58 0.5 for base58 decoding Frontend: - Login screen discovers enabled providers via GET /auth/providers - Shows social login buttons (Google, GitHub, Apple, NEAR) dynamically - NEAR button loads @hot-labs/near-connect via ESM CDN import - Wallet connection → signMessage → POST to /auth/near/verify → session - OAuth cookie-based sessions auto-detected on page load (existing flow) Configuration: - NEAR_AUTH_ENABLED=true - NEAR_AUTH_NETWORK=mainnet|testnet (defaults to mainnet) - NEAR_AUTH_RPC_URL (auto-detected from network) Closes #1807 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address second round of PR review comments (#1798) - Fix early return in with_oauth() that skipped NEAR setup and OIDC domain restrictions when no OAuth redirect providers were configured - Add active-status check before linking identity by verified email (prevents linking to suspended/deactivated accounts) - Remove inline onclick handlers from login buttons (CSP compliance) - Export SESSION_COOKIE_NAME from auth.rs, reuse in handlers and middleware - NEAR challenge returns structured message ("Sign in to IronClaw\nNonce: {nonce}") that both client and server use for signature verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address human reviewer security findings (#1798) Six fixes from serrrfirat's review: 1. Domain check now requires email_verified=true before trusting the email for domain restriction — prevents unverified emails from bypassing access control (e.g., GitHub unverified fallback) 2. First-user bootstrap race documented — concurrent first logins may both see has_any_users()=false, but the second gets member role. Acceptable tradeoff; unique constraint prevents identity duplication. 3. NEP-413 payload mismatch fixed — server now builds the exact borsh-serialized NEP-413 payload (tag + message + nonce + recipient) that the wallet signs, instead of raw message bytes 4. Token extraction priority fixed — explicit ?token= query param now takes precedence over session cookie, preventing SSE/WS user mismatch when a browser has both a cookie and a query-param token 5. NEAR domain suffix check hardened — requires exact match or dotted subdomain boundary (alice.company.near passes, evilcompany.near does not) 6. NEAR network surfaced to frontend — /auth/providers response includes near_network field, frontend wallet connector uses it instead of hardcoded mainnet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): improve login page UX when OAuth providers are enabled When OAUTH_ENABLED=true with providers configured, the login screen now shows social login buttons (Google, GitHub, Apple, NEAR) as the primary action. The token input is collapsed behind a clickable "or use a token" divider for API users. Without OAuth: unchanged — token input is the only option. With OAuth: social buttons first, token input expandable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): hide token form until providers are discovered The token input was visible by default, causing it to flash before OAuth buttons appeared. Now: - Token form starts hidden (display:none) - /auth/providers fetch determines what to show - With providers: social buttons shown, token form behind "or use a token" - Without providers (or fetch fails): token form shown as fallback - After OAuth redirect: cookie-based autoAuth skips login screen entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): replace Connected indicator with user avatar + account menu Replace the "Connected" status indicator in the header with a user avatar button that shows connection status via an overlay dot. Clicking the avatar opens a dropdown with: - Display name, email, and role - Connection status (green/red dot + text) with gateway stats - Sign out button (calls POST /auth/logout, clears session, reloads) Avatar source: - OAuth logins: profile photo from Google/GitHub/Apple (avatar_url) - Token logins: initials from display_name (colored circle) Backend: profile_get_handler now queries user_identities for avatar_url from linked OAuth accounts. Frontend: social buttons are primary when OAuth is enabled, token input collapsed behind "or use a token" divider. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): remove Connected section from dropdown, fix avatar loading - Remove the connection status section from the user dropdown (was redundant with the avatar dot) - Add update_identity_profile() to IdentityStore — updates display_name and avatar_url on re-login so avatars load for accounts created before the avatar field was wired - Call update_identity_profile() in resolve_user() when an existing identity is found (re-login path) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): restore gateway stats in dropdown, add avatar debug logging - Bring back gateway stats section in user dropdown (without the word "Connected" — just the server stats like uptime, model, channels) - Add debug tracing to Google provider (logs picture claim from id_token) and profile handler (logs identity count + avatar_url) to diagnose why avatar isn't loading for Google OAuth accounts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix Google avatar not loading, restore gateway stats - Add referrerpolicy="no-referrer" to avatar img — Google's lh3.googleusercontent.com returns 403 when Referer header is sent from a different origin - Add crossorigin="anonymous" for CORS - Use display:block explicitly instead of empty string - Add onerror fallback to initials if image fails to load - Restore gateway stats section in dropdown (without "Connected" text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix squeezed avatar image in header Add min-width/min-height and flex-shrink:0 to both the avatar button and the img element so they don't get compressed by the tab-bar flex layout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): position avatar img and initials absolutely inside button Both children were competing for flex space, causing 0px width. Now both are position:absolute inside the 32px button, layered on top of each other. The JS toggles display:block/none to show the right one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): rewrite avatar loading — CSS src selector + onload/onerror Previous approach: inline style display:none toggled by JS. Failed because display:none prevented image fetch in some browsers, and position:absolute elements competed for z-index. New approach: - No inline style on img — CSS hides it via .user-avatar-img (display:none) - CSS .user-avatar-img[src] shows it (display:block, z-index:1) - JS sets src, onload hides initials, onerror removes src as fallback - Initials always rendered first as the base layer - Removed crossorigin="anonymous" which can cause CORS failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): prefetch avatar with new Image() before showing Use a throwaway Image() to prefetch the avatar URL. Only when onload fires, set src on the real <img> and unhide it. This avoids all CSS display/src selector issues — the real img element only gets a src after the image is confirmed loadable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): set avatar src directly, set referrerPolicy in JS The new Image() prefetch was failing because the programmatic Image object didn't have referrerPolicy set. Simplify: set referrerPolicy and src directly on the real <img> element, unhide it immediately, and use onload to hide initials. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): explicit display:block on avatar img, removeAttribute hidden - Add display:block to .user-avatar-img CSS (img elements default to inline which can cause rendering issues with position:absolute) - Bump z-index to 2 to ensure img renders above initials - Use removeAttribute('hidden') instead of hidden=false - Use style.display='none' on initials instead of hidden attribute Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): swap img/initials DOM order so img paints on top Put <img> after <span> in DOM order. With both position:absolute, later elements paint on top. Combined with z-index:2 on img vs z-index:0 on initials, the avatar photo should now reliably cover the initials circle when loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): add OAuth avatar domains to Content-Security-Policy The CSP had img-src 'self' data: which blocked Google and GitHub avatar images from loading. Added: - img-src: *.googleusercontent.com, avatars.githubusercontent.com - script-src: esm.sh (for near-connect dynamic import) - connect-src: esm.sh, *.near.org (for NEAR RPC) - form-action: Google, GitHub, Apple OAuth endpoints This was the root cause of avatar images not rendering despite correct src URLs — the browser silently blocked them via CSP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): show welcome card for new OAuth users with no threads New OAuth users have no assistant thread yet, so switchToAssistant() was never called, and loadHistory() never ran to show the welcome card. Now explicitly show the welcome card when there's no current thread and no assistant thread (brand-new user). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting for new OAuth users on workspace creation New OAuth users saw an empty chat because the bootstrap greeting was only persisted when the agent loop processed the first message (take_bootstrap_pending check in agent_loop.rs:1314). But OAuth users land on the web UI without sending any message. Fix: WorkspacePool now checks take_bootstrap_pending() after seed_if_empty() and persists the GREETING.md content into the assistant conversation immediately. This runs in a background task so it doesn't block the workspace creation. The greeting is in the DB before the frontend loads threads/history. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting synchronously, not in background Move greeting persistence from tokio::spawn to the same await chain as seed_if_empty() so the greeting is guaranteed to be in the DB before the workspace is returned to the caller. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): seed bootstrap greeting in chat_threads_handler for new users The WorkspacePool approach didn't work because the workspace pool is only accessed by memory handlers — chat_threads_handler runs first when a new user loads the page. Move the greeting seed to chat_threads_handler: after get_or_create_assistant_conversation, check if the conversation has zero messages and inject the GREETING.md content. This guarantees the greeting is in the DB before the thread list is returned to the frontend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(agent): consolidate bootstrap greeting into chat_threads_handler Remove three redundant greeting insertion paths from agent_loop.rs: 1. Single-user startup (Agent::run bootstrap_thread_id) 2. Single-user SSE broadcast after startup 3. Multi-tenant message handler (take_bootstrap_pending on first msg) Also remove the dead WorkspacePool greeting code in server.rs. The single source of truth is now chat_threads_handler: when the assistant conversation is created with zero messages, GREETING.md is inserted. This works for all auth modes (token, OAuth, OIDC) and both single-user and multi-tenant deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: downgrade workspace seed log from info to debug Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian's blocking review items 1. Add Google iss validation — set_issuer(&["https://accounts.google.com"]) to match Apple's issuer check. Prevents cross-provider id_token acceptance. 2. Convert oauth_rate_limiter from global RateLimiter to per-IP PerUserRateLimiter(20, 60). Extracts client IP from X-Forwarded-For header. One user retrying no longer locks out all OAuth for everyone. Also: - sanitize_redirect now rejects backslash (/\) open redirect vector - All auth handlers extract headers for per-IP rate limiting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): stop inserting greeting on every page load The previous check used list_conversations_with_preview with limit=1 and defaulted to is_empty=true when the assistant thread wasn't in the result (unwrap_or(true)). This caused the greeting to be inserted on every chat_threads_handler call. Fix: use list_conversation_messages_paginated(assistant_id, None, 1) to directly check if the assistant conversation has any messages. Only insert the greeting when the message list is truly empty. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for bootstrap greeting and cookie auth Five tests covering the greeting behavior and OAuth session auth: 1. test_greeting_inserted_once_for_new_user — verifies greeting appears exactly once and is not duplicated on second page load 2. test_greeting_not_duplicated_on_rapid_calls — 5 concurrent /api/chat/threads requests produce exactly 1 greeting 3. test_each_user_gets_own_greeting — multi-user: Alice and Bob each get their own assistant thread with separate greetings 4. test_cookie_auth_works_for_threads — cookie-based auth (ironclaw_session=token) works for protected endpoints 5. test_existing_conversation_no_greeting — pre-populated conversations are not overwritten with the greeting These tests would have caught the unwrap_or(true) bug that caused greeting re-insertion on every page load. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix near-connect CDN URL (package is v0.x, not v1) The @hot-labs/near-connect package is version 0.11.1 — there is no v1 release. The @1 version specifier returned 404 from esm.sh. Changed to @0.11 which resolves correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): support base64 encoding for NEAR wallet signatures NEAR wallets (e.g. HOT) may return signatures and public keys in base64 format, not just base58/hex. Added base64 standard and URL-safe decoding to decode_multiformat(), which is used by both decode_near_public_key and decode_near_signature. Also: - Added debug logging to near_verify_handler to trace credential formats - Updated CSP img-src to allow wallet logos (raw.githubusercontent.com, jsdelivr.net, near.org, pages.near.org) - Added CSP frame-src for near-connect wallet sandboxes (iframes) - Added blob: to img-src for inline wallet icons Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): widen CSP connect-src and img-src for NEAR wallet resources near-connect fetches wallet manifests from raw.githubusercontent.com and cdn.jsdelivr.net, and wallet logos from app.hot-labs.org. These were blocked by the restrictive connect-src and img-src policies. - connect-src: added raw.githubusercontent.com, *.jsdelivr.net, *.cloudflare.com - img-src: added *.hot-labs.org Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): try both NEP-413 and raw message for NEAR signature verification Different NEAR wallets may sign the full NEP-413 borsh payload or just the raw message string. Try NEP-413 first, fall back to raw message bytes. This makes verification work with HOT wallet and other wallets that may not implement the full NEP-413 serialization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): fix NEP-413 field order and try both payload layouts The NEP-413 borsh payload field order was wrong. Our implementation had tag → message → nonce → recipient → callback_url, but the NEAR docs (docs.near.org/web3-apps/backend-login) show tag → message → recipient → nonce. Now tries both field orderings (v1 and v2), plus SHA256 variants, plus raw message bytes — covering all known wallet implementations. Tests updated: verify_near_signature tested with raw message, NEP-413 v2, and wrong-key rejection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP for wallet ecosystem — allow all HTTPS for connect/img/frame The NEAR wallet ecosystem spans dozens of domains (intear.tech, hot-labs.org, meteorwallet.app, herewallet.app, etc.) that change as new wallets are added. Whitelisting each one is a losing game. Relax CSP to allow all HTTPS for: - connect-src: wallet manifests, wallet JS modules, RPC endpoints - img-src: wallet logos from various CDNs - frame-src: wallet sandbox iframes script-src remains restricted to specific CDNs (jsdelivr, cloudflare, esm.sh) — this is the security-critical directive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow unsafe-inline scripts for NEAR wallet sandbox iframes NEAR wallet sandboxes (MeteorWallet, etc.) use inline scripts inside their iframe sandboxes. The CSP script-src blocked these, preventing wallets from loading. Added 'unsafe-inline' to script-src. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP style-src and font-src for wallet iframes NEAR wallet sandboxes load fonts from rsms.me, cdnfonts.com, and embed data: font URIs. Relaxed style-src and font-src to allow all HTTPS sources and data: URIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 4 — 14 fixes serrrfirat (high/medium): 1. decode_multiformat ambiguity: replaced with context-aware decoders. NEAR pubkeys enforce ed25519: prefix + base58 (unambiguous). Signatures try base64 first (most wallets), then base58. 2. UTF-8 slicing panic: use safe_truncate() with char_indices() 3. NEAR pubkey → RPC format: re-encode decoded bytes as ed25519:{base58} for the canonical format expected by view_access_key 4. GitHub redirect_uri: now included in token exchange form body Copilot (medium/low): 5. near_network stored explicitly in GatewayState (not inferred from URL) 6. NEAR verify sets HttpOnly session cookie (consistent with OAuth flow) 7. Reuse reqwest::Client via LazyLock (no per-request allocation) 8. OAuth module doc updated to list all 4 providers 9. Rate limiter comment fixed (was stale "10 requests") 10. Profile identity error logged at warn (not silently swallowed) 11. Config doc updated for Apple/NEAR requirements 12. Test file doc comment updated to match actual coverage 13. RPC status check before JSON parse 14. GitHub token exchange includes redirect_uri Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR feedback on cookie auth and NEAR sessions * fix(auth): address code review — tighten CSP, fix races, improve security - Tighten CSP: narrow connect-src/img-src/frame-src to specific origins instead of blanket `https:` (prevents data exfiltration) - Fix greeting race: add atomic add_conversation_message_if_empty using INSERT...WHERE NOT EXISTS (both PostgreSQL and libSQL) - Case-insensitive email matching: use LOWER() in identity lookups and normalize emails to lowercase on storage - Add X-Real-IP fallback for rate limit key when X-Forwarded-For missing - Add OAuthError::SignatureVerification variant (was misusing ProfileFetch) - Fix dead branch in with_oauth (has_near check inside !has_near block) - Fix _user → user in logout_handler (variable is actually used) - Add partial index WHERE email IS NOT NULL to libSQL (match PostgreSQL) - Downgrade noisy tracing::debug to trace in profile handler - Add i18n for "Sign out" button (en + zh-CN) - Remove duplicate test_session_cookie_auth_passes test - Update E2E bootstrap tests to match new DB-based greeting architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove garbled unicode character in section comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 5 — admin race, 303 redirect, OIDC email_verified - Atomic first-user admin: create_user_with_identity now promotes to admin inside the DB transaction with UPDATE...WHERE COUNT(*)=1, eliminating the TOCTOU race where two concurrent first logins both get admin role (both PostgreSQL and libSQL) - Apple callback redirect: use 303 See Other instead of 307 Temporary so POST form_post callbacks are converted to GET on redirect - OIDC domain restriction: now requires email_verified=true before checking domain allowlist, preventing unverified emails from bypassing the restriction - Postgres add_conversation_message_if_empty: call touch_conversation after insert to match libSQL behavior and keep last_activity current - Greeting seeding: log errors instead of silently discarding with let _ Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): advisory lock for admin election, skip empty query tokens - Postgres first-user admin: add pg_advisory_xact_lock before the COUNT(*)=1 promotion to serialize concurrent transactions under READ COMMITTED isolation (prevents two admins on concurrent signup) - Empty ?token= query parameter no longer overrides a valid session cookie — trimmed empty tokens return None from query_token() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian security review — redirect, sweep, NEAR sigs Blocking issues from security review: 1. redirect_after hardened: strict URL-safe char allowlist in sanitize_redirect (blocks /%09/ and encoded separators), plus re-validation before use in handle_callback (defense in depth) 2. Sweep tasks shutdown-aware: OAuth state store and NEAR nonce store sweep loops now select on a watch channel and exit when the sender is dropped (stored in GatewayState.oauth_sweep_shutdown) 3. NEAR signature verification tightened: removed raw-message-bytes and SHA256-of-raw fallbacks that lacked nonce binding (replay risk). Only NEP-413 structured payloads (v1 + v2) are accepted. Added test_verify_near_signature_rejects_raw_message regression test. Non-blocking: 4. NEAR RPC client timeout: set 10s timeout on the static reqwest client to prevent indefinite hangs on slow RPC endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): percent-decode redirect_after before validation is_safe_redirect now percent-decodes the URL and re-validates against the // and /\ guards, preventing smuggling via %2f%2f or %5c. Added 5 regression tests covering normal paths, protocol-relative, absolute URLs, encoded smuggling, and sanitize_redirect filtering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): case-insensitive get_user_by_email, require OIDC iss/aud claims - get_user_by_email now uses LOWER() in both PostgreSQL and libSQL, matching the case-insensitive identity lookup. This ensures admin-created users with different email casing are correctly linked during OAuth account resolution. - OIDC validation now adds iss/aud to required_spec_claims when configured, rejecting JWTs that omit these claims entirely (not just mismatches). Updated two tests from assert-passes to assert-rejects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): normalize UserRecord.email to lowercase, add aria-label to avatar - UserRecord.email now lowercased on create (matching identity records), preventing case-mismatched duplicates against the UNIQUE constraint - Avatar button: added aria-label with i18n (en + zh-CN) for screen readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
Illia Polosukhin
|
5435b38eca |
feat(workspace): metadata-driven indexing/hygiene, document versioning, and patch (#1723)
* feat(workspace): metadata-driven indexing/hygiene, document versioning, and patch support Foundation for the extensible frontend system. Workspace documents now support metadata flags (skip_indexing, skip_versioning, hygiene config) via folder-level .config documents and per-file overrides, replacing hardcoded hygiene targets and indexing behavior. Key changes: - DocumentMetadata type with resolution chain (doc → folder .config → defaults) - Document versioning: auto-saves previous content on write/append/patch - Workspace patch: search-and-replace editing via memory_write tool - Hygiene rewrite: discovers cleanup targets from .config metadata instead of hardcoded daily/ and conversations/ directories - memory_read gains version/list_versions params - memory_write gains metadata/old_string/new_string/replace_all params - V14 migration adds memory_document_versions table (both PG + libSQL) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback — transaction safety, patch mode, formatting - Wrap libSQL save_version in a transaction to prevent race condition where concurrent writers could allocate the same version number - Make content optional in memory_write when in patch mode (old_string present) — LLM no longer forced to provide unused content param - Improve metadata update error handling with explicit match arms - Run cargo fmt across all files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings — write-path performance, version pruning, descriptions 1. Resolve metadata once per write: write(), append(), and patch() now call resolve_metadata() once and pass the result to both maybe_save_version() and reindex_document_with_metadata(), cutting redundant DB queries from 3-5 per write down to 1 resolution. 2. Optimize version hash check: replaced get_latest_version_number() + get_version() (2 queries) with list_versions(id, 1) (1 query) for the duplicate-hash check in maybe_save_version(). 3. Wire up version_keep_count: hygiene passes now prune old versions for documents in cleaned directories, enforcing the configured version_keep_count (default: 50). Removes the TODO comment. 4. Fix misleading tool description: patch mode works with any target including 'memory', not just custom paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: wire remaining unwired components — changed_by, layer versioning 1. changed_by now populated: all write paths pass self.user_id as the changed_by field in version records instead of None, so version history shows who made each change. 2. Layer write/append versioned: write_to_layer() and append_to_layer() now auto-version and use metadata-optimized reindexing, matching the standard write()/append() paths. 3. append_memory versioned: MEMORY.md appends now auto-version with metadata-driven skip and shared metadata resolution. 4. Remove unused reindex_document wrapper: all callers now use reindex_document_with_metadata directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive coverage for versioning, metadata, patch, and hygiene 26 new tests covering critical and high-priority gaps: document.rs (7 unit tests): - is_config_path edge cases (foo.config, empty string, .config/bar) - content_sha256 with empty string (known SHA-256 constant) - content_sha256 with unicode (multi-byte UTF-8) - DocumentMetadata merge: null overlay, nested hygiene replaced wholesale, both empty, non-object base memory.rs (2 schema tests): - memory_write schema includes patch/metadata params, content not required - memory_read schema includes version/list_versions params hygiene.rs (5 integration tests): - No .config docs → no cleanup happens - .config with hygiene disabled → directory skipped - Multiple dirs with different retention (fast=0, slow=9999) - Documents newer than retention not deleted - Version pruning during hygiene (keep_count=2, verify pruned) workspace/mod.rs (14 integration tests): - write creates version with correct hash and changed_by - Identical writes deduplicated (hash check) - Append versions pre-append content - Patch: single replacement, replace_all, not-found error, creates version - Patch with unicode characters - Patch with empty replacement string - resolve_metadata: no config (defaults), inherits from folder .config, document overrides .config, nearest ancestor wins - skip_versioning via .config prevents version creation [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address zmanian review — PG transaction safety, identity protection, perf Must-fix: 1. PostgreSQL save_version now uses a transaction with SELECT FOR UPDATE to prevent concurrent writers from allocating the same version number, matching the libSQL implementation. 2. Restore identity document protection in hygiene cleanup_directory(). MEMORY.md, SOUL.md, IDENTITY.md, etc. are now protected from deletion regardless of which directory they appear in, via is_identity_document() case-insensitive check. This restores the safety net that was removed when migrating from hardcoded to metadata-driven hygiene. Should-fix: 3. resolve_metadata() now uses find_config_documents (single query) + in-memory nearest-ancestor lookup, instead of O(depth) serial DB queries walking up the directory tree. 4. memory_write validates that at least one mode is provided (content for write/append, or old_string+new_string for patch) with a clear error message upfront, instead of relying on downstream empty checks. 5. Fixed misleading GIN index comment in V15 migration. 9. Added "Fail-open: versioning failures must not block writes" comments to all `let _ = self.maybe_save_version(...)` call sites. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt * fix: address Copilot review — DoS prevention, no-op skip, duplicate hygiene Security: - Reject empty old_string in both workspace.patch() and memory_write tool to prevent pathological .matches("") behavior (DoS vector) Correctness: - Remove duplicate hygiene spawn in multi-user heartbeat — was running both via untracked tokio::spawn AND inside the JoinSet, causing double work and immediate skip via global AtomicBool guard - Disallow layer param in patch mode — patch always targets the default workspace scope; combining with layer could silently patch the wrong document - Restore trim-based whitespace rejection for non-patch content validation (was broken when refactoring required fields) Performance: - Short-circuit write() when content is identical to current content, skipping versioning, update, and reindex entirely - Normalize path once at start of resolve_metadata instead of only for config lookup (prevents missed document metadata on unnormalized paths) Cleanup: - Remove duplicate tests/workspace_versioning_integration.rs (same tests already exist in workspace/mod.rs versioning_tests module) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: eliminate flaky hygiene tests caused by global AtomicBool contention All hygiene tests that used run_if_due() were flaky when running concurrently because they competed for the global RUNNING AtomicBool guard. Rewrote them to test the underlying components directly: - metadata_driven_cleanup_discovers_directories: now uses find_config_documents() + cleanup_directory() directly - multiple_directories_with_different_retention: now uses cleanup_directory() per directory directly - cleanup_respects_cadence: rewritten as a sync unit test that validates state file + timestamp logic without touching the global guard Verified stable across 3 consecutive runs (3793 tests, 0 failures). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata ordering, PG locking, hygiene safety 1. Metadata applied BEFORE write/patch (#10-11,15): metadata param is now set via get_or_create + update_metadata before the write/patch call, so skip_indexing/skip_versioning take effect for the same operation instead of only subsequent ones. 2. Layer write doc ID (#13-14): metadata no longer re-reads after write since it's applied upfront. Removes the stale-scope risk. 3. Version param overflow (#16): validates version is 1..i32::MAX before casting, returns InvalidParameters on out-of-range. 4. Hygiene protection list (#18): added HYGIENE_PROTECTED_PATHS that includes MEMORY.md, HEARTBEAT.md, README.md (missing from IDENTITY_PATHS). cleanup_directory now uses is_protected_document() which checks both lists with case-insensitive matching. 5. PG FOR UPDATE on empty table (#22-24): now locks the parent memory_documents row (SELECT 1 FROM memory_documents WHERE id=$1 FOR UPDATE) before computing MAX(version), which works even when no version rows exist yet. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata merge, retention guard, migration ordering 1. **Metadata merge in memory_write tool**: incoming metadata is now merged with existing document metadata via `DocumentMetadata::merge()` instead of full replacement, so setting `{hygiene: {enabled: true}}` no longer silently drops a previously-set `skip_versioning: true`. 2. **Minimum retention_days**: `HygieneMetadata.retention_days` is now clamped to a minimum of 1 day during deserialization, preventing an LLM from writing `retention_days: 0` and causing mass-deletion on the next hygiene pass. 3. **Migration version ordering**: renumbered document_versions migration to come after staging's already-deployed migrations (PG: V15→V16, libSQL: 15→17). Documented the convention that new migrations must always be numbered after the highest version on staging/main. 4. **Duplicate doc comment**: removed duplicated line on `reindex_document_with_metadata`. 5. **HygieneSettings**: added `version_keep_count` field to persist the setting through the DB-first config resolution chain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip metadata pre-apply when layer is specified, clean up stale comment 1. When a layer is specified, skip the metadata pre-apply via get_or_create — it operates on the primary scope and would create a ghost document there while the actual content write targets the layer's scope. 2. Removed stale "See review comments #10-11,15" reference; the surrounding comment already explains the rationale. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use BEGIN IMMEDIATE for libSQL save_version to serialize writers The default DEFERRED transaction only acquires a write lock at the first write statement (INSERT), not at the SELECT. Two concurrent writers could both read the same MAX(version) before either inserts, causing a UNIQUE violation. BEGIN IMMEDIATE acquires the write lock upfront, matching the existing pattern in conversations.rs. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: document trust boundary on metadata/versioning WorkspaceStore methods These methods accept bare document UUIDs without user_id checks at the DB layer. The Workspace struct (the only caller) always obtains UUIDs through user-scoped queries first. Document this trust boundary explicitly on the trait so future implementors/callers know not to pass unverified UUIDs from external input. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address new Copilot review comments — ghost doc, param validation, overflow 1. Skip metadata pre-apply in patch mode to avoid creating a ghost empty document via get_or_create when the document doesn't exist, which would change a "not found" error into "old_string not found". 2. Validate list_versions and version as mutually exclusive in memory_read to avoid ambiguous behavior (list_versions silently won). 3. Clamp version_keep_count to i32::MAX before casting to prevent overflow on extreme config values. 4. Mark daily_retention_days and conversation_retention_days as deprecated in HygieneSettings — retention is now per-folder via .config metadata. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: apply metadata in patch mode via read(), reorder libSQL migrations 1. Metadata is no longer silently ignored in patch mode — uses workspace.read() (which won't create ghost docs) instead of skipping entirely, so skip_versioning/skip_indexing flags take effect for patches on existing documents. 2. Reorder INCREMENTAL_MIGRATIONS to strictly ascending version order (16 before 17) to match iteration order in run_incremental(). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove duplicate is_patch_mode, add TODO comments for known limitations - Remove duplicate `is_patch_mode` binding in memory_write (was computed at line 280 and again at line 368). - Document multi-scope hygiene edge case: workspace.list() includes secondary scopes but workspace.delete() is primary-only, causing silent no-ops for cross-scope entries. - Document O(n) reads in version pruning as acceptable for typical directory sizes. - Add TODO on WorkspaceError::SearchFailed catch-all for future cleanup into more specific variants. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reindex on no-op writes for metadata changes, use read_primary in patch 1. write() no longer fully short-circuits when content is unchanged — it still resolves metadata and reindexes so that metadata-driven flags (e.g. skip_indexing toggled via memory_write's metadata param) take effect immediately even without a content change. 2. Patch-mode metadata pre-apply now uses workspace.read_primary() instead of workspace.read() to ensure we target the same scope that patch() operates on, preventing cross-scope metadata mutation in multi-scope mode. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Zaki Manian
|
27fa292b33 |
fix(security): block cross-channel approval thread hijacking (#1590)
* fix(security): block cross-channel approval thread hijacking (#1485) Add source_channel to Thread and verify channel authorization before allowing approval messages to target threads by UUID. The web gateway channel is allowed as a trusted approval UI. Threads without source_channel (deserialized from older DB records) are permitted for backward compatibility. Closes #1485 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: run cargo fmt https://claude.ai/code/session_01Mdiz3XwyZcjqMkqicaynGs * fix(security): address review feedback on source_channel - hydrate_thread_from_db now passes message.channel as source_channel instead of None, ensuring DB-hydrated threads get proper channel auth - Replace is_none_or (unstable) with map_or(true, ...) for MSRV compat - Add "gateway" to trusted approval channels alongside "web" - Document why bootstrap thread uses None for source_channel Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(clippy): use is_none_or instead of map_or for Option check is_none_or is stable since Rust 1.82 and preferred by clippy over map_or(true, ...) pattern. https://claude.ai/code/session_012nbbEyFXjDwdZZrHg7gFNK * fix(security): persist source_channel to DB, harden cross-channel authorization Address PR #1590 review feedback: 1. Persist source_channel to DB: Add source_channel column to conversations table in both PostgreSQL (V14 migration) and libSQL (incremental migration + base schema). Add get_conversation_source_channel trait method to ConversationStore with both backend implementations. 2. Fix hydrate_thread_from_db: Read source_channel from DB instead of stamping the requesting message's channel, preventing channel confusion after server restart. 3. Reject reserved WASM channel names: Validate that WASM channels cannot register as "web", "gateway", "cli", or "repl" to prevent authorization bypass via name spoofing. 4. Require pending_approval exists: Authorization check now verifies thread.pending_approval.is_some() before allowing approval-shaped messages to target a thread. 5. Fail-closed for None source_channel: Use "__bootstrap__" sentinel for bootstrap threads (authorized from any channel). None now means "deny by default" instead of "allow by default". 6. Extract and test authorization predicate: is_approval_authorized() helper with 6 unit tests covering same-channel, cross-channel blocked, web/gateway always allowed, None denied, and bootstrap sentinel. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve merge conflicts from staging rebase - Fix Thread::with_id calls to include source_channel parameter - Fix ensure_conversation calls to include source_channel parameter - Bump libsql source_channel migration to V15 (V14 taken by users) - Remove stale conflict markers - Fix clippy warning in users.rs https://claude.ai/code/session_01Esh8QQzHACYyfsVwCb479F * style: fix cargo fmt formatting https://claude.ai/code/session_01Ci7CAdGaHhssYdio7wxVvd * fix(security): address review feedback on cross-channel approval checks 1. thread_ops.rs: Remove .or(Some(&*message.channel)) fallback in maybe_hydrate_thread() so that when source_channel is NULL in the DB, it stays None rather than being stamped with the requesting channel. This preserves the fail-closed behavior of is_approval_authorized(). 2. libsql_migrations.rs: Remove source_channel from base SCHEMA to eliminate duplicate column definition. The column is now added solely by V14 migration, preventing fresh databases from failing on startup. 3. wasm/setup.rs: Expand RESERVED_CHANNEL_NAMES to cover all built-in channels (http, signal, slack-relay, secret_save) and add a dynamic collision check against already-registered channel names passed from the startup sequence. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(security): harden cross-channel approval authorization - Fix migration number collisions (V14 already taken by users migration; rename to V15 for PostgreSQL, bump to 16 for libSQL) - Extract TRUSTED_APPROVAL_CHANNELS constant to replace hardcoded "web"/"gateway" in is_approval_authorized(); WASM setup imports it - Add __bootstrap__ sentinel to WASM reserved channel names to prevent impersonation granting universal approval rights - Fix TenantScope::ensure_conversation passing None for source_channel, which silently blocked approvals for tenant-created threads - Add 11 regression tests: authorization logic, WASM reserved name validation, libSQL source_channel DB round-trip and upsert invariant Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address code review findings for cross-channel approval security 1. Add "telegram" to WASM channel name blocklist -- bundled channels like telegram were claimable by malicious WASM modules that load before the bundled one, bypassing cross-channel approval auth. 2. Make V16 libSQL migration (ADD COLUMN source_channel) idempotent -- the runner now checks pragma_table_info before executing ALTER TABLE, preventing startup failures if the base schema already includes the column. 3. Replace silent .unwrap_or(None) in thread hydration with explicit match on DB result -- legacy threads without stored source_channel now log a warning, and DB errors log an error. Both cases remain fail-closed (approvals denied) but are no longer silent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com> |
||
|
Illia Polosukhin
|
8f8cb7f7b1 |
feat: DB-backed user management, admin secrets provisioning, and multi-tenant isolation (#1626)
* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling
Finishes the remaining isolation work from phases 2–4 of #59:
Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.
Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.
Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use selected_model setting key to match /model command persistence
The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override
Three follow-up fixes for multi-tenant isolation:
1. Multi-user heartbeat now runs memory hygiene per user before each
heartbeat check, matching single-user heartbeat behavior.
2. /model command in multi-tenant mode only persists to per-user
settings (selected_model) without calling set_model() on the shared
LlmProvider. The per-request model_override in the dispatcher reads
from the same setting. Added multi_tenant flag to AgentConfig
(auto-detected from GATEWAY_USER_TOKENS).
3. RigAdapter now supports per-request model overrides by injecting the
model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
API servers use last-key-wins for duplicate JSON keys, so the override
takes effect via serde's flatten serialization order.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review — cost model attribution, heartbeat concurrency, pruning
Fixes from review comments on #1614:
- Cost tracking now uses the override model name (not active_model_name)
when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: /status ownership, model persistence scoping, heartbeat robustness
Addresses second round of PR review on #1614:
- /status <job_id> DB path now validates job.user_id == requesting user
before returning data (was missing ownership check, security fix).
- persist_selected_model takes user_id param instead of owner_id, and
skips .env/TOML writes in multi-tenant mode (these are shared global
files). handle_system_command now receives user_id from caller.
- JoinSet collection handles Err(JoinError) explicitly instead of
silently dropping panicked tasks.
- Notification forwarder extracts owner_id from response metadata in
multi-tenant mode for per-user routing instead of broadcasting to
the agent owner.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap
Round 3 review fixes:
- Cost tracking passes None for cost_per_token when model override is
active, letting CostGuard look up pricing by model name instead of
using the default provider's rates (serrrfirat).
- fire_manual() now uses per-user workspace, matching spawn_fire()
pattern (serrrfirat).
- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).
- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
the LLM provider (serrrfirat + Copilot).
- Fixed inject_model_override doc comment accuracy (Copilot).
- Added comment explaining multi-tenant notification routing priority
(Copilot).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: user-scoped webhook endpoint for multi-tenant isolation
Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.
The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.
Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
services that can't send bearer tokens)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(db): add UserStore trait with users, api_tokens, invitations tables
Foundation for DB-backed user management (#1605):
- UserRecord, ApiTokenRecord, InvitationRecord types in db/mod.rs
- UserStore sub-trait (17 methods) added to Database supertrait
- PostgreSQL migration V14__users.sql (users, api_tokens, invitations)
- libSQL schema + incremental migration V14
- Full implementations for both PgBackend (via Store delegation) and
LibSqlBackend (direct SQL in libsql/users.rs)
- authenticate_token JOINs api_tokens+users with active/non-revoked
checks; has_any_users for bootstrap detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(web): DB-backed auth, user/token/invitation API handlers
Adds the web gateway layer for DB-backed user management (#1605):
Auth refactor:
- CombinedAuthState wraps env-var tokens (MultiAuthState) + optional
DbAuthenticator for DB-backed token lookup with LRU cache (60s TTL,
1024 max entries)
- auth_middleware tries env-var tokens first, then DB fallback
- From<MultiAuthState> impl for backward compatibility
- main.rs wires with_db_auth when database is available
API handlers (12 new endpoints):
- /api/admin/users — CRUD: create, list, detail, update, suspend, activate
- /api/tokens — create (returns plaintext once), list, revoke
- /api/invitations — create, list, accept (creates user + first token)
Token creation: 32 random bytes → hex plaintext, SHA-256 hash stored.
Invitation accept: validates hash + pending + not expired, creates
user record and first API token atomically.
All test files updated for CombinedAuthState type change.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: startup env-var user migration + UserStore integration tests
Completes the DB-backed user management feature (#1605):
- Startup migration: when GATEWAY_USER_TOKENS is set and the users
table is empty, inserts env-var users + hashed tokens into DB.
Logs deprecation notice when DB already has users.
- hash_token made pub for reuse in migration code.
- 10 integration tests for UserStore (libsql file-backed):
- has_any_users bootstrap detection
- create/get/get_by_email/list/update user lifecycle
- token create → authenticate → revoke → reject cycle
- suspended user tokens rejected
- wrong-user token revoke returns false
- invitation create → accept → user created
- record_login and record_token_usage timestamps
- libSQL migration: removed FK constraints from V14 (incompatible
with execute_batch inside transactions). Tables in both base SCHEMA
and incremental migration for fresh and existing databases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: remove GATEWAY_USER_TOKENS, fix review feedback
GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.
Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
via db.has_any_users() in app.rs)
Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add role-based access control (admin/member)
Adds a `role` field (admin|member) to user management:
Schema:
- `role TEXT NOT NULL DEFAULT 'member'` added to users table in both
PostgreSQL V14 migration and libSQL schema/incremental migration
- UserRecord gains `role: String` field
- UserIdentity gains `role: String` field, populated from DB in
DbAuthenticator and defaulting to "admin" for single-user mode
Access control:
- AdminUser extractor: returns 403 Forbidden if role != "admin"
- /api/admin/users/* handlers: require AdminUser (create, list,
detail, update, suspend, activate)
- POST /api/invitations: requires AdminUser (only admins can invite)
- User creation accepts optional "role" param (defaults to "member")
- Invitation acceptance creates users with "member" role
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(web): add Users admin tab to web UI
Adds a Users tab to the web gateway UI for managing users, tokens,
and roles without needing direct API calls.
Features:
- User list table with ID, name, email, role, status, created date
- Create user form with display name, email, role selector
- Suspend/activate actions per user
- Create API token for any user (shows plaintext once with copy button)
- Role badges (admin highlighted, member muted)
- Non-admin users see "Admin access required" message
- Keyboard shortcut: Cmd/Ctrl+5 switches to Users tab
CSS:
- Reuses routines-table styles for the user list
- Badge, token-display, btn-small, btn-danger, btn-primary components
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move Users to Settings subtab, bootstrap admin user on first run
- Moved Users from top-level tab to Settings sidebar subtab (under
Skills, before Theme toggle)
- On first startup with empty users table, automatically creates an
admin user from GATEWAY_USER_ID config with a corresponding API
token from GATEWAY_AUTH_TOKEN. This ensures the owner appears in
the Users panel immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: user creation shows token, + Token works, no password save popup
Three UI/UX fixes:
1. Create user now generates an initial API token and shows it in a
copy-able banner instead of triggering the browser's password save
dialog. Uses autocomplete="off" and type="text" for email field.
2. "+ Token" button works: exposed createTokenForUser/suspendUser/
activateUser on window for inline onclick handlers in dynamically
generated table rows. Token creation uses showTokenBanner helper.
3. Admin token creation: POST /api/tokens now accepts optional
"user_id" field when the requesting user is admin, allowing
token creation for other users from the Users panel.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use event delegation for user action buttons (CSP compliance)
Inline onclick handlers are blocked by the Content-Security-Policy
(script-src 'self' without 'unsafe-inline'). Switched to data-action
attributes with a delegated click listener on the users table.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add i18n for Users subtab, show login link on user creation
- Added 'settings.users' i18n key for English and Chinese
- Token banner now shows a full login link (domain/?token=xxx)
with a Copy Link button, plus the raw token below
- Login link works automatically via existing ?token= auto-auth
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: token hash mismatch — hash hex string, not raw bytes
Critical auth bug: token creation hashed the raw 32 bytes
(hasher.update(token_bytes)) but authentication hashed the hex-encoded
string (hash_token(candidate) where candidate is the hex string the
user sends). This meant newly created tokens could never authenticate.
Fixed all 4 token creation sites (users, tokens, invitations create,
invitations accept) to use hash_token(&plaintext_token) which hashes
the hex string consistently with the auth lookup path.
Removed now-unused sha2::Digest imports from handlers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: remove invitation system
The invitation flow is redundant — admin create user already generates
a token and shows a login link. Invitations add complexity without
value until email integration exists.
Removed:
- InvitationRecord struct and 4 UserStore trait methods
- invitations table from V14 migration (postgres + both libsql schemas)
- PostgreSQL Store methods (create/get/accept/list invitations)
- libSQL UserStore invitation methods + row_to_invitation helper
- invitations.rs handler file (212 lines)
- /api/invitations routes (create, list, accept)
- test_invitation_lifecycle test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: user deletion, self-service profile, per-user job limits, usage API
Four multi-tenancy improvements:
1. User deletion cascade (DELETE /api/admin/users/{id}):
Deletes user and all data across 11 user-scoped tables (settings,
secrets, routines, memory, jobs, conversations, etc.). Admin only.
2. Self-service profile (GET/PATCH /api/profile):
Users can read and update their own display_name and metadata
without admin privileges.
3. Per-user job concurrency (MAX_JOBS_PER_USER env var):
Scheduler checks active_jobs_for(user_id) before dispatch.
Prevents one user from exhausting all job slots.
4. Usage reporting (GET /api/admin/usage?user_id=X&period=day|week|month):
Aggregates LLM costs from llm_calls via agent_jobs.user_id.
Returns per-user, per-model breakdown of calls, tokens, and cost.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add TenantCtx for compile-time tenant isolation
Implements zmanian's architectural proposal from #1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.
TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.
AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).
TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.
Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR #1626 review feedback — bounded LRU cache, admin auth, FK cleanup
- Replace HashMap with lru::LruCache in DbAuthenticator so the token
cache is hard-bounded at 1024 entries (evicts LRU, not just expired)
- Gate admin user endpoints (list/detail/update/suspend/activate) with
AdminUser extractor so members get 403 instead of full access
- Add api_tokens to libSQL delete_user cleanup list to prevent orphaned
tokens (libSQL has no FK cascade)
- Add regression tests for all three fixes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update CA certificates in runtime Docker image
Ensures the root certificate bundle is current so TLS handshakes
to services like Supabase succeed on Railway.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: resolve CI failures — formatting, no-panics check
- Run cargo fmt on test code
- Replace .expect() with const NonZeroUsize in DbAuthenticator
- Add // safety: comments for test-only code in multi_tenant.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: switch PostgreSQL TLS from rustls to native-tls
rustls with rustls-native-certs fails TLS handshake on Railway's
slim container (empty or stale root cert store). native-tls delegates
to OpenSSL on Linux which handles system certs more reliably.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Adding user management api
* feat: admin secrets provisioning API + API documentation
- Add PUT/GET/DELETE /api/admin/users/{id}/secrets/{name} endpoints for
application backends to provision per-user secrets (AES-256-GCM encrypted)
- Add secrets_store field to GatewayState with builder wiring
- Create docs/USER_MANAGEMENT_API.md with full API spec covering users,
secrets, tokens, profile, and usage endpoints
- Update web gateway CLAUDE.md route table
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add CatchPanicLayer to capture handler panics
Without this, panics in async handlers silently drop the connection
and the edge proxy returns a generic 503. Now panics are caught,
logged, and returned as 500 with the panic message.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address second-round review — transactional delete, overflow, error logging
- C1: Wrap PostgreSQL delete_user() in a transaction so partial cleanup
can't leave users in a half-deleted state
- M2: Add job_events to delete cleanup (both backends) — FK to
agent_jobs without CASCADE would cause FK violation
- H1/M4: Cap expires_in_days to 36500 before i64 cast (tokens + secrets)
- H2: Validate target user exists before creating admin token to prevent
orphan tokens on libSQL
- H3: Log DB errors in DbAuthenticator::authenticate() instead of
silently swallowing them as 401
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert to rustls with webpki-roots fallback for PostgreSQL TLS
native-tls/OpenSSL caused silent crashes (segfaults in C code) during
DB writes on Railway containers. Switch back to rustls but add
webpki-roots as a fallback when system certs are missing, which was
the original TLS handshake failure on slim container images.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update Cargo.lock for rustls + webpki-roots
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* debug: add /api/debug/db-write endpoint to diagnose user insert failure
Temporary diagnostic endpoint that tests DB INSERT to users table
with full error logging. No auth required. Will be removed after
debugging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: use cargo-chef in Dockerfile for dependency caching
Splits the build into planner/deps/builder stages. Dependencies are
only recompiled when Cargo.toml or Cargo.lock change. Source-only
changes skip straight to the final build stage.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* debug: add tracing to users_create_handler
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard created_by FK in user creation handler
The auth identity user_id (from owner_id scope) may not match any
user row in the DB, causing a FK violation on the created_by column.
Check that the referenced user exists before setting created_by.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: collapse GATEWAY_USER_ID into IRONCLAW_OWNER_ID
Remove the separate GATEWAY_USER_ID config. The gateway now uses
IRONCLAW_OWNER_ID (config.owner_id) directly for auth identity,
bootstrap user creation, and workspace scoping.
Previously, with_owner_scope() rebinds the auth identity to owner_id
while keeping default_sender_id as the gateway user_id. This caused
a FK constraint violation when creating users because the auth
identity ("default") didn't match any user in the DB ("nearai").
Changes:
- Remove GATEWAY_USER_ID env var and gateway_user_id from settings
- Remove user_id field from GatewayConfig
- Add owner_id parameter to GatewayChannel::new()
- Remove with_owner_scope() method
- Remove default_sender_id from GatewayState
- Remove sender override logic in chat/approval handlers
- Remove debug endpoint and tracing from prior debugging
- Update all tests and E2E fixtures
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: hide Users tab for non-admins, remove auth hint text
- Fetch /api/profile after login and hide the Users settings tab
when the user's role is not admin
- Remove the "Enter the GATEWAY_AUTH_TOKEN" hint from the login page
since tokens are now managed via the admin panel, not .env files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review feedback (auth 503, token expiry, CORS PATCH)
- DB auth errors now return 503 instead of 401 so outages are
distinguishable from invalid tokens (serrrfirat H3)
- Cap expires_in_days to 36500 before i64 cast to prevent negative
duration from u64 overflow (serrrfirat H1)
- Add PATCH to CORS allowed methods for profile/user update
endpoints (Copilot)
- Stop leaking panic details in CatchPanicLayer response body
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: harden multi-tenant isolation — review fixes from #1614
- Add conversation ownership checks in TenantScope: add_conversation_message,
touch_conversation, list_conversation_messages (+ paginated),
update_conversation_metadata_field, get_conversation_metadata now return
NotFound for conversations not owned by the tenant (cross-tenant data leak)
- Fix multi-user heartbeat: clear notify_user_id per runner so notifications
persist to the correct user, not the shared config target
- Move hygiene tasks into bounded JoinSet instead of unbounded tokio::spawn
- Revert send_notification to private visibility (only used within module)
- Use effective_model_name() for cost attribution in dispatcher so providers
that ignore per-request model overrides report the actual model used
- Fix inject_model_override doc comment; add 3 unit tests
- Fix heartbeat doc comment ("routines" not "active routines")
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add Jobs, Cost, Last Active columns to admin Users table
Add UserSummaryStats struct and user_summary_stats() batch query to the
UserStore trait (both PostgreSQL and libSQL backends). The admin users
list endpoint now fetches per-user aggregates (job count, total LLM
spend, most recent activity) in a single query and includes them inline
in the response. The frontend Users table displays three new columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review comments and CI formatting failures
CI fixes:
- cargo fmt fixes in cli/mod.rs and db/tls.rs
Security/correctness (from Copilot + serrrfirat + pranavraja99 reviews):
- Token create: reject expires_in_days > 36500 with 400 instead of silent clamp
- Token create: return 404 when admin targets non-existent user
- User create: map duplicate email constraint violations to 409 Conflict
- User create: remove unnecessary DB roundtrip for created_by (use AdminUser directly)
- DB auth: log warn on DB lookup failures instead of silently swallowing errors
- libSQL: add FK constraints on users.created_by and api_tokens.user_id
Config fixes:
- agent.multi_tenant: resolve from AGENT_MULTI_TENANT env var instead of hardcoding false
- heartbeat.multi_tenant: fix doc comment to match actual env-var-based behavior
UI fix:
- showTokenBanner: pass correct title ("Token created!" vs "User created!")
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address remaining review comments (round 2)
- Secrets handlers: normalize name to lowercase before store operations,
validate target user_id exists (returns 404 if not found)
- libSQL: propagate cost parsing errors instead of unwrap_or_default()
in both user_usage_stats and user_summary_stats
- users_list_handler: propagate user_summary_stats DB errors (was
silently swallowed with unwrap_or_default)
- loadUsers: distinguish 401/403 (admin required) from other errors
- Docs: fix users.id type (TEXT not UUID), remove "invitation flow"
from V14 migration comment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: i18n for Users tab, atomic user+token creation, transactional delete_user
i18n:
- Add 31 translation keys for all Users tab strings (en + zh-CN)
- Wire data-i18n attributes on HTML elements (headings, buttons, inputs,
table headers, empty state)
- Replace all hard-coded strings in app.js with I18n.t() calls
Atomic user+token creation:
- Add create_user_with_token() to UserStore trait
- PostgreSQL: wraps both INSERTs in conn.transaction() with auto-rollback
- libSQL: wraps in explicit BEGIN/COMMIT with ROLLBACK on error
- Handler uses single atomic call instead of two separate operations
Transactional delete_user for libSQL:
- Wrap multi-table DELETE cascade in BEGIN/COMMIT transaction
- ROLLBACK on any error to prevent partial cleanup / inconsistent state
- Matches the PostgreSQL implementation which already used transactions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert V14 migration to match deployed checksum [skip-regression-check]
Refinery checksums applied migrations — editing V14__users.sql after
it was already applied causes deployment failures. Revert the cosmetic
comment changes (added in
|
||
|
Henry Park
|
878a67cdb6 |
Refactor owner scope across channels and fix default routing fallback (#1151)
* refactor: add explicit owner scope across channels * fix: tighten routine owner target routing * fix: address owner scope review feedback * Fix owner-scope onboarding and event trigger isolation * Tighten routing fallback and wizard owner validation * fix: address owner-scope follow-up review * fix: tighten owner-scope follow-up details * fix: import Channel trait in telegram test * fix: normalize http webhook sender ids * fix: address remaining owner-scope review issues * fix: reconcile config rebase fallout * fix: reconcile extension manager rebase drift * fix: address current copilot review regressions * fix: restore clippy matrix after rebase |
||
|
Henry Park
|
873322f2fb |
fix: staging CI review issues (batch 1) (#883)
* fix: address staging-ci-review issues (batch 1) - #811: Fix unreachable error handling in worker — restructure .await? to explicit match on nested Result so token budget errors are properly logged and marked as failed - #813: Combine metadata + token budget into single update_context() call to prevent concurrent worker observing partial state - #814: Persist max_tokens and total_tokens_used to both PostgreSQL and libSQL backends — add V12 migration, update save_job/get_job - #815: Cap user-supplied max_tokens at configured max_tokens_per_job to prevent budget bypass via metadata injection - #869: Release locks before async I/O in webhook handler (http.rs) and SIGHUP handler (main.rs) to prevent blocking concurrent requests Fixes: #811, #813, #814, #815, #869 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR #883 review feedback - Fix min(user_val, 0) bug: guard for unlimited config (max_tokens_per_job == 0) - Remove duplicate columns from libSQL base SCHEMA (v12 migration is sole source) - Use get_i64() helper for consistency in libsql/jobs.rs - Add regression tests for scheduler token budget capping Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
9f71bd0d44 |
feat: unified thread model for web gateway (#607)
* feat: unified thread model for web gateway Every piece of activity (user chat, routine run, heartbeat alert, external channel message) now lives in its own thread, properly isolated, with meaningful titles and visual distinction. Key changes: - Add `channel` field to ConversationSummary and ThreadInfo so the gateway can distinguish thread origins (gateway, telegram, routine, heartbeat). - Add `list_conversations_all_channels` to Database trait (both postgres and libsql) so chat_threads_handler shows cross-channel threads. - Routine runs get a persistent conversation per routine via `get_or_create_routine_conversation`; notifications carry thread_id. - Heartbeat gets a persistent conversation via `get_or_create_heartbeat_conversation`; HeartbeatRunner accepts an optional Database store and binds notifications to the thread. - Fix broadcast() in web gateway to propagate response.thread_id instead of hardcoding empty string. - Fix isCurrentThread(null) returning true (the core notification leak bug) — now returns false so events without a thread_id don't leak into the active thread. - Rewrite frontend thread sidebar: meaningful titles with channel-specific fallbacks, relative timestamps instead of turn counts, channel badges for non-gateway threads, unread notification dots, read-only indicator for external channel threads. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — TOCTOU races, stale comment, debounce, broadcast warning - Fix TOCTOU race in get_or_create_routine_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_routine unique index + SELECT-back. - Fix TOCTOU race in get_or_create_heartbeat_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_heartbeat unique index + SELECT-back. - Fix TOCTOU race in get_or_create_routine_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Fix TOCTOU race in get_or_create_heartbeat_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Add V11 migration with partial unique indexes for postgres. - Add matching unique indexes to libsql schema. - Update stale comment on isCurrentThread (said "always shown" but logic now returns false for missing thread_id). - Debounce loadThreads() on off-thread SSE events to prevent request storms. - Log warning in broadcast() when thread_id is None (clients will drop it). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: sort in-memory thread fallback by updated_at descending The in-memory thread list fallback (when no DB is available) used HashMap::values() which has no guaranteed ordering. Sort by updated_at descending to match the SQL query ordering. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: retry libsql connect() on transient "unable to open database file" The cron ticker's background task occasionally fails with "unable to open database file" when creating a new SQLite connection concurrently with the main thread. Add retry with exponential backoff (50ms, 100ms, 200ms) to handle transient VFS/locking issues in libsql's local mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use ON CONFLICT with index expressions instead of named constraints PostgreSQL ON CONFLICT ON CONSTRAINT requires a named table constraint, but V11 migration creates unique indexes. Switch to the expression form (ON CONFLICT (columns) WHERE condition) which works with unique indexes. Also fix dead code in threadTitle() where thread.title was already checked on the previous line. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix rustfmt chain collapse in heartbeat.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: skip broadcast when thread_id is None instead of sending empty Clients drop SSE events with empty thread_id anyway, so avoid the unnecessary network traffic by returning early. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add libsql routine/heartbeat conversation idempotency tests Add tests proving get_or_create_routine_conversation returns the same conversation ID across multiple invocations with the same routine_id. Add debug logging to routine engine to track conversation resolution. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: show "New chat" title for empty threads - threadTitle() returns "New chat" when turn_count is 0 - Assistant thread label updates dynamically from API data - Default HTML label changed from "Assistant" to "New chat" - New threads naturally sort to top via last_activity DESC [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: thread sorting, routine isolation, and UI polish - Fix libsql timestamp format mismatch causing broken thread sort order. SQLite defaults used `datetime('now')` (space-separated) while Rust code used RFC3339 (T-separated), breaking string-based ORDER BY. All INSERTs now use RFC3339, and queries use `datetime()` to normalize comparison. - Route manual routine triggers through RoutineEngine.fire_manual() instead of injecting as regular chat messages, so routines always run in their dedicated conversation thread. - Add RoutineEngineSlot to GatewayState for gateway<->engine communication. - Derive routine thread titles from conversation metadata (routine_name) instead of showing truncated UUID hashes. - Make chat_new_thread_handler persist to DB synchronously so loadThreads() sees newly created threads immediately. - Fix enableChatInput() no-op and wrong element ID in disableChatInputReadOnly(). - Fix handlers/chat.rs stale gateway-only query (use list_conversations_all_channels). - Sort in-memory threads by DateTime before converting to RFC3339 strings. - Trigger debouncedLoadThreads() on thinking/status SSE events for non-current threads so routine/heartbeat threads appear in sidebar promptly. - Remove "Threads" text from sidebar header. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: routine history display, orphaned tool_results, duplicate system messages Three independent fixes with regression tests: 1. Routine conversations now display in the web UI. build_turns_from_db_messages() handles standalone assistant messages (no preceding user message) by creating turns with empty user_input. Frontend skips empty user bubbles. 2. Worker select_tools and execute_plan paths now push an assistant_with_tool_calls message before tool execution, preventing sanitize_tool_messages from rewriting tool_results as orphaned user messages. 3. Reasoning::plan() and respond_with_tools() merge system messages from context into a single system prompt instead of creating [system, system, ...] sequences that strict LLM providers (Qwen) reject. Also: sidebar padding/spacing improvements, wider thread panel (240px). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR #607 review — RwLock held across await, missing ownership check, heartbeat config - Clone Arc<RoutineEngine> out of RwLock before .await in trigger handler - Add user_id ownership check to fire_manual() with NotAuthorized error - Wire heartbeat notify_user/notify_channel from config to AgentHeartbeatConfig Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: gitignore trace_*.json files and remove stale traces Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove trace JSON files from repo Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: proper HTTP status codes for routine errors, read-only input guard, respond thread_id - Map RoutineError::NotFound → 404, NotAuthorized → 403, Disabled → 409 - Guard enableChatInput() against re-enabling on read-only threads - Skip respond() when thread_id is None (matches broadcast() behavior) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
04c5c3fe9f |
feat: WASM extension versioning with WIT compat checks (#592)
* feat: add WASM extension versioning with WIT compat checks and CI enforcement Phase 1 — WIT Versioning & Compatibility Checks: - Version WIT packages as `package near:agent@0.2.0;` - Add `semver` crate for version parsing and comparison - Add `WIT_TOOL_VERSION` / `WIT_CHANNEL_VERSION` host constants - Add `version` and `wit_version` fields to capabilities schemas - Add `wit_version` column to `wasm_tools` DB table (both backends) - Add load-time `check_wit_version_compat()` with semver rules - Add `IncompatibleWitVersion` error variants for tools and channels - Enhance instantiation errors with WIT version mismatch hints - Update all 14 capabilities JSON and 14 registry JSON files Phase 2 — Upgrade-in-Place & Channel DB Storage: - Change tool store to DELETE-before-INSERT (one version per extension) - Create `wasm_channels` table (PostgreSQL migration + libSQL schema) - Add `WasmChannelStore` trait with PostgreSQL and libSQL backends - Add `extension_info` tool showing version, WIT version, and status - Wire `ExtensionInfoTool` into tool registry (7 extension tools) Phase 3 — CI Version-Bump Enforcement: - Add `scripts/check-version-bumps.sh` checking WIT/tool/channel versions - Add `version-check` CI job (PR-only) to `.github/workflows/test.yml` - Support `[skip-version-check]` label/commit message bypass Includes 7 regression tests for WIT version compatibility checking and 2 integration tests for WIT version annotation verification. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback for WASM extension versioning - Wrap PostgreSQL DELETE+INSERT in transactions for both tool and channel store() methods to prevent data loss on partial failure (Gemini, Copilot) - Rename StoredWasmChannelWithBinary.tool → .channel (copy-paste fix) - Remove unused WasmError::IncompatibleWitVersion variant (dead code) - Map channel loader WIT mismatch to IncompatibleWitVersion instead of generic Config error, simplify variant to single String message - Fix extension_info description to match actual returned fields - Add schema test for ExtensionInfoTool matching existing test pattern - Fix CI script to fail fast on git errors instead of silent bypass [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
097a26ace6 |
fix: harden openai-compatible provider, approval replay, and embeddings defaults (#237)
* fix: harden openai-compatible tool flow and local defaults * fix: close approval replay gaps and harden openai-compatible flow * fix: address review feedback and code improvements (takeover #112) - Make ChatCompletionResponse.id Optional<String> to handle providers that omit or null the field - Propagate HTTP client builder errors instead of silently dropping timeout configuration (openai_compatible_chat, nearai_chat) - Add EMBEDDING_DIMENSION env var with smart per-model defaults instead of hardcoding 768/1536 everywhere - Remove duplicated dimension inference logic from main.rs Co-Authored-By: panosAthDBX <panosAthDBX@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: harden src/llm/ module from crate audit findings - Replace 9x .expect() on RwLock with graceful poison recovery (nearai.rs: 7, nearai_chat.rs: 2) — eliminates production panics - Propagate HTTP client builder errors in nearai.rs instead of silently dropping timeout config (NearAiProvider::new now returns Result) - Make nearai_chat ChatCompletionResponse.id Optional<String> (mirrors openai_compatible_chat.rs fix for providers that omit id) - Make nearai_chat usage fields optional with defensive parse_usage() helper (was required u32 fields that crash on null/missing) - Truncate error responses to 512 chars in nearai_chat.rs error messages to prevent log bloat and potential data leakage - Delegate 4 missing LlmProvider methods in FailoverProvider (model_metadata, seed_response_chain, get_response_chain_id, calculate_cost) to last-used provider instead of trait defaults Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(llm): add RetryProvider, remove openai_compatible_chat, harden decorators - Add composable RetryProvider decorator wrapping any LlmProvider with exponential backoff + jitter, respecting RateLimited retry_after hints - Remove openai_compatible_chat.rs — replaced by rig adapter + RetryProvider - Remove internal retry loop from nearai.rs (was causing double-retry with external RetryProvider, up to 16 attempts instead of 4) - Remove internal retry loop from nearai_chat.rs (same issue) - Wire RetryProvider into main.rs composition chain: each provider gets its own retry wrapper before failover - Move normalize_tool_name to rig_adapter.rs for all rig-based providers - Reconcile is_retryable() vs is_transient() error classification: ModelNotAvailable no longer retryable, Json no longer transient - Fix unchecked Duration subtraction panic in circuit_breaker.rs - Make failover.rs use shared is_retryable() from retry.rs - Remove stale #[allow(dead_code)] on NearAiResponse::id (field is used) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback — error handling, dimension validation, libSQL warning - Replace response.text().await.unwrap_or_default() with proper error propagation in nearai.rs and nearai_chat.rs (4 call sites). Failures now return LlmError::RequestFailed with context instead of silently proceeding with an empty string. - Add embedding dimension validation in OllamaEmbeddings::embed_batch(): returns EmbeddingError if Ollama returns embeddings with a dimension that doesn't match the configured value. - Add runtime warning when libSQL backend is used with non-1536 embedding dimension, since the libSQL schema uses F32_BLOB(1536) and cannot store different-dimension vectors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: panosAthDbx <packux@gmail.com> Co-authored-by: panosAthDBX <127238517+panosAthDBX@users.noreply.github.com> Co-authored-by: panosAthDBX <panosAthDBX@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Illia Polosukhin
|
ced83d5b4d |
feat: Sandbox jobs (#4)
* Orchestrating jobs and running them in sandboxes * Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback - Query /v1/models API for context_length and set max_tokens to half (floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7 need much larger budgets - Guard against empty LLM content (reasoning models can burn all tokens on chain-of-thought and return content: null) - Simplify notification routing: try configured channel first, fall back to broadcast_all so heartbeat alerts always reach someone - Add ModelMetadata struct and model_metadata() to LlmProvider trait - Refactor NearAiChatProvider::list_models into shared fetch_models() - Add standalone test_heartbeat example for isolated debugging Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add job detail view with drill-down from jobs list Click a job row to see full details across four sub-tabs: Overview (metadata grid, description, state transitions timeline), Actions (expandable tool call cards with input/output JSON), Thinking (conversation messages styled by role), and Files (embedded workspace tree browser). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400 Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the content field instead of using the OpenAI tool_calls array. This XML leaks through to channels as text, and Telegram's Markdown parser chokes on the underscores, returning 400 "can't parse entities". Two fixes: - Generalize clean_response() to strip <tool_call>, <function_call>, <tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside the existing <thinking> tag stripping - Add Telegram send_message helper with parse_mode fallback: try Markdown first, retry as plain text on "can't parse entities" 400 errors Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add SystemCommand submission type for thread-state-independent commands System commands (/help, /model, /version, /tools, /ping, /debug) now bypass thread-state checks and safety validation via a dedicated Submission::SystemCommand variant. Previously these flowed through process_user_input() which blocked them during Processing/AwaitingApproval /Completed states. - Add /model [name] for runtime model switching with provider validation - Add active_model_name()/set_model() to LlmProvider trait with RwLock hot-swap in both NEAR AI providers - Rewrite /help with aligned columns grouped by category - Expand REPL tab-completion from 10 to 23 slash commands - Remove REPL-local /help interception (now handled by agent) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files The sandbox e2e pipeline (agent -> container -> built website -> browsable URL) was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need minutes, no auto-created project directory meant container output vanished, and no HTTP route to browse the built files. - Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler, worker/runtime) with the per-tool value - Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer) - Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified, so every sandbox job gets a persistent bind mount - Include `project_dir` and `browse_url` in sandbox tool output JSON - Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes to the web gateway with path traversal protection and MIME type detection - Add `mime_guess` dependency for content-type detection Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply cargo fmt to wizard.rs after merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Persist sandbox jobs in DB, fix web UI, unify job model Sandbox container jobs were invisible to the web UI because they lived only in ContainerJobManager's in-memory HashMap while the API queried ContextManager. This persists them to the agent_jobs table and fixes all six front-end bugs (empty job list, broken back button, empty actions/thinking tabs, wrong files tab, stuck status, no persistence). Key changes: - V4 migration adds project_dir and user_id columns to agent_jobs - Embedded migrations via refinery (no external CLI needed) - SandboxJobRecord CRUD in Store with fire-and-forget DB writes - Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager - Web API queries DB for sandbox jobs, merges with ContextManager direct jobs - New endpoints: restart, project file list/read with path traversal protection - Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in chat stream, source badges, restart button for failed/interrupted jobs - Gateway defaults to enabled, prints Web UI URL on startup - Stale jobs marked "interrupted" on restart for visibility and restartability Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Secure in-chat auth: tokens never touch the LLM or chat history Remove the token parameter from tool_auth so the LLM cannot pass raw API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket (auth_token) endpoints that route tokens directly to ext_mgr.auth(), completely bypassing the message pipeline, turns, history, and compaction. Web UI shows an auth card (password input + OAuth button) when the agent enters auth mode, submitted via the dedicated endpoint. CLI auth mode interception is unchanged (already secure). New StatusUpdate::AuthRequired/AuthCompleted variants propagate through all channels (SSE, WebSocket, REPL, WASM). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Add Claude Code mode for sandbox jobs Run Claude Code CLI inside Docker containers as an alternative to the standard worker mode. The bridge spawns `claude -p` with stream-json output, posts events to the orchestrator, and supports follow-up prompts via `--resume`. Key additions: - `claude-bridge` CLI subcommand and ClaudeBridgeRuntime - JobMode enum (Worker vs ClaudeCode) with per-mode container config - Orchestrator endpoints for Claude events and prompt polling - SSE event variants for real-time Claude Code streaming to frontend - Claude Code sub-tab in web UI with terminal-style output and input bar - Database migration for job_mode column and claude_code_events table - ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.) - Mode parameter on run_in_sandbox tool schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs When sandbox mode is on, the LLM would call create_job (creating a pending "direct" entry) then run_in_sandbox (creating a second "sandbox" entry), producing two jobs in the list for a single user request. Now register_job_tools() skips create_job when sandbox is enabled since run_in_sandbox already creates tracked jobs. Also improved the run_in_sandbox description to guide the LLM to use it directly and to mention wait=false for async execution. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Web gateway UI quality-of-life improvements Phase 1: Send button disabled state to prevent double-sends, copy button on code blocks, confirm() guards on destructive actions, SSE-driven job list auto-refresh, log filters re-applied on tab switch, jobEvents memory leak fix (cap at 500, cleanup after 60s). Phase 2: Toast notification system replacing chat-based system messages, memory search highlighting with centered snippets, keyboard shortcuts (Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur), activity tab toolbar with event type filter and auto-scroll toggle. Phase 3: Thread sidebar with load/switch/create, thread_id passed with messages, collapsible to hamburger. Memory inline editing with textarea, Save/Cancel, POST to /api/memory/write. Phase 4: Gateway status popover on hover (polls every 30s), extension install form (name/URL/kind), markdown rendering in memory viewer for .md files, mobile responsive layout at 768px breakpoint. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Add routines system, remove non-sandbox job mode from web UI Routines: scheduled & reactive job system with cron and event triggers, lightweight (single LLM call) and full-job execution modes, guardrails (cooldown, max concurrent, dedup), and LLM-facing tools for CRUD. Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs are now exclusively sandbox-backed (DB + container). Simplify job detail response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo), fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab event rendering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML Three fixes: 1. Chat input stays disabled after agent finishes: the "Done" status SSE event now calls enableChatInput() as a safety net when the response event is empty or lost. Same for auth_completed and cancelAuth(). 2. tool_activate never triggers auth: when activation fails due to missing authentication, it now auto-initiates the auth flow (same pattern as the web API handler). detect_auth_awaiting() also matches tool_activate results now. 3. Models like GLM-4.7 emit tool calls as XML tags in content (<tool_call>tool_list</tool_call>) instead of using the structured tool_calls array. recover_tool_calls_from_content() extracts and validates these before falling back to plain text. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Add routines web UI tab, update docs for sandbox-jobs branch Add full routines management to the web gateway (list, detail, trigger, toggle, delete) with 7 new API endpoints, response types, and frontend (HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new subsystems, config, TODOs), and README.md (architecture diagram, features, components, fix onboard command). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Bind Telegram bot to owner account during setup Without owner binding, anyone who discovers the bot can send it messages. The setup wizard now prompts the user to message their bot, captures their Telegram user ID via getUpdates, and persists it as telegram_owner_id in settings. On startup, the owner_id is injected into the WASM channel config so the existing owner restriction logic drops messages from non-owners. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Move settings from disk to PostgreSQL database Settings previously lived in three JSON files on disk (settings.json, mcp-servers.json, session.json). This made them inaccessible from the web UI and caused redundant disk reads (Settings::load() called 8+ times during startup). Now all settings live in a `settings` table (user_id + key -> JSONB) with only 4 bootstrap fields remaining on disk (database_url, pool size, secrets key source, onboard_completed) since they're needed before the DB connection exists. - Add V8 migration for settings table - Add BootstrapConfig (thin disk file) and Settings DB round-trip - Add Store CRUD methods for settings (get/set/delete/list/bulk) - Refactor Config to load from DB (env > DB > default cascade) - Add SessionManager DB persistence for session tokens - Add DB-backed MCP server config load/save functions - Add 6 settings web API endpoints (list/get/set/delete/export/import) - Add one-time disk-to-DB migration on first boot - Make CLI config commands async with DB access (disk fallback) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth - Add Workspace::seed_if_empty() to create core identity files (README, MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called on every boot without overwriting existing user edits - Remove duplicate gateway log lines from web/mod.rs (main.rs has the useful clickable ?token= URL) - Auto-authenticate from ?token= URL parameter in the web UI and strip the token from the address bar after successful auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Harden sandbox security (path traversal + orchestrator auth) Two vulnerabilities fixed: 1. project_dir path traversal: The create_job tool let the LLM specify arbitrary host paths for Docker bind mounts. Removed project_dir from the tool schema entirely, and added canonicalization + prefix validation at both resolve_project_dir() and the job_manager bind mount point. 2. Orchestrator API auth bypass: worker_auth_middleware was defined but never applied. Each handler manually called validate_token(), so any new endpoint that forgot would be publicly accessible. Applied the middleware as route_layer on all /worker/ routes, removed manual auth from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps 0.0.0.0 since containers reach host via docker bridge, not loopback). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining Implements the 4-phase plan for overhauling the web gateway chat: - Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below - Phase 2: Cursor-based history pagination with infinite scroll - Phase 3: NEAR AI previous_response_id chaining (delta-only messages), with fallback to full history on chain errors, and DB persistence of chain state across restarts - Phase 4: SSE thread isolation (events filtered by thread_id) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Add per-request HTTP timeout to WASM host, redact credentials in errors Three fixes for WASM channel reliability: 1. Per-request timeout: Add optional timeout-ms parameter to http-request in both channel and tool WIT interfaces. Telegram long-poll now specifies 35s (outliving the 30s server-side hold), while regular API calls use the 30s default. Fixes the triple-30s timeout race that caused polling failures. 2. Credential redaction: reqwest::Error includes the full URL (with injected bot tokens) in its Display output. Scrub credential values from error messages before logging or returning to WASM. 3. Webhook route registration: Remove tunnel URL gate so webhook routes are always available when webhook channels exist, not only when TUNNEL_URL is configured. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: Fix clippy warnings in WASM tools and channels - slack channel: allow dead_code on signing_secret_name (forward compat field) - gmail tool: use div_ceil() instead of manual (n+2)/3 - google-calendar tool: extract CreateEventParams/UpdateEventParams structs to fix too-many-arguments warnings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix approval flow * fix: Rebuild bundled telegram.wasm with updated WIT interface The bundled WASM binary must match the host's WIT definition. Previous binary was compiled against the old 4-arg http-request; this rebuild includes the new timeout-ms parameter. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: Load WASM channels from disk instead of bundling in binary Remove include_bytes! embedding of telegram.wasm. Channels are now loaded from their build output directories (channels-src/<name>/target/) during onboarding, then from ~/.ironclaw/channels/ at runtime. - bundled.rs: locate_channel_artifacts() finds WASM + capabilities from build output; IRONCLAW_CHANNELS_SRC env var overrides the default path - available_channel_names(): only lists channels with build artifacts - bundled_channel_names(): lists all known channels (manifest) - Setup wizard uses available_channel_names() to offer installable channels - Add *.wasm to .gitignore, remove tracked telegram.wasm Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Persist gateway auth token, fix thread hydration race, polish auth screen Three web gateway UX fixes: 1. Token persistence: Store auth token in sessionStorage so refreshing the page doesn't force re-authentication. Hide the auth screen immediately when a saved token exists to prevent flash. 2. Thread hydration: Remove the !msgs.is_empty() bail-out in maybe_hydrate_thread so that even brand-new (empty) assistant threads get hydrated with their correct DB UUID. Previously resolve_thread would mint a fresh UUID, causing messages to land in the wrong conversation and duplicate threads to appear. 3. Auth screen: Redesign as a centered card with brand, tagline, labeled input, and hint text. Also adds 34 new tests covering session/thread lifecycle, thread resolution isolation (user, channel, external ID), hydration edge cases, serialization round-trips, approval flows, and stale mapping recovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Use bindgen! for WASM tool wrapper, add dev tool loading Three changes: 1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen! instead of manual linker.root().func_wrap(). This fixes the "component imports instance 'near:agent/host', but a matching implementation was not found in the linker" error. All 6 host functions (log, now-millis, workspace-read, http-request, secret-exists, tool-invoke) are now properly registered under the near:agent/host namespace. Also adds WASI support, credential injection, and leak detection for HTTP requests made by WASM tools. 2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the loader now also scans tools-src/*/target/wasm32-wasip2/release/ for build artifacts that are newer than installed copies. This means during development you just rebuild the WASM and restart the host; no manual copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir. 3. Wire up load_dev_tools() in main.rs alongside the existing load_from_dir() call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Wire main startup and CLI to use DB-backed settings main.rs now reloads Config from the database after connecting, attaches the store to the session manager for dual-write tokens, and loads MCP servers from DB instead of disk. ExtensionManager and MCP CLI commands use DB when available with disk fallback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
235f6aae18 |
Add heartbeat integration, planning phase, and auto-repair
- Add HeartbeatConfig for proactive periodic execution with channel notifications - Add use_planning option to Worker for ActionPlan generation before tool execution - Implement tool failure tracking in database (V3 migration) - Add auto-repair via Builder for broken WASM tools in self_repair.rs - Record tool failures in Worker for self-repair tracking - Update .env.example with new configuration options Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
32bfd24154 |
Add WASM sandbox secure API extension
Extends the WASM sandbox with HTTP API capabilities, secrets management, tool aliasing, and leak detection. Key security principle: WASM never sees credentials, injection happens at host boundary. New modules: - secrets: AES-256-GCM encrypted storage with HKDF key derivation - leak_detector: Aho-Corasick + regex pattern matching for secret exfiltration - capabilities: Extended capability system (HTTP, ToolInvoke, Secrets) - allowlist: HTTP endpoint validation with glob patterns - credential_injector: Host-boundary credential injection - rate_limiter: Sliding window per-tool rate limiting - storage: WASM binary storage with BLAKE3 integrity verification Leak detection happens at two points: 1. Before HTTP request (prevents exfiltration via URL/headers/body) 2. After response (prevents exposure in outputs returned to WASM) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
3718cfa767 |
Simplify workspace to path-based storage, remove legacy code
- Consolidate all migrations into V1__initial.sql - Replace DocType enum with flexible path-based file storage - Add list_workspace_files SQL function for directory listing - Update memory tools for path-based API (memory_read, memory_write, memory_search, memory_list) - Remove unused OpenAI/Anthropic providers (NEAR AI only) - Simplify config to remove multi-provider support - Update CLAUDE.md documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
4e238e60ac |
Add workspace and memory system (OpenClaw-inspired)
Implements persistent memory for agents with hybrid search: - Database-backed workspace with PostgreSQL (not filesystem) - Memory documents: MEMORY.md, daily logs, identity files - Chunked content with FTS (tsvector) + vector (pgvector) indexes - Reciprocal Rank Fusion (RRF) for hybrid search combining BM25 and semantic - Memory tools: memory_search, memory_write, memory_read - Proactive heartbeat system for periodic execution (30 min default) - OpenAI embeddings provider (text-embedding-3-small) Key patterns from OpenClaw: - "Memory is files, not RAM" - explicit persistence required - Two-tier memory: daily logs (raw) + curated MEMORY.md - Session isolation via user_id/agent_id scoping Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
8c38566378 | Initial implementation of the agent framework |