mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
bfca5e93311f9eb4aae9f6fa9d9b293f5e7af2b5
118 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Illia Polosukhin
|
0af0267125 |
feat(engine-v2): per-project sandbox (Phases 1–7) (#2211)
* feat(engine-v2): mount-backend abstraction for per-project sandbox (Phase 1) Adds the engine-side `MountBackend` trait + minimal `WorkspaceMounts` registry and a host-side bridge interceptor that routes sandbox-eligible tool calls (`file_read`, `file_write`, `list_dir`, `apply_patch`, `shell`) through a backend when their path argument starts with `/project/`. Default behavior is unchanged: until `EffectBridgeAdapter::set_workspace_mounts(Some(...))` is called (Phase 6), the interception path is dormant. This is the first phase of the per-project sandbox plan (`docs/plans/2026-04-10-engine-v2-sandbox.md`) and a deliberately small subset of the unified Workspace VFS proposed in nearai/ironclaw#1894 — just enough abstraction so the sandbox can be a `MountBackend` rather than a special case in the bridge. When #1894's full mount table lands, the sandbox backend slots in unchanged. Engine crate (`crates/ironclaw_engine/src/workspace/`): - `mount.rs` — `MountBackend` trait, `MountError` (NotFound / InvalidPath / PermissionDenied / Io / Tool / Backend / Unsupported), `DirEntry`, `EntryKind`, `ShellOutput` - `filesystem.rs` — `FilesystemBackend`: passthrough host-fs implementation with two-layer path validation (lexical reject of absolute / `..`, then symlink-escape canonicalization). `read`/`write`/`list` fully implemented; `patch`/`shell` return `Unsupported` so the bridge falls through to the host tool until Phase 5 - `registry.rs` — `WorkspaceMounts` per-project registry with lazy `ProjectMountFactory`, longest-prefix-match resolution, cached and invalidatable Bridge (`src/bridge/sandbox/`): - `intercept.rs` — `maybe_intercept` and `SANDBOX_TOOL_NAMES`. Returns `Handled(json)` on a successful backend dispatch, `FellThrough` for non-sandbox tools, host paths, missing path params, or `Unsupported` backend ops - `effect_adapter.rs` — `workspace_mounts` field + `set_workspace_mounts` setter; interception block in `execute_action_internal` right before `execute_tool_with_safety`, gated on the optional mount table Tests (31 new): - 17 engine workspace unit tests covering trait error mapping, path safety (lexical + symlink), longest-prefix routing, and lazy factory caching - 9 bridge sandbox unit tests including `intercept_actually_dispatches_into_backend` (counting backend) which proves the interceptor reaches the backend - 5 integration tests in `tests/engine_v2_sandbox_integration.rs` driving `EffectBridgeAdapter::execute_action()` end-to-end per the "Test Through the Caller" rule (`.claude/rules/testing.md`), including a host-path-falls-through test that asserts the sandbox tempdir was not touched, and a `..`-escape test that verifies no `/etc/passwd` content leaks even after safety-layer redaction Drive-by: feature-gate two pre-existing dead-code helpers in `crates/ironclaw_skills/src/parser.rs` on `#[cfg(feature = "registry")]` to match their only call site, fixing a pre-existing clippy warning that blocked the workspace's `-D warnings` policy when `ironclaw_skills` is built with `default-features = false` (as the engine crate does). Verification: - `cargo fmt --check` clean - `cargo clippy --all --benches --tests --examples --all-features` zero warnings - 31 / 31 new tests passing; no existing tests broken Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine-v2): per-project sandbox — Phases 2–7 + live Docker e2e test Completes the per-project sandbox plan (docs/plans/2026-04-10-engine-v2-sandbox.md Phases 2–7), building on Phase 1's mount-backend abstraction (#2211). Phase 2 — Project workspace folder: - `Project.workspace_path: Option<PathBuf>` field + `with_workspace_path()` - Host-side `project_workspace_path()`, `ensure_project_workspace_dir()` (creates `~/.ironclaw/projects/<id>/` mode 0700, idempotent) - `FilesystemMountFactory` taking a `ProjectPathResolver` closure (decoupled from `Store`); wired into `EffectBridgeAdapter` via `set_workspace_mounts()` Phase 3 — Standalone daemon binary: - `src/bin/sandbox_daemon.rs` — NDJSON over stdin/stdout, health/shutdown/execute_tool - Constructs ReadFileTool/WriteFileTool/ListDirTool/ApplyPatchTool/ShellTool with `base_dir=/project` (override via `IRONCLAW_SANDBOX_BASE_DIR`) Phase 4 — Dockerfile.sandbox: - Multi-stage build: rust-slim builder (+ python3 for pyo3) compiles sandbox_daemon; debian-slim runtime with tini PID 1, common build tools, `/project` mount target Phase 5 — ProjectSandboxManager + ContainerizedFilesystemBackend: - protocol.rs: Request/Response/RpcError matching daemon wire format - transport.rs: `SandboxTransport` trait (seam for testing without Docker) - containerized_backend.rs: `ContainerizedFilesystemBackend` impls `MountBackend`, translates relative→`/project/<rel>`, maps tool-error→MountError - docker_transport.rs: real bollard exec session, serialized Mutex, lazy reconnect - lifecycle.rs: deterministic `ironclaw-sandbox-<pid>` naming, ensure_running/stop/remove - manager.rs: `ProjectSandboxManager` per-project transport cache Phase 6 — Router gating on ENGINE_V2_SANDBOX: - `engine_v2_sandbox_enabled()` helper (truthy: 1/true/yes/on) - Router selects `ContainerizedMountFactory` when enabled + Docker reachable; falls back to `FilesystemMountFactory` with warning otherwise Live e2e bugs caught and fixed: - Shell without explicit `workdir` defaulted to host (not sandbox); fixed by defaulting to `/project/` in `extract_path_param` - `ContainerizedFilesystemBackend::shell` parsed `stdout`/`stderr` but host ShellTool returns merged `output` field; fixed with fallback key lookup - SANDBOX_TOOL_NAMES only had v2 names (`file_read`/`file_write`) but host registry uses v1 names (`read_file`/`write_file`); added both aliases Tests (62 sandbox-related, all green): - 27 bridge sandbox unit tests (intercept, workspace_path, factory, protocol, lifecycle, containerized_backend with ScriptedTransport mock) - 7 containerized-backend tests (including 2 regression tests for the shell bugs) - 5 engine v2 sandbox integration tests (EffectBridgeAdapter end-to-end) - 5 daemon binary smoke tests (real subprocess + NDJSON I/O) - 17 engine workspace unit tests - 1 live Docker e2e test: agent clones nearai/ironclaw into sandbox, renames to megaclaw via sed, verifies with grep — 70s, $0.09, recorded trace committed Verification: - `cargo fmt --check` clean - `cargo clippy --all --benches --tests --examples --all-features` zero warnings - All 62 sandbox tests passing; no existing tests broken Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace .expect() with Result in DockerTransport::ensure_session CI's no-panics checker flagged the .expect("just inserted") in production code. Replace with .ok_or_else() returning MountError::Backend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: multi-tenant project paths + unify sandbox env var with v1 Two issues addressed: 1. Project workspace paths now namespace by user_id: `~/.ironclaw/projects/<user_id>/<project_id>/` instead of `~/.ironclaw/projects/<project_id>/`. Prevents filesystem collisions in multi-tenant deployments where two users could theoretically have the same project UUID. 2. Sandbox enablement now reads `SANDBOX_ENABLED` (same env var as v1 sandbox) in addition to `ENGINE_V2_SANDBOX`. Either being truthy enables the per-project sandbox. This means a single flag governs sandbox behavior regardless of engine version, while the v2-specific override remains available for transitional setups. Tests: 30 bridge sandbox unit tests passing (added multi-tenant path tests + env var combination tests). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review — TOCTOU race, shell env passthrough, canonicalize guard Three issues flagged by the code review bot on #2211: 1. TOCTOU race in WorkspaceMounts::resolve (HIGH): Added double-checked locking — re-check the cache after acquiring the write lock so two threads racing on the same project's first access don't both call factory.build(). The second thread finds the insert from the first. 2. Shell intercept ignores env parameter (MEDIUM): The shell arm in maybe_intercept was passing HashMap::new() instead of forwarding the tool call's env map. Fixed to parse parameters["env"] and pass it through to backend.shell(). 3. Canonicalization fails when root doesn't exist (MEDIUM): When self.root hasn't been created yet (first write to a new project), canonicalize_under_root would walk up to a real ancestor and the starts_with check against the non-existent root would always fail. Now skips canonicalization entirely when root doesn't exist — lexical safety is already guaranteed by safe_join. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review round 2 — apply_patch schema, content validation, dir perms, docs - Fix apply_patch schema mismatch: MountBackend::patch now takes (old_string, new_string, replace_all) matching ApplyPatchTool's actual contract. Previously sent {patch: diff} which would fail with invalid_params in the containerized daemon. - Validate file_write content param: return error instead of silently writing empty string when content is missing. - Log stderr frames from sandbox daemon at debug! instead of silently discarding them in docker_transport StreamReader. - Tighten permissions on intermediate directories created by ensure_project_workspace_dir (projects/, <user_id>/) to 0o700, not just the leaf. - Fix stale module doc in sandbox/mod.rs (referenced "Phase 5 will add" but all phases shipped). - Fix doc path mismatch: workspace path is <user_id>/<project_id>/, not <project_id>/ (workspace_path.rs, CLAUDE.md, design plan). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review round 3 — symlink safety, visibility, debug logging - Close TOCTOU window in canonicalize_under_root: re-canonicalize and verify containment when the reassembled path exists on disk - Fix list_dir_recursive: use symlink_metadata (lstat) so symlinks are detected instead of followed; validate directories against root before recursive traversal - Tighten is_mountable_path to /project/, /memory/, /home/ prefixes instead of any absolute path (defense-in-depth) - Narrow sandbox module visibility to pub(crate) and remove unused pub use re-exports - Remove concrete types (FilesystemBackend, DirEntry, EntryKind, ShellOutput) from engine crate top-level re-exports; access via ironclaw_engine::workspace:: module path - Add debug! tracing to sandbox intercept routing decisions - Add read_file/write_file v1 aliases to daemon SUPPORTED_TOOLS health response - Remove developer-local path from sandbox mod.rs doc comment - Merge staging to fix CI (user_timezone field on ThreadExecutionContext) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review round 4 — safety validation, network isolation, binary writes - Add pre-intercept safety param validation so sandbox-dispatched calls go through the same checks as host-dispatched calls (#1) - Set network_mode: "none" on sandbox containers to prevent outbound network access (#3) - Reject binary content in containerized write instead of silently corrupting via from_utf8_lossy (#5) - Cap list_dir depth to 10 to prevent unbounded traversal (#8) - Change container creation log from info! to debug! to avoid breaking REPL/TUI output (#10) - Make is_truthy case-insensitive so SANDBOX_ENABLED=True works (#11) - Return error instead of unwrap_or_default for missing container ID (#12) - Propagate set_permissions errors instead of silently ignoring (#13) - Return error for missing daemon output key instead of defaulting to empty object (#14) - Add env mutex guard in sandbox_live_e2e test (#15) - Fix rustfmt formatting for let-chain in canonicalize_under_root Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review round 5 — path traversal, error types, tests Security fixes: - Sanitize user_id in workspace path to prevent directory traversal via malicious user IDs containing `..` or `/` - Add Component::ParentDir check in ContainerizedFilesystemBackend::container_path matching the defense-in-depth approach of FilesystemBackend::safe_join Correctness: - Use MountError::Tool instead of MountError::InvalidPath for missing tool parameters (content, old_string, new_string) — fixes confusing LLM-visible error messages - Fix clippy sort_by_key suggestion in registry.rs Cleanup: - Remove spurious Notify import and dead _notify_link function New tests: - ContainerizedFilesystemBackend path traversal rejection (read + write) - container_path unit tests for safe and unsafe paths - Adversarial user_id test in workspace_path - Daemon-side path traversal test in sandbox_daemon_smoke Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review round 6 — param normalization, error types, edge cases - Normalize sandbox params via prepare_tool_params() before validation, matching the host execution path (fixes inconsistent validation) - Return ToolError::InvalidParameters instead of EngineError::Effect for sandbox param validation failures (consistent error surface) - ensure_dir checks path.is_dir() not path.exists() (rejects files) - Empty user_id returns "_anonymous" sentinel instead of empty hex string that would drop the tenant namespace via PathBuf::join("") - Restore ENGINE_V2_SANDBOX env var after sandbox live E2E test - Tighten is_mountable_path to /project/ only (no mounts for /memory/ or /home/ yet) - Add v1 tool name aliases (read_file, write_file) to SUPPORTED_TOOLS Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: unify sandbox env var — remove ENGINE_V2_SANDBOX, use SANDBOX_ENABLED only Single env var controls sandboxing for both engine versions. The transitional ENGINE_V2_SANDBOX override is removed from code, tests, docs, and Dockerfile. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: double-checked locking in transport_for, explicit stdin close in smoke test - ProjectSandboxManager::transport_for no longer holds the mutex across the Docker ensure_running await. Uses double-checked locking so concurrent projects initialize in parallel. - sandbox_daemon_smoke: explicitly take() stdin before wait_with_output so EOF is sent even without a shutdown request. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review — network mode, error types, race, protocol dedup - Change sandbox container network_mode from "none" to default bridge so git clone / cargo build / pip install work inside the container - Fix binary content rejection to use MountError::Tool instead of MountError::InvalidPath (semantic mismatch) - Fix list depth: use actual depth value instead of depth.max(1) - Fix orphan container race in transport_for by holding lock across container creation instead of double-checked locking - Deduplicate protocol types: daemon now imports from shared bridge::sandbox::protocol instead of defining its own copies - Make bridge::sandbox pub (narrow exposure: only protocol and workspace_path sub-modules are pub) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update plan doc — sandbox uses bridge networking, not network_mode=none Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
77c3821f33 |
feat(common): apply ExtensionName newtype to fan-out sites (PR 2/2) (#2617)
* feat(common): add CredentialName and ExtensionName newtypes Introduce typed identifiers for the backend-secret vs user-facing extension identity split that the Extension/Auth Invariants section of CLAUDE.md describes. Four recent PRs (#2561, #2473, #2512, #2574) have been identity- confusion bugs with the same shape: a stringly-typed value passed through multiple layers with each layer meaning a different thing. Newtypes make each of those a compile error. This is PR 1 of 2. PR 1 lands the newtypes and migrates the core auth seam (ResumeKind::Authentication, MissingCredential, ToolReadiness::NeedsAuth, LatentActionExecution::NeedsAuth, extensions/naming.rs). PR 2 will migrate AppEvent.extension_name, OAuth/pending-flow stores, TUI events, and the remaining extension_name: String fields. Wire format is unchanged — both newtypes use #[serde(transparent)] so on- wire and on-disk representations stay plain strings and legacy persisted rows keep deserializing. Validation runs at explicit construction (::new / ::try_from / ::from_str), not at deserialize time. Also adds .claude/rules/types.md codifying the "no stringly-typed internals" rule. Regression coverage: 17 new unit tests in identity.rs; existing auth_manager, router, and gate tests (130+ cases) all pass unchanged. * fix(common): address PR #2611 review feedback Four fixes from Copilot, Gemini, and Claude reviews: - **identity.rs docs**: drop reference to a non-existent `validate()` re-validation API. Document that instances represent "passed validation at some point in history" rather than "guaranteed valid right now" — by design. - **effect_adapter.rs**: the `awaiting_authorization` / `awaiting_token` gate path was using `CredentialName::from_trusted` to wrap a value read straight out of a tool's JSON output. Tool output is external/untrusted; use `CredentialName::new` (validating) with a cascade: external → tool name → `from_trusted(tool_name)` as final fallback. Closes a credential-name shape-injection vector. - **canonicalize()**: reorder checks cheapest-first against the trimmed slice so invalid inputs reject without allocating a canonicalized `String`. `replace('-', "_")` is deferred until after the structural checks pass; since `-`/`_` are both one byte, the earlier length check stays valid. - **Remove `Deref<Target = str>`** from identity newtypes, keep `AsRef<str>`. Auto-deref let `&cred_name` silently coerce to `&str`, which is exactly the implicit-conversion pattern these newtypes exist to prevent. Callers that had a `&CredentialName` where `&str` was expected now write `.as_str()` explicitly. Added a regression test for the accessor contract and updated the rule template in `.claude/rules/types.md` to document the decision. Declined one review item (Claude): the remaining `to_string()` calls inside `IdentityError` variants are on the exception path; the common invalid-input case no longer allocates twice after the canonicalize reorder, and errors must carry owned strings so they can escape the function. Regression coverage: 5035 lib tests + 18 identity tests (one new — `explicit_accessors_work`) pass. Zero clippy warnings. * feat(common): apply ExtensionName newtype to fan-out sites (PR 2/2) Follow-up to #2611. Migrates the remaining stringly-typed extension_name and credential_name fields to use the ExtensionName and CredentialName newtypes introduced in ironclaw_common::identity. Fields now typed: - AppEvent::{OnboardingState, GateRequired, ExtensionStatus}.extension_name (serde transparent — wire format unchanged) - StatusUpdate::{AuthRequired, AuthCompleted}.extension_name - TuiEvent::{AuthRequired, AuthCompleted}.extension_name (adds ironclaw_common dep to ironclaw_tui) - PendingOAuthLaunchParams.extension_name - PendingOAuthFlow.extension_name - PendingAuth.extension_name, PendingAuthPrompt.extension_name - ParsedAuthData.extension_name, selected_auth_prompt tuple - emit_auth_required_status() and Session::enter_auth_mode() parameters - event_from_configure_result() parameter - resolve_extension_for_action() and resolve_auth_gate_display_name() return types - normalize_extension_name() return type PendingAuthPrompt::new is now infallible (accepts ExtensionName directly) since the identity validator carries the non-empty invariant the constructor used to re-check. The "blank extension name" rejection test moved out — that logic lives in ironclaw_common::identity tests. Test updates use `ExtensionName::new("...").unwrap()` at construction sites and `from_trusted(...)` where a trusted upstream string is being adapted. Every site is a compile-time audit of where the type was crossing a boundary untyped. Regression coverage: existing 5034 lib tests + 26 engine_v2_gate integration tests + 40 ironclaw_common tests all pass. Zero clippy warnings across all features. * fix(web): return ExtensionName from pending_gate_extension_name Addresses Claude's review comment on #2611: the function was doing `Some(credential_name.as_str().to_string())` in the fallback branch, defeating the newtype's purpose by re-stringifying the identity. Return `Option<ExtensionName>` instead. Plumbs through `PendingGateInfo. extension_name` (wire format unchanged — `#[serde(transparent)]`). The fallback path's cross-identity conversion (credential name → extension name) is now an explicit `ExtensionName::from_trusted` call, making the boundary crossing visible at the call site. Also fixes the `Deref<Target = str>` removal fallout that followed the rebase onto the updated PR 1: call sites that relied on auto-deref (`ext.contains(...)`, `auth_manager.submit_auth_token(&cred_name, ...)`) now explicitly call `.as_str()`. * fix(router,web): address PR #2617 review feedback Four Gemini review comments, all on the boundary between credential/ extension identifiers and user input. 1. [HIGH, security] extensions_setup_submit_handler was wrapping the URL path segment in ExtensionName::from_trusted, which skips the newtype's path-traversal / invalid-character validation. That path is user-controlled (`/api/extensions/{name}/setup`). Validate with ExtensionName::new at the handler entry and return 400 on failure; downstream uses switch to .as_str() or .clone() of the validated value, and the three in-handler from_trusted sites disappear. 2. Rename resolve_auth_gate_display_name -> resolve_auth_gate_extension_name. The function returns an identifier/slug, not a human-readable display name — the old name was a leftover from when the value was a String. 3. Return Option<ExtensionName> from the renamed function. Previously the non-Authentication gate branch fabricated an ExtensionName::from_trusted(pending.action_name), which was semantically wrong (an action name is not an extension identifier) and silently defeated the type's invariants. Now it returns None for Approval/External gates, and callers thread an Option through. send_pending_gate_status accepts Option<&ExtensionName> and only uses it on the Authentication arm, with a warn! log if upstream plumbing ever reaches the arm with None. The GateRequired SSE event's extension_name is now a clean .clone() of the Option. 4. Rename auth_display_name -> extension_name on send_pending_gate_status so the parameter name matches both its type and the StatusUpdate::AuthRequired.extension_name field it feeds. Regression: new test_extensions_setup_submit_rejects_path_traversal_name at the handler tier (per .claude/rules/testing.md "Test Through the Caller, Not Just the Helper") drives the handler with malformed path segments and asserts 400 before the value reaches extension lookup or any from_trusted wrap. 5035 lib tests pass, zero clippy warnings. * docs(identity): codify web-boundary rules + add static check Three rule additions + one enforcement hook covering the identity boundary that PR #2617 review uncovered: - src/channels/web/CLAUDE.md — extend "Unified Extension Onboarding" with explicit rules: * Setup/configure/activate routes MUST validate `{name}` via `ExtensionName::new` at handler entry (return 400 on failure). * Web DTOs and handlers MUST NOT reference `CredentialName` — credential identity is backend-only; the dispatcher/auth_manager resolves it from the ExtensionName server-side. * Auth-flow extension resolution happens in *one* place (`AuthManager::resolve_extension_name_for_auth_flow`). Wrappers are thin and delegate; they must not duplicate the precedence logic or re-derive from credential prefixes. The four recent identity bugs (#2561, #2473, #2512, #2574) were duplicate- resolution drift. - src/bridge/CLAUDE.md — new module spec documenting auth_manager.rs as the single authority for auth-flow extension resolution, with the resolver's four-step precedence order and the approved wrapper call sites. - scripts/pre-commit-safety.sh — new check #8 (CREDNAME): flags `CredentialName` references in newly-added production lines under `src/channels/web/**`. Test-mod code is excluded via the existing `strip_test_mod_lines` filter. Suppression via `// web-identity-exempt: <reason>` for the rare legitimate case of reading an already-typed value off a backend struct. Smoke-tested: * baseline (current branch) — no warnings * injected violation — fires with CREDNAME warning * injected violation + `// web-identity-exempt:` — suppressed The rules and the check live at the same level — humans read the rule, CI enforces it. * fix(auth): validate user-influenced names at the resolver boundary Addresses four Copilot review comments on PR #2617 that all pointed at the same seam: the canonical `AuthManager::resolve_extension_name_for_auth_flow` returned a raw `String` whose first branch (the LLM-supplied `name` parameter on `tool_install` / `tool_activate` / `tool_auth` actions) passed through without `ExtensionName` validation. Both call sites then wrapped the result in `ExtensionName::from_trusted`, promoting an unvalidated user-influenced value to a typed identity. - **Resolver now returns `ExtensionName`.** Branch 1 validates the user-controlled name via `ExtensionName::new` and falls through on failure; branches 2–4 use `from_trusted` because their sources (tool registry hint, canonicalizer, typed credential fallback) are already trusted upstream. This consolidates validation in the single "resolve once" site documented in `src/bridge/CLAUDE.md`. - **router.rs and server.rs drop their wraps.** `resolve_extension_for_action` (router) and `pending_gate_extension_name` (server) return the resolver's typed output directly. The tool-registry fallback in router.rs (no-auth-manager path) keeps its `from_trusted` wrap since it operates on the same trusted sources as branch 2. - **`restore_selected_auth_prompt` re-validates rehydrated prompts.** `PendingAuthPrompt` is `#[serde(transparent)]`, so deserialize does not re-check the inner `ExtensionName` string. A legacy-persisted invalid name would previously have been dropped by the old `PendingAuthPrompt::new(String, ...)` empty-string rejection; now `restore_selected_auth_prompt` re-runs `ExtensionName::new` and drops + warns on failure, upgrading the old non-empty-only check to the full identity invariant. New test `test_restore_selected_auth_prompt_rejects_invalid_legacy_row` forges three invalid rows (empty / uppercase / path-traversal) straight through serde and asserts each is dropped. - **Docstring on `PendingAuthPrompt` refreshed.** The old comment claimed `::new` "trims and validates extension_name is non-empty", which is no longer true — `::new` is infallible and the invariant lives in `ExtensionName` itself. The new comment documents the split: validation runs at `ExtensionName::new` construction and at restore-from-persistence, not inside `PendingAuthPrompt`. Regression: 5063 lib tests pass (+1 new). Clippy zero warnings. * fix(ci): adapt post-merge-from-staging sites to ExtensionName Staging shipped #2640 (repl unlock) and gateway refactor commits after my last merge. The CI build picked them up via auto-merge and hit three type mismatches my branch hadn't seen: - src/channels/repl.rs:908 — new test constructs `StatusUpdate::AuthRequired { extension_name: "google_oauth_token" .to_string(), ... }`. Typed field; now `ExtensionName::new(...).unwrap()`. - src/channels/web/server.rs:1405-1424 — staging added a no-auth-manager fallback chain to `pending_gate_extension_name` that returned raw `Some(String)` on three branches. Aligned with `AuthManager::resolve_extension_name_for_auth_flow`: branch 1 (user-influenced `tool_install`/`tool_activate`/`tool_auth` `name` param) validates via `ExtensionName::new` and falls through on failure; branches 2-3 (provider-extension hint, credential-name fallback) use `from_trusted` because they're sourced from typed upstream state. Mirrors the fix applied to the canonical resolver in |
||
|
Illia Polosukhin
|
ff119531d4 |
test(replay): promote engine traces to insta-backed snapshot gate (#2621)
* test(replay): promote engine replay traces to insta-backed snapshot gate Adds a ReplayOutcome snapshot type, a replay-gate CI workflow, and a developer script wrapper for cargo-insta. Replaces unreviewable 3,000-line JSON diffs on engine changes with a YAML snapshot of the observable run shape (tool sequence, final state, retrospective analyzer issues). Why: engine v2 live-fixture traces had grown past reviewability. A single prompt-wording change could move the whole fixture, and reviewers had no way to see which behaviour actually changed. Splitting the fixture into a "replay driver" (JSON stays in tests/fixtures/) and a "regression snapshot" (YAML in tests/snapshots/) gives reviewers a narrow, stable diff to approve, while keeping the full recorded context for deterministic replay. Changes: - `tests/support/replay_outcome.rs` — ReplayOutcome + assert_replay_snapshot! macro; snapshots include retrospective analyzer output (TraceIssue severity/category) via a new `ironclaw::bridge::engine_retrospectives_for_test()` helper that runs `build_trace()` over engine threads - `tests/e2e_engine_v2.rs` — three POC snapshot tests (single_tool_echo, tool_error_recovery, zizmor_scan_v2) - `tests/e2e_bug_bash_snapshots.rs` + `tests/fixtures/llm_traces/bug_bash/` — bug-regression fixture template, mapped to open issues in the README - `.github/workflows/replay-gate.yml` — cargo insta test --check on engine/agent/LLM/tools/bridge path changes; rejects committed .snap.new - `scripts/replay-snap.sh` — review/accept/test/record wrappers around cargo-insta and IRONCLAW_RECORD_TRACE - `scripts/trace-coverage.sh` — reports EventKind variants with snapshot coverage; `--strict` mode for future CI promotion - `tests/e2e_live.rs` — `#[ignore]` swapped for `cfg_attr(not(feature="replay"), ignore)` so the replay CI job can run the scenarios without `-- --ignored` - `Cargo.toml` — new `replay = ["libsql"]` feature; insta gains the `yaml` feature - `tests/fixtures/llm_traces/README.md` — documents the two-role driver/snapshot split Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(replay): address PR #2621 review + swap cargo-insta installer Review fixes: - Replay gate was missing the bug-bash snapshot suite. Adds `tests/e2e_bug_bash_snapshots.rs` to the workflow paths trigger and the `cargo insta test --check` invocation so bug-regression snapshots are actually gated. (copilot-pull-request-reviewer) - `cargo install cargo-insta --locked` added ~40s of cold-cache compile to the gate. Swapped for `taiki-e/install-action@v2`, which downloads a precompiled binary in a few seconds. Also updated `scripts/replay-snap.sh` to *fail closed* when cargo-insta is missing instead of silently auto-installing it. (gemini-code-assist) - `engine_retrospectives_for_test` was `pub` and re-exported under the default-enabled `libsql` feature, contradicting its "not part of any public API" doc. Split the re-export, kept `reset_engine_state` as a plain `pub use`, and hid `engine_retrospectives_for_test` behind `#[doc(hidden)]` — it still needs to cross the crate boundary for integration tests (which live in a separate crate, so `#[cfg(test)]` doesn't reach them), but no longer appears in published docs. (copilot-pull-request-reviewer) - Added an explicit "caller must serialize" note on `engine_retrospectives_for_test` explaining the `ENGINE_STATE` singleton and pointing new callers at `engine_v2_test_lock()` / `reset_engine_state()`. Matches what the existing snapshot tests already do. (gemini-code-assist) Doc corrections: - `snapshot_zizmor_scan_v2` doc claimed the snapshot pinned `ApprovalNeeded` events and response wording — it doesn't. Rewrote to describe what the snapshot actually asserts (tool order, step count, retrospective issues, final state). (copilot-pull-request-reviewer) - `llm_call_count` was documented as "bucketed" but passed through verbatim. Updated the field doc to reflect the raw value. Bucketing wasn't needed because fixtures are deterministic. (copilot-pull-request-reviewer) - `src/bridge/router.rs` doc referenced a non-existent `ReplayOutcome.trace_issues` field — the struct uses `engine_threads`. Fixed the reference. (copilot-pull-request-reviewer) - `scripts/trace-coverage.sh` header claimed CI runs it with `--strict`; the workflow runs it in advisory mode. Rewrote the header to match, with a pointer for when to promote to strict. (copilot-pull-request-reviewer) No-change replies (rationale commented in the code): - `event_kind_name` uses an exhaustive `match` on `EventKind` rather than `Debug` or a `strum` derive. The compile-time exhaustiveness check is the point — adding a new engine event should force a conscious decision about how the snapshot represents it, not a silent fallthrough. Added a comment making that intent explicit. - `trace-coverage.sh` awk parser of `event.rs` is fragile — agreed, but the script is advisory and its failure mode is false negatives (uncovered variants simply aren't gated). Documented the tradeoff and the rewrite-in-Rust escape hatch in the script header. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(replay-gate): prime cache on staging, restrict PR runs to read-only The second run on PR #2621 missed the cache ("No cache found" in the rust-cache restore step) even though the workflow is wired correctly. Root cause: the repo sits close to GitHub's 10 GB per-repo cache quota (~59 entries, many >500 MB), and the LRU policy evicts PR-scoped caches before they get reused. Fix: - Add `push: [staging, main]` so the gate runs (and saves a ~1.2 GB cache under the `replay-gate` key) on every merge to the branches PRs actually target. Subsequent PRs restore from that base-branch cache — GitHub Actions permits cross-ref restore when the restoring ref's base matches the saved ref. - Set `save-if: ${{ github.event_name == 'push' }}` so PR runs only *read* the cache. Without this gate, each PR push would save its own copy and crowd out the primed base-branch cache, putting us right back in the eviction loop. Expected effect: cold-cache 9m → warm ~2-3m once staging has a run with the new workflow. Base-branch prime run still pays 9m (no regression). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(replay): drop bug-bash fixture scaffolding Replay fixtures can't reproduce the Phase 3 target bugs because the fixture *is* the LLM's output — handwriting a trace where the LLM emits a tool call doesn't test whether the real LLM would have emitted that call, only that the harness dispatches a scripted one. What `summarization_uses_tools.json` actually pinned was the happy path, not the #2541 bug. Of the 7 open bug-bash issues, only #2544 ("plans and delegates but never executes") is catchable by replay, and only via a live-recorded fixture. The other six are LLM-behavior or infra-timing bugs outside replay's reach. Rather than ship regression theater, tear out the scaffolding. Removed: - tests/e2e_bug_bash_snapshots.rs - tests/fixtures/llm_traces/bug_bash/ - tests/snapshots/replay__bug_bash_summarization_uses_tools.snap Unwired: - Replay-gate workflow paths + test list no longer mention bug_bash - scripts/replay-snap.sh test command drops the extra --test flag Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: switch to cargo-nextest with per-test timeouts Nextest runs each integration test in its own process and runs test binaries in parallel, which is a big unlock for this repo: - Engine v2 tests share a process-global `ENGINE_STATE` singleton (OnceLock), which the current test lock serialises inside a single test binary. Nextest's process-per-test model gives each test a clean state automatically, so the 16 engine_v2 tests stop running one-by-one. - Cross-binary parallelism: `cargo test --test A --test B` runs binaries in sequence; nextest runs them concurrently. Measured locally: the replay-gate test set (3 binaries, 21 tests) went from ~30s sequential to **2.7s parallel**. Adds `.config/nextest.toml` with: - `slow-timeout = 60s / terminate-after 3` in the default profile so a hung test fails fast instead of blocking the workflow-level 25- minute cap. - A `ci` profile with `fail-fast = false` (one flake shouldn't mask other failures), `failure-output = immediate-final`, `success-output = never` for readable Actions logs. - Per-test 300s override for the handful of genuinely slow scenarios (zizmor scan, e2e_thread_scheduling). Workflows updated: - `replay-gate.yml`: installs cargo-nextest via taiki-e/install-action alongside cargo-insta (one step), runs `cargo insta test --test-runner nextest` with `NEXTEST_PROFILE=ci`. - `test.yml`: all five `cargo test` invocations swapped for `cargo nextest run --profile ci`. Nextest doesn't execute doctests, so every nextest step is paired with a `cargo test --doc` follow-up to preserve coverage. Local dev is unchanged — `cargo test` still works; nextest is only required in CI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: re-trigger replay-gate workflow after nextest migration Previous push only modified workflow files and `.config/nextest.toml`; GitHub skipped the `pull_request` workflow events for that sync, so the nextest migration didn't actually get exercised in CI. Empty commit forces re-evaluation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(replay): note nextest wiring in the fixtures README Also forces a CI re-run: the previous empty commit had no matching paths, so the `pull_request.paths` filters skipped every workflow including replay-gate. Touching a file under `tests/fixtures/llm_traces/**` re-matches the filter and runs the nextest-based gate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(test): defer test.yml nextest migration Staging restructured test.yml significantly while this PR was open (matrix-config dynamic matrix, `changes` code-detection job, composite install-cargo-component action, save-if restricted to base-branch pushes). The merge into staging had heavy conflicts for every nextest-swap hunk. Rather than force a re-layering of the new staging structure on top of the nextest migration in this PR, revert test.yml to staging's current version. This PR now scopes the nextest change to just the replay-gate workflow (where it cleanly demonstrates the value) plus the shared `.config/nextest.toml` profile. Migrating the rest of test.yml to nextest is a follow-up that can rebase on the new structure without the heavy conflict surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Henry Park <henrypark133@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
9fee70906e |
feat(common): CredentialName + ExtensionName newtypes (PR 1/2) (#2611)
* feat(common): add CredentialName and ExtensionName newtypes Introduce typed identifiers for the backend-secret vs user-facing extension identity split that the Extension/Auth Invariants section of CLAUDE.md describes. Four recent PRs (#2561, #2473, #2512, #2574) have been identity- confusion bugs with the same shape: a stringly-typed value passed through multiple layers with each layer meaning a different thing. Newtypes make each of those a compile error. This is PR 1 of 2. PR 1 lands the newtypes and migrates the core auth seam (ResumeKind::Authentication, MissingCredential, ToolReadiness::NeedsAuth, LatentActionExecution::NeedsAuth, extensions/naming.rs). PR 2 will migrate AppEvent.extension_name, OAuth/pending-flow stores, TUI events, and the remaining extension_name: String fields. Wire format is unchanged — both newtypes use #[serde(transparent)] so on- wire and on-disk representations stay plain strings and legacy persisted rows keep deserializing. Validation runs at explicit construction (::new / ::try_from / ::from_str), not at deserialize time. Also adds .claude/rules/types.md codifying the "no stringly-typed internals" rule. Regression coverage: 17 new unit tests in identity.rs; existing auth_manager, router, and gate tests (130+ cases) all pass unchanged. * fix(common): address PR #2611 review feedback Four fixes from Copilot, Gemini, and Claude reviews: - **identity.rs docs**: drop reference to a non-existent `validate()` re-validation API. Document that instances represent "passed validation at some point in history" rather than "guaranteed valid right now" — by design. - **effect_adapter.rs**: the `awaiting_authorization` / `awaiting_token` gate path was using `CredentialName::from_trusted` to wrap a value read straight out of a tool's JSON output. Tool output is external/untrusted; use `CredentialName::new` (validating) with a cascade: external → tool name → `from_trusted(tool_name)` as final fallback. Closes a credential-name shape-injection vector. - **canonicalize()**: reorder checks cheapest-first against the trimmed slice so invalid inputs reject without allocating a canonicalized `String`. `replace('-', "_")` is deferred until after the structural checks pass; since `-`/`_` are both one byte, the earlier length check stays valid. - **Remove `Deref<Target = str>`** from identity newtypes, keep `AsRef<str>`. Auto-deref let `&cred_name` silently coerce to `&str`, which is exactly the implicit-conversion pattern these newtypes exist to prevent. Callers that had a `&CredentialName` where `&str` was expected now write `.as_str()` explicitly. Added a regression test for the accessor contract and updated the rule template in `.claude/rules/types.md` to document the decision. Declined one review item (Claude): the remaining `to_string()` calls inside `IdentityError` variants are on the exception path; the common invalid-input case no longer allocates twice after the canonicalize reorder, and errors must carry owned strings so they can escape the function. Regression coverage: 5035 lib tests + 18 identity tests (one new — `explicit_accessors_work`) pass. Zero clippy warnings. |
||
|
standardtoaster
|
4353493a97 |
fix(gateway): resolve assistant thread for threadless broadcasts (#2444)
* fix(gateway): resolve assistant thread for threadless broadcasts Mission notifications, self-repair alerts, and extension activation messages broadcast via channels without a thread_id. The gateway's broadcast() rejected these with MissingRoutingTarget, silently dropping the messages. Two fixes: 1. Mission notification now chains .in_thread() — the thread_id was already available on MissionNotification but not being passed through. 2. Gateway broadcast() falls back to the user's assistant conversation when thread_id is None and a DB store is available. This routes threadless messages (self-repair, extension activation) to a known thread instead of rejecting them. When no store is available, the original MissingRoutingTarget error is preserved. Fixes #2405 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: don't leak owner thread_id to notify_user in mission broadcasts When notify_user differs from the mission owner, omit .in_thread() so the gateway's broadcast() fallback resolves the recipient's own assistant thread instead of attaching the owner's thread_id. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: verify broadcast thread_id resolution and cross-user guard Address review feedback on #2444: 1. Fallback test now subscribes to SSE, verifies the emitted thread_id matches the DB assistant conversation UUID, and confirms the row exists. 2. Three new caller-level tests for the cross-user guard in handle_mission_notification: - cross-user: notify_user != user_id -> owner's thread_id is NOT attached, recipient gets their own assistant thread - same-user: notify_user is None -> owner's mission thread_id IS attached to the broadcast - explicit same-user: notify_user = Some(user_id) -> guard still matches, thread_id is attached (catches is_none() refactors) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix rustfmt in cross-user guard tests Collapse handle_mission_notification call sites to single-line form to satisfy cargo fmt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: ignore RUSTSEC-2026-0098 and bump rustls-webpki 0.103.12 RUSTSEC-2026-0098 (URI name constraint bypass in rustls-webpki) affects 0.102.8, which is pinned by the libsql 0.6.0 transitive dependency chain (libsql -> rustls 0.22 -> rustls-webpki 0.102.x). The fix (>=0.103.12) is only available on the 0.103.x line, so the 0.102.8 instance cannot be upgraded without a libsql major bump. Add the advisory to deny.toml ignore list (same rationale as the existing RUSTSEC-2026-0049 exception for the same crate/version). Also bump rustls-webpki 0.103.10 -> 0.103.12 for the non-pinned instance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: serrrfirat <f@nuff.tech> |
||
|
Zaki Manian
|
160a75e38c |
fix(llm): image detail field + /v1 base URL normalization (#2380)
* fix(llm): add image detail field, auto-append /v1 to base URL (#2378, #1934) Set detail: "auto" on ImageUrl construction so providers requiring the field (e.g. MiniMax) no longer reject vision requests. Normalize OpenAI-compatible base URLs by appending /v1 when missing, fixing 404s for local model servers (MLX, vLLM, llama.cpp) using bare URLs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(llm): scope /v1 normalization to bare host-only URLs Address review feedback: - Only append /v1 when the URL has no path component (bare scheme://host[:port]). URLs with existing paths like Zai's /api/paas/v4 or Gemini's /v1beta/openai are now left unchanged. - Use case-insensitive check for /v1 suffix to prevent double-suffixing URLs like http://localhost:8080/V1. - Document why Ollama is intentionally excluded from normalization (uses /api/chat, not /v1/chat/completions). - Add test cases for real provider URLs from providers.json (Zai, Gemini) and case-insensitive /V1. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update gimli 0.33.1 -> 0.33.0 (yanked crate) gimli v0.33.1 was yanked on crates.io, causing cargo-deny to fail. Downgrade to v0.33.0 which is the latest non-yanked release compatible with wasmtime 43. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: serrrfirat <f@nuff.tech> |
||
|
Illia Polosukhin
|
66ccafb96b |
chore(engine): update monty to v0.0.11 (#2364)
* chore(engine): update monty to v0.0.11 Bump the embedded Python interpreter (pydantic/monty) from rev 7a0d4b7 to the v0.0.11 release. Notable upstream changes: ~2x faster JSON loads/~1.6x faster dumps, filesystem mounting, Rust-side async API additions, and mount edge case fixes. No Python-level syntax changes, so the CodeAct preamble is unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): correct async/await limitation in MONTY.md `await` and `asyncio.gather()` work for tool calls and llm_query() via Monty's ExternalFuture/ResolveFutures mechanism. Only `async def` (defining custom coroutines) is unsupported. The previous wording incorrectly said async/await was not available at all. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): correct stale limitations in MONTY.md and CodeAct preamble Remove features that actually work in Monty v0.0.11 from the "not supported" lists: - async def / await / asyncio.gather() — fully supported - *expr star unpacking in assignments — fully supported - generator expressions — work (yield statements still don't) Clarify: - class: host-provided dataclasses work, user-defined classes don't - yield: generator expressions work, yield statements don't - os module: available (os.getenv, os.path), not just os.path - asyncio module: available (asyncio.gather) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): clarify os module is blocked, not available import os succeeds in Monty but the executor blocks all OsFunction calls (os.getenv, Path.*, os.environ) with OSError. Document this explicitly and remove os from the available modules list. Agents must use injected tools (shell, read_file, etc.) for OS operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: retrigger with regression check skip [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Henry Park
|
fdb0a13b91 |
chore: sync staging and main (#2337)
* [codex] Label migration PRs with DB MIGRATION (#1967) * Add DB MIGRATION PR label * Broaden DB MIGRATION label coverage * chore(ci): address DB MIGRATION label review feedback * Fix Telegram UTF-16 message splitting (#1961) * Fix Telegram UTF-16 message splitting * fix: bump telegram channel registry version * chore: bump registry versions for github tool, whatsapp and telegram channels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: undo 2 main-only commits to unblock staging-promote merge (#2297) Reverts: - |
||
|
Illia Polosukhin
|
2cc5546017 |
feat(tools): production-grade coding tools, file history, and skills (#2025)
* feat(tools): add production-grade coding tools, file history, and coding skills Add dedicated coding tools inspired by Claude Code's architecture to make IronClaw a more effective coding assistant: New tools: - GlobTool: fast file pattern matching via `glob` crate, sorted by mtime, with default exclusions (.git, node_modules, target, etc.) - GrepTool: content search wrapping ripgrep with 3 output modes (content, files_with_matches, count), pagination, and context lines - FileUndoTool: restore files to pre-modification state using in-memory file history snapshots Enhanced tools: - ReadFileTool: 10MB limit, 2000-line default, binary detection, device path blocking (/dev/zero, /proc/*/fd/*) - ApplyPatchTool: uniqueness validation (error on ambiguous matches), workspace path rejection, 10MB size limit, file history integration - WriteFileTool: file history integration for undo support Updated tool descriptions to guide LLM behavior (prefer apply_patch over write_file, always read before editing, use glob/grep instead of shell). New skills: - coding: best practices for code editing, search, and file operations - commit: git commit message generation workflow - review: code review workflow with structured checklist Shared infrastructure: - DEFAULT_EXCLUDED_DIRS constant in path_utils.rs - FileHistory module with SharedFileHistory for cross-tool snapshots 66 new tests covering all tools, edge cases, and regression scenarios. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): address PR review — security, correctness, and robustness fixes - Move device path blocking after validate_path() to prevent traversal bypass - Add /proc/kcore, /proc/kmem to blocked paths - Reject absolute patterns and '..' in glob tool, add strip_prefix defense - Wrap glob sync I/O in spawn_blocking to avoid blocking tokio executor - Sort files_with_matches globally before pagination in grep tool - Add default exclusions for node_modules/target in grep tool - Inject ctx.extra_env into rg environment matching ShellTool policy - Use per-line strip_prefix for content mode path relativization - Change FileSnapshot.content_before to Vec<u8> for binary file support - Log snapshot errors with tracing::debug instead of silently discarding - Fix skill name mismatch: code-review → review to match directory Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(skills): rename review skill directory to code-review Aligns the directory name with the manifest name (code-review) to prevent incorrect override/dedup behavior in the bundled-skill loader. The name stays "code-review" since other domains may also need review-type skills. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tools): add file edit guards — staleness detection, fuzzy matching, encoding preservation Add file_edit_guard module with production-grade safeguards for file editing: - ReadFileState tracks file reads with mtime for staleness detection - 4-level fuzzy matching fallback (exact → whitespace-normalized → quote-normalized → both) - UTF-16LE BOM detection and line ending style preservation (LF/CRLF/CR) - Read-before-edit enforcement for ApplyPatch and WriteFile tools - No-op edit rejection (old_string == new_string) - Shared state injection via Arc<RwLock<>> across ReadFile, WriteFile, ApplyPatch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): address all PR review comments — session scoping, parallelism, security - Session-scoped state: ReadFileState and FileHistory now keyed by job_id so concurrent sessions sharing the same registry don't leak state (#2025) - Parallel metadata: grep files_with_matches uses JoinSet (max 64 concurrency) instead of sequential await per file for mtime sorting - Shared env allowlist: grep_tool imports SAFE_ENV_VARS from shell.rs (made pub(crate)) instead of maintaining a divergent copy - Glob traversal: uses Component::ParentDir check instead of substring ".." match, so patterns like "foo..bar" are no longer falsely rejected - UTF-16LE in read_file: binary detection skips null-byte check for files with UTF-16LE BOM; read_file uses encoding-aware read path - Partial flag: default 2000-line truncation now marks read as partial, preventing edits against unseen content - write_file guard softened: staleness check logs warning instead of hard error (full-file replacement has lower risk than apply_patch) - Updated e2e trace to include read_file before apply_patch - Updated expected tool list in schema validation tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): use async metadata instead of blocking path.exists() in write_file Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): fix false-positive panic detection for lifetimes in char lexer The check_no_panics.py lexer misinterpreted Rust lifetimes ('static) as char literal starts, causing in_char state to persist across lines and hide all subsequent brace-delimited blocks — including #[cfg(test)] mod tests. Reset in_char at line boundaries since Rust char literals cannot span lines. https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC * test: verify MCP push works * test * chore: remove test file * style: apply cargo fmt to file.rs Collapse multi-line method chain to single line per rustfmt. https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC * style: apply cargo fmt to file.rs Collapse multi-line method chain to single line per rustfmt. https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC * fix(file-tools): harden fuzzy patch matching and undo * fix(ci): formatting + wasmtime 43 cache config compatibility After merging latest staging, cargo fmt had diffs in file tools and the wasmtime cache TOML format changed (v43 dropped the `enabled` field under `[cache]`). Also removes accidental .fmt-test artifact. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(file-tools): simplify strip_trailing_whitespace Remove redundant double-pass through .lines() — the first collect+join was a no-op since .lines() already handles line endings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): address PR review comments — security, correctness, tests - Add is_sensitive_path checks to GlobTool and GrepTool, matching the defense-in-depth posture of ReadFileTool/WriteFileTool/ListDirTool - Fix UTF-8 panicking byte-index slice in apply_patch error preview (old_string[..200] → chars().take(200)) - Add 10MB size guard on file_history snapshots to prevent memory exhaustion from snapshotting large files - Replace dead turn_number field with auto-incrementing sequence_number in FileHistory — callers no longer pass a hardcoded 0 - Fix glob mtime test flakiness by increasing sleep to 1100ms (above 1s filesystem granularity) - Fix emoji test to actually include emoji/non-ASCII content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Zaki Manian <zaki@iqlusion.io> |
||
|
Illia Polosukhin
|
f37a26f75a |
fix(engine): mission cron scheduling + timezone propagation (#1944) (#1957)
* fix(engine): compute next_fire_at for cron missions (#1944) MissionCadence::Cron missions never fired automatically because next_fire_at was initialized to None and never computed from the cron expression. The ticker checked next_fire_at <= now which was always false. - Add next_cron_fire() helper that parses cron expressions (5/6/7-field) and computes the next fire time, with optional timezone support - Compute next_fire_at in create_mission() for Cron cadence - Advance next_fire_at in fire_mission() after each successful fire - Recompute next_fire_at in resume_mission() for stale cron missions - Add regression tests covering create, fire, tick, and resume flows Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): propagate user timezone to missions and CodeAct scripts (#1944) The LLM had to ask users for timezone because it wasn't available in context. Now: - Add user_timezone to ThreadExecutionContext (from thread metadata) - Store user timezone in thread metadata when received from channel - Auto-inject timezone into mission_create cron cadence from context - Expose user_timezone as a Monty/CodeAct context variable - Document user_timezone in the CodeAct preamble prompt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: introduce ValidTimezone strict type in ironclaw_common (#1944) Address PR review feedback: timezone strings were stored and propagated without validation. Now: - Add ValidTimezone newtype in ironclaw_common that validates IANA timezone strings at construction (parse returns None for empty/invalid) - MissionCadence::Cron.timezone is now Option<ValidTimezone> - ThreadExecutionContext.user_timezone is now Option<ValidTimezone> - Bridge router validates timezone before storing in thread metadata - next_cron_fire() takes Option<&ValidTimezone> — no silent fallback - CodeAct scripting validates on read, falls back to "UTC" for missing - Fix orchestrator doc comment (bridge router, not ConversationManager) - Tighten test assertion to require strictly future next_fire_at - Add ValidTimezone unit tests (parse, serde roundtrip, empty/invalid) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove clone_on_copy for ValidTimezone (clippy) ValidTimezone is Copy, so .clone() is unnecessary. Clippy CI caught this. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: lenient timezone deserialization + bootstrap backfill (#1944) Address second round of PR review feedback: - Add deserialize_option_lenient() in ironclaw_common so persisted missions with invalid timezone strings deserialize as None instead of failing the whole record - Apply lenient deserializer to MissionCadence::Cron.timezone field - Backfill next_fire_at in bootstrap_project() for legacy cron missions that predate the scheduling fix (next_fire_at was None) - Remove unused chrono-tz direct dep from ironclaw_engine (now via ironclaw_common) - Add tests for lenient deserialization (valid, invalid, null, missing) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: collapse nested if in bootstrap_project (clippy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): address PR review — pre-spawn timezone, update_mission scheduling, polish Addresses 6 unanswered review comments on #1957: - (high, serrrfirat) router.rs: user_timezone was set after start_thread, so the executor's in-memory thread never saw it on the first turn. Threaded user_timezone through handle_user_message -> spawn_thread_with_history via a new initial_metadata param so it lands on the thread before the background task starts. source_channel is routed through the same path (had the same latent bug). - (high, serrrfirat / Copilot) update_mission: Manual -> Cron left next_fire_at = None and the mission stayed inert; cron expression edits kept firing on the old schedule. update_mission now recomputes next_fire_at for Cron and clears it for non-cron cadences. - (low, Copilot) parse_cadence: dropped trimmed.contains(' ') so cron expressions with tab/newline separators are detected. - (medium, Copilot) bootstrap_project: backfill save_mission failure now logs at debug! instead of being silently swallowed. - (low, Copilot) codeact_preamble: cron timezone is a default, not automatic — explicit timezone param overrides. Plus self-review polish: - normalize_cron_expression rejects 4-field (and other off-count) input up front with a clear error instead of falling through to the cron parser. - fire_mission has a comment explaining the catch-up semantics (next_fire_at recomputed from now(), missed windows coalesced). - New unit tests for next_cron_fire with an explicit America/New_York timezone, plus 3 update_mission regression tests for Manual->Cron, Cron->Manual, and cron expression change. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): DST/timezone tests, InvalidCadence error, lenient drop log Addresses the latest review on #1957: - Add EngineError::InvalidCadence and switch next_cron_fire to use it. Cron parse failures are validation errors, not store errors — callers can now map them to user-facing messages without misclassification. - Add DST regression tests in types/mission.rs covering the two tricky cases the PR exists to enable: * spring-forward gap (30 2 * * * on a missing-local-hour day must not land in the [02:00, 03:00) window) * fall-back overlap (30 1 * * * on a doubled-hour day must consistently round-trip) * 30-day window straddling spring-forward must contain both 13:00 and 14:00 UTC fires for an "9am NY" schedule Plus normalize_seven_field_cron and an InvalidCadence error path test. - Add a tz-positive test in runtime/mission.rs that creates a cron mission with America/New_York and asserts the resulting next_fire_at lands at UTC 13/14 (NY 09:00) rather than UTC 09 — exercising the full MissionManager → next_cron_fire chain end-to-end. - ironclaw_common::deserialize_option_lenient now logs at debug! when it drops an invalid IANA timezone string to None, so a typo in fresh user config is observable in logs even though the record loads. Adds tracing as a direct dep of ironclaw_common. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): suppress no-panics check on DST test helper The check_no_panics.py CI script has a pre-existing lexer bug where a lifetime apostrophe in this file (`Formatter<'_>` on line 38) puts its char-state lexer into character mode permanently, breaking brace tracking and so failing to detect that the new `schedule_after` test helper lives inside `#[cfg(test)] mod tests`. The script normally only checks added lines so the latent bug is invisible — my new helper exposed it. Use the script's documented `// safety:` per-line escape hatch to suppress the false positives. The helper is unambiguously a test-only helper and the panics are intentional in test context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): log all next_cron_fire failure modes in bootstrap backfill PR review (Copilot): bootstrap_project only patched the legacy mission when next_cron_fire returned Ok(Some(next)). The Ok(None) case (cron expression with no upcoming fire times — e.g. a year-restricted expression in the past) and the Err(_) case (invalid expression) both fell through silently, leaving the mission Active with next_fire_at = None and no log signal — exactly the silent-stuck-mission scenario this PR is supposed to prevent. Match all three branches and emit a debug! log on each path so an operator can see which legacy missions need attention. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): close resume_mission TOCTOU window and stop fire_mission from orphaning threads Two medium-severity issues from PR review (serrrfirat): 1. **resume_mission TOCTOU.** The previous flow was `update_mission_status(Active)` followed by a separate `load + mutate next_fire_at + save` round-trip — leaving an extra interleave window where a concurrent `update_mission`/`fire_mission` write could be silently clobbered by the second save's stale reload. Use the mission already loaded for the ownership check, set `status = Active` and `next_fire_at`, and do a single `save_mission`. 2. **fire_mission orphan thread.** The thread was spawned, then `next_cron_fire(...)?` and `save_mission(...)?` ran. Both could propagate Err *after* the thread was already running, leaving: - no entry in `thread_history` - `threads_today` not incremented (budget bypassed) - no outcome watcher installed Two narrow fixes: - Install `spawn_mission_outcome_watcher` *before* `save_mission`. The watcher only depends on `thread_id` (it joins via ThreadManager and reloads the mission record itself), so a transient store error no longer abandons the running thread. - Replace `next_cron_fire(...)?` with match-and-log: a parse error on a corrupt persisted expression now preserves the existing `next_fire_at` and emits a `debug!` instead of aborting fire and unwinding past the spawn. Regression tests: - `fire_mission_with_corrupt_cron_expression_does_not_orphan_thread` - `resume_mission_preserves_concurrent_field_changes` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): Paranoid Architect review — orphan/re-fire, TOCTOU, cron docs, resume tz Addresses 4 of 5 findings from serrrfirat's Paranoid Architect review on #1957 (the 5th is declined with rationale in the PR thread). 1. (HIGH) fire_mission save_mission failure non-fatal + tick() per-mission isolation. The previous code propagated `save_mission(...)?` after the thread was spawned, leaving an orphan thread *and* — for cron cadences — leaving next_fire_at un-advanced, causing the next tick to re-fire the same mission in a runaway loop. Now save is best-effort: log a `debug!` on failure and still return Ok(Some(thread_id)). The watcher is already installed (from the earlier round). Separately, tick() now catches per-mission load and fire failures with match-and-continue so a single bad mission doesn't abort the entire tick cycle. 2. (MED) bootstrap_project backfill TOCTOU: previously list_all_missions then a deferred save_mission could clobber concurrent writes with the stale snapshot. Now re-load the mission immediately before save and only patch if next_fire_at is still None — narrows the window significantly without adding a new Store trait method. Residual race documented inline. 3. (MED) 6-field cron ambiguity: the `0 9 * * * 2027` form could be read as either "sec min hr dom mon dow" (our normalizer's interpretation, matching the `cron` crate's native format) or Quartz-style "min hr dom mon dow year". Documented the assumed format in the doc comment, added a regression test pinning the interpretation, and noted it in the CodeAct preamble so users know to use the explicit 7-field form for year-bounded schedules. 4. (MED) user_timezone propagation on inject/resume: only set on new thread spawn before. Now the resume path writes the fresh tz to thread metadata before resume_thread() reloads from store, so the resumed execution sees the up-to-date value. The inject path is documented as a known limitation — updating live in-memory state in a running ExecutionLoop requires a new signal type (out of scope). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): close latent next_fire_at bypass in ensure_* mission helpers PR review (serrrfirat): two private mission-creation helpers built a Mission via `Mission::new() + save_mission()` directly, skipping the `next_fire_at` computation that `create_mission` performs. Today every caller passes `OnSystemEvent` so the bug never bites — but a future caller passing `Cron` would silently re-introduce the original `next_fire_at = None` bug that #1944 fixes, and the bootstrap backfill wouldn't help (it only triggers for Active+Cron with `next_fire_at = None` *after* a process restart). Add the same Cron-cadence guard to both helpers: - `ensure_self_improvement_mission` - `ensure_mission_by_metadata` Regression test `ensure_mission_by_metadata_with_cron_cadence_computes_next_fire_at` exercises the cron path through the private helper and asserts the computed `next_fire_at` is set and in the future. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): final PR #1957 review pass — Ok(None) cron + tick cooldown Addresses the remaining open review threads on PR #1957: 1. types/mission.rs:391 — DST test reference comment now matches the actual `2027-03-13 00:00 UTC` anchor instead of the stale `22:00 UTC` wording (Copilot 3050772614 / 3055762609). 2. runtime/manager.rs::set_thread_metadata — was best-effort and silently swallowed store errors, while the conversation.rs:250 caller comment claimed the resumed thread was guaranteed to see the new value. Now returns Result<(), EngineError>; conversation.rs logs on failure and the comment is honest about the contract (Copilot 3051471863 + serrrfirat 3056444137). 3. types/mission.rs::next_cron_fire_required — new helper that maps Ok(None) to EngineError::InvalidCadence. Used by create_mission, update_mission cadence-change, resume_mission, and the two ensure_*_mission helpers. fire_mission and bootstrap_project keep the existing grace (logged) since the thread/data is already in flight (Copilot 3051471912/47/85 + serrrfirat 3056443657). 4. runtime/mission.rs::tick — when save_mission fails after a successful spawn, the persisted next_fire_at stays in the past and every subsequent 60s tick re-fires the same mission, spawning duplicates up to the daily budget. Added an in-memory `last_fire_attempt` map armed by fire_mission (regardless of save outcome) and consulted by tick to enforce a 90s per-mission cooldown (serrrfirat 3056443083). Regression tests: - create_mission_rejects_unschedulable_cron - update_mission_rejects_switch_to_unschedulable_cron - resume_mission_rejects_unschedulable_cron - tick_cooldown_suppresses_re_fire_on_save_failure All four use a 7-field year-locked cron (`0 0 0 1 1 * 2020`) to exercise the Ok(None) path deterministically. Mission test count: 49 → 53. Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test` green. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): prune fire cooldown map + reject Quartz-style 6-field cron Addresses two follow-up review threads on PR #1957: 1. `last_fire_attempt` was insert-only — completed/paused missions left stale entries that accumulated over the process lifetime. Now: - `pause_mission` and `complete_mission` drop the entry explicitly. - `tick` opportunistically prunes any entry whose cooldown window has elapsed, catching stragglers from non-graceful transitions (crash recovery, direct store edits) so the map can never grow unbounded. Regression test: `pause_and_complete_drop_cooldown_entry`. (serrrfirat 3057568698) 2. 6-field cron with a year-shaped trailing field (e.g. `0 9 * * * 2027`) was silently misinterpreted as `sec min hr dom mon dow=2027` instead of the Quartz-style "9am daily in 2027" the caller almost certainly meant. `normalize_cron_expression` now rejects this pattern with a clear `InvalidCadence` error pointing at the explicit 7-field form (`0 0 9 * * * 2027`). The 4-digit year heuristic is bounded to 1970-2099 so out-of-range numerics fall through unchanged. The existing pinning test for `* `-terminated 6-field input is unaffected. Regression test: `six_field_cron_with_year_shaped_last_field_is_rejected`. (serrrfirat 3057569413) Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test` green (308 engine tests pass). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge,engine): parse_cadence prefix ordering + year-stable cron tests Addresses three Copilot review threads on PR #1957: 1. `parse_cadence` (src/bridge/effect_adapter.rs) checked the cron heuristic (`split_whitespace().count() >= 5`) BEFORE the explicit `event:` / `webhook:` prefixes. An input like `event: a b c d e` silently became a `Cron` cadence with a parse-error downstream rather than the `OnEvent` the user requested. Reordered to check explicit prefixes first, falling through to cron only when none match. Regression test `parse_cadence_event_prefix_with_multi_token_pattern` covers `event:` + `webhook:` + a real cron control case. (Copilot 3057828107) 2. `next_cron_fire_respects_timezone` asserted `in_ny.year() >= 2026`, which is time-dependent (fails before 2026, tautology after). Replaced with `assert!(in_ny > Utc::now())` so the test stays stable across calendar years. (Copilot 3057828177) 3. `update_mission_cron_expression_change_recomputes_next_fire_at` used `0 0 1 1 *` ("once a year on Jan 1") as `before` and asserted `after < before`. Race around New Year's: the yearly schedule's next fire could land within seconds and invert the ordering. Switched to a year-locked 7-field cron (`0 0 0 1 1 * 2099`) so the next fire is deterministically far in the future regardless of run date. (Copilot 3057828212) Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test --lib` green. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): outcome processor reconciles fire accounting on save failure Addresses Copilot review thread on PR #1957: When `fire_mission`'s post-spawn `save_mission` fails, the persisted mission is left missing the new `thread_id` (in `thread_history`), the `threads_today` increment, the `last_fire_at` stamp, and — for cron cadences — the advanced `next_fire_at`. The in-memory `last_fire_attempt` cooldown holds the runaway re-fire path closed for ~90 s, but once it elapses tick would re-fire against the still-stale persisted state. Worse, when the spawned thread later completes, the outcome watcher loaded the **stale** mission, mutated only `approach_history`/`status`, and saved — permanently overwriting any chance to record the missing fields. `process_mission_outcome_and_notify` now reconciles those fields the first time it sees a `thread_id` that isn't in `thread_history`: - Append `thread_id` to `thread_history` - Bump `threads_today` (saturating) - Stamp `last_fire_at = now` as a conservative approximation - For cron missions, recompute `next_fire_at` if it's None-or-past The reconcile is idempotent: a replay with the same `thread_id` is a no-op. Achieves eventual consistency for transient store failures even after retries are exhausted, and handles the permanent-failure case that a save-side retry loop alone could not. Regression test: `outcome_processor_reconciles_missing_fire_accounting` covers both the first-time reconcile path (history append, budget bump, last_fire_at stamp, cron next_fire_at advance) and idempotent replay (no double-count, no duplicate history entry). Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test --lib` green, ironclaw_engine 87/87 mission tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): cooldown only on save failure + reconcile to original fire instant Self-review pass on the #1944 work — addresses three findings from a fresh read of the cooldown / reconcile interaction: 1. **Tick cooldown was throttling high-frequency cron schedules.** The in-memory `last_fire_attempt` map was checked unconditionally, with the 90 s cooldown enforced even for normally-firing cron missions. For `* * * * *` (every minute) the 60 s tick interval falls inside the 90 s window, so half the events were silently dropped after every successful fire. The cooldown was only ever needed to detect a `save_mission` failure, not to enforce a global rate floor. Fix: `fire_mission` now uses a single `fire_instant` for both the persisted `Mission.last_fire_at` and the in-memory cooldown entry. Tick arms the cooldown only when the persisted `last_fire_at` does NOT equal the in-memory value — the equality is the proof that `save_mission` succeeded. On the success path the two match and the cooldown is transparent regardless of cron frequency. On the failure path the persisted side still holds the OLD value (or `None`), the mismatch fires, and re-fire is suppressed until reconcile. Regression test: `tick_does_not_throttle_high_frequency_cron_after_successful_fire` creates a `* * * * *` mission, fires it, advances `next_fire_at` into the past WITHOUT clobbering `last_fire_at`, calls tick, and asserts a new thread is spawned. The pre-existing failure-mode test was updated to explicitly clobber `last_fire_at` so it continues to model the failed-save state under the new logic. 2. **Reconcile drifted `last_fire_at` to the outcome time.** When `process_mission_outcome_and_notify` reconciled fields after a failed save, it stamped `last_fire_at = now`, which can be many seconds (or hours, for long-running mission threads) later than the actual fire instant. For users with a configured `cooldown_secs`, that drift gradually extended the cooldown window beyond what the user asked for. Fix: thread the original `fire_instant` from `fire_mission` through `spawn_mission_outcome_watcher` and `process_mission_outcome_and_notify` as `original_fire_at: Option<DateTime<Utc>>`. The reconcile path uses it to set `last_fire_at` back to the moment of the original spawn, falling back to `now` only when the watcher path is unknown (test helpers, callers without an original instant). The watcher's `original_fire_at` ALSO matches the in-memory `last_fire_attempt[mid]` value, so the cooldown's mismatch detector resolves immediately after reconcile — even before the 90 s window elapses. Regression test: `outcome_processor_reconciles_missing_fire_accounting` now passes an explicit `original_fire_at` and asserts the reconciled `last_fire_at` equals that exact instant (not `now`). 3. **Reconcile bypassed `mission.record_thread`.** It pushed directly into `thread_history` and missed the `updated_at` bump. `process_mission_outcome_and_notify` re-stamps `updated_at` later so there was no functional impact, but two paths diverging on the same field-mutation pattern is a future-bug invitation. Fix: use `mission.record_thread(thread_id)` in the reconcile branch for parity with `fire_mission`. Polish: - Documented the lenient `next_cron_fire` choice at all three call sites (`bootstrap_project` backfill, `fire_mission` post-spawn advance, reconcile path) to make the lenient/strict split easy to spot during review. - Added a clarifying comment on `update_mission`'s save-after-validate sequencing — `save_mission` is the only persistence boundary in the function, so `next_cron_fire_required`'s `Err` leaves the store untouched even though the in-memory `mission` was already mutated. Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test --lib` green, ironclaw_engine 88/88 mission tests pass (87 → 88). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): close cooldown race + corrupt-cron runaway Self-review pass on the cooldown rework — addresses the two failure modes a fresh read of the success/failure detection turned up plus a documentation gap on the equality check. 1. **TOCTOU race between save_mission success and cooldown insert.** The previous order was `save_mission(...)` → `last_fire_attempt.insert(...)`. A concurrent tick observing the gap saw the freshly-persisted `last_fire_at = fire_instant` AND no in-memory entry, evaluated the mismatch check to false, and (if `next_fire_at` was still in the past from any other code path) re-fired immediately. Fix: insert into `last_fire_attempt` BEFORE calling `save_mission`. While save is in flight, the in-memory map has `fire_instant` and the persisted record still has the OLD `last_fire_at`, so a concurrent tick sees a mismatch and arms the cooldown. Once save lands the values match (success) or stay mismatched (failure) — both correct. Regression test: `fire_mission_arms_cooldown_before_save_mission`. Adds an optional save-mission gate to `TestStore` (via oneshot channel + Notify), spawns `fire_mission` in a task, waits for save to enter the gate, and asserts `last_fire_attempt[mid]` is already populated while save is parked. 2. **Corrupted cron expression bypassed the cooldown.** When `next_cron_fire(expression)` returned `Err` (corrupt persisted expression), the previous code stamped `last_fire_at = fire_instant` anyway. Save then succeeded with `last_fire_at` matching the in-memory value, so tick's mismatch detector saw "save succeeded" and the cooldown was never armed. With `next_fire_at` still in the past (preserved because the cron crate couldn't compute a new one), every tick re-fired the same mission until `max_threads_per_day` was exhausted — same root cause shape as #1944. Fix: track `cron_advanced: bool` in `fire_mission`. When the cron advance fails, deliberately leave `last_fire_at` at its OLD value. The in-memory `last_fire_attempt[mid]` is still set to `fire_instant`, so the in-memory vs persisted mismatch arms the cooldown via the exact same code path as a save failure — no new signal needed. After 90 s the cooldown elapses and tick can retry, bounded by `max_threads_per_day`, in case the corruption resolves. Regression test: `tick_does_not_re_fire_corrupted_cron_within_cooldown_window`. Creates a cron mission, corrupts the persisted expression, sets `next_fire_at` to the past, fires once, then asserts a subsequent tick returns no new spawn AND that the persisted `last_fire_at` was deliberately left unset (proving the mismatch-arming path). 3. **Documented the equality check's precision requirement.** The `mission.last_fire_at != Some(*in_mem_last)` comparison is load-bearing and assumes the `Store` round-trips `DateTime<Utc>` without precision loss. The bridge's in-memory cache and JSON persistence both preserve nanoseconds; a future Postgres-backed store using `TIMESTAMPTZ` would silently break the success-path detection (microsecond truncation). Added a long comment at the tick check pointing future store implementers at the requirement so the next backend addition can either preserve precision or relax the comparison to "within one microsecond" before landing. Verified: cargo fmt clean, cargo clippy --all-targets --all-features zero warnings, full `cargo test --lib` green, ironclaw_engine 90/90 mission tests pass (88 → 90). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
4147c6d587 |
feat(gateway): extract gateway frontend into ironclaw_gateway crate with widget system (#1725)
* feat(frontend): extract frontend into ironclaw_frontend crate with widget extension system
Moves all frontend static assets (app.js, style.css, index.html, i18n/*,
theme-init.js, favicon.ico) from src/channels/web/static/ into a dedicated
ironclaw_frontend crate. The crate also adds:
- Layout configuration types (branding, tab order, chat features, per-widget config)
- Widget manifest types with named slot system (tab, chat_header, sidebar, etc.)
- CSS scoping utility (auto-prefixes selectors with [data-widget="id"])
- Bundle assembly (injects layout config, widgets, and custom CSS into HTML)
- Frontend API endpoints (GET/PUT layout, list widgets, serve widget files)
- Browser-side IronClaw.registerWidget() API with authenticated fetch,
event subscription, theme access, and i18n
Widgets are stored in workspace at frontend/widgets/{id}/ and served via
the API. Layout config is stored at frontend/layout.json. The agent can
create/edit both using existing memory_write/memory_read tools.
Gateway handlers now reference ironclaw_frontend::assets constants instead
of include_str!() with local paths, completing the separation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CI failures — license, rust-version, formatting, manifest warnings
- Add license = "MIT OR Apache-2.0" to ironclaw_frontend Cargo.toml (cargo-deny)
- Fix rust-version to 1.92 to match other crates
- Log warning for invalid widget manifests instead of silent skip
- Run cargo fmt across all files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(frontend): structured data cards + chat renderer API for rich message rendering
Agent responses containing JSON/structured data (like mission results,
status objects) now render as styled cards with labeled fields, status
badges, and monospaced IDs instead of raw text.
Built-in rendering:
- Detects inline JSON objects (including Python-style single quotes)
- Renders as data cards with key-value rows
- Status/state fields get colored badges (success/error/pending)
- UUIDs rendered in monospace
Extensible via widgets:
- IronClaw.registerChatRenderer({ id, match, render, priority })
- First matching renderer wins (priority ordering)
- Renderer gets the content element to mutate in place
Also adds ChatRenderer variant to WidgetSlot enum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(frontend): hash-based URL navigation for page refresh persistence
Navigation state is now encoded in window.location.hash so refreshing
the page (or sharing a URL) restores the current view:
#/chat → chat tab, assistant thread
#/chat/{threadId} → specific conversation
#/memory/{path/to/file} → memory browser with file open
#/jobs/{jobId} → job detail view
#/routines/{id} → routine detail view
#/settings/{subtab} → settings sub-tab (extensions, etc.)
#/logs → logs tab
Hooked into all navigation functions: switchTab, switchThread,
switchToAssistant, createNewThread, readMemoryFile, openJobDetail,
closeJobDetail, openRoutineDetail, closeRoutineDetail,
switchSettingsSubtab.
Thread restore is deferred until loadThreads() completes (async),
then the pending thread ID is matched against the loaded thread list.
Browser back/forward buttons work via hashchange listener.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(frontend): auto-open README.md when first visiting Memory tab
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(frontend): preserve URL hash across page refresh
Two bugs caused the hash to reset on Cmd+R:
1. Auth URL cleanup (replaceState) stripped the hash fragment —
now preserves it via cleaned.hash
2. restoreFromHash() called switchTab() which called updateHash()
overwriting the full hash before the detail was restored —
now suppresses hash updates during the entire restore sequence
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(frontend): seed frontend/README.md with customization guide for agent
The agent didn't know it could customize the frontend via workspace writes.
Now seeds frontend/README.md on first boot with a guide covering:
- Layout config (branding, colors, tab order) via frontend/layout.json
- Custom CSS via frontend/custom.css with common variable names
- Widget creation (manifest + index.js + style.css)
- API endpoints
Also seeds frontend/.config with skip_indexing: true so frontend assets
aren't chunked/embedded for search.
When a user says "change the color scheme to red", the agent can now
discover frontend/README.md via memory_tree, read the guide, and write
the appropriate layout.json or custom.css.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(frontend): wire workspace-aware serving for index.html and style.css
The index_handler and css_handler now read from workspace to apply
frontend customizations on page load:
- index_handler: reads frontend/layout.json, discovers widgets in
frontend/widgets/*, reads frontend/custom.css, then calls
assemble_index() to inject branding colors, layout config,
widget scripts, and custom CSS into the base HTML.
Falls back to embedded HTML if no customizations exist.
- css_handler: appends frontend/custom.css from workspace after
the embedded base stylesheet.
This completes the end-to-end flow:
Agent writes frontend/layout.json → user refreshes → sees changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(frontend): wire up remaining widget system gaps
Audit-driven fixes for the widget extension system:
1. Widget tab panel ID: panels now get id="tab-{widgetId}" so
switchTab() can find and activate them
2. Widget JS auth: inline widget JS in assembled HTML instead of
<script src> to protected endpoint (browser script tags can't
send Authorization headers)
3. Layout config: fully implement tab ordering, default_tab,
chat.suggestions, chat.image_upload application
4. SSE event forwarding: wrap EventSource.addEventListener to
intercept all named events and dispatch to widget subscribers
via IronClaw.api._dispatch()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(frontend): XSS prevention, widget queue drain, code-block false positives
Security (2 XSS fixes):
1. HTML-escape branding title in assemble_index() to prevent
<script>alert(1)</script> injection via layout.json
2. Escape </script> in inlined widget JS to prevent script tag
breakout — uses <\/script> replacement
3. Escape widget IDs in HTML attributes via escape_html_attr()
Correctness:
4. Drain _widgetInitQueue after DOM is ready — widgets registered
before tab-bar exists now mount correctly instead of silently
failing
5. Skip inline <code> elements in upgradeInlineJson to prevent
false-positive JSON card rendering on code spans like
<code>{key: value}</code>
6. Document scope_css limitation with nested @media rules
Tests (13 new):
- XSS: title injection escaped, widget JS </script> breakout escaped,
widget ID attribute escaped
- Edge cases: escape_html basic, escape_html_attr quotes, missing
head/body tags, empty widget JS, whitespace-only custom CSS skipped
- Widget: at-rule not prefixed, declarations preserved, special chars
in widget ID, all slot variants round-trip, minimal manifest
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix clippy — collapsible if, while_let_on_iterator
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): resolve frontend clippy and formatting failures
* fix(frontend): address PR review — XSS, scope_css, cache, dedup
Security (3 XSS gaps):
1. Layout JSON injected into <script>window.__IRONCLAW_LAYOUT__</script>
is now run through escape_tag_close() — serde_json does not escape `<`
or `/`, so a branding title containing `</script>` previously broke
out of the script tag. Case-insensitive, UTF-8 safe.
2. Widget CSS and custom CSS injected into <style> tags are now escaped
the same way against `</style>` breakouts.
3. New escape_tag_close() helper handles `</script`/`</style` uniformly
(case-insensitive with tail preserved, via char-boundary walk).
Correctness:
4. scope_css now tracks brace depth via a stack that distinguishes rule
lists from declaration blocks. Selectors nested inside @media,
@supports, @container, @layer, @document, @scope are recursively
scoped. @keyframes/@font-face/@page bodies pass through opaque so
inner keyframe selectors (0%, 100%) are not prefixed. The old
single-bool parser produced unbalanced output on any nested rule.
5. WidgetInstanceConfig.enabled now defaults to true (via serde_default
+ manual Default impl). A layout entry that omits `enabled` while
setting `config` no longer silently disables the widget.
6. build_frontend_html short-circuit replaced with a
layout_has_customizations() helper covering all branding/tabs/chat
fields. The old boolean missed subtitle, logo_url, favicon_url,
default_tab, image_upload.
7. Custom CSS is now served only via /style.css (css_handler). Removed
from FrontendBundle injection to prevent double-application.
8. Dead pub index_handler/css_handler/js_handler in
handlers/static_files.rs removed — routes use private handlers in
server.rs that need GatewayState.
9. Widget file path validation is now component-based via
is_safe_segment / is_safe_relative_path. Rejects `.`, `..`, empty,
`/`, `\`, NUL in any component, plus leading `/`. MIME detection is
case-insensitive and adds .mjs / .map.
10. Layout and widget-manifest parse errors now log tracing::warn!
instead of silently falling back.
Extension system follow-ups:
11. Extracted shared widget-loading helpers (load_widget_manifests,
load_resolved_widgets, read_widget_manifest) in handlers/frontend.rs.
frontend_widgets_handler and build_frontend_html both delegate, so
widget discovery exists in exactly one place.
12. New FrontendHtmlCache in GatewayState. Cache key is derived from the
updated_at of frontend/layout.json and the frontend/widgets/
directory (max child mtime) via a single list("frontend/") call.
A cache hit skips reading every widget manifest/JS/CSS per request.
Edits invalidate naturally because list() sees the newer timestamp.
Cache survives rebuild_state() by cloning the Arc.
13. upgradeInlineJson rewritten without the nested-quantifier regex. New
_findJsonCandidates does a linear bracket scan that respects string
literals and fast-skips <code>/<pre> regions. Three hard caps bound
worst-case work (MAX_PARA_LEN=20000, MAX_SCAN=5000,
MAX_CANDIDATES=32), eliminating the catastrophic-backtracking risk.
Tests (29 new):
- bundle.rs: 5 — layout JSON / widget CSS / custom CSS <script>/<style>
breakouts, escape_tag_close case-insensitive, multi-byte safety
- widget.rs: 5 — @media inner selector scoped, nested @supports+@media,
@keyframes passthrough, sibling rules in @media, complex mix brace
balance
- layout.rs: 3 — enabled defaults true, Default impl enabled,
explicit false respected
- handlers/frontend.rs: 4 — segment allows/rejects, relative path
allows/rejects (traversal, backslash, encoded separators)
Quality gate:
- cargo fmt clean
- cargo clippy --all --benches --tests --examples --all-features →
zero warnings
- cargo test --lib -p ironclaw_frontend -p ironclaw → 4171 main +
43 frontend tests pass
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: post-merge — PairingStore::new_noop, CLI snapshot, docs
Merge of origin/staging surfaced three small follow-ups:
1. src/channels/wasm/wrapper.rs — PairingStore::new() signature changed
in staging to take (db, cache). Switch the test call site to
PairingStore::new_noop() to match other tests in the file.
2. src/cli/snapshots/..long_help_output_without_import.snap — accept
the new snapshot. Clap's render_long_help for --auto-approve now
emits an indented blank line between the short and long description;
this test was already failing on staging tip (see Staging CI run
24021660555) so the snapshot update was needed regardless of this PR.
3. src/workspace/seeds/FRONTEND.md — address new copilot comments:
- Placeholder is `{id}` (matches API path segment and manifest id
field), not `{name}`.
- Only `slot: "tab"` is actually mounted by the browser runtime.
Trim the slot list to what's implemented and mention
IronClaw.registerChatRenderer() for inline rendering. The extra
WidgetSlot variants stay in the Rust API for forward compatibility
but are no longer advertised to users until mounting is wired.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rename ironclaw_frontend → ironclaw_gateway, .system/gateway/ workspace
Two coupled renames to align frontend assets with the broader `.system/`
namespace introduced by other in-progress work:
1. Workspace folder: `frontend/` → `.system/gateway/`
- layout.json, custom.css, widgets/{id}/, README.md, .config all
move under `.system/gateway/`
- LAYOUT_PATH and WIDGETS_DIR are now constants in the handler so a
future move is a one-line change
- is_config_path test updated to use the new path
- FRONTEND.md seed rewritten to point at `.system/gateway/`
- Cache key doc comments updated to match
- No legacy or migration shim — this never shipped to prod
2. Crate: `ironclaw_frontend` → `ironclaw_gateway`
- Matches how the surrounding subsystem is called (`channels/web` is
"the gateway"). Cleaner mental model: workspace folder, crate name,
and module name all align.
- Directory renamed via `git mv` so history is preserved.
- Cargo.toml workspace member + dependency updated; package name
updated; description tweaked to "gateway frontend assets".
- All `use ironclaw_frontend::` imports rewritten in server.rs and
handlers/frontend.rs.
- Doctest in widget.rs updated to use the new crate name.
- Cargo.lock regenerated.
The HTTP API paths stay as `/api/frontend/*` since they're a public
surface; only the internal workspace path and crate name moved.
Quality gate:
- cargo fmt clean
- cargo clippy --all --benches --tests --examples --all-features →
zero warnings
- cargo test -p ironclaw_gateway → 43 unit + 1 doctest pass
- cargo test --lib -p ironclaw → 4228 pass (8 unrelated IPv6/DNS
validation failures, also failing on clean post-merge baseline)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(gateway): per-request CSP nonce for inlined widget scripts
Copilot review caught that `assemble_index()` injects two kinds of inline
`<script>` blocks (the layout-config script and per-widget module scripts),
but the gateway's CSP sets `script-src 'self' …CDNs…` with no
`'unsafe-inline'` and no nonce — so the browser silently blocks every
injected script the moment any customization is enabled. The widget
runtime would never execute on a customized index page.
Fix uses a per-request CSP nonce (W3C standard pattern):
- `crates/ironclaw_gateway/src/bundle.rs`
- New `NONCE_PLACEHOLDER` sentinel constant, re-exported from the crate root
- `assemble_index()` stamps `nonce="__IRONCLAW_CSP_NONCE__"` on every
injected `<script>` tag (both the layout-config script and each
widget's module script)
- Inline `<style>` blocks deliberately do NOT carry a nonce — the
gateway's CSP allows `'unsafe-inline'` for `style-src`, so adding
one would be dead weight; pinned with a regression test
- Three new tests verify the placeholder appears on layout + widget
scripts and is absent on widget styles
- `src/channels/web/server.rs`
- Static CSP layer now reads from a single `BASE_CSP` constant so the
static and per-response variants stay in lock-step
- New `build_csp_with_nonce(nonce)` produces the same CSP with
`'nonce-{nonce}'` added to script-src, preserving the explicit CDN
list and the strict `style-src 'self' 'unsafe-inline' …` policy
- New `generate_csp_nonce()` returns 16 random bytes hex-encoded via
OsRng — same primitive `tokens_create_handler` already uses
- `index_handler` now returns `Response` (not `impl IntoResponse`) so
it can branch:
- Workspace has no customizations → serve embedded `INDEX_HTML`
unchanged; the static CSP layer applies (no inline scripts to
authorize anyway)
- Workspace has customizations → generate fresh nonce, replace
placeholder in cached HTML, and emit a per-response
`Content-Security-Policy` header with the nonce. Setting the
header here suppresses the global `if_not_present` layer for this
response only.
- Two new unit tests pin the nonce-source position in script-src and
the format/uniqueness of `generate_csp_nonce()`
The HTML cache still works because the cached HTML contains the
placeholder (not the actual nonce); per-request substitution preserves
caching while the browser still sees a unique nonce on every page load.
Quality gate:
- cargo fmt clean
- cargo clippy --all --benches --tests --examples --all-features →
zero warnings
- cargo test -p ironclaw_gateway → 46 pass (+3 nonce tests)
- cargo test --lib -p ironclaw → 4238 pass (+2 CSP tests)
Refs: PR #1725 review by copilot-pull-request-reviewer
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(gateway): wire ko.js asset through ironclaw_gateway::assets
The merge of staging brought in a Korean i18n pack referenced via
include_str!("static/i18n/ko.js") in src/channels/web/server.rs.
After the gateway extraction the static/ directory moved into
crates/ironclaw_gateway/static/, so the legacy include_str! path
no longer resolved. Add I18N_KO_JS to ironclaw_gateway::assets and
make the i18n_ko_handler reference it like the other language packs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(e2e): add Playwright coverage for chat-driven frontend customization
Adds two end-to-end scenarios for the widget extension system shipped in
PR #1725, both driven by talking to the agent in chat:
1. **Tab bar to left side panel.** The user asks the agent to move the
tab bar; the mock LLM emits a `memory_write` tool call writing
`.system/gateway/custom.css`, and after a reload the test asserts the
served stylesheet contains the overlay, the computed flex-direction
of `.tab-bar` is `column`, and the bar is now taller than it is wide.
2. **Workspace-data widget.** The user asks the agent to create a
"Skills" widget that renders workspace skills. Two chat turns write
`.system/gateway/widgets/skills-viewer/manifest.json` and `index.js`
into the workspace. After a reload the test verifies the new tab
button appears in `.tab-bar`, switches to it, waits for the widget's
`data-testid="skills-viewer-root"` to mount, and asserts the widget
actually fetched `/api/skills` (no `skills-viewer-error` marker) and
that the panel carries the `data-widget="skills-viewer"` attribute
the gateway runtime stamps for CSS isolation.
Both tests share a `clean_customizations` fixture that wipes the
workspace overlay files before and after each run so the session-scoped
gateway server stays isolated across tests in the file (`memory_write`
treats empty content as effectively cleared, and the gateway skips
empty / unparseable widget files silently).
Supporting changes:
- **mock_llm.py**: three new `TOOL_CALL_PATTERNS` (`customize: move
tab bar to left`, `customize: create skills viewer manifest`,
`customize: install skills viewer code`) that emit one
`memory_write` call per turn — the existing one-tool-per-response
shape is preserved.
- **app.js (`_addWidgetTab`)**: fix a latent bug where widget tabs
would be queued forever because the function looked for a
`.tab-content` / `#tab-content` element that the gateway HTML never
ships. The built-in tab panels live as siblings of `.tab-bar` inside
`#app`, so we now resolve the parent off the first existing
`.tab-panel` (with `#app` as a final fallback). Without this fix the
Skills widget tab never mounts and the second scenario can't pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(e2e): support multi tool calls per response in mock_llm
The mock LLM previously emitted at most one tool call per assistant
turn. That shape silently bypasses the v2 engine and CodeAct dispatch
paths, where a single response can fan out into several parallel tool
calls (or several Python helper invocations from one script). Tests
written against that constraint were either contorted into multiple
chat turns or quietly failed to cover multi-call regressions.
Changes:
- ``TOOL_CALL_PATTERNS`` args functions may now return ``list[dict]``
instead of a single ``dict``. Each item is its own
``{"tool_name", "arguments"}`` pair, so one trigger can mix several
tools in one response. ``_normalize_tool_calls`` always wraps the
return value into a list so the dispatcher stays shape-agnostic.
- ``match_tool_call`` returns ``list[dict] | None``.
- ``_tool_call_response`` and ``_stream_tool_call`` now accept either a
single dict (legacy callers) or a list. The streaming path emits
per-tool-call header + arguments chunks with distinct ``index``
values, exercising clients' per-index merging logic the same way real
providers force them to.
- ``_find_tool_results`` collects every fresh ``role: tool`` message
after the most recent user turn (not just the first), and the
chat-completion summary path renders a multi-line acknowledgment
when more than one tool ran in a single turn. The single-result
helper is kept as a thin shim for the special-response path.
- The PR #1725 customization scenario is consolidated: instead of
three separate triggers (one memory_write each), the
``customize: install skills viewer widget`` trigger now emits *both*
the manifest and ``index.js`` writes in one assistant turn. The
``customize: move tab bar to left`` trigger stays single-call to
cover the legacy code path. The Playwright test in
``test_widget_customization.py`` is updated to a single chat turn
for the widget install — if the v2 engine ever drops the second
parallel call, the test will fail because the new tab can't mount
without both files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(gateway): address PR #1725 review feedback
Four issues raised in the 2026-04-07 review pass:
1. **Widget id / directory mismatch** (`src/channels/web/handlers/frontend.rs`).
`read_widget_manifest` now rejects widgets whose `manifest.id` does
not match the on-disk directory name. The loader uses the directory
name to compute file paths (`{WIDGETS_DIR}{dir}/index.js`) while the
layout-config gating and the public
`/api/frontend/widget/{id}/{*file}` endpoint key off `manifest.id`.
When those drift, code can be mounted from one folder under a
different id and the file API silently 404s — a correctness footgun
for widget authors and a path-confusion attack surface for the
serving handler. Fix lives in the shared helper so both
`load_resolved_widgets` and `load_widget_manifests` get it. Adds
regression tests for both the rejection and the matching path.
2/3. **`memory_write` doc examples used the wrong parameter name**
(`src/workspace/seeds/FRONTEND.md`). The seeded customization guide
showed `memory_write path=".system/gateway/..."`, but the actual tool
parameter is `target` (`src/tools/builtin/memory.rs`). As written the
examples wouldn't work if copy-pasted into a tool call. Both
examples (layout.json + custom.css) updated to `target=`.
4. **`css_handler` allocated on the hot path** (`src/channels/web/server.rs`).
The handler always called `assets::STYLE_CSS.to_string()` in the
no-overlay branches, copying the entire embedded stylesheet on
every request. Switched the local to `Cow<'static, str>` so the
common path borrows the static string and only the overlay branch
pays for an owned `format!`.
Quality gate:
- `cargo fmt` clean
- `cargo clippy --no-default-features --features libsql --tests` zero warnings
- `cargo test --no-default-features --features libsql --lib channels::web::handlers::frontend` — 6 passed (4 existing + 2 new regression tests)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(gateway): address PR #1725 paranoid-architect review
Five issues raised in the 2026-04-07 review pass:
1. **High — `</style>` breakout XSS in branding CSS-vars injection**
(`crates/ironclaw_gateway/src/bundle.rs`). Every other inline injection
point in `assemble_index()` runs through `escape_tag_close`, but the
branding `<style>` block formatted directly. A hostile color value
containing `</style>` could close the tag early and inject HTML. Now
wraps `css_vars` in `escape_tag_close(&css_vars, "</style")` for
defense in depth, with a regression test in
`test_assemble_index_branding_style_breakout_escaped`.
2. **Medium — CSS property injection via unvalidated branding colors**
(`crates/ironclaw_gateway/src/layout.rs`). `to_css_vars()` interpolated
`primary` / `accent` strings raw into `--color-primary: {};`, letting
a hostile `layout.json` break out of the `:root {}` block (e.g.
`red; } .chat-input[value^="s"] { background: url(...) }`). Added
`is_safe_css_color()` validator that accepts hex literals, modern
functional notation including `rgb(0 0 0 / 50%)`, and bare named
colors, while rejecting `;`, `{}`, `<>`, quotes, backslash, `*`
(handles both `/*` and `*/` comment markers), `url(...)`, and unknown
functions. `to_css_vars()` silently drops invalid values so the rest
of the branding config still applies. Six new unit tests cover the
accepted forms, the injection vectors, and the `to_css_vars` drop.
3. **Medium — CSP policy duplication risks silent drift**
(`src/channels/web/server.rs`). `BASE_CSP` and `build_csp_with_nonce`
re-hardcoded every directive independently, so adding a `connect-src`
to one would silently leave the other on the old policy. Extracted
per-directive constants (`STYLE_SRC`, `FONT_SRC`, `CONNECT_SRC`,
`IMG_SRC`, `FRAME_SRC`, `FORM_ACTION`) and built both flavors via a
single `build_csp(nonce: Option<&str>)` helper. `BASE_CSP_HEADER` is
now a `LazyLock<HeaderValue>` (with a safe minimal fallback to honor
the no-`.expect()` rule on the request path). Added two regression
tests: `test_base_and_nonce_csp_agree_outside_script_src` strips the
`script-src` directive from both flavors and asserts byte equality,
and `test_base_csp_header_matches_build_csp_none` locks the lazy
header to `build_csp(None)`.
4. **Medium — `_wipe_customizations` ignored HTTP status**
(`tests/e2e/scenarios/test_widget_customization.py`). The cleanup
posts now assert `status_code == 200` with `resp.text` in the
message, so an auth/server failure surfaces immediately instead of
bleeding leftover workspace state into the next test.
5. **Drive-by — pre-existing flake in `test_telegram_token_colon_preserved
_in_validation_url`** (`src/extensions/manager.rs`). The test reads
`IRONCLAW_TEST_TELEGRAM_API_BASE_URL` via `telegram_bot_api_url`
without taking the `lock_env()` mutex, so when a parallel test holds
the override the read races and the assertion sees
`http://127.0.0.1:.../bot…` instead of `https://api.telegram.org/`.
The new tests in this PR changed scheduling enough to surface the
race on every run. Fixed by acquiring the same `ScopedEnvVar` lock
and clearing the override inside the test, making it deterministic.
Quality gate:
- `cargo fmt` clean
- `cargo clippy --no-default-features --features libsql --tests` zero warnings
- `cargo test --no-default-features --features libsql --lib` — 4284 passed
- `cargo test -p ironclaw_gateway` — 50 unit + 1 doctest passed (was 46)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: nudge workflows for
|
||
|
Henry Park
|
6a8e5815b4 |
fix(wasm): upgrade Wasmtime to 43.0.1 and restore CI (#2224)
* fix(wasm): upgrade wasmtime to 43.0.1 * chore(wasm): align wasmparser with wasmtime deps |
||
|
Illia Polosukhin
|
af9b59a284 |
feat: unified tool dispatch + schema-validated workspace (#2049)
* feat(workspace): add JSON Schema validation to document metadata Add a `schema` field to `DocumentMetadata` that enables automatic content validation on workspace writes. When a document or its folder `.config` carries a JSON Schema, all write operations (write, append, patch, write_to_layer, append_to_layer) validate content against it before persisting. This is the foundation for typed system state (settings, extension configs, skill manifests) stored as workspace documents. Builds on the metadata infrastructure from #1723 — schema is inherited via the existing `.config` chain (folder → document → defaults). Refs: #640, #1894, #1937 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tools): add channel-agnostic ToolDispatcher with audit trail Introduce `ToolDispatcher` — a universal entry point for executing tools from any caller (gateway, CLI, routine engine, WASM channels). Creates lightweight system jobs for FK integrity, records ActionRecords, and returns ToolOutput. This is a third entry point alongside v1's Worker::execute_tool() and v2's EffectBridgeAdapter::execute_action(). DispatchSource::Channel(String) is intentionally string-typed — channels are interchangeable extensions that can appear at runtime. Also adds JobContext::system() factory and create_system_job() to both PostgreSQL and libSQL backends. Refs: #640 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(workspace): settings-as-workspace-documents with dual-write adapter Add WorkspaceSettingsAdapter that implements SettingsStore by reading/ writing workspace documents at _system/settings/{key}.json. During migration, dual-writes to both the legacy settings table and workspace. Reads prefer workspace, falling back to the legacy table. Known setting keys (llm_backend, selected_model, tool_permissions.*, etc.) get JSON Schemas stored in document metadata — writes are validated automatically by Phase 0's schema validation. Also adds settings_schemas.rs with compile-time schema registry and settings_path() helper. Refs: #640, #1937 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(gateway): wire ToolDispatcher into GatewayState Add tool_dispatcher field to GatewayState with with_tool_dispatcher() builder method. Create and wire the dispatcher in main.rs when both tool_registry and database are available. All 16 GatewayState construction sites updated. Per-handler migration (routing mutations through ToolDispatcher instead of direct DB calls) is deferred to follow-up PRs — each handler has complex ownership checks, cache refresh, and response types. Refs: #640 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tools): add system introspection tools (tools_list, version) Add SystemToolsListTool and SystemVersionTool as proper Tool implementations that replace hardcoded /tools and /version commands. Registered at startup via register_system_tools(). Available in both v1 and v2 engines — no is_v1_only_tool filter to worry about. Refs: #640 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(workspace): extension and skill state schemas and path helpers Add workspace path helpers and JSON Schemas for storing extension configs, extension state, and skill manifests under _system/extensions/ and _system/skills/. This establishes the workspace document structure that ExtensionManager and SkillRegistry will use as a durable persistence backend (read-through cache pattern). Runtime state (active MCP connections, WASM runtimes) stays in memory. Only durable config and activation state moves to workspace documents. Refs: #640, #1741 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback and CI failures CI fixes: - deny.toml: allow MIT-0 license required by jsonschema - workspace/document.rs: #[allow(dead_code)] on system path constants pending follow-up phases that consume them - workspace/settings_adapter.rs: remove unused chrono::Utc import - workspace/settings_adapter.rs: collapse nested if into && form Review fixes (gemini-code-assist): - tools/dispatch.rs: await save_action directly instead of fire-and-forget tokio::spawn so short-lived CLI callers cannot drop audit records before they are persisted; surface errors via tracing::warn - tools/dispatch.rs: remove DispatchSource::Agent variant — sequence_num=0 with a reused job_id would violate UNIQUE(job_id, sequence_num). Agent callers must use Worker::execute_tool() which manages sequence numbers atomically against the agent's existing job - workspace/settings_adapter.rs: validate content against the schema BEFORE the first workspace write so the initial document creation cannot bypass schema enforcement (subsequent writes are validated by the workspace resolved-metadata path established after the first write) Refs: #2049 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: unify all machine state under .system/ Rename the workspace prefix from `_system/` to `.system/` (Unix dot-prefix convention for hidden internal state) and migrate v2 engine state from `engine/` to `.system/engine/` so all machine-managed state lives under one root. New layout: .system/ ├── settings/ (per-user settings as workspace docs) ├── extensions/ (extension config + activation state) ├── skills/ (skill manifests) └── engine/ ├── README.md (auto-generated index) ├── knowledge/ (lessons, skills, summaries, specs, issues) ├── orchestrator/ (Python orchestrator versions, failures, overlays) ├── projects/ (project files + nested missions/) └── runtime/ (threads, steps, events, leases, conversations) The inner `.runtime/` dot-prefix is dropped under `.system/engine/` since `.system/` itself is the hidden marker; no double-hiding needed. The `ENGINE_PREFIX` constant in `workspace::document::system_paths` is declared as the canonical convention; bridge `store_adapter` continues to define per-subdirectory constants below it for ergonomic interpolation. No legacy migration code — pre-production rename. Refs: #2049 * fix(pr-2049): security, correctness, and robustness fixes from review Critical security: - dispatch.rs: redact sensitive params before persisting ActionRecord (was leaking plaintext secrets into the audit log for tools with sensitive_params()) - settings_schemas.rs: validate settings keys against path traversal (reject /, \, .., leading ., empty, length > 128, non-alphanumeric); wire validation into all settings_adapter read/write/delete paths Data correctness: - history/store.rs + libsql/jobs.rs: write status as JobState::Completed .to_string() ('completed' snake_case) instead of 'Completed'; system jobs were round-tripping as Pending in parse_job_state() - settings_adapter.rs: fix .system/.config metadata to set skip_versioning: false (was true) — descendants inherit this via find_nearest_config, so the previous value silently disabled versioning for ALL .system/** documents, contradicting the audit- trail intent - workspace/mod.rs: add resolve_metadata_in_scope; use it in write_to_layer / append_to_layer so non-primary layer writes resolve schema/indexing/versioning from the target layer's .config chain instead of the primary user_id's. Also pass &scope (not &self.user_id) to maybe_save_version so versions are attributed to the correct scope Pipeline parity: - dispatch.rs: add SafetyLayer to ToolDispatcher; mirror Worker pipeline (prepare_tool_params -> validator -> redact -> timeout -> sanitize output) so dispatch path gets the same safety guarantees as the agent worker. Sanitized output is now stored in ActionRecord.output_sanitized instead of duplicating raw JSON Robustness: - settings_adapter.rs: propagate update_metadata errors in ensure_system_config and write_to_workspace (was silently ignored via let _ =, leaving schemas/skip_indexing unenforced) - settings_adapter.rs: set_all_settings now collects the first workspace write error and returns it after the legacy write completes, so partial-migration state is observable - settings_schemas.rs: rewrite llm_custom_providers schema to match CustomLlmProviderSettings (id/name/adapter/base_url/default_model/ api_key/builtin instead of stale name/protocol/base_url/model) Build: - Cargo.toml: jsonschema with default-features = false to avoid pulling a second reqwest major version Docs: - db/mod.rs: docstring for create_system_job uses 'completed' snake_case - workspace/document.rs: clarify .system/ versioning ("by default ARE versioned; individual files may opt out via skip_versioning") - settings_adapter.rs: clarify per-key reads prefer workspace, aggregate reads stay on legacy during migration - tools/builtin/system.rs: trim doc to match implemented scope (system_tools_list, system_version) - channels/web/mod.rs: move stale 'sweep tasks managed by with_oauth' comment back to oauth_sweep_shutdown line Refs: #2049 * docs+ci: enforce 'everything goes through tools' principle Document the core design principle from #2049 in two places so future contributors (human and AI) discover it during development: - CLAUDE.md: new "Everything Goes Through Tools" section near the "Adding a New Channel" guide. Includes the rule, the rationale (audit trail, safety pipeline parity, channel-agnostic surface, agent parity), and a pointer to the detailed rule file. - .claude/rules/tools.md: full pattern with required/forbidden examples, the list of layers that ARE exempt (Worker::execute_tool, v2 EffectBridgeAdapter, tool implementations themselves, background engine jobs, read-aggregation queries), and how to annotate intentional exceptions. Also extends `paths` to cover src/channels/** and src/cli/** so it surfaces when those files are edited. Enforce with a new pre-commit safety check (#7) in scripts/pre-commit-safety.sh: - Scans newly added lines under src/channels/web/handlers/*.rs and src/cli/*.rs for direct touches of state.{store, workspace, workspace_pool, extension_manager, skill_registry, session_manager}. - Suppress with a trailing `// dispatch-exempt: <reason>` comment on the same line, matching the existing `// safety:` convention. - Only checks added lines (`+` in the diff), so existing untouched handlers don't trip the check during incremental migration. The check fires only for new code: handlers that haven't been migrated yet (52 existing direct accesses across 12 handler files) won't break unmodified, but any new line that bypasses the dispatcher will be flagged at commit time. Refs: #2049 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pr-2049): address Copilot review on workspace schema layer - workspace::extension_state: extension/skill path helpers now reuse the canonical name validators (`canonicalize_extension_name`, `validate_skill_name`) instead of a weak `replace('/', "_")`. Names containing `..`, `\`, NUL, or other escapes are now rejected at the helper boundary, eliminating a path-traversal foothold for callers. Helpers return `Result<String, PathError>`. Regression tests added. - workspace::settings_adapter::ensure_system_config: now idempotent across upgrades. If `.system/.config` already exists with stale metadata (e.g. an older `skip_versioning: true` from before fix #3042846635), it is repaired to the expected inherited values instead of being left silently broken. Regression test added. - workspace::settings_adapter::write_to_workspace: lazily seeds `.system/.config` via a `OnceCell`, so callers no longer need to remember to invoke `ensure_system_config()` at startup before any setting write. Regression test added. - workspace::settings_adapter::delete_setting: workspace delete failures are now logged via `tracing::warn!` instead of being silently dropped. We still don't propagate the error — the legacy table is the source of truth during migration and a stale workspace doc is recoverable on the next write — but partial-delete state is now observable. - workspace::schema: documented why we don't cache compiled validators yet (settings/extension/skill writes are not a hot path; revisit if schema validation moves into a frequent write path). [skip-regression-check] schema.rs change is doc-only. * fix(pr-2049): address 4 remaining review issues 1. tool_dispatcher dropped during gateway startup src/channels/web/mod.rs: rebuild_state was initializing tool_dispatcher to None, so every subsequent with_* call zeroed the dispatcher the first caller injected. Preserve it across rebuild_state like every other field. Regression test: tool_dispatcher_survives_subsequent_with_calls. 2. WorkspaceSettingsAdapter not wired into runtime src/app.rs: Build the adapter in build_all() when workspace+db are both present, eagerly call ensure_system_config(), expose on AppComponents as settings_store, and thread it into init_extensions(...) so register_permission_tools and upgrade_tool_list receive it instead of the raw db. src/main.rs: SIGHUP handler prefers the adapter over raw db. src/workspace/mod.rs: re-export WorkspaceSettingsAdapter. 3. changed_by regression on layered writes src/workspace/mod.rs: write_to_layer and append_to_layer were passing the target layer's scope as changed_by, so version history attributed layered edits to the layer name instead of the actor. Pass self.user_id while keeping metadata resolution in the target scope. Regression test: layered_writes_record_actor_in_changed_by. 4. Legacy engine/ paths invisible after upgrade src/bridge/store_adapter.rs: Add migrate_legacy_engine_paths(), called at the start of load_state_from_workspace(), which scans list_all() for engine/... documents and rewrites them to .system/engine/... Idempotent: skips rewrites when the new path already exists, deletes the legacy duplicate either way. Three regression tests in #[cfg(all(test, feature = "libsql"))] module. Quality gate: cargo fmt, cargo clippy --all --all-features zero warnings, cargo test --all-features --lib 4313 passed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): use PUT for settings write in ownership test test_settings_written_and_readable was sending POST /api/settings/{key} but the route has been PUT since #4 (Feb 2026) — the test was returning 405 Method Not Allowed. Switch to httpx.put() so it matches the current route registration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pr-2049): address second round of review feedback Addresses the remaining unresolved PR #2049 review comments from serrrfirat and ilblackdragon. ## Changes ### ToolDispatcher — integration coverage + log level - src/tools/dispatch.rs: add two libsql-gated integration tests for the full dispatch pipeline: (a) persist an ActionRecord with sensitive params redacted in the audit row while the tool still sees the raw value, sanitized output populated; (b) honor the per-tool execution_timeout() and record a failure action. - Tests use a raw-SQL helper to find system-category jobs since list_agent_jobs_for_user intentionally filters them out. - Replace warn! with debug! on audit persistence failure — dispatch is reachable from interactive CLI/REPL sessions where warn!/info! output corrupts the terminal UI (CLAUDE.md Code Style → logging). ### WorkspaceSettingsAdapter — log level - src/workspace/settings_adapter.rs: same warn! → debug! fix on the delete_setting workspace failure path, for the same REPL reason. ### Schema validation — surface all errors - src/workspace/schema.rs: switch from jsonschema::validate to validator_for + iter_errors so users fixing a malformed setting see every violation in one round instead of playing whack-a-mole. Also distinguishes "invalid schema" from "invalid content" errors. - Regression tests: multiple_errors_are_all_reported and invalid_schema_is_distinguished_from_invalid_content. ### create_system_job — started_at + row growth docs - src/db/libsql/jobs.rs and src/history/store.rs: include started_at in the INSERT (set to the same instant as created_at/completed_at) so duration queries don't see NULL and "started but not completed" filters don't misclassify these rows. Fixed in both backends. - Add doc comments on both impls warning about row growth per dispatch call. Deleting rows would violate "LLM data is never deleted" (CLAUDE.md); if listing-query performance becomes a concern, prefer a partial index (WHERE category != 'system') over deletion. ### Lib test repair - src/channels/web/server.rs: extensions_setup_submit_handler Err branch now sets resp.activated = Some(false) so clients and the regression test see an explicit `false` rather than `null`. Also rename the test's fake channel to snake_case (test_failing_channel) so it matches the canonicalize-extension-names behavior from PR #2129 — previously the test was passing a dashed name and getting "Capabilities file not found" instead of the intended activation failure. ## Not addressed (false positive / deferred) - dispatch.rs:177 output_raw/output_sanitized swap — verified against ActionRecord::succeed(Option<String>, Value, Duration) and the worker's call site at job.rs:704; argument order is correct. - settings_adapter.rs:186 TOCTOU window — author self-classified as "Low / completeness" and no other code path writes to .system/settings/** without going through write_to_workspace. - schema.rs recompilation caching — deferred per earlier review. ## Quality gate - cargo fmt - cargo clippy --all --benches --tests --examples --all-features zero warnings - cargo test --all-features --lib: 4387 passed, 0 failed, 3 ignored Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pr-2049): address third round of review feedback Addresses unresolved comments from serrrfirat's "Paranoid Architect Review" and Copilot's third pass on the engine-state migration. ## src/workspace/settings_adapter.rs ### HIGH — Cross-tenant data leak through owner-scoped Workspace `Workspace` is constructed for a single user_id at AppBuilder time. Without gating, `set_setting("user_B", key, val)` would dual-write into the **owner's** workspace, and a subsequent `user_A.get_setting(...)` would return user_B's value: a real cross-user data leak. Fix: - Add `gate_user_id` field set to `workspace.user_id()` at construction. - All `SettingsStore` methods that touch the workspace now check `workspace_allowed_for(user_id)` first; non-owner callers fall through to the legacy table only — preserving their pre-#2049 behavior. - This matches the long-term plan: per-user settings live in the legacy table until a per-user `WorkspaceSettingsAdapter` (one per WorkspacePool entry) is wired up; admin/global settings go through the workspace-backed path so they pick up schema validation. Regression test: `workspace_settings_are_owner_gated_in_multi_tenant_mode` asserts (a) owner's workspace doc is not overwritten by a non-owner write, (b) each user reads back their own legacy value, and (c) a non-owner with no legacy entry must NOT see the owner's workspace value bleeding through. ### MEDIUM — Dual-write order Reverse `set_setting` and `set_all_settings` to write legacy first, workspace second. The legacy table is the source of truth during migration (it backs aggregate `list_settings` reads), so writing it first guarantees those readers always see a consistent value even if the workspace write fails. Failed workspace writes are self-healing on the next per-key read-miss. ### MEDIUM — `ensure_system_config_lazy` double-execution race Replace the manual `get()`/`set()` pattern with `OnceCell::get_or_try_init`. Two concurrent first-callers no longer both run `ensure_system_config()`. Functionally equivalent (idempotent either way) but no longer wasteful. ## src/bridge/store_adapter.rs ### MEDIUM — Migration drops document metadata (S3) `migrate_legacy_engine_paths` previously copied only `doc.content`, silently dropping the `metadata` column. Now calls `ws.update_metadata(new_doc.id, &doc.metadata)` after each write to preserve schema/skip_indexing/hygiene flags. Logged-not-fatal: content has already been moved, metadata loss is recoverable. Regression test: `migration_preserves_document_metadata` seeds a doc with custom metadata and asserts it survives the rewrite. ### MEDIUM — `ws.exists()` swallowed transient errors (Copilot) `unwrap_or(false)` on the existence check could cause the migrator to overwrite an existing `.system/engine/...` doc when storage hiccups. Now propagates the error (counts as failed step + `continue`), per Copilot's exact suggested patch. ### LOW — `list_all()` runs every startup (Copilot) Add a cheap preflight: `ws.list("engine")` first; only fall through to the recursive `list_all()` discovery when the directory listing returns at least one entry. Steady-state startups (post-migration) skip the full workspace scan entirely. Regression test: `migration_preflight_skips_full_scan_when_no_legacy_paths` asserts unrelated and already-migrated documents are untouched. ### MEDIUM — Counter undercount on `already_present` (S5) When `already_present` is true the legacy duplicate is still deleted, but the previous code skipped the `migrated += 1` increment, undercounting in debug logs. Fixed: `migrated` now counts every successful path migration including the already-present case. ### Documented — Version-history loss is acceptable scope (C1) Read-write-delete pattern means `memory_document_versions.document_id ON DELETE CASCADE` drops the legacy doc's version chain. Documented in the function-level doc comment as intentional + bounded: - v2 engine state is runtime state (rewritten on every mutation), not user-curated data - v2 was newly introduced in this PR — no production deployment with pre-existing curated history at risk - A path-preserving rename op would need new trait methods on both backends; out of scope for fix-forward. If a future caller needs history-preserving rename, it should be added to the storage layer properly, not bolted onto migration. ## Quality gate - cargo fmt - cargo clippy --all --benches --tests --examples --all-features zero warnings - cargo test --all-features --lib: 4390 passed, 0 failed, 3 ignored (+3 new tests on top of round 2) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pr-2049): address fourth round of review feedback Two latent issues flagged by serrrfirat in the latest review pass: 1. **Null schema permanently locks documents** (`src/workspace/schema.rs`). `serde_json` deserializes a metadata field of `"schema": null` as `Some(Value::Null)`, not `None`, so the upstream `if let Some(schema) = &metadata.schema` check passes through to `validate_content_against_schema`. There, `validator_for(Value::Null)` errors out and every subsequent write to that document is blocked — a latent DoS. Added an explicit `schema.is_null()` early-return guard at the top of the validator, plus a regression test (`null_schema_is_treated_as_no_op`) that asserts even non-JSON content passes when the schema is null. 2. **System job titles were raw source labels** (`src/history/store.rs`, `src/db/libsql/jobs.rs`). `create_system_job` set `title = source`, so any UI rendering `agent_jobs.title` would display dispatched system jobs as `channel:gateway` / `system` / etc. instead of a human-readable label. Both PostgreSQL and libSQL backends now write `format!("System: {source}")`. Updated the two dispatch integration tests that pinned the old format. Schema-recompilation comment (`schema.rs:47`) was acknowledged as "acceptable for now" by the reviewer; existing NOTE in the source already documents the caching trade-off and upgrade path, so no code change. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pr-2049): address fifth round of review feedback Eight comments from Copilot + serrrfirat. Real fixes for the load-bearing gaps; doc clarifications for the rest where the existing behavior is intentional. **Real code changes** - `src/tools/dispatch.rs` — enforce `tool.parameters_schema()` (JSON Schema) in the dispatch path. Previously the SafetyLayer validator only checked for injection patterns; channel/CLI/routine callers could pass arbitrary shapes and only discover the mismatch (or worse, silently malformed behavior) inside the tool itself. Now we run `jsonschema::validate(&tool.parameters_schema(), &normalized_params)` after the injection check, with a permissive-empty-schema fast path so tools that haven't yet declared a schema aren't penalised. Regression test `dispatch_rejects_params_violating_tool_schema` asserts a required-field violation is rejected before the tool is invoked. - `src/workspace/settings_adapter.rs` — `write_to_workspace` now calls `schema_for_key(key)` once and reuses the resolved schema for both pre-write validation and post-write metadata persistence (was called twice). Eliminates duplicate work and removes a theoretical divergence window if the schema registry ever became non-deterministic. - `src/workspace/settings_adapter.rs` — `ensure_system_config` now also rewrites the `.config` document content when its metadata is repaired, not just the metadata column. The metadata column is the inheritance source of truth, but having the doc's content silently diverge from it confuses anyone reading the doc directly to understand which inherited flags are active. - `src/error.rs` + `src/workspace/settings_schemas.rs` — new `WorkspaceError::InvalidPath { path, reason }` variant. Path/key rejection (path-traversal, character set, length) now surfaces as `InvalidPath`, not `SchemaValidation` — callers and downstream UIs can distinguish "your settings *key* has bad characters" from "your settings *value* failed JSON-Schema validation" without string-matching error messages. `validate_settings_key` returns the new variant; the one match site in `settings_adapter.rs::write_to_workspace` is updated. Regression test `validate_settings_key_returns_invalid_path_variant`. **Documentation-only fixes** - `src/tools/dispatch.rs` — clarify in the `dispatch()` doc-comment that `sanitize_tool_output` runs only against the persisted ActionRecord payload, NOT against the value returned to the caller. This mirrors `Worker::execute_tool` (the agent loop also receives the raw output so reasoning can be reproduced from history). Channels that forward dispatcher output to end users must run their own boundary sanitization at the channel edge. - `src/history/store.rs` + `src/db/libsql/jobs.rs` — `create_system_job` doc updated to explicitly state that system job timestamps do NOT reflect tool execution time (the row is INSERTed before the tool runs, with all three timestamps pinned to "now"). Consumers that need execution duration must read `job_actions.duration_ms` for the associated action rows. Restructuring to a two-phase INSERT+UPDATE was rejected: the audit row must be durable even if the dispatcher panics mid-tool, and the second write would double per-dispatch DB cost. - `src/workspace/schema.rs` — added baseline regression test `moderately_complex_schema_compiles_within_budget` that pins schema compile + validate latency for a moderately deep nested schema at <500ms wall-clock. Guards against orders-of-magnitude regressions from a future `jsonschema` upgrade or accidentally pathological schema construction. Hard limits on schema complexity are deferred (the real defense today is keeping schema-bearing paths under `.system/`, which is system-controlled). **Acknowledged, no change** - libSQL `create_system_job` unbounded row growth — already documented as intentional in the existing comment block, with the mitigation path spelled out (partial index on `WHERE category != 'system'` for listing queries). Rate-limiting dispatch would silently drop user-initiated actions, which is worse than unbounded retention. The "LLM data is never deleted" rule (CLAUDE.md) explicitly applies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
980d60ea45 |
[codex] Stabilize auth readiness and gate flows (#2050)
* Unify extension readiness and refresh dynamic tool leases * Fix v2 OAuth refresh and scope legacy credential fallback * Stabilize auth readiness and gate flows * Tighten auth token submission and OAuth fallback * Expose tool registry database handle * Handle expired runtime credentials in auth preflight * Fix E2E regressions on extension lifecycle branch * Normalize OAuth auth descriptors and flow launchers * Address review feedback on gate routing and latent actions * Apply formatter cleanup in tests * Address auth API review follow-ups * Generalize Google auth fallback and bundle alias metadata * Skip MCP OAuth when Authorization header is configured * Re-emit pending approval gates on follow-up * Open OAuth auth links in a new tab * Move shared OAuth runtime into auth module * Fix CI lint failures after staging merge * Unify OAuth resume and user greeting lifecycle * Ignore E2E virtualenv * Repair staging-merge build break in extension lifecycle paths The previous merge of staging into extension-lifecycle (commit |
||
|
firat.sertgoz
|
8aa094125b |
feat(engine): restage skill repair learning loop on staging (#1962)
* feat(engine): add skill repair learning loop * fix(engine): guard skill repair mission updates * fix(engine): persist skill repair provenance * fix(engine): address skill-repair PR review feedback - Fix hex formatting: iterate GenericArray bytes individually instead of relying on Display impl which produces debug-like output - Always recompute content hash from actual doc.content when archiving a revision to prevent drift from out-of-band writes - Prune repair history on rollback to remove records for versions newer than the one being restored - Combine collect_error_messages + collect_observed_actions into a single pass (collect_errors_and_actions) to avoid redundant event iteration - Document bounded revision eviction policy (cap at 10) - Add comment clarifying concurrent skill-repair / error-diagnosis triggers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): constrain skill repair updates * fix(engine): keep insights on completed threads * style(engine): satisfy fmt and clippy on mission.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> |
||
|
firat.sertgoz
|
482ee57c5f |
feat(tui): port full-featured Ratatui terminal UI onto staging (#1973)
* feat: port ratatui tui onto staging * Add TUI model picker for /model * Fix TUI CI lint failures * Format /tools output as vertical list * Restore TUI approval modal on thread switch * Re-emit pending approval events on follow-up messages * Improve TUI thread handling and activity UI * Sort TUI resume conversations by activity * fix(tui): address PR review feedback * Add TUI thread detail modal for activity sidebar * feat(tui): improve conversation scrolling UX - Mouse wheel: 1-line increments (was 3-line jumps) - PageUp/PageDown: full-page scroll based on viewport height (was 5 lines) - Add scrollbar widget on conversation right edge (track │, thumb ┃) - Add "↓ N more ↓ End to return" indicator when scrolled up - Add auto-follow (pinned_to_bottom) that disengages on scroll-up and re-engages when reaching bottom or pressing End - Clamp scroll offset to valid range (can't scroll past content) - Add End key binding to jump to bottom Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tui): use engine context pressure data for status bar The context bar was using cumulative session tokens (total_input + total_output) which grow unboundedly across turns, making the bar always show 100% after a few exchanges. Now uses the actual context window usage from ContextPressure events when available, falling back to cumulative tokens only before the first engine update arrives. Also syncs context_window from the engine's max_tokens so the limit reflects the real model capability instead of name-based heuristics. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tui): render markdown in thread detail modal The thread detail modal was displaying raw markdown text (plain line splitting). Now uses render_markdown() for proper formatting of headers, lists, bold, code blocks, etc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tui): hydrate sidebar with engine threads and routines at startup The TUI sidebar was empty until the first user message because EngineThreadList and RoutineUpdate events were only sent after processing a message. Now sends initial data right before the message loop so the activity panel shows existing threads and routines immediately on startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tui): use owner_id for engine thread hydration at startup list_engine_threads filters by user_id, so passing "" matched no threads. Now uses self.owner_id() which matches the TUI channel's user_id, so threads are visible in the sidebar immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tui): fix CI — type errors and formatting in TUI tests Wrap `started_at` and `updated_at` in `Some(...)` to match `Option<DateTime<Utc>>` after upstream struct change, and run `cargo fmt` on files with formatting drift. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): resolve clippy warnings — collapsible ifs and needless borrow Collapse three nested `if` blocks into `if && let` chains and remove a needless `&` on the `process_list_threads` call, all in agent_loop.rs. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): add live_harness.rs with updated StatusUpdate patterns The live_harness.rs file was added to staging after this branch diverged. When CI merges the PR into staging, the file uses old StatusUpdate patterns that don't account for the new `detail` and `call_id` fields added by this branch. Add the file with `..` rest patterns to fix the merge-time compile errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
37a7de43f3 |
[codex] Move safety benches into ironclaw_safety crate (#1954)
* Move safety benches into ironclaw_safety crate * Annotate benchmark JSON unwraps for panic check |
||
|
Illia Polosukhin
|
4c9a985bac |
feat(engine): Unified Thread-Capability-CodeAct execution engine (v2 architecture) (#1557)
* v2 architecture phase 1 * feat(engine): Phase 2 — execution loop, capability system, thread runtime Add the core execution engine to ironclaw_engine crate: - CapabilityRegistry: register/get/list capabilities and actions - LeaseManager: async lease lifecycle (grant, check, consume, revoke, expire) - PolicyEngine: deterministic effect-level allow/deny/approve - ThreadTree: parent-child relationship tracking - ThreadSignal/ThreadOutcome: inter-thread messaging via mpsc - ThreadManager: spawn threads as tokio tasks, stop, inject messages, join - ExecutionLoop: core loop replacing run_agentic_loop() with signals, context building, LLM calls, action execution, and event recording - Structured executor (Tier 0): lease lookup → policy check → effect execution - Tool intent nudge detection - MemoryStore + RetrievalEngine stubs for Phase 4 - Full 8-phase architecture plan in docs/plans/ - CLAUDE.md spec for the engine crate 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 3 — Monty Python executor with RLM pattern Add CodeAct execution (Tier 1) using the Monty embedded Python interpreter, following the Recursive Language Model (RLM) pattern from arXiv:2512.24601. Key additions: - executor/scripting.rs: Monty integration with FunctionCall-based tool dispatch, catch_unwind panic safety, resource limits (30s, 64MB, 1M allocs) - LlmResponse::Code variant + ExecutionTier::Scripting - Context-as-variables (RLM 3.4): thread messages, goal, step_number, previous_results injected as Python variables — LLM context stays lean while code accesses data selectively - llm_query(prompt, context) (RLM 3.5): recursive subagent calls from within Python code — results stored as variables, not injected into parent's attention window (symbolic composition) - Compact output metadata between code steps instead of full stdout - MontyObject ↔ serde_json::Value bidirectional conversion - Updated architecture plan with RLM design principles 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): RLM best-practices enhancements from cross-reference analysis Cross-referenced our implementation against the official RLM (alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation. Key enhancements: - FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching all three reference implementations. Code can signal completion at any point, not just via return value. - llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn, matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch. - Output truncation increased to 8000 chars (from 120), matching Prime Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT]. - Step 0 orientation preamble: auto-injects context metadata (message count, total chars, goal, last user message preview) before first code step, matching fast-rlm's auto-print pattern. - Error-to-LLM flow: Python parse errors, runtime errors, NameErrors, OS errors, and async errors now flow back as stdout content instead of terminating the step, enabling LLM self-correction on next iteration. Only VM panics (catch_unwind) terminate as EngineError. 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): update architecture plan with RLM cross-reference learnings Comprehensive update after cross-referencing against official RLM (alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), Prime Intellect (verifiers/RLMEnv), rlm-rs (zircote/rlm-rs), and Google ADK RLM. Changes: - Mark Phases 1-3 as DONE with commit refs and test counts - Add "Key Influences" section documenting all reference implementations - Phase 3: full table of implemented RLM features with sources - Phase 3: "Remaining gaps" table with which phase addresses each - Phase 4: expanded with compaction (85% context), rlm_query() (full recursive sub-agent), dual model routing, budget controls (USD, timeout, tokens, consecutive errors), lazy loading, pass-by-reference - Add "RLM Execution Model" cross-cutting section - Add "Implementation Progress" tracking table - Remove stale "TO IMPLEMENT" markers (all Phase 3 work is done) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 4 — budget controls, compaction, reflection pipeline Budget enforcement in ExecutionLoop: - max_tokens_total: cumulative token limit, checked before each iteration - max_duration: wall-clock timeout for entire thread - max_consecutive_errors: consecutive error steps threshold (resets on success, matching official RLM behavior) - All produce ThreadOutcome::Failed with descriptive messages Context compaction (from RLM paper, 85% threshold): - estimate_tokens(): char-based estimation (chars/4, matching RLM) - should_compact(): triggers when tokens >= threshold_pct * context_limit - compact_messages(): asks LLM to summarize progress, replaces history with [system, summary, continuation_note], preserves intermediate results - Configurable via ThreadConfig: model_context_limit, compaction_threshold Dual model routing: - LlmCallConfig gains depth field (0=root, 1+=sub-call) - Implementations can route to cheaper models for sub-calls - ExecutionLoop passes thread depth to every LLM call Reflection pipeline (reflection/pipeline.rs): - reflect(thread, llm): analyzes completed thread via LLM - Produces Summary doc (always), Lesson doc (if errors), Issue doc (if failed) - Builds transcript from thread messages + error events - Returns ReflectionResult with docs + token usage ThreadConfig extended with: max_tokens_total, max_consecutive_errors, model_context_limit, enable_compaction, compaction_threshold, depth, max_depth. 78 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 5 — conversation surface separated from execution Conversation is now a UI layer, not an execution boundary. Multiple threads can run concurrently within one conversation; threads can outlive their originating conversation. New types (types/conversation.rs): - ConversationSurface: channel + user + entries + active_threads - ConversationEntry: sender (User/Agent/System) + content + origin_thread_id - ConversationId, EntryId (UUID newtypes) - EntrySender enum (User, Agent{thread_id}, System) ConversationManager (runtime/conversation.rs): - get_or_create_conversation(channel, user) — indexed by (channel, user) - handle_user_message() — injects into active foreground thread or spawns new - record_thread_outcome() — adds agent/system entries, untracks completed threads - get_conversation(), list_conversations() This enables the key architectural insight: a user can ask "what's the weather?" while a deployment thread is still running. Both produce entries in the same conversation. 85 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): simplify execution tiers — Monty-only for CodeAct/RLM Restructure phases 6-8 to clarify execution model: - Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker Python runtimes for LLM-generated code. - WASM sandbox is for third-party tool isolation (existing infra, Phase 8) - Docker containers are for thread-level isolation of high-risk work (Phase 8) - Two-phase commit moves to Phase 6 (integration) at the adapter boundary Phase renumbering: - Old Phase 6 (Tier 2-3) → removed as separate phase - Old Phase 7 (integration) → Phase 6 - Old Phase 8 (cleanup) → Phase 7 - New Phase 8: WASM tools + Docker thread isolation (infra integration) Updated progress table: Phases 1-5 marked DONE with test counts and commits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 6 — bridge adapters for main crate integration Strategy C parallel deployment: when ENGINE_V2=true env var is set, user messages route through the engine instead of the existing agentic loop. All existing behavior is unchanged when the flag is off. Bridge module (src/bridge/): - LlmBridgeAdapter: wraps LlmProvider as engine LlmBackend, converts ThreadMessage↔ChatMessage, ActionDef↔ToolDefinition, depth-based model routing (primary vs cheap_llm) - EffectBridgeAdapter: wraps ToolRegistry+SafetyLayer as EffectExecutor, routes tool calls through existing execute_tool_with_safety pipeline - InMemoryStore: HashMap-backed Store impl (no DB tables needed yet) - EngineRouter: is_engine_v2_enabled() + handle_with_engine() that builds engine from Agent deps and processes messages end-to-end Integration touchpoint (4 lines in agent_loop.rs): After hook processing, before session resolution, check ENGINE_V2 flag and route UserInput through the engine path. Accessor visibility widened: llm(), cheap_llm(), safety(), tools() changed from pub(super) to pub(crate) for bridge access. 85 engine tests + main crate clippy clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): add user message and system prompt to thread before execution The ExecutionLoop was sending empty messages to the LLM because the thread was spawned with the user's input as the goal but no messages. Fixes: - ThreadManager.spawn_thread() now adds the goal as an initial user message before starting the execution loop - ExecutionLoop.run() injects a default system prompt if none exists Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): match existing LLM request format to prevent 400 errors The LLM bridge was missing several defaults that the existing Reasoning.respond_with_tools() sets: - tool_choice: "auto" when tools are present (required by some providers) - max_tokens: 4096 (default) - temperature: 0.7 (default) - When no tools (force_text): use plain complete() instead of complete_with_tools() with empty tools array — matches existing no-tools fallback path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): persist conversation context across messages The engine was creating a fresh ThreadManager and InMemoryStore per message, losing all context between turns. A follow-up question like "what are the latest 10 issues?" had no memory of the prior "how many issues" response. Fixes: - EngineState (ThreadManager, ConversationManager, InMemoryStore) now persists across messages via OnceLock, initialized on first use - ConversationManager builds message history from prior conversation entries (user messages + agent responses) and passes it to new threads - ThreadManager.spawn_thread_with_history() accepts initial_messages that are prepended before the current user message - System notifications (thread started/completed) are filtered out of the history (not useful as LLM context) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): enable CodeAct/RLM mode with code block detection The engine now operates in CodeAct/RLM mode: System prompt (executor/prompt.rs): - Instructs LLM to write Python in ```repl fenced blocks - Documents available tools as callable Python functions - Documents llm_query(), llm_query_batched(), FINAL() - Documents context variables (context, goal, step_number, previous_results) - Strategy guidance: examine context, break into steps, use tools, call FINAL() Code block detection (bridge/llm_adapter.rs): - extract_code_block() scans LLM text responses for ```repl or ```python blocks - When detected, returns LlmResponse::Code instead of LlmResponse::Text - The ExecutionLoop routes Code responses through Monty for execution No structured tool definitions sent to LLM: - Tools are described in the system prompt as Python functions - The LLM call sends empty actions array, forcing text-mode responses - This ensures the LLM writes code blocks (CodeAct) instead of structured tool calls (which would bypass the REPL) 85 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(engine): add 8 CodeAct/RLM E2E tests with mock LLM Comprehensive test coverage for the Monty Python execution path: - codeact_simple_final: Python code calls FINAL('answer') → thread completes - codeact_tool_call_then_final: code calls test_tool() → FunctionCall suspends VM → MockEffects returns result → code resumes → FINAL() - codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15') with no tool calls — pure Python in Monty - codeact_multi_step: first step prints output (no FINAL), second step sees output metadata and calls FINAL — tests iterative REPL flow - codeact_error_recovery: first step has NameError → error flows to LLM as stdout → second step recovers with FINAL — tests error transparency - codeact_context_variables_available: code accesses `goal` and `context` variables injected by the RLM context builder - codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times → 3 FunctionCall suspensions → all results collected → FINAL - codeact_llm_query_recursive: code calls llm_query('prompt') → VM suspends → MockLlm provides sub-agent response → result returned as Python string variable 93 tests passing (85 prior + 8 new), zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): detect code blocks in plain completion path + multi-block support Two bugs fixed: 1. The no-tools completion path (used by CodeAct since we send empty actions) returned LlmResponse::Text without checking for code blocks. Code blocks were rendered as markdown text instead of being executed. 2. extract_code_block now: - Handles bare ``` fences (skips non-Python languages) - Collects ALL code blocks in the response and concatenates them (models often split code across multiple blocks with explanation) - Tries markers in order: ```repl, ```python, ```py, then bare ``` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(bridge): add 11 regression tests for code block extraction Covers the exact failure modes discovered during live testing: - extract_repl_block: standard ```repl fenced block - extract_python_block: ```python marker - extract_py_block: ```py shorthand - extract_bare_backtick_block: bare ``` with Python content - skip_non_python_language: ```json should NOT be extracted - no_code_blocks_returns_none: plain text, no fences - multiple_code_blocks_concatenated: two ```repl blocks with explanation between them → concatenated with \n\n - mixed_thinking_and_code: model outputs explanation + two ```python blocks (the Hyperliquid case) → both extracted - repl_preferred_over_bare: ```repl takes priority over bare ``` - empty_code_block_skipped: empty fenced block returns None - unclosed_block_returns_none: no closing ``` returns None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): detect FINAL() in text responses + regression tests Models sometimes write FINAL() outside code blocks — as plain text after an explanation. The Hyperliquid case: model outputs a long analysis then FINAL("""...""") at the end, not inside ```repl fences. Fixes: - extract_final_from_text(): regex-based FINAL detection in text responses, matching the official RLM's find_final_answer() fallback - Handles: double-quoted, single-quoted, triple-quoted, unquoted, nested parens - Checked in LlmResponse::Text handler BEFORE tool intent nudge (FINAL takes priority) 9 new tests: - codeact_final_in_text_response: FINAL("answer") in plain text - codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text - final_double_quoted, final_single_quoted, final_triple_quoted, final_unquoted, final_with_nested_parens, final_after_long_text, no_final_returns_none 102 tests passing (93 + 9 new), zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add crate extraction & cleanup roadmap Documents architectural recommendations from the engine v2 design process for future reference: - Root directory consolidation (channels-src + tools-src → extensions/) - Crate extraction tiers: zero-coupling (estimation, observability, tunnel), trivial-coupling (document_extraction, pairing, hooks), medium-coupling (secrets, MCP, db, workspace, llm, skills), heavy-coupling (web gateway, agent, extensions) - src/ module reorganization into logical groups (core, persistence, infra, media, support) - main.rs/app.rs slimming targets (100/500 lines after migration) - WASM module candidates (document_extraction) and non-candidates (REPL, web gateway → separate crates instead) - Priority ordering for extraction work - Tracks completed items (ironclaw_safety, ironclaw_engine, transcription move) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): live progress status updates via event broadcast Engine v2 now shows live progress in the CLI (and any channel): - "Thinking..." when a step starts - Tool name + success/error when actions execute - "Processing results..." when a step completes Implementation: - ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256) - ExecutionLoop.emit_event() writes to thread.events AND broadcasts - ThreadManager.subscribe_events() returns a receiver - Router uses tokio::select! to listen for events while waiting for thread completion, forwarding them as StatusUpdate to the channel This replaces the polling approach with zero-latency event streaming. Agent.channels visibility widened to pub(crate) for bridge access. 102 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): include tool results in code step output for LLM context The LLM was ignoring tool results and answering from training data because the compact output metadata didn't include what tools returned. Tool results lived only as ActionResult messages (role: Tool) which some providers flatten or the model ignores. Now the code step output includes: - stdout from Python print() statements - [tool_name result] with the actual output (truncated to 4K per tool) - [tool_name error] for failed tools - [return] for the code's return value - Total output truncated to 8K chars to prevent context bloat This ensures the model sees web_search results, API responses, etc. in the next iteration and can reason about them instead of hallucinating. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): add debug/trace logging for CodeAct execution Three verbosity levels for debugging the engine: RUST_LOG=ironclaw_engine=debug: - LLM call: message count, iteration, force_text - LLM response: type (text/code/action_calls), token usage - Code execution: code length, action count, had_error, final_answer - Text response: length, FINAL() detection RUST_LOG=ironclaw_engine=trace: - Full message list sent to LLM (role, length, first 200 chars each) - Full code block being executed - stdout preview (first 500 chars) - Per-tool results (name, success, first 300 chars of output) - Text response preview (first 500 chars) Usage: ENGINE_V2=true RUST_LOG=ironclaw_engine=debug cargo run ENGINE_V2=true RUST_LOG=ironclaw_engine=trace cargo run Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): execution trace recording + retrospective analysis Enable with ENGINE_V2_TRACE=1 to get full execution traces and automatic issue detection after each thread completes. Trace recording (executor/trace.rs): - build_trace(): captures full thread state — messages (with full content), events, step count, token usage, detected issues - write_trace(): writes JSON to engine_trace_{timestamp}.json - log_trace_summary(): logs summary + issues at info/warn level Retrospective analyzer detects 8 issue categories: - thread_failure: thread ended in Failed state - no_response: no assistant message generated - tool_error: specific tool failures with error details - code_error: Python errors (NameError, SyntaxError, etc.) in output - missing_tool_output: tool results exist but not in system messages - excessive_steps: >10 steps (may be stuck in loop) - no_tools_used: single-step answer without tools (hallucination risk) - mixed_mode: text responses without code blocks (prompt not followed) Thread state now saved to store after execution completes (for trace access after join_thread). Usage: ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run # After each message: trace JSON + issue log in terminal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): wire reflection pipeline + trace analysis into thread lifecycle After every thread completes, ThreadManager now automatically runs: 1. Retrospective trace analysis (non-LLM, always): - Detects 8 issue categories (tool errors, code errors, missing outputs, excessive steps, hallucination risk, etc.) - Logs issues at warn level when found 2. Trace file recording (when ENGINE_V2_TRACE=1): - Writes full JSON trace to engine_trace_{timestamp}.json 3. LLM reflection (when enable_reflection=true): - Calls reflection pipeline to produce Summary, Lesson, Issue docs - Saves docs to store for future context retrieval - Enabled by default in the bridge router All three run inside the spawned tokio task after exec.run() completes, before saving the final thread state. No external wiring needed. Removed duplicate trace recording from the router — it's now handled by ThreadManager automatically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): convert tool name hyphens to underscores for Python compatibility Root cause from trace analysis: the LLM writes `web_search()` (valid Python identifier) but the tool registry has `web-search` (with hyphen). The EffectBridgeAdapter couldn't find the tool → "Tool not found" error → model fabricated fake data instead. Fixes: - available_actions(): converts tool names from hyphens to underscores (web-search → web_search) so the system prompt lists valid Python names - execute_action(): tries the original name first, then falls back to hyphenated form (web_search → web-search) for tool registry lookup - Same conversion in router's capability registry builder Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): parse JSON tool output to prevent double-serialization From trace analysis: web_search returned a JSON string, which was wrapped as serde_json::json!(string) creating a Value::String containing JSON. When Monty got this as MontyObject::String, the Python code couldn't index it with result['title'] → TypeError. Fix: try parsing the tool output string as JSON first. If valid, use the parsed Value (becomes a Python dict/list). If not valid JSON, keep as string. This means web_search results are directly indexable in Python: results = web_search(query="...") print(results["results"][0]["title"]) # works now Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): persist variables across code steps via `state` dict Monty creates a fresh runtime per code step, so variables are lost between steps. This caused the model to re-paste tool results from system messages, wasting tokens. Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that accumulates across steps: - Tool results stored by tool name: state["web_search"] = {results...} - Return values stored: state["last_return"], state["step_0_return"] - Injected as a `state` Python variable in each new MontyRun Now the model can do: Step 1: results = web_search(query="...") # tool result saved in state Step 2: data = state["web_search"] # access previous result summary = llm_query("summarize", str(data)) FINAL(summary) System prompt updated to document the `state` variable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): add state hint on code errors + retrieval engine integration When code fails with NameError/UnboundLocalError (model trying to access variables from a previous step), the error output now includes: [HINT] Variables don't persist between code blocks. Use the `state` dict to access data from previous steps. Available keys: ["web_search", "last_return"] This teaches the model to use `state["web_search"]` instead of `result` after a NameError, reducing wasted steps from 3-4 to 1. Also integrates RetrievalEngine into context building and ThreadManager: - build_step_context() now accepts optional RetrievalEngine to inject relevant memory docs (Lessons, Specs, Playbooks) into LLM context - RetrievalEngine uses keyword matching with doc-type priority scoring - Memory docs from reflection (Phase 4) now feed back into future threads Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove trace files and add to .gitignore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): replace web_fetch example with web_search in CodeAct prompt The system prompt example used web_fetch(url="...") which doesn't exist as a tool. The model learned from the example and tried web_fetch, getting "Tool not found". Changed to web_search(query="...") which is an actual registered tool. Found via trace analysis — reflection pipeline correctly identified this as a "Tool Name Correction" spec doc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(engine): extract prompt templates to markdown files Prompt templates moved from inline Rust strings to plain markdown files at crates/ironclaw_engine/prompts/ for easy inspection and iteration: - prompts/codeact_preamble.md — main instructions, special functions, context variables, rules - prompts/codeact_postamble.md — strategy section Loaded at compile time via include_str!(), so no runtime file I/O. Edit the .md files and rebuild to iterate on prompts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): replace byte-index slicing with char-safe truncation Panic: 'byte index 80 is not a char boundary; it is inside ''' when tool output contained multi-byte UTF-8 characters (smart quotes from web search results). Fixed 4 unsafe byte-index slices: - thread.rs:281: message preview &content[..80] → chars().take(80) - loop_engine.rs:556: tool output &str[..4000] → chars().take(4000) - loop_engine.rs:579: output tail &str[len-8000..] → chars().skip() - scripting.rs:82: stdout tail &str[len-N..] → chars().skip() All now use .chars().take() or .chars().skip() which respect character boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on user-supplied or external strings." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): fix false positive missing_tool_output warning in trace analyzer The check was looking for "[" + "result]" in System-role messages only, but tool output metadata is added with patterns like "[shell result]" and may appear in messages with any role. Changed to scan all messages for " result]" or " error]" patterns regardless of role. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): update architecture plan with Phase 6 status and approval flow design Phase 6 updated to reflect what was actually built: - Bridge adapters (LLM, Effect, InMemoryStore, Router) — all done - Integration touchpoint (4 lines in handle_message) — done - Live progress via broadcast events — done - Conversation persistence across messages — done - Trace recording + retrospective analysis — done - 8 bugs found and fixed via trace analysis — documented Phase 6 remaining work documented: - Approval flow: detailed 5-step design (send to channel, pause thread, route response, resume execution, always handling) with v1 reference - Database persistence (InMemoryStore → real DB tables) - Acceptance testing (TestRig + TraceLlm fixtures) - Two-phase commit for high-stakes effects Progress table updated: Phase 6 marked as DONE (partial), 134 tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add self-improving engine design plan Designs a system where the engine debugs and improves itself, based on the pattern observed in the last session: 5 consecutive bug fixes all followed trace → read → identify → edit → test, using tools the engine already has access to. Three levels of self-improvement: - Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply. - Level 2 (Config): adjust defaults/mappings. Branch + test + PR. - Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR. Architecture: Self-improvement Mission spawns a Reflection thread that reads traces, reads source, proposes fixes, validates via cargo test, and either auto-applies (Level 1) or creates a PR (Level 2-3). Includes: fix pattern database (seeded from our 8 debugging session fixes), feedback loop diagram, safety model, implementation phases (A through D), and what exists vs what's new. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add engine v2 security model and audit Comprehensive security analysis of engine v2 covering: Threat model: 4 attacker profiles (malicious input, prompt injection via tools, poisoned memory, supply chain). Current state audit: 9 controls working (Monty sandbox, safety layer, policy engine, leases, provenance, events) and 9 gaps identified. Critical finding: ALL tools granted by default — CodeAct code can call shell, write_file, apply_patch without approval. Proposed fix: 3-tier tool classification (auto/approve-once/always-approve). CodeAct-specific threats: tool call amplification, prompt injection via search results, data exfiltration via tool chains, Monty escape. Self-improvement security: poisoned trace attacks, memory poisoning via reflection. Mitigations: edit validation, frequency caps, audit trail, auto-rollback, reflection output scanning. 6-layer security architecture proposed: input validation, capability gating, output sanitization, execution sandboxing, self-improvement controls, observability. Prioritized implementation plan with severity/effort ratings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(security): cross-reference v1 controls — use, don't reinvent Updated security plan with detailed audit of ALL existing v1 security controls and how they map to engine v2 bridge gaps: Key finding: v1 already has solutions for every security gap identified. The bridge just needs to wire them in: - Tool::requires_approval() exists but bridge doesn't call it - safety.wrap_for_llm() exists but tool results enter context unwrapped - RateLimiter exists but bridge doesn't check rate limits - BeforeToolCall hooks exist but bridge doesn't run them - redact_params() exists but bridge doesn't redact sensitive params - Shell risk classification (Low/Medium/High) is inherited but ignored Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter, not new security infrastructure. The bridge is the security boundary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): add missions, reliability tracker, reflection executor, and provenance-aware policy - Add Mission type and MissionManager for recurring thread scheduling - Add ReliabilityTracker for per-capability success/failure/latency tracking - Add reflection executor that spawns CodeAct threads for post-completion reflection - Extend PolicyEngine with provenance-aware taint checking (LLM-generated data requires approval for financial/external-write effects) - Extend Store trait with mission CRUD methods - Add conversation surface tracking, compaction token fix, context memory injection - Wire new modules through lib.rs re-exports and bridge adapters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): wire v1 security controls into engine v2 adapter Zero engine crate changes. All security controls enforced at the bridge boundary in EffectBridgeAdapter: 1. Tool approval (v1: Tool::requires_approval): - Checks each tool's approval requirement with actual params - Always → returns EngineError::LeaseDenied (blocks execution) - UnlessAutoApproved → checks auto_approved set, blocks if not approved - Never → proceeds - Per-session auto_approved HashSet (for future "always" handling) 2. Hook interception (v1: BeforeToolCall): - Runs HookEvent::ToolCall before every execution - HookOutcome::Reject → blocks with reason - HookError::Rejected → blocks with reason - Hook errors → fail-open (logged, execution continues) 3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm): - Leak detection: API keys in tool output are redacted - Policy enforcement: content policy rules applied - Length truncation: output capped at 100KB - XML boundary protection: prevents injection via tool output 4. Sensitive param redaction (v1: redact_params): - Tool's sensitive_params() consulted before hooks see parameters - Redacted params sent to hooks, original params used for execution 5. available_actions() now sets requires_approval based on each tool's default approval requirement, so the engine's PolicyEngine can gate tools it hasn't seen before. 6. Actual execution timing measured via Instant::now() (replaces placeholder Duration::from_millis(1)). Accessor visibility: hooks() widened to pub(crate). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): implement tool approval flow for engine v2 Adds a complete approval flow that mirrors v1 behavior, using the existing v1 security controls (Tool::requires_approval, auto-approve sets, StatusUpdate::ApprovalNeeded). ## How it works ### Step 1: Tool blocked at execution When the LLM's code calls a tool (e.g., `shell("ls")`): 1. EffectBridgeAdapter.execute_action() looks up the Tool object 2. Calls tool.requires_approval(¶ms) — returns ApprovalRequirement 3. If Always → EngineError::LeaseDenied (always blocks) 4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set, returns EngineError::LeaseDenied 5. If Never → proceeds to execution ### Step 2: Engine returns NeedApproval The LeaseDenied error propagates through: - CodeAct path: becomes Python RuntimeError, code halts, thread returns NeedApproval with action_name + parameters - Structured path: same via ActionResult.is_error ### Step 3: Router stores pending approval - PendingApproval { action_name, original_content } stored on EngineState - StatusUpdate::ApprovalNeeded sent to channel (shows approval card in CLI/web with tool name, parameters, yes/always/no buttons) - Returns text: "Tool 'shell' requires approval. Reply yes/always/no." ### Step 4: User responds handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2: - 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes original message (tool now passes the approval check on second run) - 'always' → same + logs for session persistence - 'no' → returns "Denied: tool was not executed." ### Key design choice Instead of pausing/resuming mid-execution (which needs engine changes to freeze/restore the Monty VM state), we auto-approve the tool and re-run the full message. The EffectBridgeAdapter's auto_approved set persists across runs, so the second execution passes immediately. This trades one extra LLM call for zero engine modifications. ## Files changed - src/bridge/router.rs: PendingApproval struct, handle_approval(), NeedApproval → StatusUpdate::ApprovalNeeded conversion - src/bridge/mod.rs: export handle_approval - src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2 - src/bridge/effect_adapter.rs: fmt fixes 151 tests passing, clippy + fmt clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): demote trace/reflection logging from info to debug INFO-level log output from background tasks (trace analysis, reflection) corrupts the REPL terminal UI. The trace summary, issue warnings, and reflection doc previews were printing mid-approval-card, breaking the interactive display. Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!. Trace analysis and reflection results now only show when RUST_LOG=ironclaw_engine=debug is set. Also added logging discipline rule to global CLAUDE.md: - info! → user-facing status the REPL intentionally renders - debug! → internal diagnostics (traces, reflection, engine internals) - Background tasks must NEVER use info! — it breaks the TUI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): demote all router info! logging to debug! "engine v2: initializing" and "engine v2: handling message" were printing at INFO level, corrupting the REPL UI. All router logging now uses debug! — only visible with RUST_LOG=ironclaw=debug. Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(safety): demote leak detector warn-action logs from warn! to debug! The leak detector's Warn-action matches (high_entropy_hex pattern on web search results containing commit SHAs, CSS colors, URL hashes) were logging at warn! level, corrupting the REPL UI with lines like: WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5 These are informational false positives — real leaks use LeakAction::Redact which silently modifies the content. Warn-action matches only log for debugging purposes and should not appear in production output. Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): strengthen CodeAct prompt to prevent shallow text answers The model was answering "Suggested 45 improvements" as a brief text summary from training data without actually searching or listing them. The trace showed: no code block, no tool calls, no FINAL(). Prompt changes: - Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with plain text only." (was: "Always write code... plain text for brief explanations") - Rule 2 (NEW): "NEVER answer from memory or training data alone. Always use tools to get real, current information before answering." - Rule 3: FINAL answer "should be detailed and complete — not just a summary like 'found 45 items'" - Rule 8 (NEW): "Include the actual content in your FINAL() answer, not just a count or summary. Users want to see the details." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): persist reflection docs to workspace for cross-session learning Replaces InMemoryStore with HybridStore: - Ephemeral data (threads, steps, events, leases) stays in-memory - MemoryDocs (lessons, specs, playbooks from reflection) persist to the workspace at engine/docs/{type}/{id}.json On engine init, load_docs_from_workspace() reads existing docs back into the in-memory cache. This means: - Lessons learned in session 1 are available in session 2 - The RetrievalEngine injects relevant past lessons into new threads - The engine genuinely improves over time as reflection accumulates Workspace paths: engine/docs/lessons/{uuid}.json engine/docs/specs/{uuid}.json engine/docs/playbooks/{uuid}.json engine/docs/summaries/{uuid}.json engine/docs/issues/{uuid}.json No new database tables. Uses existing workspace write/read/list. workspace() accessor widened to pub(crate). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): adapt to execute_tool_with_safety params-by-value change Staging merge changed execute_tool_with_safety to take params by value instead of by reference (perf optimization from PR #926). Updated bridge adapter to clone params before passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): add web gateway integration plan to Phase 6 Documents three gaps between engine v2 and the web gateway: 1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent) 2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB) 3. No cross-channel visibility (REPL ↔ web messages invisible to each other) Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1 conversation tables after thread completion. Prerequisite: AppEvent extraction PR (in progress separately). Also updated DB persistence status: HybridStore with workspace-backed MemoryDocs is now implemented (partial persistence). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): document routine/job gap and SIGKILL crash scenario Routines are entirely v1 — not hooked up to engine v2. When a user asks "create a routine" as natural language, engine v2 tries to call routine_create via CodeAct, but the tool needs RoutineEngine + Database refs that the bridge's minimal JobContext doesn't provide. This caused a SIGKILL crash during testing. Options documented: block routine tools in v2 (short term), pass refs through context (medium), replace with Mission system (long term). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: extract AppEvent to crates/ironclaw_common SseEvent was defined in src/channels/web/types.rs but imported by 12+ modules across agent, orchestrator, worker, tools, and extensions — it had become the application-wide event protocol, not a web transport concern. Create crates/ironclaw_common as a shared workspace crate and move the enum there as AppEvent. Also move the truncate_preview utility which was similarly leaked from the web gateway into agent modules. - New crate: crates/ironclaw_common (AppEvent, truncate_preview) - Rename SseEvent → AppEvent, from_sse_event → from_app_event - web/types.rs re-exports AppEvent for internal gateway use - web/util.rs re-exports truncate_preview - Wire format unchanged (serde renames are on variants, not the enum) Aligned with the event bus direction on refactor/architectural-hardening where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): integrate with web gateway via AppEvent + v1 conversation DB Three changes to make engine v2 visible in the web gateway: 1. SSE event streaming (AppEvent broadcast): - ThreadEvent → AppEvent conversion via thread_event_to_app_event() - Events broadcast to SseManager during the poll loop - Covers: Thinking, ToolCompleted (success/error), Status, Response - Web gateway receives real-time progress without any gateway changes 2. Conversation persistence to v1 database: - After thread completes, writes user message + agent response to v1 ConversationStore via add_conversation_message() - Uses get_or_create_assistant_conversation() for per-user per-channel - Web gateway reads from DB as usual — chat history appears 3. Final response broadcast: - AppEvent::Response with full text + thread_id sent via SSE - Web gateway renders the response in the chat UI New EngineState fields: sse (Option<Arc<SseManager>>), db (Option<Arc<dyn Database>>). Both populated from Agent.deps. Agent.deps visibility widened to pub(crate). Depends on: ironclaw_common crate with AppEvent type (PR #1615). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): complete Phase 6 — v1-only tool blocking, rate limiting, call limits Three security/stability improvements in EffectBridgeAdapter: 1. V1-only tool blocking: - routine_create, create_job, build_software (and hyphenated variants) return helpful error: "use the slash command instead" - Filtered out of available_actions() so system prompt doesn't list them - Prevents crash from tools needing RoutineEngine/Scheduler refs 2. Per-step tool call limit: - Max 50 tool calls per code block (AtomicU32 counter) - Prevents amplification: `for i in range(10000): shell(...)` - Returns "call limit reached, break into multiple steps" 3. Rate limiting: - Per-user per-tool sliding window via RateLimiter - Checks tool.rate_limit_config() before every execution - Returns "rate limited, try again in Ns" Architecture plan updated: - Gateway integration: DONE - Routines: BLOCKED (gracefully, with slash command fallback) - Rate limiting: DONE - Call limit: DONE - Phase 6 status: DONE (remaining: acceptance tests, two-phase commit) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Mission system design — goal-oriented autonomous threads Missions replace routines with evolving, knowledge-accumulating autonomous agents. Unlike routines (fixed prompt, stateless), Missions: - Generate prompts from accumulated Project knowledge (lessons, playbooks, issues from prior threads) - Adapt approach when something fails repeatedly - Track progress toward a goal with success criteria - Self-manage: pause when stuck, complete when goal achieved Architecture: MissionManager with cron ticker spawns threads via ThreadManager. Meta-prompt built from mission goal + Project MemoryDocs via RetrievalEngine. Reflection feeds back automatically. 6-step implementation plan: cron trigger, meta-prompt builder, bridge wiring, CodeAct tools, progress tracking, persistence. Includes two worked examples: daily tech news briefing (ongoing) and test coverage improvement (goal-driven, self-completing). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): extend Mission types with webhook/event triggers + evolving strategy Mission types updated to support external activation sources: MissionCadence expanded: - Cron { expression, timezone } — timezone-aware scheduling - OnEvent { event_pattern } — channel message pattern matching - OnSystemEvent { source, event_type } — structured events from tools - Webhook { path, secret } — external HTTP triggers (GitHub, email, etc.) - Manual — explicit triggering only The engine defines trigger TYPES. The bridge implements infrastructure (cron ticker, webhook endpoints, event matchers). GitHub issues, PRs, email, Slack events all use the generic Webhook cadence — no special-casing in the engine. Webhook payload injected as state["trigger_payload"] in the thread's Python context. Mission struct extended: - current_focus: what the next thread should work on (evolving) - approach_history: what we've tried (for adaptation) - max_threads_per_day / threads_today: daily budget - last_trigger_payload: webhook/event data for thread context Plan updated with trigger type table and webhook integration design. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): implement MissionManager execution with meta-prompts The MissionManager now builds evolving meta-prompts and processes thread outcomes for continuous learning: fire_mission() upgraded: - Loads Project MemoryDocs via RetrievalEngine for context - Builds meta-prompt from: goal, current_focus, approach_history, project knowledge docs, trigger payload, thread count - Spawns thread with meta-prompt as user message - Background task waits for completion and processes outcome - Daily thread budget enforcement (max_threads_per_day) Meta-prompt structure: # Mission: {name} Goal: {goal} ## Current Focus (evolves between threads) ## Previous Approaches (what we've tried) ## Knowledge from Prior Threads (lessons, playbooks, issues) ## Trigger Payload (webhook/event data if applicable) ## Instructions (accomplish step, report next focus, check goal) Outcome processing: - Extracts "next focus:" from FINAL() response → updates current_focus - Detects "goal achieved: yes" → completes mission - Records accomplishment in approach_history - Failed threads recorded as "FAILED: {error}" Cron ticker: - start_cron_ticker() spawns tokio task, ticks every 60s - Checks active Cron missions, fires those past next_fire_at 151 tests passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): wire MissionManager into engine v2 for CodeAct access Missions are now callable from CodeAct Python code: ```python # Create a daily briefing mission result = mission_create( name="Tech News", goal="Daily AI/crypto/software news briefing", cadence="0 9 * * *" ) # List all missions missions = mission_list() # Manually fire a mission mission_fire(id="...") # Pause/resume mission_pause(id="...") mission_resume(id="...") ``` Implementation: - MissionManager created on engine init, cron ticker started - EffectBridgeAdapter intercepts mission_* function calls before tool lookup and routes to MissionManager - parse_cadence() handles: "manual", cron expressions, "event:pattern", "webhook:path" - Mission functions documented in CodeAct system prompt - MissionManager set on adapter via set_mission_manager() after init (avoids circular dependency) System prompt updated with mission_create, mission_list, mission_fire, mission_pause, mission_resume documentation. 151 tests passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): map routine_* calls to mission operations in v2 When the model calls routine_create, routine_list, routine_fire, routine_pause, routine_resume, or routine_delete, the bridge now routes them to the MissionManager instead of blocking with an error. Mapping: routine_create → mission_create (with cadence parsing) routine_list → mission_list routine_fire → mission_fire routine_pause → mission_pause routine_resume → mission_resume routine_update → mission_pause/resume (based on params) routine_delete → mission_complete (marks as done) Routine tools removed from v1-only blocklist and restored in available_actions(). The model can use either "routine" or "mission" vocabulary — both work. Still blocked: create_job, cancel_job, build_software (need v1 Scheduler/ContainerJobManager refs). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(engine): add E2E mission flow tests — 7 new tests Comprehensive mission lifecycle tests: - fire_mission_builds_meta_prompt_with_goal: verifies thread spawned with project context and recorded in history - outcome_processing_extracts_next_focus: "Next focus: X" in FINAL() response → mission.current_focus updated - outcome_processing_detects_goal_achieved: "Goal achieved: yes" → mission status transitions to Completed - mission_evolves_via_direct_outcome_processing: 3-step evolution: step 1 sets focus to "db module", step 2 evolves to "tools module", step 3 detects goal achieved → mission completes. Tests the full learning loop without background task timing dependencies. - fire_with_trigger_payload: webhook payload stored on mission and threads_today counter incremented - daily_budget_enforced: max_threads_per_day=1 → first fire succeeds, second returns None 157 tests passing (151 prior + 6 new mission E2E). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): self-improving engine via Mission system Wire the self-improvement loop as a Mission with OnSystemEvent cadence, inspired by karpathy/autoresearch's program.md approach. The mission fires when threads complete with issues, receives trace data as trigger payload, and uses tools directly to diagnose and fix problems. Key changes: Engine self-improvement (Phase A+B from design doc): - Add fire_on_system_event() to MissionManager for OnSystemEvent cadence - Add start_event_listener() that subscribes to thread events and fires matching missions when non-Mission threads complete with trace issues - Add ensure_self_improvement_mission() with autoresearch-style goal prompt (concrete loop steps, not vague instructions) - Add process_self_improvement_output() for structured JSON fallback - Seed fix pattern database with 8 known patterns from debugging - Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now async + Store-aware, appends learned rules from prompt_overlay docs) - Pass Store to ExecutionLoop for overlay loading Bridge review fixes (P1/P2): - Scope engine v2 SSE events to requesting user (broadcast_for_user) - Per-user pending approvals via HashMap instead of global Option - Reset tool-call limit counter before each thread execution - Only persist auto-approval when user chose "always", not one-off "yes" - Remove dead store/mission_manager fields from EngineState Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add checkpoint-based engine thread recovery * feat(engine): add Python orchestrator module and host functions Add the orchestrator infrastructure for replacing the Rust execution loop with versioned Python code. This commit adds the module and host functions without switching over — the existing Rust loop is unchanged. New files: - orchestrator/default.py: v0 Python orchestrator (run_loop + helpers) - executor/orchestrator.rs: host function dispatch, orchestrator loading from Store with version selection, OrchestratorResult parsing Host functions exposed to orchestrator Python via Monty suspension: __llm_complete__, __execute_code_step__ (nested Monty VM), __execute_action__, __check_signals__, __emit_event__, __add_message__, __save_checkpoint__, __transition_to__, __retrieve_docs__, __check_budget__, __get_actions__ Also makes json_to_monty, monty_to_json, monty_to_string pub(crate) in scripting.rs for cross-module use. Design doc: docs/plans/2026-03-25-python-orchestrator.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): switch ExecutionLoop::run() to Python orchestrator Replace the 900-line Rust execution loop with a ~80-line bootstrap that loads and runs the versioned Python orchestrator via Monty VM. The orchestrator Python code (orchestrator/default.py) is the v0 compiled-in version. Runtime versions can override it via MemoryDoc storage (orchestrator:main with tag orchestrator_code). Key fixes during switchover: - Use ExtFunctionResult::NotFound for unknown functions so Monty falls through to Python-defined functions (extract_final, etc.) - Move helper function definitions above run_loop for Monty scoping - Use FINAL result value (not VM return value) in Complete handler - Rename 'final' variable to 'final_answer' to avoid Python keyword Status: 171/177 tests pass. 6 remaining failures are step_count and token tracking bookkeeping — the orchestrator manages these internally but doesn't yet update the thread's counters via host functions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): all 177 tests pass with Python orchestrator - Increment step_count and track tokens in __emit_event__("step_completed") so thread bookkeeping matches the old Rust loop behavior - Remove double-counting of tokens in bootstrap (orchestrator handles it) - Match nudge text to existing TOOL_INTENT_NUDGE constant - Fix FINAL result propagation (use stored final_result, not VM return) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): orchestrator versioning, auto-rollback, and tests Add version lifecycle for the Python orchestrator: - Failure tracking via MemoryDoc (orchestrator:failures) - Auto-rollback: after 3 consecutive failures, skip the latest version and fall back to previous (or compiled-in v0) - Success resets the failure counter - OrchestratorRollback event for observability Update self-improvement Mission goal with Level 1.5 instructions for orchestrator patches — the agent can now modify the execution loop itself via memory_write with versioned orchestrator docs. 12 new tests: version selection (highest wins), rollback after failures, rollback to default, failure counting/resetting, outcome parsing for all 5 ThreadOutcome variants. 189 tests pass, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add engine v2 architecture, self-improvement, and dev history Three new docs for contributors: - engine-v2-architecture.md: Two-layer architecture (Rust kernel + Python orchestrator), five primitives, execution model with nested Monty VMs, bridge layer, memory/reflection, missions, capabilities - self-improvement.md: Three improvement levels (prompt/orchestrator/ config/code), autoresearch-inspired Mission loop, versioned orchestrator with auto-rollback, fix pattern database, safety model - development-history.md: Summary of 6 Claude Code sessions that built the system, key design decisions and debugging moments, architecture evolution from 900-line Rust loop to Python orchestrator Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): complete v2 side-by-side integration with gateway API Wire engine v2 into the full submission pipeline and expose threads, projects, and missions through the web gateway REST API. Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear submissions to engine v2 when ENGINE_V2=true. Previously only UserInput and ApprovalResponse were handled; all other control commands fell through to disconnected v1 sessions. Bridge query layer — add 11 read-only query functions and 6 DTO types so gateway handlers can inspect engine state (threads, steps, events, projects, missions) without direct access to the EngineState singleton. Gateway endpoints — new /api/engine/* routes: GET /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events GET /projects, /projects/{id} GET /missions, /missions/{id} POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume SSE events — add ThreadStateChanged, ChildThreadSpawned, and MissionThreadSpawned AppEvent variants. Expand the bridge event mapper to forward StateChanged and ChildSpawned engine events to the browser. Engine crate — add ConversationManager::clear_conversation() for /new and /clear commands. Code quality — replace 10 .expect() calls with proper error returns, remove dead AgentConfig.engine_v2 field, log silent init errors, fix duplicate doc comment, improve fallthrough documentation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): empty call_id on ActionResult and trace analyzer false positives Fix structured executor not stamping call_id onto ActionResult — the EffectExecutor trait doesn't receive call_id, so the structured executor must copy it from the original ActionCall after execution. Empty call_id caused OpenAI-compatible providers to reject the next LLM request with "Invalid 'input[2].call_id': empty string". Fix trace analyzer false positives: - code_error check now only scans User-role code output messages (prefixed with [stdout]/[stderr]/[code ]/Traceback), not System prompt which contains example error text - missing_tool_output check now recognizes ActionResult messages as valid tool output (Tier 0 structured path) - Add NotImplementedError to detected code error patterns New trace checks: - empty_call_id: detect ActionResult messages with missing/empty call_id before they reach the LLM API (severity: Error) - llm_error: extract LLM provider errors from Failed state reason - orchestrator_error: extract orchestrator errors from Failed state Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): add Missions tab to gateway UI Add a full Missions page to the web gateway with list view, detail view, and action buttons (Fire, Pause, Resume). Backend: add /api/engine/missions/summary endpoint returning counts by status (active/paused/completed/failed). Frontend: - New "Missions" tab between Jobs and Routines - Summary cards showing mission counts by status - Table with name, goal, cadence type, thread count, status, actions - Detail view with goal, cadence, current focus, success criteria, approach history, spawned thread list, and action buttons - Fire/Pause/Resume actions with toast notifications - i18n support (English + Chinese) - CSS following the existing routines/jobs patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): eagerly initialize engine v2 at startup The gateway API endpoints (/api/engine/missions, etc.) call bridge query functions that return empty results when the engine state hasn't been initialized yet. Previously, initialization only happened lazily on the first chat message via handle_with_engine(). Now when ENGINE_V2=true, the engine is initialized in Agent::run() before channels start, so the self-improvement mission and other engine state is available to gateway API endpoints immediately. Also rename get_or_init_engine → init_engine and make it public so it can be called from agent_loop.rs at startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): improve mission detail with markdown goal and thread table - Goal rendered as full-width markdown block instead of plain-text meta item (uses existing renderMarkdown/marked) - Current focus and success criteria also rendered as markdown - Spawned threads shown as a clickable table with goal, type, state, steps, tokens, and created date instead of a UUID list - Clicking a thread row opens an inline thread detail view showing metadata grid and full message history with markdown rendering - Back button returns to the mission detail view - Backend: mission detail now returns full thread summaries (goal, state, step_count, tokens) instead of just thread IDs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): close SSE connections on page unload to prevent connection starvation The browser limits concurrent HTTP/1.1 connections per origin to 6. Without cleanup, SSE connections from prior page loads linger after refresh/navigation, eating into the pool. After 2-3 refreshes, all 6 slots are consumed by stale SSE streams and new API fetch calls queue indefinitely — the UI shows "connected" (SSE works) but data never loads. Add a beforeunload handler that closes both eventSource (chat events) and logEventSource (log stream) so the browser can reuse connections immediately on page reload. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): support multiple gateway tabs by reducing SSE connections Each browser tab opened 2 SSE connections (chat events + log events). With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the pool and couldn't load any data. Three changes: 1. Lazy log SSE — only connect when the logs tab is active, disconnect when switching away. Most users rarely view logs, so this saves a connection slot per tab. 2. Visibility API — close SSE when the browser tab goes to background (user switches to another tab), reconnect when it becomes visible. Background tabs don't need real-time events. 3. Combined with the existing beforeunload cleanup, this means: - Active foreground tab: 1 connection (chat SSE only, +1 if logs tab) - Background tabs: 0 connections - Closed/refreshed tabs: 0 connections (beforeunload cleanup) This allows many gateway tabs to coexist within the 6-connection limit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): route messages to correct conversation by thread scope Messages sent from a new conversation in the gateway always appeared in the default assistant conversation because handle_with_engine ignored the thread_id from the frontend. Two fixes: 1. Engine conversation scoping — when the message carries a thread_id (from the frontend's conversation picker), use it as part of the engine conversation key: "gateway:<thread_id>" instead of just "gateway". This creates a distinct engine conversation per v1 thread, so messages don't cross-contaminate. 2. V1 dual-write targeting — write user messages and assistant responses to the v1 conversation matching the thread_id (via ensure_conversation), not the hardcoded assistant conversation. Falls back to the assistant conversation when no thread_id is present (e.g., default chat). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): richer activity indicators for engine v2 execution The gateway UI showed only generic "Thinking..." during engine v2 execution with no visibility into CodeAct code execution, tool calls, or reflection. Now the event mapping produces detailed status updates: Step lifecycle: - "Calling LLM..." when a step starts (was "Thinking...") - "Step complete — N in / M out tokens" when done (was "Processing...") Tool execution: - Emit ToolStarted + ToolCompleted SSE events so the frontend renders proper tool cards with spinner → checkmark/error transitions - Duration shown in parameters field (e.g., "42ms") CodeAct visibility: - "Executing code..." when assistant produces a code block - "Code executed" / "Code executed (no output)" for successful runs - "Code error — retrying..." when Monty raises an exception Reflection: - "Reflecting on execution..." when post-thread analysis starts - "Reflection complete — N insight(s) saved" when done Also refactored thread_event_to_app_event → thread_event_to_app_events (returns Vec<AppEvent>) to support emitting ToolStarted before ToolCompleted in a single event handler pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): resolve tool names as callable stubs in CodeAct runtime When LLM-generated code calls `mission_list()` or any tool function, Monty's Python execution model first resolves the name (`mission_list`) as a NameLookup before invoking it as a FunctionCall. The NameLookup handler always returned Undefined, causing NameError before the function call could dispatch to the effect executor. Fix: before starting the Monty VM, collect all known tool names from the effect executor's available_actions(). In the NameLookup handler, if the name matches a known tool, return a MontyObject::Function stub instead of Undefined. Monty then yields FunctionCall for the stub, which dispatches to the normal tool execution pipeline. This enables CodeAct code to call any registered tool as a Python function: mission_list(), mission_create(), routine_list(), web_search(), memory_search(), etc. — all without explicit imports or __execute_action__ boilerplate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): consolidate action execution, remove reflection, add learning missions Three major changes to the v2 engine: 1. **Consolidated action execution** — `handle_execute_action` in Rust is now the single source of truth for lease lookup, policy check, lease consumption, action execution, event emission, and ActionResult message recording. The Python orchestrator no longer duplicates event/message logic. This fixes the empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant messages (Codex "No tool call found" error). 2. **Removed reflection system** — Deleted the per-thread reflection pipeline (pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection, enable_reflection config, and all 3 reflection event kinds. Learning is now handled entirely by event-driven missions that fire selectively. 3. **Three learning missions** replace reflection: - `self-improvement` — fires on trace issues (error diagnosis, prompt fixes) - `playbook-extraction` — fires on successful 5+ step threads (reusable procedures) - `conversation-insights` — fires every 5 threads per project (user preferences, domain knowledge, workflow patterns) Additional fixes: - llm_query()/llm_query_batched() always include system message (Codex compat) - handle_llm_complete adds assistant message with structured action_calls for Tier 0 responses (prevents "No tool call found" errors) - Gateway broadcasts without thread_id emit as Status events instead of being dropped - Comprehensive tests for call_id propagation and trace analysis (17 new tests) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(skills): extract ironclaw_skills crate and integrate with v2 engine Extract the skills system into a standalone `ironclaw_skills` crate (following the ironclaw_safety pattern) and wire it into the v2 engine for deterministic skill selection, CodeAct code injection, and confidence tracking. **ironclaw_skills crate** (94 tests): - Core types: SkillManifest, ActivationCriteria, LoadedSkill, SkillTrust - V2 types: V2SkillMetadata, CodeSnippet, SkillMetrics, V2SkillSource - Deterministic 4-phase selector (gating→scoring→budget→attenuation) - apply_confidence_factor() for extracted skill scoring - SKILL.md parser, validation/escaping, gating, registry, catalog - Feature-gated: catalog (reqwest), registry (filesystem) **Engine integration** (14 new tests): - DocType::Skill with retrieval weight 0.45 - SkillSelector bridges MemoryDoc→LoadedSkill for shared scoring - SkillTracker for usage/version/rollback confidence tracking - System prompt injection via <skill> XML blocks - CodeAct snippet injection via Monty NameLookup - Skill extraction mission replaces playbook extraction - ThreadManager.set_skill_selector() for runtime wiring **Bridge + migration**: - skill_migration.rs: v1 SKILL.md → v2 MemoryDoc (idempotent) - init_engine() migrates v1 skills, builds SkillSelector - src/skills/mod.rs → re-export shim **E2E test** (tests/engine_v2_skill_codeact.rs): - Full CodeAct loop: skill selected → LLM returns Python code → Monty executes http() → mock returns canned GitHub JSON → FINAL() terminates → thread completes with canned data - GitHub SKILL.md in skills/github/ as reference implementation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Documenting research around how to extend to more integrations * docs: update engine-v2-architecture for missions and skills - Replace "Reflection Pipeline" with "Learning Missions" (self-improvement, skill-extraction, conversation-insights) - Add "Skills System" section covering ironclaw_skills crate, deterministic selection pipeline, CodeAct integration, confidence tracking, v1 migration - Update MemoryDoc types table (add Skill, remove Playbook as primary) - Update Integration Scaling section: Skills replace Capabilities-as-knowledge as the concrete implementation - Update example from Capability YAML to SKILL.md format with credentials - Fix thread state machine (remove Reflecting state) - Update key files table and test counts - Add self-improvement feedback loop diagram Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: clean up legacy playbook references in engine crate - Rename PLAYBOOK_MIN_STEPS/ACTIONS → SKILL_EXTRACTION_MIN_STEPS/ACTIONS - Fix pattern DB uses DocType::Note instead of DocType::Playbook - Update CLAUDE.md: skill-extraction mission, DocType list, module map Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(skills): credential specs in skill frontmatter, HTTP tool hardening, mission leases Skills can now declare API credentials in YAML frontmatter (SkillCredentialSpec, SkillCredentialLocation, SkillOAuthConfig, ProviderRefreshStrategy). Valid specs are registered into SharedCredentialRegistry at startup; the HttpTool auto-injects credentials for matching hosts — same zero-exposure model as WASM tools. HTTP tool security hardening: - Block LLM-provided auth headers for hosts with registered credentials - Return structured authentication_required error for missing credentials - Strip sensitive response headers (Set-Cookie, WWW-Authenticate, Authorization) - Scan response body through LeakDetector before returning to LLM Mission capability leases: registered mission_create/list/fire/pause/resume/delete as a "missions" capability so threads receive leases. Removed routine_* aliases from effect adapter — descriptions mention "routine" for LLM intent mapping. Includes 10 integration tests (tests/skill_credential_injection.rs) covering the full pipeline: YAML parsing → validation → registry → HttpTool wiring → per-user isolation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(engine): remove legacy Playbook doc type, superseded by Skill Drop DocType::Playbook variant and all references — playbook extraction mission was already renamed to skill extraction in the previous session. Updates CLAUDE.md, architecture docs, context builder, retrieval weights, mission comments, and store adapter path mapping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(engine): move skill selection and injection to Python orchestrator Skill selection was in Rust (SkillSelector in loop_engine.rs) — now it's in the Python orchestrator where the self-improvement mission can evolve it. Rust provides data access via two new host functions: - __list_skills__() — loads DocType::Skill MemoryDocs from Store - __record_skill_usage__(doc_id, success) — confidence tracking Python orchestrator handles everything else: - score_skill() — keyword/tag/confidence scoring (~40 lines) - select_skills() — budget-aware top-N selection (~15 lines) - format_skills() — XML block injection into system prompt (~20 lines) - Injection at step 0 with active_skill_ids stored in state Removed from Rust: - SkillSelector field + builder on ExecutionLoop and ThreadManager - format_skills_section() from prompt.rs - Rust-side skill injection block in loop_engine.rs - SkillSelector wiring in bridge/router.rs E2E test updated: skills stored in TestStore, Python orchestrator finds them via __list_skills__() and injects based on goal keywords. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: annotate v1-only code for removal after migration Mark modules and functions that exist solely for the v1 agent with "remove after v1 migration" notes: - src/skills/mod.rs ��� shim, attenuation, credential registration - src/skills/attenuation.rs — trust-based tool filtering (v1 only) - ironclaw_skills: selector, gating, registry, catalog modules - ironclaw_engine: skill_selector.rs (superseded by Python orchestrator) - src/bridge/skill_migration.rs — one-time v1→v2 conversion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(engine): remove unused skill_selector.rs Rust-side skill selection was moved to the Python orchestrator in |
||
|
Illia Polosukhin
|
5c35b58ff1 |
feat(auth): direct OAuth/social login with Google, GitHub, Apple, and NEAR wallet (#1798)
* feat(auth): add direct OAuth/social login with Google and GitHub (#1771) Add optional OAuth authentication so users can sign in directly via Google or GitHub without requiring admin-created tokens or a reverse-proxy SSO setup. On successful OAuth, the system creates or links a user via the existing UserStore, issues an API token, and sets it as an HttpOnly cookie — reusing the existing DbAuthenticator for subsequent requests. Key changes: - user_identities table (PostgreSQL V15 + libSQL migration) for linking external provider accounts to internal users - IdentityStore trait with dual-backend implementations - OAuthProvider trait with Google (OIDC id_token) and GitHub (API-based) provider implementations - In-memory CSRF + PKCE state store with TTL and capacity bounds - Cookie-based session extraction in auth middleware - User resolution: existing identity → email linking → new account creation - All behind OAUTH_ENABLED=true flag; existing auth paths unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add email domain restrictions for OAuth and OIDC login (#1771) Add configurable email domain restrictions so admins can limit OAuth and OIDC login to specific organizations: - OAUTH_ALLOWED_DOMAINS: comma-separated list of allowed email domains, applied to all OAuth providers and OIDC (e.g., company.com,partner.org) - GOOGLE_ALLOWED_HD: restrict Google login to a specific Workspace domain via the `hd` authorization parameter + server-side validation - Domain check enforced in both the OAuth callback handler and the OIDC JWT middleware path (extracts email claim from validated JWT) - Add setup documentation in .env.example with step-by-step instructions for configuring Google and GitHub OAuth credentials Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR review — security hardening and cleanup (#1798) Fixes from Gemini and Copilot review: Security (critical/high): - Validate `aud` claim in Google id_token to prevent token substitution - Sanitize `redirect_after` to relative paths only (prevent open redirects) Correctness (medium): - Remove orphaned token: replace `create_user_with_identity_and_token` with `create_user_with_identity` — single token created in callback handler - Logout now revokes the API token (not just clears cookie) - Session tokens expire after 30 days (matching cookie lifetime) - Store decoded Google claims in raw_profile (not JWT string) - Propagate GitHub email fetch errors instead of swallowing - Fix `list_identities_for_user` to propagate row iteration errors Cleanup (low): - Extract `SESSION_COOKIE_NAME` constant - Add Secure flag to logout cookie clearing - Add single-quote escaping in error_page HTML - Fix garbled unicode in auth.rs comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add Apple Sign In provider (#1807) Add Apple Sign In as an OAuth provider alongside Google and GitHub. Apple-specific handling: - JWT client_secret generation (ES256-signed, team_id/key_id/private_key) - response_mode=form_post — Apple POSTs the callback instead of GET - POST callback route added alongside existing GET route - User name extracted from Apple's `user` form field (sent only on first authorization) and merged into the profile - id_token decoded with aud + issuer validation - email_verified handles both boolean and string "true"/"false" formats Configuration: - APPLE_CLIENT_ID, APPLE_TEAM_ID, APPLE_KEY_ID - APPLE_PRIVATE_KEY_PATH (file) or APPLE_PRIVATE_KEY_PEM (inline) - Setup instructions added to .env.example Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add NEAR wallet login via NEP-413 signature verification (#1807) Add NEAR wallet authentication as a fourth login method alongside Google, GitHub, and Apple. Unlike OAuth, NEAR uses a challenge-response flow with Ed25519 signature verification. Backend: - GET /auth/near/challenge — generate a random nonce (32 bytes hex) - POST /auth/near/verify — verify Ed25519 signature + NEAR RPC access key check, then issue session token via existing user resolution pipeline - NearNonceStore: in-memory nonce store with 5-min TTL and replay protection - Supports both base58 (NEAR standard) and hex key/signature encoding - New dependency: bs58 0.5 for base58 decoding Frontend: - Login screen discovers enabled providers via GET /auth/providers - Shows social login buttons (Google, GitHub, Apple, NEAR) dynamically - NEAR button loads @hot-labs/near-connect via ESM CDN import - Wallet connection → signMessage → POST to /auth/near/verify → session - OAuth cookie-based sessions auto-detected on page load (existing flow) Configuration: - NEAR_AUTH_ENABLED=true - NEAR_AUTH_NETWORK=mainnet|testnet (defaults to mainnet) - NEAR_AUTH_RPC_URL (auto-detected from network) Closes #1807 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address second round of PR review comments (#1798) - Fix early return in with_oauth() that skipped NEAR setup and OIDC domain restrictions when no OAuth redirect providers were configured - Add active-status check before linking identity by verified email (prevents linking to suspended/deactivated accounts) - Remove inline onclick handlers from login buttons (CSP compliance) - Export SESSION_COOKIE_NAME from auth.rs, reuse in handlers and middleware - NEAR challenge returns structured message ("Sign in to IronClaw\nNonce: {nonce}") that both client and server use for signature verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address human reviewer security findings (#1798) Six fixes from serrrfirat's review: 1. Domain check now requires email_verified=true before trusting the email for domain restriction — prevents unverified emails from bypassing access control (e.g., GitHub unverified fallback) 2. First-user bootstrap race documented — concurrent first logins may both see has_any_users()=false, but the second gets member role. Acceptable tradeoff; unique constraint prevents identity duplication. 3. NEP-413 payload mismatch fixed — server now builds the exact borsh-serialized NEP-413 payload (tag + message + nonce + recipient) that the wallet signs, instead of raw message bytes 4. Token extraction priority fixed — explicit ?token= query param now takes precedence over session cookie, preventing SSE/WS user mismatch when a browser has both a cookie and a query-param token 5. NEAR domain suffix check hardened — requires exact match or dotted subdomain boundary (alice.company.near passes, evilcompany.near does not) 6. NEAR network surfaced to frontend — /auth/providers response includes near_network field, frontend wallet connector uses it instead of hardcoded mainnet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): improve login page UX when OAuth providers are enabled When OAUTH_ENABLED=true with providers configured, the login screen now shows social login buttons (Google, GitHub, Apple, NEAR) as the primary action. The token input is collapsed behind a clickable "or use a token" divider for API users. Without OAuth: unchanged — token input is the only option. With OAuth: social buttons first, token input expandable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): hide token form until providers are discovered The token input was visible by default, causing it to flash before OAuth buttons appeared. Now: - Token form starts hidden (display:none) - /auth/providers fetch determines what to show - With providers: social buttons shown, token form behind "or use a token" - Without providers (or fetch fails): token form shown as fallback - After OAuth redirect: cookie-based autoAuth skips login screen entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): replace Connected indicator with user avatar + account menu Replace the "Connected" status indicator in the header with a user avatar button that shows connection status via an overlay dot. Clicking the avatar opens a dropdown with: - Display name, email, and role - Connection status (green/red dot + text) with gateway stats - Sign out button (calls POST /auth/logout, clears session, reloads) Avatar source: - OAuth logins: profile photo from Google/GitHub/Apple (avatar_url) - Token logins: initials from display_name (colored circle) Backend: profile_get_handler now queries user_identities for avatar_url from linked OAuth accounts. Frontend: social buttons are primary when OAuth is enabled, token input collapsed behind "or use a token" divider. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): remove Connected section from dropdown, fix avatar loading - Remove the connection status section from the user dropdown (was redundant with the avatar dot) - Add update_identity_profile() to IdentityStore — updates display_name and avatar_url on re-login so avatars load for accounts created before the avatar field was wired - Call update_identity_profile() in resolve_user() when an existing identity is found (re-login path) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): restore gateway stats in dropdown, add avatar debug logging - Bring back gateway stats section in user dropdown (without the word "Connected" — just the server stats like uptime, model, channels) - Add debug tracing to Google provider (logs picture claim from id_token) and profile handler (logs identity count + avatar_url) to diagnose why avatar isn't loading for Google OAuth accounts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix Google avatar not loading, restore gateway stats - Add referrerpolicy="no-referrer" to avatar img — Google's lh3.googleusercontent.com returns 403 when Referer header is sent from a different origin - Add crossorigin="anonymous" for CORS - Use display:block explicitly instead of empty string - Add onerror fallback to initials if image fails to load - Restore gateway stats section in dropdown (without "Connected" text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix squeezed avatar image in header Add min-width/min-height and flex-shrink:0 to both the avatar button and the img element so they don't get compressed by the tab-bar flex layout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): position avatar img and initials absolutely inside button Both children were competing for flex space, causing 0px width. Now both are position:absolute inside the 32px button, layered on top of each other. The JS toggles display:block/none to show the right one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): rewrite avatar loading — CSS src selector + onload/onerror Previous approach: inline style display:none toggled by JS. Failed because display:none prevented image fetch in some browsers, and position:absolute elements competed for z-index. New approach: - No inline style on img — CSS hides it via .user-avatar-img (display:none) - CSS .user-avatar-img[src] shows it (display:block, z-index:1) - JS sets src, onload hides initials, onerror removes src as fallback - Initials always rendered first as the base layer - Removed crossorigin="anonymous" which can cause CORS failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): prefetch avatar with new Image() before showing Use a throwaway Image() to prefetch the avatar URL. Only when onload fires, set src on the real <img> and unhide it. This avoids all CSS display/src selector issues — the real img element only gets a src after the image is confirmed loadable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): set avatar src directly, set referrerPolicy in JS The new Image() prefetch was failing because the programmatic Image object didn't have referrerPolicy set. Simplify: set referrerPolicy and src directly on the real <img> element, unhide it immediately, and use onload to hide initials. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): explicit display:block on avatar img, removeAttribute hidden - Add display:block to .user-avatar-img CSS (img elements default to inline which can cause rendering issues with position:absolute) - Bump z-index to 2 to ensure img renders above initials - Use removeAttribute('hidden') instead of hidden=false - Use style.display='none' on initials instead of hidden attribute Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): swap img/initials DOM order so img paints on top Put <img> after <span> in DOM order. With both position:absolute, later elements paint on top. Combined with z-index:2 on img vs z-index:0 on initials, the avatar photo should now reliably cover the initials circle when loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): add OAuth avatar domains to Content-Security-Policy The CSP had img-src 'self' data: which blocked Google and GitHub avatar images from loading. Added: - img-src: *.googleusercontent.com, avatars.githubusercontent.com - script-src: esm.sh (for near-connect dynamic import) - connect-src: esm.sh, *.near.org (for NEAR RPC) - form-action: Google, GitHub, Apple OAuth endpoints This was the root cause of avatar images not rendering despite correct src URLs — the browser silently blocked them via CSP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): show welcome card for new OAuth users with no threads New OAuth users have no assistant thread yet, so switchToAssistant() was never called, and loadHistory() never ran to show the welcome card. Now explicitly show the welcome card when there's no current thread and no assistant thread (brand-new user). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting for new OAuth users on workspace creation New OAuth users saw an empty chat because the bootstrap greeting was only persisted when the agent loop processed the first message (take_bootstrap_pending check in agent_loop.rs:1314). But OAuth users land on the web UI without sending any message. Fix: WorkspacePool now checks take_bootstrap_pending() after seed_if_empty() and persists the GREETING.md content into the assistant conversation immediately. This runs in a background task so it doesn't block the workspace creation. The greeting is in the DB before the frontend loads threads/history. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting synchronously, not in background Move greeting persistence from tokio::spawn to the same await chain as seed_if_empty() so the greeting is guaranteed to be in the DB before the workspace is returned to the caller. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): seed bootstrap greeting in chat_threads_handler for new users The WorkspacePool approach didn't work because the workspace pool is only accessed by memory handlers — chat_threads_handler runs first when a new user loads the page. Move the greeting seed to chat_threads_handler: after get_or_create_assistant_conversation, check if the conversation has zero messages and inject the GREETING.md content. This guarantees the greeting is in the DB before the thread list is returned to the frontend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(agent): consolidate bootstrap greeting into chat_threads_handler Remove three redundant greeting insertion paths from agent_loop.rs: 1. Single-user startup (Agent::run bootstrap_thread_id) 2. Single-user SSE broadcast after startup 3. Multi-tenant message handler (take_bootstrap_pending on first msg) Also remove the dead WorkspacePool greeting code in server.rs. The single source of truth is now chat_threads_handler: when the assistant conversation is created with zero messages, GREETING.md is inserted. This works for all auth modes (token, OAuth, OIDC) and both single-user and multi-tenant deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: downgrade workspace seed log from info to debug Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian's blocking review items 1. Add Google iss validation — set_issuer(&["https://accounts.google.com"]) to match Apple's issuer check. Prevents cross-provider id_token acceptance. 2. Convert oauth_rate_limiter from global RateLimiter to per-IP PerUserRateLimiter(20, 60). Extracts client IP from X-Forwarded-For header. One user retrying no longer locks out all OAuth for everyone. Also: - sanitize_redirect now rejects backslash (/\) open redirect vector - All auth handlers extract headers for per-IP rate limiting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): stop inserting greeting on every page load The previous check used list_conversations_with_preview with limit=1 and defaulted to is_empty=true when the assistant thread wasn't in the result (unwrap_or(true)). This caused the greeting to be inserted on every chat_threads_handler call. Fix: use list_conversation_messages_paginated(assistant_id, None, 1) to directly check if the assistant conversation has any messages. Only insert the greeting when the message list is truly empty. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for bootstrap greeting and cookie auth Five tests covering the greeting behavior and OAuth session auth: 1. test_greeting_inserted_once_for_new_user — verifies greeting appears exactly once and is not duplicated on second page load 2. test_greeting_not_duplicated_on_rapid_calls — 5 concurrent /api/chat/threads requests produce exactly 1 greeting 3. test_each_user_gets_own_greeting — multi-user: Alice and Bob each get their own assistant thread with separate greetings 4. test_cookie_auth_works_for_threads — cookie-based auth (ironclaw_session=token) works for protected endpoints 5. test_existing_conversation_no_greeting — pre-populated conversations are not overwritten with the greeting These tests would have caught the unwrap_or(true) bug that caused greeting re-insertion on every page load. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix near-connect CDN URL (package is v0.x, not v1) The @hot-labs/near-connect package is version 0.11.1 — there is no v1 release. The @1 version specifier returned 404 from esm.sh. Changed to @0.11 which resolves correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): support base64 encoding for NEAR wallet signatures NEAR wallets (e.g. HOT) may return signatures and public keys in base64 format, not just base58/hex. Added base64 standard and URL-safe decoding to decode_multiformat(), which is used by both decode_near_public_key and decode_near_signature. Also: - Added debug logging to near_verify_handler to trace credential formats - Updated CSP img-src to allow wallet logos (raw.githubusercontent.com, jsdelivr.net, near.org, pages.near.org) - Added CSP frame-src for near-connect wallet sandboxes (iframes) - Added blob: to img-src for inline wallet icons Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): widen CSP connect-src and img-src for NEAR wallet resources near-connect fetches wallet manifests from raw.githubusercontent.com and cdn.jsdelivr.net, and wallet logos from app.hot-labs.org. These were blocked by the restrictive connect-src and img-src policies. - connect-src: added raw.githubusercontent.com, *.jsdelivr.net, *.cloudflare.com - img-src: added *.hot-labs.org Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): try both NEP-413 and raw message for NEAR signature verification Different NEAR wallets may sign the full NEP-413 borsh payload or just the raw message string. Try NEP-413 first, fall back to raw message bytes. This makes verification work with HOT wallet and other wallets that may not implement the full NEP-413 serialization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): fix NEP-413 field order and try both payload layouts The NEP-413 borsh payload field order was wrong. Our implementation had tag → message → nonce → recipient → callback_url, but the NEAR docs (docs.near.org/web3-apps/backend-login) show tag → message → recipient → nonce. Now tries both field orderings (v1 and v2), plus SHA256 variants, plus raw message bytes — covering all known wallet implementations. Tests updated: verify_near_signature tested with raw message, NEP-413 v2, and wrong-key rejection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP for wallet ecosystem — allow all HTTPS for connect/img/frame The NEAR wallet ecosystem spans dozens of domains (intear.tech, hot-labs.org, meteorwallet.app, herewallet.app, etc.) that change as new wallets are added. Whitelisting each one is a losing game. Relax CSP to allow all HTTPS for: - connect-src: wallet manifests, wallet JS modules, RPC endpoints - img-src: wallet logos from various CDNs - frame-src: wallet sandbox iframes script-src remains restricted to specific CDNs (jsdelivr, cloudflare, esm.sh) — this is the security-critical directive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow unsafe-inline scripts for NEAR wallet sandbox iframes NEAR wallet sandboxes (MeteorWallet, etc.) use inline scripts inside their iframe sandboxes. The CSP script-src blocked these, preventing wallets from loading. Added 'unsafe-inline' to script-src. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP style-src and font-src for wallet iframes NEAR wallet sandboxes load fonts from rsms.me, cdnfonts.com, and embed data: font URIs. Relaxed style-src and font-src to allow all HTTPS sources and data: URIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 4 — 14 fixes serrrfirat (high/medium): 1. decode_multiformat ambiguity: replaced with context-aware decoders. NEAR pubkeys enforce ed25519: prefix + base58 (unambiguous). Signatures try base64 first (most wallets), then base58. 2. UTF-8 slicing panic: use safe_truncate() with char_indices() 3. NEAR pubkey → RPC format: re-encode decoded bytes as ed25519:{base58} for the canonical format expected by view_access_key 4. GitHub redirect_uri: now included in token exchange form body Copilot (medium/low): 5. near_network stored explicitly in GatewayState (not inferred from URL) 6. NEAR verify sets HttpOnly session cookie (consistent with OAuth flow) 7. Reuse reqwest::Client via LazyLock (no per-request allocation) 8. OAuth module doc updated to list all 4 providers 9. Rate limiter comment fixed (was stale "10 requests") 10. Profile identity error logged at warn (not silently swallowed) 11. Config doc updated for Apple/NEAR requirements 12. Test file doc comment updated to match actual coverage 13. RPC status check before JSON parse 14. GitHub token exchange includes redirect_uri Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR feedback on cookie auth and NEAR sessions * fix(auth): address code review — tighten CSP, fix races, improve security - Tighten CSP: narrow connect-src/img-src/frame-src to specific origins instead of blanket `https:` (prevents data exfiltration) - Fix greeting race: add atomic add_conversation_message_if_empty using INSERT...WHERE NOT EXISTS (both PostgreSQL and libSQL) - Case-insensitive email matching: use LOWER() in identity lookups and normalize emails to lowercase on storage - Add X-Real-IP fallback for rate limit key when X-Forwarded-For missing - Add OAuthError::SignatureVerification variant (was misusing ProfileFetch) - Fix dead branch in with_oauth (has_near check inside !has_near block) - Fix _user → user in logout_handler (variable is actually used) - Add partial index WHERE email IS NOT NULL to libSQL (match PostgreSQL) - Downgrade noisy tracing::debug to trace in profile handler - Add i18n for "Sign out" button (en + zh-CN) - Remove duplicate test_session_cookie_auth_passes test - Update E2E bootstrap tests to match new DB-based greeting architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove garbled unicode character in section comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 5 — admin race, 303 redirect, OIDC email_verified - Atomic first-user admin: create_user_with_identity now promotes to admin inside the DB transaction with UPDATE...WHERE COUNT(*)=1, eliminating the TOCTOU race where two concurrent first logins both get admin role (both PostgreSQL and libSQL) - Apple callback redirect: use 303 See Other instead of 307 Temporary so POST form_post callbacks are converted to GET on redirect - OIDC domain restriction: now requires email_verified=true before checking domain allowlist, preventing unverified emails from bypassing the restriction - Postgres add_conversation_message_if_empty: call touch_conversation after insert to match libSQL behavior and keep last_activity current - Greeting seeding: log errors instead of silently discarding with let _ Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): advisory lock for admin election, skip empty query tokens - Postgres first-user admin: add pg_advisory_xact_lock before the COUNT(*)=1 promotion to serialize concurrent transactions under READ COMMITTED isolation (prevents two admins on concurrent signup) - Empty ?token= query parameter no longer overrides a valid session cookie — trimmed empty tokens return None from query_token() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian security review — redirect, sweep, NEAR sigs Blocking issues from security review: 1. redirect_after hardened: strict URL-safe char allowlist in sanitize_redirect (blocks /%09/ and encoded separators), plus re-validation before use in handle_callback (defense in depth) 2. Sweep tasks shutdown-aware: OAuth state store and NEAR nonce store sweep loops now select on a watch channel and exit when the sender is dropped (stored in GatewayState.oauth_sweep_shutdown) 3. NEAR signature verification tightened: removed raw-message-bytes and SHA256-of-raw fallbacks that lacked nonce binding (replay risk). Only NEP-413 structured payloads (v1 + v2) are accepted. Added test_verify_near_signature_rejects_raw_message regression test. Non-blocking: 4. NEAR RPC client timeout: set 10s timeout on the static reqwest client to prevent indefinite hangs on slow RPC endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): percent-decode redirect_after before validation is_safe_redirect now percent-decodes the URL and re-validates against the // and /\ guards, preventing smuggling via %2f%2f or %5c. Added 5 regression tests covering normal paths, protocol-relative, absolute URLs, encoded smuggling, and sanitize_redirect filtering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): case-insensitive get_user_by_email, require OIDC iss/aud claims - get_user_by_email now uses LOWER() in both PostgreSQL and libSQL, matching the case-insensitive identity lookup. This ensures admin-created users with different email casing are correctly linked during OAuth account resolution. - OIDC validation now adds iss/aud to required_spec_claims when configured, rejecting JWTs that omit these claims entirely (not just mismatches). Updated two tests from assert-passes to assert-rejects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): normalize UserRecord.email to lowercase, add aria-label to avatar - UserRecord.email now lowercased on create (matching identity records), preventing case-mismatched duplicates against the UNIQUE constraint - Avatar button: added aria-label with i18n (en + zh-CN) for screen readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
rajulbhatnagar
|
d12b8bd7ec |
feat: Add ACP (Agent Client Protocol) job mode for delegating to any compatible coding agent (#1600)
* feat: add ACP (Agent Client Protocol) job mode for delegating to any compatible coding agent Add a third container job mode (`JobMode::Acp`) that spawns any ACP-compliant agent (Goose, Codex, Gemini CLI, Cline, Copilot, etc.) as a subprocess inside a Docker container and communicates via the standard ACP protocol (JSON-RPC over stdio). **Bridge runtime** (`src/worker/acp_bridge.rs`): - Spawns agent subprocess, performs ACP handshake (initialize → session → prompt) - Translates ACP SessionNotification events to IronClaw's JobEventPayload stream - Auto-approves permissions (Docker container is the security boundary) - Supports follow-up prompts from the orchestrator - Detects agent process exit via oneshot channel to prevent infinite polling **User configuration** (mirrors MCP server pattern): - `ironclaw acp add/list/remove/toggle/test` CLI commands - DB-backed persistence with `~/.ironclaw/acp-agents.json` disk fallback - Per-agent `enabled` flag + global `ACP_ENABLED` toggle - `ironclaw acp test` spawns agent, verifies ACP handshake, reports capabilities **System integration**: - `ExtensionKind::AcpAgent` in extension manager (12 match arms) - `agent_name` parameter in CreateJobTool resolves agent from AcpAgentsFile - Mode stored as `"acp:<agent_name>"` for restart support - Doctor validation, status display, boot screen, app startup logging - Web UI: extension install mapping, job restart, follow-up prompt support Closes #1506 * test: add comprehensive ACP test coverage (22 new tests) Bridge: ToolCall, ToolCallUpdate, thought-image, max_turn_requests, session_id propagation, text_from_content_block, multibyte truncation. Config: AcpModeConfig defaults, settings resolution, env overrides. Job tool: schema includes "acp" mode + agent_name, mode="acp" requires agent_name parameter, JobMode::Acp as_str/display. Job manager: JobMode::Acp as_str/display, acp_memory_limit_mb default. CLI: parse_env_var valid/invalid/equals-in-value, command variants. * refactor: make IronClawAcpClient reusable for CLI test command Extract AcpEventSink trait so the same Client implementation (permission auto-approval, event translation) is shared between the container bridge (posts to orchestrator HTTP API) and the CLI test command (prints to stdout). Also extracts ironclaw_init_request() to avoid duplicating the ACP handshake parameters between bridge and test command. * fix(sandbox): use host.docker.internal on all platforms for orchestrator URL The orchestrator host was hardcoded to 172.17.0.1 on Linux, which is only correct for the default Docker bridge network. Environments with custom bridge IPs break container-to-host connectivity. Since all containers already set extra_hosts with host-gateway, using host.docker.internal works on all platforms and network configurations. * fix ACP PR review feedback * fix ACP DB error fallback * fix clippy after staging merge --------- Co-authored-by: Rajul Bhatnagar <brajul@amazon.com> Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
ironclaw-ci[bot]
|
33e3ce0a1a |
Merge pull request #1745 from nearai/release-plz-2026-03-30T00-59-23Z
chore(ironclaw): release v0.24.0 |
||
|
Henry Park
|
419b47dcff |
Merge pull request #1698 from nearai/staging-promote/7234700c-23635804857
chore: promote staging to main (2026-03-27 07:25 UTC) |
||
|
synner88
|
368d2f5238 |
feat(gateway): OIDC JWT authentication for reverse-proxy deployments (#1463)
* feat(gateway): add OIDC JWT authentication for reverse-proxy deployments
Add an optional OIDC JWT auth path to the web gateway, enabling
deployments behind identity-aware proxies like AWS ALB with Okta/Cognito.
When GATEWAY_OIDC_ENABLED=true, the gateway reads a signed JWT from a
configurable HTTP header (default: x-amzn-oidc-data), fetches the
signing key from a JWKS endpoint, and verifies the signature + claims.
Auth flow: Bearer token → OIDC JWT → query-string token → 401.
Key design decisions:
- Split signature verification from claim extraction to handle AWS ALB's
non-standard base64 padding (ALB includes '=' padding in JWT segments,
but jsonwebtoken's decode() strips it, changing the signing input).
We verify against the original token text, then extract claims from a
normalized copy.
- JWKS keys cached for 1 hour with per-kid granularity.
- Supports both ALB-style per-key PEM URLs ({kid} placeholder) and
standard JWKS endpoints.
- DER-to-raw ECDSA signature conversion for IdPs that use DER encoding.
- Frontend auto-detects proxy auth via /api/gateway/status probe,
skipping the login screen when OIDC is active.
Configuration (env vars):
GATEWAY_OIDC_ENABLED=true
GATEWAY_OIDC_HEADER=x-amzn-oidc-data (default)
GATEWAY_OIDC_JWKS_URL=https://public-keys.auth.elb.us-east-1.amazonaws.com/{kid}
GATEWAY_OIDC_ISSUER=https://example.okta.com (optional)
GATEWAY_OIDC_AUDIENCE=my-client-id (optional)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Address code review feedback on OIDC auth PR
- Add EdDSA PEM key parsing support (was falling through to RSA)
- Fix issuer validation: remove set_issuer(&[]) else branch that
rejected all tokens when GATEWAY_OIDC_ISSUER is unset
- Make missing `sub` claim a validation error instead of silently
defaulting to "unknown"
- Extract initApp() in app.js so OIDC auto-auth actually initializes
the UI (was calling undefined function)
- Add regression tests for sub claim and issuer validation fixes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Harden OIDC auth: address claude[bot] security review
- SSRF: URL-encode kid before substituting into JWKS URL template
- Cache bounds: cap key cache at 64 entries, evict expired + oldest
- DER parsing: support long-form length encoding (>= 128 bytes),
validate component lengths against expected curve size
- Production safety: replace .expect() with Result in OidcState::from_config
- Fetch backoff: cache failed JWKS fetches for 10s to prevent retry storms
- Body limit: cap JWKS responses at 256 KB to prevent OOM from rogue endpoint
- Add regression tests for DER long-form, kid encoding, cache bounds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(auth): add regression test for OIDC identity resolution
Add two integration tests that exercise the full OIDC middleware path
through to AuthenticatedUser extraction:
- test_oidc_auth_inserts_user_identity_for_handler: sends a valid OIDC
JWT through the middleware and verifies the handler receives the sub
claim as user_id. Returns 401 if identity insertion is missing —
verified by temporarily removing the insert and confirming failure.
- test_oidc_auth_user_gets_member_role: confirms OIDC-authenticated
users receive role=member (not admin).
Uses a seed_key() test helper on OidcState to pre-populate the key
cache with an HS256 secret, avoiding the need for an HTTP JWKS mock.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(auth): comprehensive OIDC test coverage for edge cases
Add 17 new OIDC tests covering middleware integration, auth priority,
invalid JWTs, issuer/audience validation, and key cache behavior:
Middleware auth priority & fallthrough:
- Bearer works when OIDC configured but header absent
- Bearer takes priority when both Bearer and OIDC header present
- Bad OIDC signature returns 401 (not 500)
- Invalid OIDC doesn't block valid bearer auth
- No auth at all with OIDC configured → 401
Expired / invalid JWT edge cases:
- Expired JWT (exp in the past) rejected
- JWT without kid header rejected
- Malformed JWTs rejected (empty, 2-part, 4-part, garbage)
- Non-string sub claim (integer) rejected
- Empty-string sub passes auth (documented behavior)
- Missing sub rejected through full middleware path
Issuer / audience validation:
- Matching issuer accepted, wrong issuer rejected
- Matching audience accepted, wrong audience rejected
- Missing iss/aud when configured: passes (jsonwebtoken v9 behavior,
documented with notes on potential hardening)
Key cache:
- Expired cache entries not served
- Fetch failure backoff blocks retry within 10s
- Backoff expiry allows retry
- Cache max entries constant verified
Also adds shared test helpers (encode_test_jwt, test_oidc_state,
oidc_auth_state, oidc_test_app) to reduce boilerplate.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): resolve formatting and no-panics check failures
- Run cargo fmt to wrap long assert lines in OIDC tests
- Add // safety: test helper comments to suppress false positives
from check_no_panics.py (unwraps in #[cfg(test)] helper fns)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: synner88 <29090601+synner88@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com>
|
||
|
Feri Muhammad
|
a8e83210ff |
feat(discord): add gateway channel flow in wasm (#944)
* feat(discord): restore gateway channel flow in wasm * chore(discord): bump channel version to 0.2.1 * fix(discord): address review feedback on gateway channel PR - Add #[serde(default)] to DiscordMessageMetadata for backward compat with old Option<String> serialized metadata - Restore mention polling alongside Gateway (on_poll processes gateway events first, then runs poll_for_mentions if configured) - Update on_respond to handle source_message_id with message_reference for mention-poll reply threading - Implement Gateway presence status: dnd before pairing, online after - Implement Gateway resume (OP 6) with session_id tracking, falling back to fresh identify on Invalid Session (OP 9) - Extract WebsocketSessionState and spawn_websocket_poll to reduce nesting in start_websocket_runtime - Simplify should_apply_dm_pairing tautology - Remove completed plan docs - Fix clippy items_after_test_module in extensions handler Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(discord): address review findings in gateway channel PR - Fix gateway presence always showing "online" by filtering empty owner_id strings from workspace store reads - Fix interaction followup using POST instead of PATCH to /messages/@original, which left deferred "thinking" state unresolved - Restore mention-poll pagination (up to 5 pages of 100 messages) - Remove dead ed25519-dalek and hex dependencies from WASM crate - Remove unused _channel_id parameter from remember_processed_id - Clean up redundant let binding in send_pairing_reply Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(discord): address second-round review findings - Log warning when gateway event queue JSON fails to deserialize instead of silently returning empty (zmanian review item 1) - Defer presence update from OP 10 Hello to after OP 0 READY, per Discord gateway protocol which requires READY before non-Identify commands (zmanian review item 2) - Add 0-25% random jitter to websocket reconnect backoff per Discord's reconnection recommendations (zmanian suggestion) - Extract WebsocketPollContext struct to replace 19-parameter spawn_websocket_poll function (zmanian suggestion) - Document intent bitmask 4609 = GUILDS + GUILD_MESSAGES + DIRECT_MESSAGES in capabilities JSON (zmanian suggestion) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zhyaoyu <zhyaoyu@aliyun.com> Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Joseph Bloggs
|
de5a1c7b0d |
fix(worker): replace script -qfc with pty-process for injection-safe PTY (#1678)
- Add pty-process crate (MIT, tokio async support) for PTY allocation - Spawn claude CLI with pty-process::Command::arg() chaining instead of building a shell string for script -qfc - Eliminates all shell injection surfaces: prompt, model, session_id are passed via execve, never interpreted by a shell - Keep stderr on separate pipe to prevent NDJSON parse breakage (pty-process attaches PTY to all fds by default) - Gate PTY behind #[cfg(unix)] with direct-spawn fallback for Windows CI - Read stdout from PTY master (implements tokio::io::AsyncRead) - Add regression tests: arg vector construction + PTY allocation Addresses review feedback from zmanian and gemini-code-assist. Co-authored-by: j-bloggs <j-bloggs@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
8f8cb7f7b1 |
feat: DB-backed user management, admin secrets provisioning, and multi-tenant isolation (#1626)
* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling
Finishes the remaining isolation work from phases 2–4 of #59:
Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.
Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.
Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use selected_model setting key to match /model command persistence
The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override
Three follow-up fixes for multi-tenant isolation:
1. Multi-user heartbeat now runs memory hygiene per user before each
heartbeat check, matching single-user heartbeat behavior.
2. /model command in multi-tenant mode only persists to per-user
settings (selected_model) without calling set_model() on the shared
LlmProvider. The per-request model_override in the dispatcher reads
from the same setting. Added multi_tenant flag to AgentConfig
(auto-detected from GATEWAY_USER_TOKENS).
3. RigAdapter now supports per-request model overrides by injecting the
model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
API servers use last-key-wins for duplicate JSON keys, so the override
takes effect via serde's flatten serialization order.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR review — cost model attribution, heartbeat concurrency, pruning
Fixes from review comments on #1614:
- Cost tracking now uses the override model name (not active_model_name)
when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: /status ownership, model persistence scoping, heartbeat robustness
Addresses second round of PR review on #1614:
- /status <job_id> DB path now validates job.user_id == requesting user
before returning data (was missing ownership check, security fix).
- persist_selected_model takes user_id param instead of owner_id, and
skips .env/TOML writes in multi-tenant mode (these are shared global
files). handle_system_command now receives user_id from caller.
- JoinSet collection handles Err(JoinError) explicitly instead of
silently dropping panicked tasks.
- Notification forwarder extracts owner_id from response metadata in
multi-tenant mode for per-user routing instead of broadcasting to
the agent owner.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap
Round 3 review fixes:
- Cost tracking passes None for cost_per_token when model override is
active, letting CostGuard look up pricing by model name instead of
using the default provider's rates (serrrfirat).
- fire_manual() now uses per-user workspace, matching spawn_fire()
pattern (serrrfirat).
- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).
- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
the LLM provider (serrrfirat + Copilot).
- Fixed inject_model_override doc comment accuracy (Copilot).
- Added comment explaining multi-tenant notification routing priority
(Copilot).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: user-scoped webhook endpoint for multi-tenant isolation
Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.
The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.
Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
services that can't send bearer tokens)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(db): add UserStore trait with users, api_tokens, invitations tables
Foundation for DB-backed user management (#1605):
- UserRecord, ApiTokenRecord, InvitationRecord types in db/mod.rs
- UserStore sub-trait (17 methods) added to Database supertrait
- PostgreSQL migration V14__users.sql (users, api_tokens, invitations)
- libSQL schema + incremental migration V14
- Full implementations for both PgBackend (via Store delegation) and
LibSqlBackend (direct SQL in libsql/users.rs)
- authenticate_token JOINs api_tokens+users with active/non-revoked
checks; has_any_users for bootstrap detection
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(web): DB-backed auth, user/token/invitation API handlers
Adds the web gateway layer for DB-backed user management (#1605):
Auth refactor:
- CombinedAuthState wraps env-var tokens (MultiAuthState) + optional
DbAuthenticator for DB-backed token lookup with LRU cache (60s TTL,
1024 max entries)
- auth_middleware tries env-var tokens first, then DB fallback
- From<MultiAuthState> impl for backward compatibility
- main.rs wires with_db_auth when database is available
API handlers (12 new endpoints):
- /api/admin/users — CRUD: create, list, detail, update, suspend, activate
- /api/tokens — create (returns plaintext once), list, revoke
- /api/invitations — create, list, accept (creates user + first token)
Token creation: 32 random bytes → hex plaintext, SHA-256 hash stored.
Invitation accept: validates hash + pending + not expired, creates
user record and first API token atomically.
All test files updated for CombinedAuthState type change.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: startup env-var user migration + UserStore integration tests
Completes the DB-backed user management feature (#1605):
- Startup migration: when GATEWAY_USER_TOKENS is set and the users
table is empty, inserts env-var users + hashed tokens into DB.
Logs deprecation notice when DB already has users.
- hash_token made pub for reuse in migration code.
- 10 integration tests for UserStore (libsql file-backed):
- has_any_users bootstrap detection
- create/get/get_by_email/list/update user lifecycle
- token create → authenticate → revoke → reject cycle
- suspended user tokens rejected
- wrong-user token revoke returns false
- invitation create → accept → user created
- record_login and record_token_usage timestamps
- libSQL migration: removed FK constraints from V14 (incompatible
with execute_batch inside transactions). Tables in both base SCHEMA
and incremental migration for fresh and existing databases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: remove GATEWAY_USER_TOKENS, fix review feedback
GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.
Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
via db.has_any_users() in app.rs)
Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add role-based access control (admin/member)
Adds a `role` field (admin|member) to user management:
Schema:
- `role TEXT NOT NULL DEFAULT 'member'` added to users table in both
PostgreSQL V14 migration and libSQL schema/incremental migration
- UserRecord gains `role: String` field
- UserIdentity gains `role: String` field, populated from DB in
DbAuthenticator and defaulting to "admin" for single-user mode
Access control:
- AdminUser extractor: returns 403 Forbidden if role != "admin"
- /api/admin/users/* handlers: require AdminUser (create, list,
detail, update, suspend, activate)
- POST /api/invitations: requires AdminUser (only admins can invite)
- User creation accepts optional "role" param (defaults to "member")
- Invitation acceptance creates users with "member" role
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(web): add Users admin tab to web UI
Adds a Users tab to the web gateway UI for managing users, tokens,
and roles without needing direct API calls.
Features:
- User list table with ID, name, email, role, status, created date
- Create user form with display name, email, role selector
- Suspend/activate actions per user
- Create API token for any user (shows plaintext once with copy button)
- Role badges (admin highlighted, member muted)
- Non-admin users see "Admin access required" message
- Keyboard shortcut: Cmd/Ctrl+5 switches to Users tab
CSS:
- Reuses routines-table styles for the user list
- Badge, token-display, btn-small, btn-danger, btn-primary components
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move Users to Settings subtab, bootstrap admin user on first run
- Moved Users from top-level tab to Settings sidebar subtab (under
Skills, before Theme toggle)
- On first startup with empty users table, automatically creates an
admin user from GATEWAY_USER_ID config with a corresponding API
token from GATEWAY_AUTH_TOKEN. This ensures the owner appears in
the Users panel immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: user creation shows token, + Token works, no password save popup
Three UI/UX fixes:
1. Create user now generates an initial API token and shows it in a
copy-able banner instead of triggering the browser's password save
dialog. Uses autocomplete="off" and type="text" for email field.
2. "+ Token" button works: exposed createTokenForUser/suspendUser/
activateUser on window for inline onclick handlers in dynamically
generated table rows. Token creation uses showTokenBanner helper.
3. Admin token creation: POST /api/tokens now accepts optional
"user_id" field when the requesting user is admin, allowing
token creation for other users from the Users panel.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use event delegation for user action buttons (CSP compliance)
Inline onclick handlers are blocked by the Content-Security-Policy
(script-src 'self' without 'unsafe-inline'). Switched to data-action
attributes with a delegated click listener on the users table.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add i18n for Users subtab, show login link on user creation
- Added 'settings.users' i18n key for English and Chinese
- Token banner now shows a full login link (domain/?token=xxx)
with a Copy Link button, plus the raw token below
- Login link works automatically via existing ?token= auto-auth
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: token hash mismatch — hash hex string, not raw bytes
Critical auth bug: token creation hashed the raw 32 bytes
(hasher.update(token_bytes)) but authentication hashed the hex-encoded
string (hash_token(candidate) where candidate is the hex string the
user sends). This meant newly created tokens could never authenticate.
Fixed all 4 token creation sites (users, tokens, invitations create,
invitations accept) to use hash_token(&plaintext_token) which hashes
the hex string consistently with the auth lookup path.
Removed now-unused sha2::Digest imports from handlers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: remove invitation system
The invitation flow is redundant — admin create user already generates
a token and shows a login link. Invitations add complexity without
value until email integration exists.
Removed:
- InvitationRecord struct and 4 UserStore trait methods
- invitations table from V14 migration (postgres + both libsql schemas)
- PostgreSQL Store methods (create/get/accept/list invitations)
- libSQL UserStore invitation methods + row_to_invitation helper
- invitations.rs handler file (212 lines)
- /api/invitations routes (create, list, accept)
- test_invitation_lifecycle test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: user deletion, self-service profile, per-user job limits, usage API
Four multi-tenancy improvements:
1. User deletion cascade (DELETE /api/admin/users/{id}):
Deletes user and all data across 11 user-scoped tables (settings,
secrets, routines, memory, jobs, conversations, etc.). Admin only.
2. Self-service profile (GET/PATCH /api/profile):
Users can read and update their own display_name and metadata
without admin privileges.
3. Per-user job concurrency (MAX_JOBS_PER_USER env var):
Scheduler checks active_jobs_for(user_id) before dispatch.
Prevents one user from exhausting all job slots.
4. Usage reporting (GET /api/admin/usage?user_id=X&period=day|week|month):
Aggregates LLM costs from llm_calls via agent_jobs.user_id.
Returns per-user, per-model breakdown of calls, tokens, and cost.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add TenantCtx for compile-time tenant isolation
Implements zmanian's architectural proposal from #1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.
TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.
AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).
TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.
Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR #1626 review feedback — bounded LRU cache, admin auth, FK cleanup
- Replace HashMap with lru::LruCache in DbAuthenticator so the token
cache is hard-bounded at 1024 entries (evicts LRU, not just expired)
- Gate admin user endpoints (list/detail/update/suspend/activate) with
AdminUser extractor so members get 403 instead of full access
- Add api_tokens to libSQL delete_user cleanup list to prevent orphaned
tokens (libSQL has no FK cascade)
- Add regression tests for all three fixes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update CA certificates in runtime Docker image
Ensures the root certificate bundle is current so TLS handshakes
to services like Supabase succeed on Railway.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: resolve CI failures — formatting, no-panics check
- Run cargo fmt on test code
- Replace .expect() with const NonZeroUsize in DbAuthenticator
- Add // safety: comments for test-only code in multi_tenant.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: switch PostgreSQL TLS from rustls to native-tls
rustls with rustls-native-certs fails TLS handshake on Railway's
slim container (empty or stale root cert store). native-tls delegates
to OpenSSL on Linux which handles system certs more reliably.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Adding user management api
* feat: admin secrets provisioning API + API documentation
- Add PUT/GET/DELETE /api/admin/users/{id}/secrets/{name} endpoints for
application backends to provision per-user secrets (AES-256-GCM encrypted)
- Add secrets_store field to GatewayState with builder wiring
- Create docs/USER_MANAGEMENT_API.md with full API spec covering users,
secrets, tokens, profile, and usage endpoints
- Update web gateway CLAUDE.md route table
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add CatchPanicLayer to capture handler panics
Without this, panics in async handlers silently drop the connection
and the edge proxy returns a generic 503. Now panics are caught,
logged, and returned as 500 with the panic message.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address second-round review — transactional delete, overflow, error logging
- C1: Wrap PostgreSQL delete_user() in a transaction so partial cleanup
can't leave users in a half-deleted state
- M2: Add job_events to delete cleanup (both backends) — FK to
agent_jobs without CASCADE would cause FK violation
- H1/M4: Cap expires_in_days to 36500 before i64 cast (tokens + secrets)
- H2: Validate target user exists before creating admin token to prevent
orphan tokens on libSQL
- H3: Log DB errors in DbAuthenticator::authenticate() instead of
silently swallowing them as 401
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert to rustls with webpki-roots fallback for PostgreSQL TLS
native-tls/OpenSSL caused silent crashes (segfaults in C code) during
DB writes on Railway containers. Switch back to rustls but add
webpki-roots as a fallback when system certs are missing, which was
the original TLS handshake failure on slim container images.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update Cargo.lock for rustls + webpki-roots
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* debug: add /api/debug/db-write endpoint to diagnose user insert failure
Temporary diagnostic endpoint that tests DB INSERT to users table
with full error logging. No auth required. Will be removed after
debugging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: use cargo-chef in Dockerfile for dependency caching
Splits the build into planner/deps/builder stages. Dependencies are
only recompiled when Cargo.toml or Cargo.lock change. Source-only
changes skip straight to the final build stage.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* debug: add tracing to users_create_handler
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard created_by FK in user creation handler
The auth identity user_id (from owner_id scope) may not match any
user row in the DB, causing a FK violation on the created_by column.
Check that the referenced user exists before setting created_by.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: collapse GATEWAY_USER_ID into IRONCLAW_OWNER_ID
Remove the separate GATEWAY_USER_ID config. The gateway now uses
IRONCLAW_OWNER_ID (config.owner_id) directly for auth identity,
bootstrap user creation, and workspace scoping.
Previously, with_owner_scope() rebinds the auth identity to owner_id
while keeping default_sender_id as the gateway user_id. This caused
a FK constraint violation when creating users because the auth
identity ("default") didn't match any user in the DB ("nearai").
Changes:
- Remove GATEWAY_USER_ID env var and gateway_user_id from settings
- Remove user_id field from GatewayConfig
- Add owner_id parameter to GatewayChannel::new()
- Remove with_owner_scope() method
- Remove default_sender_id from GatewayState
- Remove sender override logic in chat/approval handlers
- Remove debug endpoint and tracing from prior debugging
- Update all tests and E2E fixtures
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: hide Users tab for non-admins, remove auth hint text
- Fetch /api/profile after login and hide the Users settings tab
when the user's role is not admin
- Remove the "Enter the GATEWAY_AUTH_TOKEN" hint from the login page
since tokens are now managed via the admin panel, not .env files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review feedback (auth 503, token expiry, CORS PATCH)
- DB auth errors now return 503 instead of 401 so outages are
distinguishable from invalid tokens (serrrfirat H3)
- Cap expires_in_days to 36500 before i64 cast to prevent negative
duration from u64 overflow (serrrfirat H1)
- Add PATCH to CORS allowed methods for profile/user update
endpoints (Copilot)
- Stop leaking panic details in CatchPanicLayer response body
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: harden multi-tenant isolation — review fixes from #1614
- Add conversation ownership checks in TenantScope: add_conversation_message,
touch_conversation, list_conversation_messages (+ paginated),
update_conversation_metadata_field, get_conversation_metadata now return
NotFound for conversations not owned by the tenant (cross-tenant data leak)
- Fix multi-user heartbeat: clear notify_user_id per runner so notifications
persist to the correct user, not the shared config target
- Move hygiene tasks into bounded JoinSet instead of unbounded tokio::spawn
- Revert send_notification to private visibility (only used within module)
- Use effective_model_name() for cost attribution in dispatcher so providers
that ignore per-request model overrides report the actual model used
- Fix inject_model_override doc comment; add 3 unit tests
- Fix heartbeat doc comment ("routines" not "active routines")
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add Jobs, Cost, Last Active columns to admin Users table
Add UserSummaryStats struct and user_summary_stats() batch query to the
UserStore trait (both PostgreSQL and libSQL backends). The admin users
list endpoint now fetches per-user aggregates (job count, total LLM
spend, most recent activity) in a single query and includes them inline
in the response. The frontend Users table displays three new columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address review comments and CI formatting failures
CI fixes:
- cargo fmt fixes in cli/mod.rs and db/tls.rs
Security/correctness (from Copilot + serrrfirat + pranavraja99 reviews):
- Token create: reject expires_in_days > 36500 with 400 instead of silent clamp
- Token create: return 404 when admin targets non-existent user
- User create: map duplicate email constraint violations to 409 Conflict
- User create: remove unnecessary DB roundtrip for created_by (use AdminUser directly)
- DB auth: log warn on DB lookup failures instead of silently swallowing errors
- libSQL: add FK constraints on users.created_by and api_tokens.user_id
Config fixes:
- agent.multi_tenant: resolve from AGENT_MULTI_TENANT env var instead of hardcoding false
- heartbeat.multi_tenant: fix doc comment to match actual env-var-based behavior
UI fix:
- showTokenBanner: pass correct title ("Token created!" vs "User created!")
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address remaining review comments (round 2)
- Secrets handlers: normalize name to lowercase before store operations,
validate target user_id exists (returns 404 if not found)
- libSQL: propagate cost parsing errors instead of unwrap_or_default()
in both user_usage_stats and user_summary_stats
- users_list_handler: propagate user_summary_stats DB errors (was
silently swallowed with unwrap_or_default)
- loadUsers: distinguish 401/403 (admin required) from other errors
- Docs: fix users.id type (TEXT not UUID), remove "invitation flow"
from V14 migration comment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: i18n for Users tab, atomic user+token creation, transactional delete_user
i18n:
- Add 31 translation keys for all Users tab strings (en + zh-CN)
- Wire data-i18n attributes on HTML elements (headings, buttons, inputs,
table headers, empty state)
- Replace all hard-coded strings in app.js with I18n.t() calls
Atomic user+token creation:
- Add create_user_with_token() to UserStore trait
- PostgreSQL: wraps both INSERTs in conn.transaction() with auto-rollback
- libSQL: wraps in explicit BEGIN/COMMIT with ROLLBACK on error
- Handler uses single atomic call instead of two separate operations
Transactional delete_user for libSQL:
- Wrap multi-table DELETE cascade in BEGIN/COMMIT transaction
- ROLLBACK on any error to prevent partial cleanup / inconsistent state
- Matches the PostgreSQL implementation which already used transactions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert V14 migration to match deployed checksum [skip-regression-check]
Refinery checksums applied migrations — editing V14__users.sql after
it was already applied causes deployment failures. Revert the cosmetic
comment changes (added in
|
||
|
ironclaw-ci[bot]
|
52551f0ef4 |
chore(ironclaw): release v0.23.0 (#1658)
Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> |
||
|
Henry Park
|
ab67f02886 | fix: publish ironclaw_safety 0.2.0 (#1659) | ||
|
ironclaw-ci[bot]
|
0b4e7c761b |
chore: release v0.22.0 (#1601)
Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> |
||
|
Henry Park
|
bb24952622 | Merge branch 'main' into staging-promote/455f543b-23329172268 | ||
|
Henry Park
|
b400c2a711 |
Merge pull request #1499 from nearai/staging-promote/9603fefd-23364438978
chore: promote staging to staging-promote/d3b69e7b-23359661011 (2026-03-20 22:04 UTC) |
||
|
serrrfirat
|
67a025e2fa | fix(deps): unblock promotion PR #1451 cargo-deny | ||
|
Illia Polosukhin
|
706c3a1b47 |
refactor: extract AppEvent to crates/ironclaw_common (#1615)
* refactor: extract AppEvent to crates/ironclaw_common SseEvent was defined in src/channels/web/types.rs but imported by 12+ modules across agent, orchestrator, worker, tools, and extensions — it had become the application-wide event protocol, not a web transport concern. Create crates/ironclaw_common as a shared workspace crate and move the enum there as AppEvent. Also move the truncate_preview utility which was similarly leaked from the web gateway into agent modules. - New crate: crates/ironclaw_common (AppEvent, truncate_preview) - Rename SseEvent → AppEvent, from_sse_event → from_app_event - web/types.rs re-exports AppEvent for internal gateway use - web/util.rs re-exports truncate_preview - Wire format unchanged (serde renames are on variants, not the enum) Aligned with the event bus direction on refactor/architectural-hardening where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: add AppEvent::event_type() helper, deduplicate match blocks Address Gemini review: extract the variant→string match into a single method on AppEvent, replacing the duplicated 22-arm matches in sse.rs and types.rs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename leftover sse vars/tests to match AppEvent rename Address Copilot review: rename sse_event vars to app_event in orchestrator/api.rs and ws.rs, rename test functions from test_ws_server_from_sse_* to test_ws_server_from_app_event_*, and update stale SSE comments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: add Deserialize to AppEvent, round-trip test, fix stale comments Address zmanian review: - Add Deserialize derive to AppEvent so downstream consumers can deserialize incoming events - Add event_type_matches_serde_type_field test that round-trips every variant through serde and asserts event_type() matches the serialized "type" field — catches drift between serde renames and the manual match - Add round_trip_deserialize test for basic Serialize/Deserialize parity - Update remaining "SSE" references in comments across server.rs, manager.rs, ws_gateway_integration.rs, and worker/job.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Reid
|
485d1568c4 |
feat(cli): add ironclaw models subcommands (list/status/set/set-provider) (#1043)
* feat(cli): add ironclaw models subcommands (list/status/set/set-provider) Implements model management CLI (part of #83): - `models list [provider] [--verbose] [--json]` — list providers; fetches live model list from the provider API when a specific provider is given - `models status [--json]` — show active provider/model - `models set <model>` — set default model with validation - `models set-provider <id> [--model <name>]` — set provider with alias normalization - fix conflicts * fix(deps): update tar to 0.4.45 (RUSTSEC-2026-0067, RUSTSEC-2026-0068) --------- Co-authored-by: firat.sertgoz <f@nuff.tech> |
||
|
Illia Polosukhin
|
a09c023642 |
feat(ux): complete UX overhaul — design system, onboarding, web polish (#1277)
* feat(ux): complete UX overhaul — design system, boot screen, onboarding, web polish Shared design system: CSS custom properties for spacing, typography, transitions, and color tokens used across web UI and boot screen. Boot screen: compact feature-tags line showing enabled subsystems (db, tools, routines, heartbeat, skills, sandbox, embeddings) at a glance. Downgrade startup info logs (libSQL, webhook, workspace seed) to debug level since the boot screen now covers this. Onboarding wizard: model picker with live API fetch, provider-aware auth flow, improved error recovery and progress display. Web UI: ARIA attributes, welcome card, streaming debounce, connection status banner, skeleton loaders, send cooldown. CLI: doctor command enhancements, status command cleanup, REPL banner consolidation, shared fmt module. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(ux): Apple-level design refinements — spring physics, glass morphism, chat polish Merge staging theme support (dark/light/system toggle) and layer UX polish on top: spring-physics motion, glass morphism depth, chat experience improvements, and responsive mobile refinements. Design system: - Restore and extend design token system (spacing, typography, timing, easing) with legacy aliases for theme compatibility - Add shadow tiers, accent glow, glass morphism, spring easing tokens - Tokens defined in both dark (:root) and light ([data-theme="light"]) Micro-interactions (Phase 2): - Spring-overshoot message entry animation (slideUp) - Spring-scale button press on all interactive buttons - Tab crossfade animation, tool card smooth accordion (max-height) - Modal scale(0.95) + blur(8px) entry, toast spring slide - Sidebar width crossfade, card hover lift Visual depth (Phase 3): - Tab bar glass morphism + surface highlight + sliding indicator - Active tab accent background pill - Assistant message accent left border, user message bubble tail - Floating input area (rounded + shadow + margin) Chat polish (Phase 4): - Smooth streaming cursor (cursorPulse), message hover timestamps - Time separators (Today/Yesterday/date) - Textarea smooth auto-expand, send button glow Settings & forms (Phase 5): - iOS-style toggle switches for boolean settings - Input focus glow, save feedback spring animation - Welcome card with gradient background + proper spacing - Sticky settings group headers with glass backdrop Accessibility & mobile (Phase 6): - Animated focus ring, prefers-reduced-motion global kill-switch - Touch target audit (44px min), mobile bottom-sheet modals - Mobile bottom tab bar, toast redesign (icon + border + countdown) - Thread hover translateX, badge in_progress pulse Bug fixes: - Gateway/TEE popover z-index (tab-bar z-index: 200, popovers 500) - Connection lost banner as fixed top bar instead of flex child - Sidebar collapse keeps toggle + new thread buttons visible - Downgrade noisy startup logs (db, webhook, vector) to debug - Remove green dot pulse animation on connected status - Deduplicate confirm-modal in HTML, add tab-indicator div Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): mobile layout improvements — sidebar toggle, settings drill-down, tab bar polish - Fix mobile sidebar toggle: use expanded-mobile class instead of collapsed, add backdrop overlay, auto-close on thread select, outside-click dismiss - Settings: replace cramped horizontal tabs with drill-down navigation (category list → detail view → back button) - Bottom tab bar: add glass morphism, hide theme toggle, flip tab indicator to top edge - Keep thread toggle button visible in collapsed 36px sidebar strip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(repl): interactive approval selector and transient status lines - Replace ASCII-art approval box with clean horizontal rule card - Add inquire-based interactive selector for tool approvals (↑↓ + Enter) - Selector runs directly from send_status via spawn_blocking, with stdin_locked flag to prevent readline from competing for stdin - Transient thinking/tool-started lines: each replaces the previous, all erased before final output (no clutter left in scrollback) - Esc in selector sends denial so agent never gets stuck Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: widen TurnCost token fields to u64 and remove unused variable - Change input_tokens/output_tokens from u32 to u64 in StatusUpdate::TurnCost, SseEvent::TurnCost, and the thread_ops emit site to avoid truncation on large conversations - Remove unused _routine_engine_for_loop binding in agent_loop.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: reduce startup log noise — demote info to debug Demote routine startup messages (builder, WASM tools, tunnel, WASM channels) from info to debug so the default log output stays clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow CDN scripts in CSP connect-src directive Add cdn.jsdelivr.net and cdnjs.cloudflare.com to connect-src so the browser can fetch marked.js and DOMPurify without CSP violations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix cargo fmt in repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): gate turn_cost SSE handler on current thread Prevents cost badge from attaching to the wrong message when switching threads or receiving events from background threads. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: retrigger CI * fix: add missing extension_manager to webhook EngineContext The webhook trigger path added in #736 was missing the extension_manager field introduced by #1453. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: ignore RUSTSEC-2026-0049 rustls-webpki CRL advisory Low impact — requires compromised CA to exploit. Tracked for upstream rustls-webpki upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(routines): use fields.join for cron normalization Use split_whitespace fields instead of re-trimming the original string to avoid preserving extra internal whitespace in cron expressions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(repl): Apple-style approval card — clean vertical flow - Drop verbose tool description (the command IS the decision surface) - Unified vertical pipe layout: ◆ header → │ params → │ selector - Selector options show keyboard shortcuts inline: Approve (y) - Compact help message, answered state uses └ to close the flow - No horizontal rules, no blank-line padding — just breathing room Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(repl): replace inquire with crossterm for approval selector Drop the inquire dependency (which pulled in crossterm 0.25, duplicating the existing 0.28). The 3-option approval selector is now built directly with crossterm raw mode — same UX, zero new dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(deps): upgrade crossterm 0.28 → 0.29, eliminate duplication termimad (via crokey) uses crossterm 0.29. Upgrading our direct dependency from 0.28 to 0.29 collapses to a single crossterm version in the dependency tree. Also migrated termimad::crossterm:: references to the direct crossterm import. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review comments — box_top off-by-one, smart_truncate overflow, mobile theme toggle - Fix box_top() fill calculation: was off-by-one, producing boxes 1 char too wide (fmt.rs) - Fix smart_truncate(): account for "..." in the budget so output never exceeds max_chars (repl.rs) - Move theme toggle to settings sidebar on mobile instead of display:none, so mobile users can still switch themes (style.css, index.html, app.js) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review — retry duplication, CSP connect-src, deny color - Remove failed message before retry to prevent duplicate user messages - Revert connect-src to 'self' — CDN hosts only need script-src - Use red for Deny confirmation in REPL approval selector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
189fc031e3 | Merge branch 'staging' into fix/musl-installer-targets | ||
|
ilblackdragon@gmail.com
|
0d1a5c210b |
fix(deps): patch rustls-webpki vulnerability (RUSTSEC-2026-0049)
Update rustls-webpki 0.103.9 → 0.103.10. Exempt 0.102.8 which is pinned by libsql's transitive dependency on an older rustls chain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ironclaw-ci[bot]
|
91a241a3c7 |
chore: release v0.21.0 (#1472)
Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> |
||
|
ironclaw-ci[bot]
|
7dc3c6d067 |
chore: release v0.20.0 (#1310)
Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> |
||
|
brajul
|
bca8bbc8ed |
fix: update Cargo.lock and pin musl CI runners
Address review feedback: - Regenerate Cargo.lock to reflect rig-core reqwest-rustls switch, removing openssl-sys and native-tls from the dependency tree - Add github-custom-runners entries for musl targets |
||
|
github-actions[bot]
|
1ad1335fea |
chore: release v0.19.0 (#973)
Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> |
||
|
Nick Stebbings
|
f618166ad8 |
feat(heartbeat): fire_at time-of-day scheduling with IANA timezone (#1029)
* feat(heartbeat): fire_at time-of-day scheduling with IANA timezone support - HEARTBEAT_FIRE_AT=HH:MM — fire heartbeat at a specific time of day instead of on a rolling interval; format is 24h HH:MM (e.g. "14:00") - HEARTBEAT_TIMEZONE=Region/City — IANA timezone name for fire_at (e.g. "Pacific/Auckland", "America/New_York"). Defaults to UTC. - When fire_at is set, interval_secs is ignored - Config also readable from settings.toml [heartbeat] section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(heartbeat): wire fire_at + timezone into HeartbeatConfig runner Missed file from heartbeat scheduling commit. HeartbeatConfig struct in agent/heartbeat.rs now carries fire_at: Option<NaiveTime> and timezone: Tz so the runner can schedule against a fixed time of day. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add chrono-tz dependency for heartbeat fire_at timezone support The chrono-tz crate was used in the heartbeat fire_at commits but its Cargo.toml entry was lost during rebase conflict resolution. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: rustfmt fix for chained method call Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(heartbeat): add fire_at scheduling and DST safety tests - test_default_config_has_no_fire_at: interval-based default unchanged - test_with_fire_at_builder: builder sets time and timezone - test_duration_until_next_fire_is_bounded: result always 1s–24h - test_duration_until_next_fire_dst_timezone_no_panic: US Eastern DST - test_resolved_tz_defaults_to_utc: missing timezone falls back to UTC - test_resolved_tz_parses_iana: IANA string resolves correctly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(heartbeat): restore drift-free interval, add settings.json fallback for fire_at - Interval path: restore tokio::time::interval (drift-free) instead of tokio::time::sleep which drifts by loop body execution time - fire_at config: fall back to settings.heartbeat.fire_at when HEARTBEAT_FIRE_AT env var is not set, consistent with other settings Addresses Gemini Code Assist review feedback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: IronClaw <deploy@agentiff.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
ZeroTrust
|
1b59eb6b39 |
feat: Reuse Codex CLI OAuth tokens for ChatGPT backend LLM calls (#693)
* feat: add Codex auth.json token reuse for LLM authentication When LLM_USE_CODEX_AUTH=true, IronClaw reads the Codex CLI's auth.json (default ~/.codex/auth.json) and extracts the API key or OAuth access token. This lets IronClaw piggyback on a Codex login without implementing its own OAuth flow. New env vars: - LLM_USE_CODEX_AUTH: enable Codex auth fallback (default: false) - CODEX_AUTH_PATH: override path to auth.json * fix: handle ChatGPT auth mode correctly Switch base_url to chatgpt.com/backend-api/codex when auth.json contains ChatGPT OAuth tokens. The access_token is a JWT that only works against the private ChatGPT backend, not the public OpenAI API. Refactored codex_auth.rs to return CodexCredentials (token + is_chatgpt_mode) instead of just a string key. * fix: Codex auth takes highest priority over secrets store When LLM_USE_CODEX_AUTH=true, Codex credentials are now loaded before checking env vars or the secrets store overlay. Previously the secrets store key (injected during onboarding) would shadow the Codex token. * feat: Responses API provider for ChatGPT backend - New CodexChatGptProvider speaks the Responses API protocol - Auto-detects model from /models endpoint (gpt-4o -> gpt-5.2-codex) - Adds store=false (required by ChatGPT backend) - Error handling with timeout for HTTP 400 responses - Message format translation: Chat Completions -> Responses API - SSE response parsing for text, tool calls, and usage stats - 7 unit tests for message conversion and SSE parsing * fix: SSE parser uses item_id instead of call_id for tool call deltas The Responses API sends function_call_arguments.delta events with item_id (e.g. fc_...) not call_id (e.g. call_...). The parser now keys pending tool calls by item_id from output_item.added and tracks call_id separately for result matching. * fix: strip empty string values from tool call arguments gpt-5.2-codex fills optional tool parameters with empty strings (e.g. timestamp: ""), which IronClaw's tool validation rejects. Strip them before passing to tool execution. * fix: prevent apiKey mode fallback to ChatGPT token When auth_mode is explicitly 'apiKey' but the key is missing/empty, do not fall through to check for a ChatGPT access_token. This prevents returning credentials with is_chatgpt_mode: true and routing to the wrong LLM provider. * refactor: reuse single reqwest::Client across model discovery and LLM calls Create Client once in with_auto_model, pass &Client to fetch_default_model, and move it into the provider struct. Eliminates the redundant Client::new() that wasted a connection pool. * fix: bump client_version to 1.0.0 to unlock gpt-5.3-codex and gpt-5.4 The /models endpoint gates newer models behind client_version. Version 0.1.0 only returns up to gpt-5.2-codex, while 1.0.0+ also returns gpt-5.3-codex and gpt-5.4. * feat: user-configured LLM_MODEL takes priority over auto-detection Fetch the full model list from /models endpoint. If LLM_MODEL is set, validate it against the supported list and warn with available models if not found. If LLM_MODEL is not set, auto-detect the highest-priority model. Also bumps client_version to 1.0.0 to unlock gpt-5.3/5.4. * fix: add 10s timeout to model discovery HTTP request Prevents startup from blocking indefinitely if chatgpt.com is slow or unreachable. Uses reqwest per-request timeout. * docs: add private API warning for ChatGPT backend endpoint The chatgpt.com/backend-api/codex endpoint is private and undocumented. Add warning in module docs and a runtime log on first use to inform users of potential ToS implications. * feat: implement OAuth 401 token refresh for Codex ChatGPT provider On HTTP 401, if a refresh_token is available, the provider now automatically refreshes the access token via auth.openai.com/oauth/token (same protocol as Codex CLI) and retries the request once. Refreshed tokens are persisted back to auth.json. Changes: - codex_auth: read refresh_token, add refresh_access_token() and persist_refreshed_tokens() - codex_chatgpt: RwLock for api_key, 401 detection + retry in send_request, send_http_request helper - config/llm: thread refresh_token/auth_path through RegistryProviderConfig - llm/mod: pass refresh params to with_auto_model * refactor: lazy model detection via OnceCell, remove block_in_place Model is no longer resolved during provider construction. Instead, resolve_model() uses tokio::sync::OnceCell to lazily fetch from /models on the first LLM call. This eliminates the block_in_place + block_on workaround in create_codex_chatgpt_from_registry. - with_auto_model (async) -> with_lazy_model (sync constructor) - resolve_model() added with OnceCell-based lazy init - build_request_body takes model as parameter - model_name() returns resolved or configured_model as fallback * feat: support multimodal content (images) in Codex ChatGPT provider message_to_input_items now checks content_parts for user messages. ContentPart::Text maps to input_text and ContentPart::ImageUrl maps to input_image, matching the Responses API format used by Codex CLI. Falls back to plain text when content_parts is empty. Also updates client_version to 0.111.0 for /models endpoint. Adds test: test_message_conversion_user_with_image * refactor: move codex_auth module from src/ to src/llm/ codex_auth is only used by the LLM layer (codex_chatgpt provider and config/llm). Moving it under src/llm/ reflects its actual scope. - Remove pub mod codex_auth from lib.rs - Add pub mod codex_auth to llm/mod.rs - Update imports: super::codex_auth, crate::llm::codex_auth * Fix codex provider style issues * Use SecretString throughout codex auth refresh flow * Use SecretString for codex access tokens * Reuse provider client for codex token refresh * Stream Codex SSE responses incrementally * Fix Windows clippy and SQLite test linkage * Trigger checks after regression skip label * Tighten codex auth module handling |
||
|
Illia Polosukhin
|
15ab156d62 |
feat: add Criterion benchmarks for safety layer hot paths (#836)
* feat: add Criterion benchmarks for safety layer hot paths Add benchmark suite using Criterion.rs for performance-critical paths: - benches/safety_check.rs: Sanitizer (clean/adversarial), Validator (normal/long/tool params), LeakDetector (clean/secrets/HTTP scan) - benches/tool_dispatch.rs: JSON parsing, schema validation patterns, tool output serialization CI compiles benchmarks on every PR to prevent regressions. Run locally with: cargo bench Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add bench-compile to CI roll-up job Include bench-compile in the run-tests roll-up job's needs array so benchmark compilation failures block PRs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add black_box to benchmarks, use real SafetyLayer pipeline - Wrap all benchmark inputs in criterion::black_box to prevent compiler optimization from skewing results - Replace generic JSON benchmarks in tool_dispatch.rs with actual SafetyLayer pipeline benchmarks (sanitize_tool_output, wrap_for_llm, scan_inbound_for_secrets) - Keep JSON parsing benchmarks for tool parameter overhead measurement Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: apply cargo fmt to benchmark files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: copy benches/ in Dockerfile to fix manifest parse error Cargo.toml references [[bench]] targets that must exist for manifest parsing to succeed. Add COPY benches/ to the Docker build stage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: re-trigger CI after adding skip-regression-check label Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review comments on criterion benchmarks - Move header string allocations outside b.iter() closure in http_request_scan to avoid measuring allocation overhead - Add .unwrap() to serde_json::from_str results in JSON parsing benchmarks to catch invalid JSON instead of silently benchmarking error construction - Add comment explaining why benches/ COPY is needed in Dockerfile ([[bench]] entries require source files for cargo manifest parsing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update Cargo.lock with criterion dependencies Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(bench): build secret-like strings at runtime to avoid CI secret scanners Construct AWS key and GitHub token patterns via format!() concatenation so the literal strings don't appear in source and trigger push protection or secret scanning in CI pipelines. The resulting strings still match LeakDetector patterns for valid benchmarking. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: rename tool_dispatch bench, drop async_tokio, replace JSON benchmarks 1. Rename `tool_dispatch.rs` → `safety_pipeline.rs` to match actual content (SafetyLayer pipeline benchmarks). 2. Drop unused `async_tokio` feature from criterion dependency. 3. Replace serde_json::from_str benchmarks (third-party only) with Validator::validate_tool_params exercising IronClaw's recursive validation on simple, complex, and deeply nested JSON inputs. 4. Add `--all-features` to CI bench-compile to match clippy/test convention and verify both DB backends. Addresses zmanian's review feedback on PR #836. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nige
|
5f0ed66a6b |
perf(routines): avoid full message history clone each tool iteration (#1172)
* perf(routines): bound tool-loop history snapshot clone cost * test(ci): annotate snapshot assertions for no-panics matcher * test(ci): keep no-panics suppression on single-line assertion * test(ci): keep snapshot tail assert single-line for no-panics * Update src/agent/routine_engine.rs Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore(deps): bump yanked uds_windows in lockfile for cargo-deny --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> |
||
|
Illia Polosukhin
|
f776d96395 |
fix: remove all inline event handlers for CSP script-src compliance (#1063)
* chore: promote staging to main (2026-03-10 15:19 UTC) (#865)
* fix: Channel HTTP: server doesn't start after config change (no hot-r… (#779)
* fix: Channel HTTP: server doesn't start after config change (no hot-reload)
* review fixes
* review fixes
* fix linter
* fix code style
* fix: prevent session lock contention blocking message processing (#783)
* fix: prevent session lock contention blocking message processing
## Problem
After container restart, POST /api/chat/send returns 202 ACCEPTED but messages
don't appear in conversation_messages and agent never responds. Messages get
stuck in "stale state" after restart.
Root cause: Session lock was held for entire duration of chat_threads_handler
and chat_history_handler, including during slow database queries. This blocked
the agent loop from acquiring the session lock to process incoming messages,
causing them to hang indefinitely.
## Solution
1. **Release session lock early in chat_threads_handler**: Only acquire lock
when reading active_thread at response time, not during DB queries for
thread list. DB operations no longer block message processing.
2. **Release session lock early in chat_history_handler**: Only acquire lock
when accessing in-memory thread state, not during paginated DB queries or
thread ownership checks. DB operations no longer block message processing.
3. **Add comprehensive logging**: Track message flow from receipt through
session resolution, thread hydration, and state transitions. Helps diagnose
future issues:
- Message queued to agent loop (chat_send_handler)
- Processing message from channel (handle_message)
- Hydrating thread from DB (maybe_hydrate_thread)
- Resolving session and thread (resolve_thread)
- Checking thread state (process_user_input)
- Persisting user message (persist_user_message)
## Impact
- Message processing no longer blocks on session lock contention
- API response times for thread list/history queries unaffected (DB queries
still happen, but lock is not held)
- Better diagnostics for future debugging
## Testing
- All 2756 tests pass
- Code compiles with zero clippy warnings
- No changes to user-facing API or behavior, only lock timing
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* security: redact PII from info-level logs
Downgrade user_id and channel logging to debug level to prevent exposing
Personally Identifiable Information (PII) in production logs.
The user_id field can contain sensitive information such as phone numbers
(e.g., for Signal messages). Logging PII in cleartext at the info level
creates a security and privacy risk, as these logs may be stored in
persistent storage, indexed by log management systems, or accessible to
unauthorized personnel.
Changes:
- Info level: logs only message_id (UUID) for tracking
- Debug level: logs user_id, channel, thread_id for troubleshooting
This maintains debugging capability for developers while protecting user
privacy in production logs.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
* chore: sync main into staging (#855)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)
Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).
- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
* fix: Chat input is hidden in mobile browser mode (#877)
* fix: stop XML-escaping tool output content (#598) (#874)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)
Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).
- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: stop XML-escaping tool output content in wrap_for_llm (#598)
Remove content escaping that corrupted JSON in tool output. The
<tool_output> structural boundary is preserved but content now passes
through raw, fixing downstream parse failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(safety): allow empty string tool params (#848)
* fix(safety): allow empty string tool params
* fix(safety): preserve heuristic checks and add path context to tool validation
This follow-up refactor addresses PR review feedback by restoring
heuristic checks (whitespace ratio, character repetition) for tool
parameter validation and improving error reporting.
Changes:
- Restored heuristic warnings in validate_non_empty_input so they apply
to both user input and tool parameters (when non-empty).
- Refactored check_strings to recursively build and pass JSON paths
(e.g., "metadata.tags[1]").
- Updated validation errors to use the specific JSON path as the field
name instead of the generic "input".
- Added regression tests for whitespace/repetition warnings and JSON
path reporting in tool parameters.
This ensures the safety layer remains semantically neutral about empty
strings (fixing the memory_tree path: "" issue) while maintaining
rigorous protection and providing better developer ergonomics.
* style: run cargo fmt
* perf: optimize release and dist build profiles (#843)
* perf: optimize release and dist build profiles
Add [profile.release] with strip=true and panic="abort" for smaller,
faster release binaries. Upgrade [profile.dist] from lto="thin" to
lto="fat" with codegen-units=1 for maximum optimization in CI releases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove panic=abort from release profile
Reviewers (zmanian, Copilot, Gemini) correctly flagged that panic=abort
in the release profile would kill the entire process on any tokio task
panic, breaking fault isolation for the long-running server. Removed
from release profile entirely.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add PR template with risk assessment (#837)
* feat: add PR template with risk assessment and review tracks
Add a pull request template that includes summary, change type,
validation checklist, security/database impact sections, blast radius,
and rollback plan. Update CONTRIBUTING.md with review track definitions
(A/B/C) based on change risk level.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: expand CONTRIBUTING.md with setup, workflow, and guidelines
Add getting started, development workflow, code style summary,
database change guidance, and dependency management sections.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add fuzzing targets for untrusted input parsers (#835)
* feat: add fuzzing targets for untrusted input parsers
Add cargo-fuzz infrastructure with 5 fuzz targets exercising
security-critical code paths:
- fuzz_safety_sanitizer: Aho-Corasick + regex injection detection
- fuzz_safety_validator: Input validation (length, encoding, patterns)
- fuzz_leak_detector: Secret leak scanning (API keys, tokens)
- fuzz_tool_params: Tool parameter JSON validation
- fuzz_config_env: TOML/JSON config parsing
Each target exercises real IronClaw business logic with invariant
assertions. Includes corpus directories and setup documentation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve fuzz targets to exercise real IronClaw code paths
- fuzz_config_env: exercise SafetyLayer end-to-end (sanitize, validate,
policy check) instead of generic TOML/JSON parsing
- fuzz_tool_params: add validate_tool_schema coverage alongside
validate_tool_params
- Add "fuzz" to workspace exclude in root Cargo.toml
- Update README descriptions to match actual target behavior
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace redundant detect() call with meaningful invariant assertion
Replace the double sanitize()+detect() call with an assertion that
critical severity warnings always trigger content modification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: rewrite fuzz_config_env to exercise IronClaw safety code directly
Replace SafetyLayer wrapper usage with direct Sanitizer, Validator, and
LeakDetector instantiation and invocation. Adds meaningful consistency
assertions (non-empty output, valid-means-no-errors, scan/clean agreement).
Removes the config construction that was only exercising struct instantiation.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(wasm): run leak scan before credential injection in tools wrapper (#791)
* fix(wasm): run leak scan before credential injection in tools wrapper
The tools WASM wrapper runs the LeakDetector on HTTP request headers
AFTER inject_host_credentials() has already substituted real secrets
(e.g., xoxb- Slack bot tokens). This causes the leak detector to
flag the tool's own legitimate outbound API calls as secret exfiltration.
Move the scan to run on raw_headers before any credential injection,
matching the fix already applied to the channels wrapper in #421.
Fixes the same class of bug as #421 (which only fixed channels/wasm/wrapper.rs).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf: inline leak scan to avoid Vec allocation on every HTTP request
Address review feedback: instead of cloning all header keys/values into
a Vec to pass to scan_http_request(), iterate over raw_headers directly
using scan_and_clean(). This also provides more specific error messages
(URL vs header vs body).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: fix cargo fmt formatting in leak scan loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(setup): drain residual terminal events before secret input (#747) (#849)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)
Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).
- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip the regression check
[skip-regression-check]
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
* feat(agent): add context size logging before LLM prompt (#810)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(agent): add context size logging before LLM prompt
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
* fix: preserve text before tool-call XML in forced-text responses (#852)
* fix: preserve text before tool-call XML in forced-text responses (#789)
Local models (Qwen3, DeepSeek, GLM) emit <tool_call> XML even when no
tools are available (force_text mode). The existing strip_xml_tag()
discards everything from an unclosed opening tag onward, producing an
empty string that triggers the "I'm not sure how to respond" fallback.
Add truncate_at_tool_tags() — a code-region-aware pre-processing step
that truncates at the first tool-call XML tag BEFORE clean_response()
runs, preserving all useful text before the tag. Protect all 7
clean_response() call sites. Case-insensitive matching handles models
that emit <TOOL_CALL> or <Tool_Call> variants.
Secondary fix: add has_native_thinking() model detection to skip
<think>/<final> system prompt injection for models with built-in
reasoning (Qwen3, QwQ, DeepSeek-R1, GLM-Z1, etc.), preventing
thinking-only responses that clean to empty.
Wire with_model_name(active_model_name()) at all 9 production sites
that construct Reasoning, so the runtime model name (not static config)
drives system prompt generation.
126 new/updated tests covering truncation edge cases, code-block
awareness, Unicode, case-insensitivity, StubLlm integration for
complete/plan/evaluate_success/respond_with_tools paths, model
detection, and conditional system prompt generation.
Closes #789
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Copilot review — unclosed-only truncation, ASCII case folding
- truncate_at_tool_tags() now only truncates at UNCLOSED tool tags;
properly closed tags (e.g. <tool_call>...</tool_call>) are left intact
for clean_response() to strip normally, preserving any text after them
- Switch from to_lowercase() to to_ascii_lowercase() to prevent byte
offset misalignment with non-ASCII characters whose lowercase form
has different byte length (e.g. Kelvin sign U+212A)
- Add closing_tag_for() helper to derive closing tags from open patterns
- Fix doc comment: "fenced markdown code blocks or inline code spans"
(not "indented", which find_code_regions() doesn't detect)
- Add regression tests: closed vs unclosed for each tag variant,
Unicode + case-insensitive offset safety, and mixed closed/unclosed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: minor review items — consistent ascii_lowercase, closing_tag_for tests
- Switch has_native_thinking() from to_lowercase() to to_ascii_lowercase()
for consistency with truncate_at_tool_tags() approach
- Add unit tests for closing_tag_for(): standard tags, space-suffixed
patterns, pipe-delimited tags, and exhaustive coverage of all
TOOL_TAG_PATTERNS entries
- Add test for mixed closed+unclosed tags of different types
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Feat/docker shell edition (#804)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(mcp): strip top-level null params before forwarding to MCP servers (#795)
* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)
Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).
- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(mcp): strip top-level null params before forwarding to MCP servers
LLMs frequently emit `"field": null` for optional parameters in tool
calls. Many MCP servers reject explicit nulls for fields that should
simply be absent — e.g. Notion returns 400 for `"sort": null` in a
search call, expecting the field to be omitted entirely.
Strip top-level null keys from the params object before calling
`call_tool()`. Only top-level keys are stripped; nested nulls are
preserved since they may be semantically meaningful.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add event-triggered routines and workflow skill templates (#756)
* Add event-triggered routines and workflow skill templates
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback for event_emit security and quality
Security fixes:
- Require approval (UnlessAutoApproved) for event_emit, matching routine_fire
- Enable sanitization on event_emit payload (external JSON reaches LLM)
- Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id
Correctness fixes:
- Rename source → event_source in event_emit for consistency with routine_create
- Use json_value_as_filter_string for filter parsing (handles numbers/booleans)
- Case-insensitive matching for event source and event_type
- Add debug logging for missing filter keys in payload
- Fix skill_install_routine_webhook_sim test missing .with_skills()
- Fix schema_validator test for event_emit payload properties
Code quality:
- Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout)
- Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs
- Add test section headers in e2e_routine_heartbeat.rs
- Clarify event_emit description to specify system_event routines only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: make routine_system_event_emit test create routine before emitting
- Add routine_create step to trace fixture so event_emit has a matching
routine to fire
- Assert fired_routines > 0, not just key presence (Copilot review)
- Add .with_auto_approve_tools(true) since event_emit now requires approval
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: renumber test headers after system_event test insertion
Test 4 was duplicated (routine_cooldown and heartbeat_findings).
Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: merge staging and add missing RoutineEngine args in test
RoutineEngine::new on staging requires `tools` and `safety` params.
Update system_event_trigger_matches_and_filters test to pass them.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address new Copilot review comments
- Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim
test so event_emit doesn't block on approval
- Fix module-level doc comment for event_emit to specify system_event trigger
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deduplicate json_value_as_string helper
Remove private `json_value_as_string` from routine_engine.rs and use
the identical public `json_value_as_filter_string` from routine.rs,
eliminating divergence risk. (Copilot review)
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: enable WASM credential injection in No-DB environments (#845)
* fix(wasm): enable credential injection in no-DB environments via env var fallback
When a secrets store is unavailable (e.g. no-DB mode), WASM channel
credentials were silently not injected, causing channels to start without
credentials. Fix by:
- Changing `inject_channel_credentials_from_secrets` to accept
`Option<&dyn SecretsStore>` — secrets store is tried first when present
- Adding env var fallback (`inject_env_credentials`) for credentials not
covered by the secrets store
- Enforcing a channel-name prefix security check on env var names to
prevent WASM channels from reading unrelated host credentials
(e.g. `AWS_SECRET_ACCESS_KEY`)
- Extracting pure `resolve_env_credentials` helper for testability
- Adding case-insensitive prefix matching for secrets store lookup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wasm): inject credentials at startup when no secrets store (setup.rs path)
The startup path (setup_wasm_channels -> register_channel) was guarded by
`if let Some(secrets) = secrets_store`, so in No-DB mode credentials were
never injected and the channel started without them.
Fix by:
- Changing inject_channel_credentials to accept Option<&dyn SecretsStore>
- Always calling it (removing the if-let guard) — env var fallback runs
even when secrets_store is None
- Adding channel-name prefix security check to the env var fallback path
(e.g. TELEGRAM_ for channel "telegram"), consistent with manager.rs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(test): correct misleading comment on ICTEST1_UNRELATED_OTHER placeholder
* fix(wasm): guard against empty channel name in credential injection
An empty channel_name would produce prefix "_", allowing any env var
starting with "_" to pass the security check and be injected. Add an
early-return guard in resolve_env_credentials, inject_env_credentials,
and inject_channel_credentials. Add a test to cover this path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: lizican123 <lizican123@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: promote to main (#878)
* fix: replace unsafe env::set_var with thread-safe inject_single_var in SIGHUP handler
Fixes race condition where SIGHUP handler modifies global environment variables
while other threads may be reading them via Config::from_env().
Changes:
- Replace unsafe { std::env::set_var() } with ironclaw::config::inject_single_var()
- Uses INJECTED_VARS mutex instead of unsafe global state modification
- All reads via optional_env() check the thread-safe overlay first
- Prevents data races between SIGHUP reload and concurrent config reads
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: spawn webhook restart as background task to avoid blocking I/O across lock
Prevents holding Mutex lock during async I/O operations (TcpListener::bind,
task shutdown). The SIGHUP handler no longer blocks webhook processing during
listener restart.
Changes:
- Read old_addr and drop lock immediately
- Spawn restart_with_addr() as background task via tokio::spawn
- Lock is only held during the actual restart operation, not the signal handler
Benefits:
- SIGHUP handler returns immediately without blocking
- Webhook requests not delayed by listener restart I/O
- Lock contention significantly reduced
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: add graceful shutdown mechanism for SIGHUP handler background task
Prevents unbounded loop without cancellation token. The SIGHUP handler now
listens for a shutdown signal and exits cleanly during graceful termination.
Changes:
- Create broadcast channel for shutdown signaling
- SIGHUP handler uses tokio::select! to wait for shutdown or SIGHUP
- Send shutdown signal to all background tasks after agent.run() completes
- Ensures clean task lifecycle and no orphaned background tasks
Benefits:
- Proper task cancellation during graceful shutdown
- Follows Tokio best practices for background task management
- No background tasks orphaned when runtime shuts down
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* refactor: replace stringly-typed parameter filtering with typed enum and single helper
Fixes DRY violation where unsupported parameter filtering was duplicated across
rig_adapter.rs and anthropic_oauth.rs using string contains checks.
Changes:
- Add UnsupportedParam typed enum in provider.rs (Temperature, MaxTokens, StopSequences)
- Create strip_unsupported_completion_params() helper function
- Create strip_unsupported_tool_params() helper function
- Update rig_adapter.rs to use shared helpers
- Update anthropic_oauth.rs to use shared helpers
- Replace 60+ lines of duplicate stringly-typed logic
Benefits:
- Type safety: parameter names checked at compile time
- Single source of truth: adding a new param updates one place
- Reduced maintenance burden: no duplicate logic to keep in sync
- Better code clarity: named enum variant is self-documenting
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* docs: clarify intentional parameter asymmetry between completion and tool requests
Add documentation explaining why strip_unsupported_tool_params does not handle
StopSequences: the field doesn't exist in ToolCompletionRequest.
Changes:
- Add clarifying comments to strip_unsupported_tool_params()
- Explain why StopSequences is only in CompletionRequest
- Note that ToolCompletionRequest only supports Temperature and MaxTokens
- Inline comment confirms no action needed for StopSequences
This addresses the appearance of incomplete implementation without changing logic,
as the asymmetry is intentional and correct (ToolCompletionRequest lacks the field).
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* perf: isolate webhook_secret to reduce lock contention on hot path
Move webhook_secret from shared HttpChannelState RwLock into its own Arc<RwLock<>>.
This eliminates contention between secret validation and other state operations.
Changes:
- Change webhook_secret field type from RwLock<Option<SecretString>> to Arc<RwLock<Option<SecretString>>>
- Update initialization in HttpChannel::new()
- Update comments to explain isolation rationale
Benefits:
- Reduce lock contention on webhook request hot path (secret validation)
- Rarely-changing field (SIGHUP only) isolated from frequent state accesses
- Other state operations (tx, pending_responses) no longer wait behind secret reads
- Minimal code change: only field declaration and initialization
The Arc wrapper allows cloning the RwLock handle to separate concerns. With this
change, every webhook request acquires its own isolated lock for secret validation,
not the shared HttpChannelState lock. This scales better under high request volume.
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: prevent partial state corruption on SIGHUP restart failure
Ensure atomicity of configuration reload: if webhook listener restart fails,
secret update is skipped to prevent inconsistent state.
Changes:
- Wait for restart_with_addr() to complete (don't spawn background task)
- Track restart result with restart_failed flag
- Only update secret if restart succeeded or wasn't needed
- Ensure listener and secret stay synchronized
Problem addressed:
- Before: restart spawned as background task, secret updated immediately
- If restart failed, secret was changed but listener still on old address
- This left system in inconsistent state (partial corruption)
Solution:
- Make restart blocking (SIGHUP handler can wait, it's not on request hot path)
- Atomically update secret only after successful restart
- Flag prevents race between restart and secret update
Benefits:
- Configuration changes are atomic (both succeed or both fail together)
- No partial state corruption on restart failure
- Failed restarts don't silently leave inconsistent state
- Secret and listener address stay in sync
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* refactor: generalize hot-secret-swapping with ChannelSecretUpdater trait
Decouple SIGHUP handler from HTTP channel internals by introducing a trait
for channels that support zero-downtime secret updates.
Changes:
- Add ChannelSecretUpdater trait in channels/channel.rs
- Implement ChannelSecretUpdater for HttpChannelState
- Export trait from channels module
- Update SIGHUP handler to use trait-based secret updater collection
- Replace explicit HTTP channel knowledge with generic updater loop
Benefits:
- SIGHUP handler no longer depends on HttpChannelState details
- Tight coupling removed: main.rs doesn't need HTTP channel imports
- Extensible: new channels can opt-in by implementing the trait
- Scalable: multiple channels supported without main.rs changes
- Maintainable: adding channels requires only trait implementation, not SIGHUP handler edits
Pattern:
- ChannelSecretUpdater trait defines the interface for all updaters
- Channels that support hot-secret-swapping implement the trait
- SIGHUP handler loops through all registered updaters generically
Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* feat: validate parameter names at deserialization time, not just tests
Add custom serde deserializer for unsupported_params that validates parameter
names at runtime when loading providers.json (or user overrides).
Changes:
- Add unsupported_params_de module with custom deserializer
- Only allows: "temperature", "max_tokens", "stop_sequences"
- Invalid parameter names cause immediate deserialization error
- Update ProviderDefinition to use custom deserializer
- Enhanced test with explicit parameter name validation
- Add new test that verifies invalid parameters are rejected
Problem solved:
- Before: Invalid param names (e.g., "temperrature") silently ignored
- Now: Rejected at deserialization time with clear error message
- Prevents runtime failures caused by typos in configuration
Example error:
unsupported parameter name 'temperrature': must be one of: temperature, max_tokens, stop_sequences
Benefits:
- Fail-fast: errors caught when loading config, not at runtime
- Clear feedback: error message lists valid parameter names
- Type safety: validators run during deserialization
- Configuration errors detected immediately, not silently ignored
Verification:
- All 2,788 tests pass (including new validation test)
- Zero clippy warnings
- Code compiles successfully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
* merge: resolve conflicts for PR #800 and #822 into staging (#881)
* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)
- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)
- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: persist user_id in save_job and expose job_id on routine runs (#709)
* feat: persist worker events to DB and fix activity tab rendering
In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.
Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: wire RoutineEngine into gateway for direct manual trigger firing
Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.
Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove stale gateway_state argument from Agent::new test call sites
The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — restore sandbox source filter, remove blank lines
- Revert removal of `source = 'sandbox'` filter in all SandboxStore
queries (8 sites across PG and libSQL). Sandbox-specific APIs should
stay scoped to sandbox jobs; unified job listing for the Jobs tab
should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
formatting CI failure.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review — regenerate Cargo.lock, add user_id regression test
- Regenerate Cargo.lock from main's lockfile to eliminate dependency
version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
and get_job in the libSQL backend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: remove trailing blank line in libsql jobs.rs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Postgres-side regression test for user_id persistence in save_job
Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: unify three agentic loops into single AgenticLoop engine (#654)
Replace three independent copy-pasted agentic loops (dispatcher, worker,
container runtime) with a single shared engine in `agentic_loop.rs` that
all consumers customize via the `LoopDelegate` trait.
Phase 1 — Shared engine (`src/agent/agentic_loop.rs`, 205 lines):
- `run_agentic_loop()` owns the core LLM → tool exec → repeat cycle
- `LoopDelegate` trait (Send + Sync, &dyn dispatch) with 6 hook points
- Tool intent nudge logic consolidated (was duplicated in 3 files)
- Iteration limit + force-text behavior preserved
Phase 2 — Three delegate implementations:
- `ChatDelegate` (dispatcher.rs): 3-phase approval flow, hooks, cost
guard, context compaction, skill attenuation, interruption
- `JobDelegate` (worker/job.rs): planning pre-loop phase, parallel
JoinSet exec, mark_completed/stuck/failed, SSE streaming, self-repair
- `ContainerDelegate` (worker/container.rs): sequential tool exec,
HTTP-proxied LLM, container-safe tools, credential injection
Phase 3 — File moves and cleanup:
- Delete `src/agent/worker.rs` — job logic moved to `src/worker/job.rs`
- Rename `src/worker/runtime.rs` → `src/worker/container.rs`
- Re-export `Worker`/`WorkerDeps` from `crate::worker` in `agent/mod.rs`
- Update `scheduler.rs` imports to new worker location
Shared helpers (`src/tools/execute.rs`):
- `execute_tool_with_safety()` replaces 4 copies of validate → timeout
→ execute → serialize
- `process_tool_result()` replaces 3 copies of sanitize → wrap →
ChatMessage (also used by thread_ops.rs approval resume paths)
Net result: -2,408 lines, zero duplicated loop logic, single code path
for tool intent nudge and completion detection.
Closes #654
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review feedback from Copilot
1. scheduler.rs: Replace `unwrap_or` fallback with proper error
propagation when parsing tool output JSON — surfaces bugs instead
of silently changing the output type.
2. worker/job.rs: Drop MutexGuard before the cancellation `.await` in
`check_signals()` to avoid holding a lock across an async I/O call
(prevents `await_holding_lock` lint).
3. worker/job.rs: Restore consecutive rate-limit counter
(MAX_CONSECUTIVE_RATE_LIMITS = 10) so sustained rate limiting marks
the job stuck with "Persistent rate limiting" instead of silently
burning through max_iterations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: incorporate staging changes — token budget tracking + mark_failed
Merge staging's changes into the refactored JobDelegate:
- Add token budget tracking in call_llm (update_context/add_tokens)
- mark_stuck → mark_failed for iteration cap and rate-limit exhaustion
(aligns with staging's #788 fix)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address zmanian's PR review — eliminate type erasure, clean up
Address all 6 review points from zmanian on PR #800:
1. Replace LoopOutcome::Custom(Box<dyn Any>) with typed
LoopOutcome::NeedApproval(Box<PendingApproval>) — eliminates
type erasure and downcast, resolves clippy large_enum_variant.
2. Remove dead max_tool_iterations field from ChatDelegate struct.
3. Add on_tool_intent_nudge() hook to LoopDelegate trait with
implementations in Job and Container delegates for observability.
4. Fix SSE events in job worker to emit raw sanitized content
instead of XML-wrapped <tool_output> tags.
5. Remove 4 duplicate completion tests from job.rs that were
already covered by the shared util module.
6. Avoid logging full tool results — use result_size_bytes in
debug logs (execute.rs, job.rs).
Also updates path references in CLAUDE.md, COVERAGE_PLAN.md,
and add-sse-event.md command.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(doctor): expand diagnostics from 7 to 16 health checks
* test: add unit tests for agentic_loop and execute shared modules
Add 16 tests covering the two new critical shared modules:
agentic_loop.rs (10 tests):
- Text response exits loop immediately
- Tool call → text response continuation
- LoopSignal::Stop exits before LLM call
- LoopSignal::InjectMessage adds user message to context
- Max iterations terminates with LoopOutcome::MaxIterations
- Tool intent nudge fires twice then caps
- before_llm_call early exit bypasses LLM
- truncate_for_preview: short string, long string, multibyte safety
execute.rs (6 tests):
- execute_tool_with_safety success path
- Missing tool returns ToolError::NotFound
- Tool execution failure propagates
- Per-tool timeout enforcement (50ms)
- process_tool_result XML wrapping on success
- process_tool_result error formatting
All 2,777 unit tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address code review — 9 issues across agentic loop, job worker, container
CRITICAL fixes:
- Rate-limit exhaustion now returns Err(LlmError::RateLimited) instead of
Ok(Text("")), stopping the loop immediately with no ghost iteration.
Below-threshold retries still use Text("") with an explicit empty-string
guard in handle_text_response to skip injection.
- check_signals drains the entire message channel before returning,
prioritizing Stop over UserMessage. Previously returned early on first
UserMessage, silently dropping any queued Stop or additional messages.
- check_signals now detects all non-progressing job states (Cancelled,
Failed, Stuck, Completed, Submitted, Accepted) instead of only
Cancelled and Failed.
HIGH fixes:
- Error path in process_tool_result_job applies truncate_for_preview to
bound error strings in SSE/DB events (was unbounded).
- Document Send+Sync lifetime constraint on LoopDelegate trait.
- Test mock before_llm_call refactored from double-lock to single lock
acquisition, eliminating deadlock risk on refactor.
MEDIUM fixes:
- CompletionReport includes actual iteration count via shared
Arc<Mutex<u32>> tracker (was hardcoded 0).
- process_tool_result_job return type changed from Result<bool> to
Result<()> — the bool was always false (dead API).
- Deduplicate truncate in container.rs; now uses truncate_for_preview
from agentic_loop.
Verified: 0 clippy warnings, 2781 tests pass, cargo fmt clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
Co-authored-by: Umesh Kumar Singh <brijbiharisingh1971@outlook.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
* Revert "Feat/docker shell edition" + fix fmt/clippy (#886)
* Revert "Feat/docker shell edition (#804)"
This reverts commit
|
||
|
Illia Polosukhin
|
5a62ceaa99 |
refactor: extract safety module into ironclaw_safety crate (#1024)
* refactor: extract safety module into ironclaw_safety crate Move prompt injection defense, input validation, secret leak detection, and safety policy enforcement into a standalone crate under crates/. The safety module was a leaf dependency with no async, no database, and no other ironclaw traits — only pure computation with pattern matching. SafetyConfig (2 fields) moves into the crate; env-var resolution stays in ironclaw's config module as a free function. src/safety/mod.rs becomes a thin re-export so all existing `crate::safety::*` imports keep working. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update CLAUDE.md for ironclaw_safety crate extraction Add guidance to migrate imports from crate::safety to ironclaw_safety when touching files. Update project structure to reflect crates/ dir. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move safety fuzz targets into ironclaw_safety crate Split fuzz infrastructure: - crates/ironclaw_safety/fuzz/ — 5 safety-only targets (sanitizer, validator, leak_detector, credential_detect, config_env) depending only on ironclaw_safety for faster builds - fuzz/ — keeps fuzz_tool_params which needs ironclaw::tools Add seed corpus files (51 total) covering each pattern family: sanitizer injection patterns, validator edge cases, leak detector secret formats, credential detect HTTP param shapes. Add new fuzz_credential_detect target exercising params_contain_manual_credentials with arbitrary JSON. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — single-pass XML escaping and versioned path dep Rewrite escape_xml_attr from chained .replace() to single-pass char iteration (O(n) instead of O(4n) with intermediate allocations). Add version = "0.1.0" to ironclaw_safety path dep to satisfy cargo-deny wildcards = "deny". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
f05896fe6a |
Migrate GitHub webhook normalization into github tool (#758)
* Add event-triggered routines and workflow skill templates * Add generic host-verified webhook ingress for tools * Migrate GitHub webhook normalization into github tool * Bump github tool registry version * Stabilize trace E2E test rig and approval behavior * Add reusable gateway workflow harness with mock LLM server (#762) * Add reusable gateway workflow test harness with mock LLM server * Fix clippy issues in workflow harness * Stabilize trace E2E test rig and approval behavior * Address PR review feedback on gateway workflow harness - Extract shared TestChannelHandle into test_channel.rs with name override support, eliminating ~55 lines of duplication between test_rig.rs and gateway_workflow_harness.rs - Remove redundant RoutineEngine creation that was immediately overwritten by Agent::run() - Replace flaky sleep(500ms) with polling loop for routine run count check - Use components.context_manager instead of creating a fresh ContextManager for job tools, ensuring agent and tools share the same instance Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix import ordering in gateway_workflow_harness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Address PR #758 review feedback - Fix header_value to use fully case-insensitive lookup (iterate with to_ascii_lowercase) instead of checking only exact/lower/upper variants - Change comment_id from u32 to u64 to handle GitHub's billion-range IDs - Remove handle_webhook from LLM-facing JSON schema to prevent direct invocation bypassing HMAC verification - Rename enrichment keys from repository/sender to repository_name/ sender_login to preserve original JSON objects in webhook payloads - Remove put_string_normalized helper (no longer needed) - Replace no-op tests (test_validate_event_in_create_pr_review, test_validate_merge_method) with test_header_value_case_insensitive - Add README docs for 6 undocumented actions (list_issue_comments, create_issue_comment, list_pull_request_comments, reply_pull_request_comment, get_pull_request_reviews, get_combined_status) - Add comment explaining max_tool_calls <= 8 bound in e2e test - Fix gateway workflow harness: add webhook_capability with secret auth to MockGithubWebhookTool, matching staging's hardened webhook security - Fix merge artifacts: remove duplicate test function, orphaned code fragment in e2e_routine_heartbeat [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix formatting in gateway workflow harness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Address Copilot review: filter keys, pr_number fallback, feature gate, version alignment - Update SKILL.md and workflow-routines.md templates to use `repository_name` and `sender_login` (matching enriched payload field names) - Mark webhook HMAC secret as required in SKILL.md prerequisites - Fall back to `/issue/number` for `pr_number` on issue_comment PR webhooks - Gate `gateway_workflow_harness` module behind `#[cfg(feature = "libsql")]` - Align tool version to 0.2.1 in Cargo.toml and capabilities.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Henry Park
|
fe82469904 |
fix(ci): WASM WIT compat sqlite3 duplicate symbol conflict (#953)
* fix(ci): use explicit features in WASM WIT compat test to avoid sqlite3 symbol conflicts The `import` feature (added in #903) brings in `rusqlite[bundled]` which conflicts with `libsql-ffi` — both bundle SQLite C code, causing duplicate symbol linker errors. Use explicit features matching the test matrix instead of `--all-features`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace rusqlite with libsql in import module to fix sqlite3 symbol conflict The `import` feature used `rusqlite[bundled]` which bundled its own SQLite C code, conflicting with `libsql-ffi` (also bundles SQLite). This caused duplicate `sqlite3_*` symbol linker errors when both features were enabled via `--all-features`. Replace `rusqlite` with `libsql` (already a dependency) in the import reader. The `import` feature now implies `libsql`. This eliminates the duplicate symbol conflict and allows `--all-features` to compile cleanly. Also restores `--all-features` in the WASM WIT compat CI test (now safe) and converts all import test helpers from rusqlite to libsql. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply cargo fmt formatting fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Nick Pismenkov
|
26068db24b |
feat: Import OpenClaw memory, history and settings (#903)
* feat: Import OpenClaw memory, history and settings * review fixes * fix: address remaining code quality issues 1. Remove dead import_conversation() function - replaced by import_conversation_atomic() 2. Improve non-UTF-8 filename handling in list_agent_dbs() - log warning instead of silent 'unknown' 3. Remove emojis from CLI output per project style guide Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> |
