lywsvip/ironclaw
1
0
Fork 0
mirror of https://github.com/nearai/ironclaw.git synced 2026-09-02 23:56:24 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,325 Commits 1,240 Branches 71 Tags
staging
Commit Graph

18 Commits

Author SHA1 Message Date
firat.sertgoz
ab38a0b234 feat(bridge): workspace-backed project registration + adapter improvements (#2533) 
* feat(projects): workspace-backed project registration; migrate commitments into projects/commitments/

[cherry-pick-target: feat/projects-workspace-backed]

Replace the parallel `.system/engine/projects/*.json` schema with
workspace-backed project registration. Writing any file under
`projects/<slug>/` is now the declaration that the project exists —
the engine auto-registers it on `memory_write`, and `mission_create`
can reference it by slug. The model reasons about projects through
normal workspace APIs instead of a hidden sidecar schema.

Engine + bridge

- `ProjectId::from_slug(user_id, slug)` derives a stable v5 UUID;
  `Project::new` routes through it so constructing the same project
  twice returns the same ID (no duplicates).
- `slugify_simple` in `ironclaw_engine::types` — pure slug, no UUID
  suffix, reverses cleanly from a `projects/<slug>/` directory name.
- Project metadata moves from `.system/engine/projects/{slug}--{id8}/
  project.json` to user-facing `projects/<slug>/.project.json`.
  One-shot startup migration copies legacy files over, idempotent.
- `HybridStore::load_projects_from_workspace` scans `projects/*/` and
  synthesizes a stub `Project` for bare directories, so a write under
  `projects/foo/` surfaces immediately on restart.
- `EffectBridgeAdapter::ensure_project_for_memory_write` hook runs
  after a successful `memory_write`: if the target is under
  `projects/<slug>/...`, finds-or-creates the project and splices
  `project_id` into the tool output (enables
  `{{call-N.project_id}}` template refs).
- Extract `resolve_project_ref` helper from the inline block in
  `handle_mission_call` — now used by both `mission_create`'s
  `project_id` param and future project-aware tools.

Skills (13 files)

- Mechanical `commitments/` → `projects/commitments/` across the nine
  commitment-domain skills (commitment-setup, -triage, -digest,
  decision-capture, delegation-tracker, idea-parking,
  tech-debt-tracker, product-prioritization, security-review).
- Four persona setup skills (ceo-setup, developer-setup,
  trader-setup, content-creator-setup) gain an explicit "declare the
  project" step (write `projects/commitments/AGENTS.md` with
  persona-specific operating principles) and pass
  `project_id: "commitments"` on every `mission_create`. Setup
  markers move to `projects/commitments/.<persona>-setup-complete`.
- `ceo-setup` gets a v0.4.0 rewrite that also installs two dashboard
  widgets under `projects/commitments/.system/widgets/`:
  `commitments-this-week` (overdue / due / completed counts) and
  `delegations-waiting` (delegation list with stale-at-2-days flag).
  Both poll `projects/commitments/widgets/state.json`, refreshed by
  the triage mission each run.

Tests

- Three new unit tests in `bridge::effect_adapter::tests`:
  `extract_project_slug_recognizes_project_paths`,
  `extract_project_slug_rejects_degenerate_targets`,
  `project_new_is_deterministic_from_user_and_slug`.
- Update `tests/e2e_live_personas.rs` path assertions
  (`workspace_paths`, `read_under`, `verify_setup_landed`,
  `DEV_SETUP_CHECKS` needles, two workflow turn messages) to the new
  `projects/commitments/` prefix.
- Add a diagnostic dump in `run_turn` when a persona workflow turn
  times out with no response, so live-test hangs surface the
  captured status events instead of an opaque panic.

No backcompat for the old flat `commitments/` layout — pre-production
deployment, nothing in the wild depends on it.

* fix: adapt cherry-picked project registration to staging API surface

Add missing struct fields (engine_store, skill_registry) and setter
methods to EffectBridgeAdapter, expose MissionManager::store() accessor,
add sync_v1_skill_to_store to skill_migration, and remove references
to fields/methods not yet on staging (Project::goals/metrics,
LiveTestHarnessBuilder::with_skills_dir, V2SkillMetadata::bundle_path).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): address review — drop slug-prefix fallback, harden tests (#2533)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): address PR #2533 review — slug-consistency, migration hardening, caller-level tests

- synth_bare_project now normalizes the raw dir name via slugify_simple
  before ProjectId::from_slug, matching Project::new. Returns Option so
  unsluggable dirs (`---`, `!!!`) don't produce phantom projects.
- migrate_legacy_project_jsons upgraded to warn! and moves unparseable
  legacy project.json aside as project.broken.json so the user can
  recover instead of the engine masking the loss on every boot.
- Document project_slug's engine-internal (mission-path, UUID-suffixed)
  scope vs project_dir's user-facing (no-UUID) scope so the two slug
  schemes aren't conflated in future edits.
- Drop unused ProjectId param from project_dir / project_path.
- Trim Project::new docstring per CLAUDE.md style.

Tests added (19):
- types::project: slug variant collapse, unicode, empty-slug stability,
  run/edge normalization
- store_adapter unit: project_slug_for_name contract, project_dir/path,
  synth_bare_project↔Project::new ID equivalence across 12 weird names,
  unsluggable-dir rejection, cross-user isolation
- store_adapter migration_tests (libsql): bare-dir load, metadata over
  synth, non-canonical skip, weird-slug collapse, user-edit preservation,
  broken-JSON move-aside
- effect_adapter caller-level: drives execute_action("memory_write")
  for canonical / idempotent / non-projects / nested / weird-slug /
  cross-user / pathological targets per .claude/rules/testing.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-20 23:23:15 +09:00
firat.sertgoz
532e07fd07 fix: prevent immediate requests creating missions (#2328) 
* fix: prevent immediate requests creating missions

* fix: address review findings (iteration 1)

* fix: use prefix stem matching for scheduling intent words

Addresses review feedback: "monitoring" now matches the "monitor" stem,
"routinely" matches "routin", etc. Replaces exact word matching with
starts_with prefix matching so morphological variants are caught without
maintaining an exhaustive word list. Adds regression test for
"set up monitoring now" being correctly allowed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix cargo fmt alignment in SCHEDULE_STEMS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): add caller-level tests for immediate mission rejection (#2328)

Address henrypark133 review: the `should_reject_immediate_mission_create`
predicate was only covered by helper-level unit tests. Per the "Test
Through the Caller" rule, add three caller-level tests that drive
`EffectBridgeAdapter::execute_action` end-to-end:

- Reject path: foreground + immediate goal → EngineError::Effect
- Allow path: foreground + scheduling intent → mission created
- Alias path: routine_create → mission_create alias also rejected

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): remove useless .into_iter() flagged by clippy 1.95

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): resolve clippy 1.95 collapsible-match and useless-conversion lints

Collapse nested `if` into match arm guards per clippy::collapsible_match
(new in Rust 1.95). Replace `.sort_by(|a, b| b.1.cmp(&a.1))` with
`.sort_by_key(|x| Reverse(x.1))` per clippy::unnecessary_sort_by.

Affected crates: ironclaw (main), ironclaw_engine, ironclaw_tui,
ironclaw_skills.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): add thread_goal to ThreadExecutionContext in gate integration test

The merge from staging introduced a new test that constructs
ThreadExecutionContext without the thread_goal field added by this PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): resolve clippy lint and 3 test failures

- Add #[allow(clippy::too_many_arguments)] on register_startup_channels
- Extract extension name from tool_install params in pending_gate_extension_name fallback
- Isolate re_resolve_llm tests from user config.toml via temp file
- Mark propagate_approval test #[ignore] (requires prebuilt telegram WASM)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
 2026-04-20 15:58:12 +09:00
Illia Polosukhin
f2c4c258dd docs(skills): clarify /search/issues returns issues and PRs (#2713) 
* docs(skills): clarify /search/issues returns issues and PRs

Add a bullet noting the unified /search/issues endpoint returns both
issues and pull requests and there is no /search/pulls endpoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(skills): simplify /search/issues bullet per review

Addresses gemini-code-assist review on PR #2713: trim redundancy with
the section header and focus the note on the absence of a
/search/pulls endpoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 15:22:33 +09:00
Illia Polosukhin
77e746f683 feat(portfolio): complete tool, tests, widget, and share-gains flow (#2368) 
* feat(portfolio): complete tool, tests, widget, and share-gains flow

Portfolio WASM tool with full pipeline:
- Indexer (fixture, dune, dune-replay backends)
- Analyzer (6 protocol classifiers, health extraction, stablecoin detection)
- Strategy filter (yield-floor, health-guard, LP impermanent-loss-watch)
- Intent builder (fixture + solver backends, bounded checks, leg bundling)
- Format (suggestion markdown, progress metric, widget state)

172 unit tests covering all modules including edge cases:
- filter.rs: 33 tests (yield floor, health guard, LP watch, helpers)
- bounded.rs: 16 tests (slippage, cost, chain allowlist, multi-leg)
- parser.rs: 18 tests (delimiters, YAML, kind inference, real strategies)
- fixture.rs: 14 tests (slippage calc, ID formats, payload structure)
- analyzer: 18 tests (stablecoin detection, health extraction, debt/yield)
- format.rs: 16 tests (totals, empty states, progress windowing)
- widget.rs: 10 tests (rendering, intents, non-ready filtering)
- types: 16 tests (parse_decimal, ChainSelector serde)
- 14 YAML replay scenarios + 4 live Dune API tests (ignored by default)

Share-gains feature:
- Gateway-level IronClaw.api.share() modal with X, LinkedIn, Facebook,
  copy-to-clipboard, and download buttons
- Portfolio widget generates SVG card showing gains (APY, annual savings,
  moves found) — no addresses or balances exposed
- "Share gains" button appears only when portfolio has positive delta

E2E Playwright tests (11 scenarios):
- Skill discovery via API and settings UI
- Chat integration (keyword + wallet address triggering)
- Widget rendering with pre-seeded state (positions, totals, suggestions)
- Share button visibility (present with gains, absent without)
- Share modal lifecycle (opens with card image, social buttons, closes)

Supporting changes:
- E2E conftest: SKILLS_DIR points to workspace skills/
- Mock LLM: canned responses for portfolio/defi and wallet address patterns
- Skill YAML, registry entry, capabilities JSON, 3 strategy docs, 4 scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(portfolio): address PR review — XSS, OnceLock, bounded checks, docs

Addresses review comments from #2368:

- XSS: widget renders all interpolated fields through escapeHtml();
  share modal creates <img> via DOM API with data:image/ prefix check
- OnceLock: protocol registry parsed once via std::sync::OnceLock
- to_ascii_lowercase() for wallet address lookups (fixture + dune_replay)
- bounded.rs: reject empty value_usd in single-leg slippage check
- fixture.rs: compute min_out amount and value_usd separately
- fixture.rs: clarify expires_at=0 comment (fixture = no expiry)
- schema.json: add "dune-replay" to source enum
- parser.rs: fix doc comment re kind inference (defaults, not inferred)
- live_tests.rs: fix log placeholder (raw_count vs classified.len())
- intent.rs: expand kind comment to match SCHEMA.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(portfolio): escape remaining innerHTML fields, add tests, WASM build

- Escape delta_vs_last_run_usd and next_mission_run in widget innerHTML
- Add fixture test with amount != value_usd (stETH: 3.5 tokens / $12250)
  to verify the review fix separating amount from value_usd
- Add empty-legs test for bundling.rs order_legs
- Add comment explaining multi-leg empty value_usd tolerance in bounded.rs
- WASM component builds successfully (754K release binary)
  via: cargo component build --release --target wasm32-wasip2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(portfolio): address second-round PR review comments

- Tighten share image validation to data:image/png only (was data:image/*)
- Add ClipboardItem existence check to prevent runtime errors in some browsers
- Fix SCHEMA.md to correctly attribute invariant enforcement (bounded.rs vs bundling.rs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(portfolio): NEAR support end-to-end with engine v2 quality fixes

Add full NEAR Protocol support to the portfolio tool: scan via FastNEAR +
Intear, classify positions through new protocols (Linear, Meta Pool, Rhea
lending, Rhea LP), match against new NEAR-specific yield strategies, and
build intent bundles. Plus assorted infrastructure fixes uncovered while
exercising the v2 / CodeAct path.

Indexer
- New `near` source: FastNEAR `/v1/account/{id}/full` + Intear
  `/list-token-price` (235 KB, vs `/tokens` at 3.2 MB which exceeded fuel).
- New `near-replay` source for offline fixture replay.
- `auto` source dispatches per address: `0x...` → Dune, `*.near`/`*.tg` →
  NEAR backend. Mixed lists are split and merged.
- `classify_near_token()` tags known NEAR DeFi contracts (Linear, Meta
  Pool, Rhea/Burrow, Rhea/Ref) with proper `protocol_id`. Default for
  unknown FT contracts is `wallet`.
- Dust filter raised from \$0.01 → \$1 to keep wallets like `root.near`
  from passing 100+ micro-cap positions through the analyzer.
- Dune `value_usd` now accepts both string and number (Dune started
  returning floats).

Analyzer
- New protocols: `wallet`, `near-staking`, `linear`, `meta-pool`,
  `rhea-lending`, `rhea-lp`. Wallet positions are no longer silently
  dropped (the prior bug that made root.near show "meteor-private" only).

Strategies
- New `near-staking-yield`, `near-lending-yield`, `near-lp-yield` —
  match wallet/staking/LP positions on `chain == "near"`.
- `StrategyAppliesTo` gains `chains` and `tokens` filters.

Tool API
- `propose.strategies` is now optional → falls back to bundled defaults
  (3 EVM + 3 NEAR strategies).
- `propose.config` is now optional → falls back to `ProjectConfig::default()`.
- `build_intent.config` optional with default.
- `propose` recovers from stringified positions (common LLM mistake of
  calling `json.dumps()` first) and returns a clearer error message.
- Capability `dune_api_key` marked `optional: true` — NEAR-only and
  fixture flows no longer block on a missing Dune key.
- Default source is now `auto`.

WASM runtime
- Default fuel limit raised 10M → 500M across config, settings, channel
  runtime, and ResourceLimits. Production was using 10M (config path)
  while tests used `ResourceLimits::DEFAULT_FUEL_LIMIT` (was 100M) — the
  divergence masked the real fuel exhaustion. The 235 KB Intear parse
  uses ~27M fuel, so 500M provides ample headroom.
- Wrapper now logs fuel consumption at debug level for diagnostics.

Engine v2 / CodeAct UX
- Preamble: 3 new rules
  - Never reconstruct tool results manually — reference variables.
  - Never paste Python code outside `\`\`\`repl` or `FINAL(answer)`.
  - Chain tool calls in a single block.
  - Pass native Python objects to tools, never `json.dumps()` first.
- Postamble: explicit good/bad chaining example + `FINAL()` answer
  quality guidance (no terse counts).
- Orchestrator: when an action result exceeds 500 chars, the truncated
  preview now tells the LLM the full result is in `state['<tool>']`
  to discourage manual reconstruction.

Skill (`skills/portfolio/SKILL.md`)
- Step 4 (Propose): explicit anti-patterns for fabricated positions,
  strategy-name-only strings, and `floor_apy` percentage integers.
- Step 5 (Rank): allows informational LLM-only suggestions when
  `propose` returns no `ready` proposals.
- Step 6 (Build intents): explicit skip when no `ready` proposals;
  documents required `plan` shape (`legs`, `expected_out`,
  `expected_cost_usd`, `proposal_id`).
- Step 8 (Summarize): require detailed Markdown output, not counts.

Tests
- `tests/e2e_wasm_portfolio.rs` (5 tests): scan, propose, full pipeline
  via `TestRigBuilder` with canned HTTP — exercises real wasmtime sandbox
  with fuel metering.
- `tests/e2e_live_portfolio.rs` (2 tests, live-only via `IRONCLAW_LIVE_TEST=1`):
  end-to-end via `LiveTestHarness` against real LLM + real FastNEAR/Intear,
  with `engine_v2(true)`. Requires `--test-threads=1` due to a v2
  thread-registry race.
- Portfolio unit tests: 183 pass (added NEAR indexer parsers, dispatch
  auto-detection, new strategy filter cases).
- Live portfolio tests: 10 pass against real APIs.
- Updated `hostile/fake-token-dust` scenario for the new "wallet"
  protocol behaviour.

Bug fixes uncovered along the way
- `intents/bounded.rs`: epsilon raised to 0.005 to tolerate the 2-decimal
  truncation in `intents/fixture.rs` (intent bundles previously failed
  the slippage check on synthetic targets).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(portfolio): address review findings from #2368

Correctness:
- bounded.rs: multi-leg slippage now checks the terminal leg (matching
  plan.expected_out.chain), not just single-leg bundles. Regression
  tests added for the bypass and for a multi-leg bundle with min_out=0
  on the terminal leg.
- bounded.rs: reject zero/negative/NaN/infinite expected_out (would
  make min_required = 0 and every leg pass vacuously).
- indexer/mod.rs: is_near_address now validates NEAR account rules
  (2..64 chars, lowercase, separators). Previously any non-0x string
  (empty, whitespace, emoji, SQL injection) passed.
- indexer/mod.rs: scan_auto rejects addresses that are neither valid
  EVM nor valid NEAR, instead of silently routing them to Dune.

Code quality:
- bundling.rs: replace .expect("indegree") and .expect("leg by id")
  with explicit error returns.
- fixture.rs: replace .unwrap() on plan.legs.last() with an Err path.
- types/mod.rs: pub use → pub(crate) use (crate-internal only).
- dune.rs / near.rs: warn (via host::log at Warn level) when a
  non-zero amount has a missing/zero value_usd, so silent undercounts
  surface in diagnostics rather than being invisible.

Security:
- gateway config.js: hoist the data:image/png prefix check to the top
  of IronClaw.api.share() so both img.src and a.href are gated.
- gateway config.js: add noopener,noreferrer to window.open features
  on share popups to close reverse-tabnabbing surface.
- widget/index.js: extend escapeXml to also escape apostrophes.

Infrastructure:
- limits.rs: TODO comment noting that 500M fuel default is driven by
  one tool (portfolio/near) and follow-up should add a per-tool
  override so the global default can stay tighter.
- test_portfolio.py: silent-return on missing widget tab converted to
  pytest.skip via shared _open_portfolio_tab_or_skip helper, so a
  regression that removes widget registration fails loudly instead of
  passing silently.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(portfolio): address follow-up review comments

- lib.rs: BuildIntent.solver now defaults to "fixture" (a valid
  value), not "auto" (unrecognized by intents::build — was shipping
  the default straight into an "Unknown intent solver: 'auto'" error
  whenever the caller omitted the field).
- capabilities.json: update discovery_summary to reflect that
  strategies/config on propose and config/solver on build_intent are
  optional. Stale text had propose requiring both positions and
  strategies.
- limits.rs + config/wasm.rs: fix the fuel-limit doc comments. The
  prior value in limits.rs was 100M (not 10M — that was the config
  path). Clarify both paths converged at 500M in #2368.
- config.js (share modal): add aria-label, aria-modal, role=dialog,
  aria-labelledby for the modal and explicit aria-label on every
  icon-only share button. Mark decorative SVGs aria-hidden. Toast
  becomes role=status with aria-live=polite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-20 14:47:51 +09:00
Illia Polosukhin
e3df3ec4ae feat(skills): setup-marker lifecycle, chain-loading, and live GitHub workflow test (#2268) 
* chore: gitignore live test fixture containing recorded credentials

The github_dev_workflow live test records HTTP exchanges including
the github_token Bearer header. GitHub push protection correctly
blocks this. The fixture is only useful locally for replay; the
test skips gracefully without it.

* test: add live test for github developer workflow

Adds tests/e2e_github_dev_workflow.rs — a multi-turn live/replay test that
drives the developer-assistant + github-workflow skills end-to-end against
a synthetic nearai/ironclaw repository:

  1. Setup           — installs the wf-* mission set (excluding
                       wf-staging-review per the implement-but-don't-
                       auto-merge autonomy contract)
  2. Issue opened    — synthetic github.issue.opened webhook payload
  3. Maintainer LGTM — pr.comment.created from a maintainer
  4. PR review       — non-maintainer review comment
  5. CI failure      — failing check_run
  6. Approval        — maintainer approval; asserts NO merge call ever
                       fires across the whole session
  7. Digest          — status report referencing the issue/PR

Webhook payloads are injected via TestRig::send_message with a
[GITHUB WEBHOOK] frame that matches what a real webhook→channel
adapter would emit. The mission OnSystemEvent firing path is covered
separately by mission.rs unit tests; this test exercises skill
behavior given the right inputs.

Adds two helpers to tests/support/live_harness.rs:
  - trace_contains_tool_call(name, needle)
  - assert_trace_contains_tool_call(name, needle, ctx)

Both scan ToolStarted.detail and ToolResult.preview for case-insensitive
substring matches, so behavior tests can assert *what the agent
actually called* without scraping the recorded trace JSON.

Drive-by cleanups from the extension-lifecycle merge:
  - thread_ops.rs: drop orphaned RecordingStatusChannel + helper that
    came from a dropped extension-lifecycle test variant
  - bridge/router.rs: clippy needless_borrow on PendingGate args
  - skills/mod.rs: SkillManifest no longer has metadata field; add
    requires: GatingRequirements::default() to test fixture
  - cargo fmt fallout in recording.rs / live_mission.rs / trace_llm.rs

The test is #[ignore]-tagged (live tier) and skips gracefully in replay
mode until tests/fixtures/llm_traces/live/github_dev_workflow_full_loop.json
is recorded with IRONCLAW_LIVE_TEST=1. Compile coverage is automatic
via the existing test matrix; live execution follows the same pattern
as e2e_live_personas.rs (manual recording + commit fixture).

cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --test e2e_github_dev_workflow -- -D warnings: clean
cargo test --features libsql --test e2e_github_dev_workflow -- --ignored: passes (skips, fixture missing)

* test(harness): add pre-seed secrets + diagnostic activity dump

Three additions to make the github_dev_workflow live test runnable:

1. **TestRigBuilder::with_secret(name, value)** — pre-seed credentials
   in the SecretsStore before the agent starts. The kernel pre-flight
   auth gate fires when a skill with a credential spec activates (e.g.
   the github skill needs github_token); without a stored credential
   the agent gets stuck in 'Authentication required' mode and can't
   make progress. Tests inject a fake/dummy value so the gate is
   satisfied — the test isn't actually hitting the credentialed API.

   Implementation: AppComponents.secrets_store is captured during
   build_all() and any pre-seeded (name, value) pairs are written via
   secrets_store.create() with user_id = config.owner_id. Already-exists
   errors are silenced so the helper is idempotent on seeded DBs.

2. **LiveTestHarnessBuilder::with_secret** — forwards to
   TestRigBuilder::with_secret. Plumbed through both build_live and
   build_replay so the same fixture works in both modes.

3. **dump_activity helper in e2e_github_dev_workflow.rs** — formats
   captured StatusUpdate stream (skill activations + every tool
   started/completed/result) to stderr. Used as a pre-assertion
   diagnostic so failing live runs surface the agent's actual tool
   sequence instead of an opaque panic on a workspace check.

Test relaxations from running this against the real LLM:
- verify_setup_landed accepts either developer-assistant OR
  github-workflow as the active skill (the deterministic selector
  picks based on keyword scoring + token budget; both routes are
  valid since github-workflow owns the mission templates)
- final required-skills check drops developer-assistant in favor of
  github-workflow + github (the orchestrator persona is optional)
- setup turn now pre-seeds github_token via with_secret

cargo check --features libsql --tests: clean

* test: rewrite github_dev_workflow as fully real live integration

Pivots the test from synthetic webhook simulation to a real end-to-end
integration test against the real nearai/ironclaw repo. Per project
owner: 'fully real live tests doing useful work on github repo... test
everything like it's live while recording all interactions to debug
what doesn't work and improve that'.

## Why the rewrite

The previous synthetic-event version injected fake GitHub payloads as
channel messages. With a real github_token in scope, the agent
attempted to fetch the fake issue 99001, got a 404, and helpfully
created 3 real issues + 3 real comments on nearai/ironclaw to
"reconcile" the discrepancy. The synthetic approach didn't surface
realistic failure modes anyway (auth gates, payload format mismatches,
rate limits), so we go all-in on real artifacts.

## New flow (2 turns + real artifact lifecycle)

1. Setup turn — agent installs the wf-* mission set for nearai/ironclaw
2. Test (NOT the agent) creates a real issue via direct REST API with
   the title "[live-test {timestamp}] Add /metrics Prometheus endpoint"
   and a real feature-request body.
3. Triage turn — test asks agent to triage issue #N. Agent reads via
   github skill, generates a plan, posts a real comment back.
4. Verification — test polls api.github.com/issues/N/comments and
   asserts at least one new comment exists since baseline. Comment
   bodies are logged to stderr for human review (the most useful
   debug output for iterating on skill quality).
5. Cleanup — std::panic::catch_unwind wraps the body so cleanup runs
   regardless of pass/fail. Closes the issue with a final "live test
   complete" comment. If cleanup itself fails, the issue URL is
   printed for manual recovery.

## Test infrastructure additions

- TestRig.get_secret(name) — read decrypted secrets back from the
  rig's SecretsStore. Required so the test can read the github_token
  the harness pre-seeded via with_secrets(["github_token"]).
- TestRig captures secrets_store + owner_id from AppComponents during
  build (needed for get_secret).
- github_api submodule inside the test file — direct REST helpers for
  create_issue, list_issue_comments, post_issue_comment, close_issue.
  Uses reqwest directly so the test has guaranteed GitHub access
  regardless of skill selection / tool gating.

## Recording

- LLM trace fixture: tests/fixtures/llm_traces/live/github_dev_workflow_full_loop.json (65K)
- Session log: github_dev_workflow_full_loop.log (5.9K)
- Both committed so future runs can replay deterministically without
  hitting real GitHub.

## What's NOT covered yet

Dropped from the previous version (can be added back as follow-ups):
- PR creation flow (agent opens a real PR with a real branch + real
  code change)
- CI failure simulation (would need a real failing CI run)
- Mission OnSystemEvent firing via real webhooks (needs an HTTP
  server registered as a GitHub webhook)
- Maintainer approval flow

This first version validates the most valuable slice: setup → react
to real issue → produce real comment → cleanup. If the agent's
comment quality is good, we expand from here.

cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --test e2e_github_dev_workflow -- -D warnings: clean
Live recording: passed in 85.9s
  - Created issue #2185
  - Agent posted 2 comments (full plan + follow-up)
  - Closed issue #2185

* feat(skills): one-time setup-marker exclusion + rename persona skills to *-setup

The persona orchestrator skills (developer-assistant, ceo-assistant,
trader-assistant, content-creator-assistant) are pure first-time
onboarding flows — their entire body is Steps 1-N of workspace setup,
mission registration, and calibration memory writes. After those steps
run successfully, there is nothing left for the skill to do, but the
deterministic selector kept evaluating them on every conversation
turn, burning ~3000 tokens of activation budget for work already
completed and risking partial re-runs of setup steps.

This commit makes setup skills opt-in to one-time activation:

## Mechanism: setup_marker exclusion

New optional field on ActivationCriteria:

  activation:
    setup_marker: commitments/.developer-setup-complete

Before scoring, the selector caller (Agent::select_active_skills)
collects every distinct setup_marker referenced by loaded skills,
checks the workspace for each via Workspace::exists(), and passes
the set of satisfied markers into prefilter_skills. Any skill whose
marker is in the satisfied set is excluded from scoring entirely
(returns None from the filter map, skipping the score_skill call).

The selector check is opt-in: skills without a setup_marker are
unaffected. Reactive operational skills (commitment-triage,
decision-capture, github, github-workflow, etc.) keep activating
on every matching message as before.

Tests:
- 4 unit tests in crates/ironclaw_skills/src/selector.rs covering
  marker present/absent, marker mismatch, and skill-without-marker
  unaffected paths
- All 152 ironclaw_skills tests pass
- Live e2e_github_dev_workflow run on real nearai/ironclaw passes
  (issue #2186 created, comment posted, closed) in 88s

## Rename: *-assistant → *-setup

Per project owner: 'rename persona skills to -setup skills to make
it explicit they are called once'. The -assistant suffix obscured
the lifecycle — these are not always-on assistants, they are
one-time onboarding wizards.

Renamed directories (via git mv) and updated SKILL.md `name:`
fields:
- skills/ceo-assistant            → skills/ceo-setup
- skills/content-creator-assistant → skills/content-creator-setup
- skills/developer-assistant       → skills/developer-setup
- skills/trader-assistant          → skills/trader-setup

All four now declare `setup_marker: commitments/.<name>-setup-complete`
and have a new final 'Step N: Mark setup complete' instructing the
agent to write the marker via memory_write after confirming setup
with the user. Different personas have different markers so they
remain independently triggerable in separate workspaces.

Cross-references updated:
- tests/e2e_live_personas.rs (4 persona test invocations)
- tests/e2e_github_dev_workflow.rs (doc comments)
- tests/e2e/LIVE_TOOL_FAILURES.md (1 reference)
- crates/ironclaw_skills/src/types.rs (doc comment example)

## Bump: SKILLS_MAX_CONTEXT_TOKENS default 4000 → 6000

The previous default was so tight that a setup skill (3000 tokens)
plus its companion github-workflow (2000) plus github (2000) would
overflow at 7000. Reactive operational skills like
commitment-triage, decision-capture, tech-debt-tracker often got
budget-evicted. With setup skills now excluded after onboarding,
the freed budget plus the bump to 6000 lets the most useful
combinations fit comfortably (e.g. github-workflow + github +
product-prioritization is now active in the live recording, where
previously product-prioritization would have been evicted).

## Plumbing changes

- ActivationCriteria gains pub setup_marker: Option<String>
  (#[serde(default)], so existing skills are unaffected)
- prefilter_skills signature gains
  &satisfied_setup_markers: &HashSet<String> (caller passes empty
  set to disable filtering — used by all existing tests via the
  prefilter_no_markers wrapper)
- Agent::select_active_skills is now async — it needs to
  Workspace::exists() each marker. dispatcher.rs caller updated
  to .await. Snapshots the skill list under the read lock then
  drops the guard before any await to avoid holding a poisonable
  RwLock across an await point.

cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --all-targets -- -D warnings: clean
cargo test -p ironclaw_skills: 152 passed
Live e2e_github_dev_workflow run: passes (88s)

* feat(skills): chain-load companions + v2 marker exclusion + commitment-setup marker

Three orthogonal follow-ups to the skill lifecycle work.

## 1. Chain-loading via requires.skills (v1 Rust + v2 Python)

When a parent skill is selected by the scorer, its requires.skills
companions are now automatically loaded, bypassing the score filter.
Persona/bundle skills like developer-setup can finally work as
designed: the orchestrator declares which operational skills it
delegates to, and selecting the orchestrator pulls them all in.

- **v1 Rust** (crates/ironclaw_skills/src/selector.rs): extracted
  skill_token_cost() and try_select() helpers used by both the
  scored-selection loop and the new chain-loading pass. Companions
  consume the same budget and respect max_candidates. Non-transitive
  (depth 1 only) to keep behavior predictable.
- **v2 Python** (crates/ironclaw_engine/orchestrator/default.py):
  select_skills() gains an inline chain-loading pass that mirrors
  the Rust logic. Uses a name-indexed lookup built from the skill
  list passed in by handle_list_skills. No closure-over-outer-var
  tricks that Monty would reject — the inner try-add is inlined.

7 chain-load unit tests in selector.rs covering: pulls in
companions, skipped when parent not selected, respects budget,
skips companion with satisfied marker, non-transitive (depth 2
not pulled), missing companion silent, dedup across parents.

## 2. v2 setup_marker exclusion

The v2 engine's Python orchestrator handles skill selection via
handle_list_skills (Rust) -> select_skills (Python). Since
handle_list_skills already has the full project doc list in scope,
we filter there: any skill whose metadata.activation.setup_marker
is in the set of existing doc titles gets excluded before the
Python orchestrator ever sees it. Zero extra store calls — we
reuse the existing list_memory_docs_with_shared result to build
an O(1) title set.

This is the v2 parity of the v1 satisfied_setup_markers parameter
threaded through prefilter_skills. Both paths now implement the
same rule: a one-time setup skill whose marker file has been
written has finished its job and should not keep burning
activation budget.

## 3. commitment-setup gets a setup_marker

commitment-setup writes commitments/README.md as its first step,
so the marker is automatically set after a successful first run.
Added:
  activation:
    setup_marker: commitments/README.md

To re-trigger (e.g. migrate to a new schema), delete README.md
first. project-setup was NOT given a marker — it's per-repo,
invoked repeatedly, not a singleton (each call creates a new
projects/<owner>-<repo>/project.md).

## 4. Lifecycle integration test

tests/skill_setup_marker_lifecycle.rs drives a real agent turn
through the v1 selector pipeline (Agent::select_active_skills ->
Workspace::exists -> prefilter_skills) to verify that a setup
skill:
  Phase 1: activates on the first matching message (marker absent)
  Phase 2: marker file is written via workspace.write()
  Phase 3: is excluded on the second matching message

The test asserts on the captured LLM system prompt content (via
rig.captured_llm_requests) rather than on StatusUpdate events so
it's agnostic to v1/v2 path differences in how skill activations
are announced. The skill's body contains a distinctive marker
string (LIFECYCLE-TEST-SKILL-BODY-MARKER-Z7Q) — if the skill was
selected, that string appears in the system prompt; if excluded,
it doesn't.

Cover matrix after this commit:
- v1 selector: 35 unit tests + 4 setup-marker tests + 7 chain-load tests
- v2 handle_list_skills marker exclusion: 1 integration test (lifecycle)
  plus structural verification via cargo check (the filter uses the
  existing list_memory_docs API, no new store calls to test)
- v2 Python select_skills chain-load: covered by the v1 unit tests
  through shared semantic contract (both paths mirror the same
  algorithm); a direct Python-level test would require spinning up
  the Monty interpreter which is out of scope for this session.

Verification:
  cargo test -p ironclaw_skills --lib:   159 passed
  cargo test -p ironclaw_engine:         304 passed
  cargo test --features libsql --test skill_setup_marker_lifecycle: 1 passed
  cargo clippy --features libsql --tests --all-targets -- -D warnings: clean

* feat(skills): carry requires through v1→v2 migration + chain-load test

V2SkillMetadata was missing the `requires` field entirely, so the
v1→v2 skill migration silently dropped `requires.skills` and the
chain-loading code I added to the v2 Python orchestrator in the
previous commit was effectively dead code — it always read an empty
companion list.

This was caught while writing an end-to-end chain-load test: the v1
test (through the Rust selector) passes, the v2 test (through the
Python orchestrator) was failing in a way that only made sense if
the companion metadata never reached Python. Inspection confirmed
`V2SkillMetadata` had no `requires` field, only `activation`.

## Fix

1. `V2SkillMetadata` gains `pub requires: GatingRequirements` with
   `#[serde(default)]` for backwards compatibility (legacy
   MemoryDocs in existing databases deserialize with an empty
   `requires`).
2. `src/bridge/skill_migration.rs::v1_skill_to_memory_doc` now
   copies `skill.manifest.requires.clone()` into the new field.
3. Four other explicit `V2SkillMetadata { ... }` literal
   constructions updated with `requires: Default::default()`:
   - `crates/ironclaw_engine/src/memory/skill_tracker.rs` (test helper)
   - `crates/ironclaw_engine/src/runtime/mission.rs` (test helper)
   - `crates/ironclaw_skills/src/v2.rs` (serde roundtrip test)
   - `tests/engine_v2_skill_codeact.rs` (test fixture)

## New test: tests/skill_chain_load_lifecycle.rs

End-to-end lifecycle test for chain-loading. Writes three skills to
a tempdir:
- `parent-setup-test` — scored by a distinctive keyword, declares
  two companions via `requires.skills`
- `companion-one-test` / `companion-two-test` — zero-scoring on
  their own (keywords deliberately don't match)

Each skill body carries a distinctive marker string
(`CHAIN-LOAD-PARENT-BODY-J4V`, `CHAIN-LOAD-COMPANION-ONE-K5W`,
`CHAIN-LOAD-COMPANION-TWO-L6X`) that the test greps for in the
captured LLM system prompt via `rig.captured_llm_requests()`. If a
marker is present, the skill was injected into the prompt; if
absent, it wasn't.

Two test variants:
- **v1** (default rig, Rust selector path): **PASSES**. Proves the
  chain-loading pass in `prefilter_skills` correctly pulls in both
  companions despite their zero individual scores.
- **v2** (with_engine_v2, Python orchestrator path):
  **`#[ignore]`d** with a detailed explanation. The v2 engine runs
  a Python orchestrator that makes multiple LLM calls per user
  message, but the default TestRig uses a single-turn TraceLlm that
  exhausts after the first call — observing skill injection through
  the v2 path needs a multi-turn TraceLlm harness or a dedicated v2
  skill test rig. The structural wiring for v2 chain-loading
  (V2SkillMetadata.requires + skill_migration copy + Python
  select_skills chain-load pass) compiles and passes the 304-test
  engine suite, so this is a test-harness gap, not a code gap.

When the multi-turn harness exists, flipping `#[ignore]` on the v2
test will exercise the full path.

Verification:
  cargo test -p ironclaw_skills --lib:                159 passed
  cargo test -p ironclaw_engine --lib:                304 passed
  cargo test --features libsql --test skill_chain_load_lifecycle
    -- --test-threads=1:                              1 passed, 1 ignored
  cargo test --features libsql --test skill_setup_marker_lifecycle
    -- --test-threads=1:                              1 passed
  cargo clippy --features libsql --tests --all-targets -- -D warnings: clean

Also includes an updated fixture recording from the last live
`e2e_github_dev_workflow` run (issue #2204, agent posted 2 comments,
cleanup closed it). No functional difference; committed for
completeness since the fixture was modified on disk by the live run
and the test is hermetic in replay mode.

* fix: adapt thread_ops test to staging's test helper API

Use make_test_agent_with_status_channel instead of removed
make_thread_ops_test_agent, StdMutex instead of TokioMutex,
and fix String comparison direction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

* fix: remove dead try_add function and stale comments in Python orchestrator

Addresses PR #2268 review feedback: the try_add closure was defined but
never called since the logic was inlined for Monty compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reconcile test harness after staging merge

Restore our branch's test helpers (SessionTurn, finish_turns_strict,
with_skills_dir, loaded_skill_names, active_skill_names, etc.) that
staging removed, while incorporating staging's new features
(record_trace, with_no_trace_recording, secrets_store/owner_id
accessors). Bridge the API gap with finish_turns_simple for tests
using staging's (String, Vec<String>) tuple convention.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR #2268 review feedback

- live_harness: replace panic with graceful TestMode::Skipped when
  record_trace=false in replay mode; update e2e_live callers to check
  mode() != Live instead of == Replay
- test_rig: match SecretError::NotFound explicitly in get_secret(),
  return None silently instead of logging expected misses
- test_rig: replace brittle "already exists" string matching in
  pre-seed loop with get_decrypted existence check before create
- default.py: align max_context_tokens fallback from 1000 to 2000
  to match Rust ActivationCriteria default (both parent and companion)
- e2e_builtin_tool_coverage: fix routine_create_list using hardcoded
  "test-user" instead of rig.owner_id() (broke when .with_skills()
  changed channel user to config owner_id)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR #2268 review feedback (round 2)

1. Fix memory_write `path:` → `target:` in all 4 setup skill completion
   markers (developer, ceo, content-creator, trader). The `memory_write`
   tool reads `target`, not `path`, so markers were never written to the
   correct location.

2. Add setup_marker validation in enforce_limits(): max 256 chars, reject
   `..` path traversal. Prevents untrusted skills from abusing markers.

3. Fix v2 Python skill budget: default 4000 → 6000 to match v1 Rust
   config. Also port the approx_tokens > declared * 2 sanity check from
   Rust to prevent budget bypass via low max_context_tokens declarations.

4. Reorder developer-setup companion skills to put github/github-workflow
   first (critical for setup) and fix misleading budget comment in config.

5. Move AssertUnwindSafe cleanup guard in e2e GitHub test to wrap
   everything after create_issue, preventing orphaned issues on panic.

6. Scope workspace in select_active_skills to the requesting user_id so
   multi-user channels check the correct user's setup marker state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove duplicate skills_dir field from LiveTestHarnessBuilder

Both sides of the merge added the same field, resulting in a duplicate
declaration that failed compilation in test targets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CI failures and Copilot review feedback

1. Fix formatting (cargo fmt).

2. Filter existing_titles to non-Skill docs in v2 orchestrator so setup
   markers don't collide with skill doc titles of the same name.

3. Fix stale doc comment in types.rs (commitments/README.md →
   commitments/.developer-setup-complete).

4. Fix misleading comment on v2 requires field — the full
   GatingRequirements struct is preserved, not just the companion list.

5. Match SecretError::NotFound explicitly in test_rig pre-seed loop
   instead of catching all errors — other errors (DB, crypto) now
   surface instead of triggering a blind create.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-18 14:32:52 +09:00
Illia Polosukhin
ab8d64cbfc feat: new-project skill and template ref resolution for parallel tool calls (#2353) 
* feat(gateway): project metrics dashboard, mission scheduling UI, and new-project skill

Adds project metrics types, mission cadence scheduling via gateway,
and a /new-project skill for creating autonomous projects with goals,
metrics, and missions. Includes gateway frontend enhancements for
project views with metrics and goal tracking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): resolve template refs in parallel tool calls and rewrite new-project skill

Two fixes from trace analysis (trace_20260411T133641.json):

1. Skill rewrite: new-project skill now instructs the model to use
   memory_write + mission_create directly instead of referencing
   nonexistent project_create/project_update tools. Includes goals
   and metrics when appropriate. Instructs sequential execution.

2. Template ref resolution: some OpenAI-format models (e.g. Qwen)
   emit {{call_id.field}} references in parallel tool call arguments.
   Added resolution pass in LlmBridgeAdapter that scans ActionCall
   parameters for these patterns and resolves them from prior tool
   results in the conversation history.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): add project detail page screenshot test

Playwright test that seeds mock project data via page.route() API
interception, navigates to the Projects tab, drills into a project,
and captures a screenshot showing goals, missions, and activity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add project detail screenshot for PR

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review — remove project tools, fix IDOR, scope widgets, add tests

- Remove project_create/project_update/project_list tools and capability
  registration (skill uses memory_write + mission_create only)
- Add ownership check on mission_create project_id override to prevent IDOR
- Reject non-UUID project_id values explicitly instead of silent fallback
- Add goals field to ProjectOverviewEntry so frontend drill-in renders them
- Propagate store errors in overview instead of unwrap_or_default masking failures
- Scope project widget CSS server-side via scope_css (prevents style leakage)
- Fix template ref doc comment to match partial resolution semantics
- Fix E2E mock widget response shape (bare array, not wrapped object)
- Call crBackToOverview() on tab switch to tear down project widgets
- Add caller-level test for template ref resolution through LlmBridgeAdapter
- Clean up stale cargo-deny advisory ignores, add RUSTSEC-2026-0097 (rand)
- Run cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve project slugs in mission_create, fix widget CSS comments

- mission_create now accepts project name/slug (not just UUID) by matching
  against the user's projects — fixes the skill's slug-based project_id
- Fix misleading CSS comment in app.js (CSS is scoped server-side)
- Fix style variable hoisting issue in widget mounting
- Log workspace.list() errors instead of silently swallowing them

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review round 3 — slug matching, template injection, N+1 queries

- Remove over-broad `starts_with` slug prefix matching in mission_create
  project_id resolution — require exact name/slug match only (serrrfirat)
- Fix slug generation inconsistency: frontend.rs now uses
  is_ascii_alphanumeric() matching effect_adapter.rs (serrrfirat)
- Prevent second-order template injection: resolve_template_refs now
  advances past resolved content instead of re-scanning from position 0,
  and skips unresolvable refs instead of breaking (serrrfirat)
- Parallelize N+1 overview queries: per-project thread/mission fetches
  now use tokio::try_join! + futures::try_join_all (serrrfirat, Copilot)
- Add two new security tests for template ref resolution

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-18 01:37:42 +09:00
firat.sertgoz
27d53f5153 docs(skills): code-review v2 + GitHub endpoint fixes + minor text updates (#2528) 
* feat(skills): paranoid-architect code-review skill v2

Rewrite the code-review skill from a 6-bullet checklist into a
paranoid-architect workflow that handles both local diffs and GitHub
PRs end-to-end:

- Two input shapes: local `git diff` or `owner/repo N` /
  `github.com/.../pull/N` URLs.
- Step 1 wraps GitHub fetches in `async def` + `FINAL(await ...)` to
  avoid the closure-capture quirk that kept tripping LLMs (see the
  paired codeact preamble update); reads metadata, diff, and files
  via three sequential awaits instead of `asyncio.gather`.
- Step 2 reads each changed file in full (raw media type, no base64
  module needed) so reviews account for surrounding context.
- Step 3 runs the change through six lenses: correctness, edge cases,
  security (with a real adversarial checklist), test coverage, docs,
  architecture.
- Step 4 renders findings as a severity table and asks which to post.
- Step 5 posts line-level comments via the PR comments endpoint with
  the captured head SHA, falling back to issue comments for
  multi-file findings.

Bumps `requires.skills` to include `github` so the activation pulls
in the GitHub API recipes via the chain-loader.

Adds a live e2e test (`e2e_live_code_review.rs`) plus a recorded
trace fixture (PR #2483) so the workflow is replayable without
hitting GitHub.

* docs(github): clarify search endpoints, response envelope, @me queries

LLMs kept inventing a `search_issues` action and looping over
`/repos/{owner}/{repo}/pulls` for "my PRs" queries. Clarify the
GitHub tool surface in three places:

- `tools-src/github/src/lib.rs` and `registry/tools/github.json`:
  enumerate the three real search actions and call out that
  `search_issues_pull_requests` covers both. Add the canonical
  `is:pr author:@me sort:updated-desc` recipe for cross-repo "my PRs".

- `skills/github/SKILL.md`: add an "Authenticated User & Cross-Repo
  Queries" section with copy-paste recipes for `@me`, the search
  endpoints with proper URL encoding, and the response-envelope
  contract (`body` is parsed JSON for application/json, raw `str` for
  diff endpoints — never call `json.loads()` on it, never write
  `.get("body", body)` as a fallback).

* fix: resolve CI failures — clippy useless_conversion + missing test harness methods

- Remove `.into_iter()` on `details` in catalog.rs (clippy::useless_conversion)
- Add `with_skills_dir` to `LiveTestHarnessBuilder` for e2e_live_code_review test
- Add `active_skill_names` to `TestRig` extracting from SkillActivated status events

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address zmanian + gemini review — URL encoding, multi-line comments, description trimming (#2528)

- URL-encode file paths in GitHub API content URLs
- Add start_line/start_side to multi-line comment example
- Add 'locally' keyword override for mode detection
- Trim overly long schema descriptions
- Remove duplicated /search/issues note from Common Mistakes
- Fetch PR title from trace fixture instead of hard-coding

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): propagate skills_dir into TestRig config (#2528)

LiveTestHarnessBuilder::with_skills_dir() stored a PathBuf but only
used it as an is_some() flag — the actual SkillRegistry always pointed
at an empty temp directory. Now the stored path flows through
TestRigBuilder::with_skills_dir() into config.skills.local_dir and
the SkillRegistry constructor.

Also generalizes the hardcoded nearai/ironclaw repo name in the
github skill's response-handling example to {owner}/{repo}.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-17 23:11:59 +09:00
Illia Polosukhin
764e586717 feat(engine): LLM council via per-call model override in CodeAct (#2320) 
* feat(engine): LLM council via per-call model override in CodeAct

Extend `llm_query()` and `llm_query_batched()` with a `model=` (and
`models=` for the batched variant) keyword so CodeAct can route
individual sub-queries to specific LLMs. The "LLM council" pattern
becomes a skill — the agent broadcasts the same prompt across a
parallel array of models and synthesizes the responses — with no new
tool, dispatch path, or capability boundary.

- Add `model: Option<String>` to `LlmCallConfig`; thread it through
  `LlmBridgeAdapter` onto `CompletionRequest.model` /
  `ToolCompletionRequest.model` so providers that honor per-request
  overrides (NEAR AI, Anthropic OAuth, GitHub Copilot, Bedrock) pick
  it up. Other providers fall back to their configured model.
- `handle_llm_query` extracts a `model` arg; `handle_llm_query_batched`
  accepts either `model="..."` (broadcast) or `models=[...]` (parallel
  array, length-validated against `prompts`).
- `__llm_complete__` host fn extracts `model` from explicit_config so
  the Python orchestrator can also forward it.
- Update CodeAct preamble docs so the agent sees the new parameters.
- Add `skills/llm-council/SKILL.md` with the council pattern,
  recommended NEAR AI model line-ups, and a synthesis example.

Tests: 5 new scripting tests (model kwarg forwarding, default `None`,
`models=` broadcast, single-`model=` broadcast, length-mismatch error)
and 2 new bridge tests (config.model → CompletionRequest.model on both
the no-tools and with-tools paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): address llm-council review feedback

Address three review comments on the LLM council PR:

1. Test coverage for the orchestrator entry point. Add two tests
   driving `handle_llm_complete` directly with explicit_config
   containing `model` (and a control case without it). Closes the
   "test through the caller, not just the helper" gap — the previous
   tests only exercised `handle_llm_query`, leaving the parallel
   `__llm_complete__` host fn path unverified.

2. Loud failure on non-string entries in `models=[...]`. Previously
   `models=[1, 2]` was silently coerced via `monty_to_string` to
   `["1", "2"]`. Now returns `TypeError` with the offending value,
   matching the existing length-mismatch handling style.

3. `None` slots in `models=[...]` are no longer backfilled by the
   singular `model=` kwarg. Each slot is authoritative: a `None`
   means "no override for this prompt" (use the configured default).
   Mixing the two would have been surprising — the docs now spell
   out the contract explicitly. Add a regression test that passes
   both `models=[None, "gpt-4o"]` and `model="claude-..."` and
   asserts the None slot stays None.

Also update `skills/llm-council/SKILL.md` to default to a 4-model
council of `anthropic/claude-opus-4-6`, `google/gemini-3-pro`,
`zai-org/GLM-latest`, `openai/gpt-5.4`. Per-call provider errors
already flow through the existing `Ok(Err(e))` arm as
`"Error: ..."` strings — the batch never fails as a whole, so
unavailable models just surface in their own slot.

Tests: 4 new (2 orchestrator, 2 scripting), all 346 engine unit
tests pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): strict optional-string parsing for llm_query model= kwarg

Address Copilot review on PR #2320. Four comments, all valid:

1 & 2. `model` and `single_model` were extracted via `extract_string_arg`,
   which calls `monty_to_string` — that coerces `MontyObject::None` to the
   literal string "None" and stringifies non-string values (ints become
   "1", etc.). So `llm_query(prompt="hi", model=None)` would silently
   route every call to a bogus model ID called "None".

   Add a strict `extract_optional_string_kwarg` helper that returns
   `Ok(None)` for missing/`None`, `Ok(Some(s))` for strings, and a
   `TypeError` for anything else. Use it in both `handle_llm_query` and
   `handle_llm_query_batched` for the `model=` kwarg. Regression tests
   cover: `model=None` → no override, `model=<int>` → TypeError, and the
   same two cases on the batched path.

3. The `models=` list-type error message said "list of strings" but we
   accept `None` entries. Updated to "list of str or None".

4. SKILL.md claimed the batched call "never raises". It does — for
   argument validation errors (wrong types, length mismatch). Clarified
   that per-model failures return as `"Error: ..."` strings, but
   argument validation still raises.

Tests: 4 new regression tests, all 4687 main-crate and 358 engine tests
pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): positional args for llm_query_batched + correct provider docs

Address two review comments from serrrfirat on PR #2320.

1. `llm_query_batched` silently dropped positional `context`/`model`/
   `models` args. The documented signature is
   `llm_query_batched(prompts, context=None, model=None, models=None)`,
   but the extractors were hardcoded to kwargs only (`&[]` for args).
   A call like `llm_query_batched(prompts, None, "gpt-4o")` routed to
   the default model — silent contract violation.

   Thread the real `args` slice into each extractor with the documented
   positional indices: context=1, model=2, models=3. Positional
   `MontyObject::None` at any of those slots now correctly means "no
   override". Added 3 regression tests:
   - `llm_query_batched_honors_positional_context_and_model`
   - `llm_query_batched_honors_positional_models_list`
   - `llm_query_batched_positional_none_for_models_is_no_override`

2. SKILL.md claimed Bedrock honors per-request model overrides, but
   `bedrock.rs::complete()` unconditionally uses
   `self.current_model_id()` and ignores `request.model`. Also, the
   default 4-model prefixed lineup (`anthropic/...`, `google/...`,
   `openai/...`) only works on aggregator backends like NEAR AI — a
   direct Anthropic OAuth or Copilot provider honors `set_model` but
   can only switch between models within its own vendor.

   Rewrite the SKILL.md preamble with a provider capability table
   (dropping Bedrock from "honors it" and adding cross-vendor routing
   as a separate column), and add per-backend default lineups:
   NEAR AI (prefixed cross-vendor), Anthropic OAuth (Anthropic tiers
   only), Copilot (Copilot-exposed models). For backends that don't
   honor `model=` at all (Bedrock, raw OpenAI/Ollama/Tinfoil), the
   skill now instructs the agent to tell the user and fall back to a
   single-model answer.

Tests: 3 new regression tests, all 4687 main-crate and 361 engine tests
pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 09:23:05 +03:00
Illia Polosukhin
2cc5546017 feat(tools): production-grade coding tools, file history, and skills (#2025) 
* feat(tools): add production-grade coding tools, file history, and coding skills

Add dedicated coding tools inspired by Claude Code's architecture to make
IronClaw a more effective coding assistant:

New tools:
- GlobTool: fast file pattern matching via `glob` crate, sorted by mtime,
  with default exclusions (.git, node_modules, target, etc.)
- GrepTool: content search wrapping ripgrep with 3 output modes
  (content, files_with_matches, count), pagination, and context lines
- FileUndoTool: restore files to pre-modification state using in-memory
  file history snapshots

Enhanced tools:
- ReadFileTool: 10MB limit, 2000-line default, binary detection, device
  path blocking (/dev/zero, /proc/*/fd/*)
- ApplyPatchTool: uniqueness validation (error on ambiguous matches),
  workspace path rejection, 10MB size limit, file history integration
- WriteFileTool: file history integration for undo support

Updated tool descriptions to guide LLM behavior (prefer apply_patch over
write_file, always read before editing, use glob/grep instead of shell).

New skills:
- coding: best practices for code editing, search, and file operations
- commit: git commit message generation workflow
- review: code review workflow with structured checklist

Shared infrastructure:
- DEFAULT_EXCLUDED_DIRS constant in path_utils.rs
- FileHistory module with SharedFileHistory for cross-tool snapshots

66 new tests covering all tools, edge cases, and regression scenarios.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: apply cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): address PR review — security, correctness, and robustness fixes

- Move device path blocking after validate_path() to prevent traversal bypass
- Add /proc/kcore, /proc/kmem to blocked paths
- Reject absolute patterns and '..' in glob tool, add strip_prefix defense
- Wrap glob sync I/O in spawn_blocking to avoid blocking tokio executor
- Sort files_with_matches globally before pagination in grep tool
- Add default exclusions for node_modules/target in grep tool
- Inject ctx.extra_env into rg environment matching ShellTool policy
- Use per-line strip_prefix for content mode path relativization
- Change FileSnapshot.content_before to Vec<u8> for binary file support
- Log snapshot errors with tracing::debug instead of silently discarding
- Fix skill name mismatch: code-review → review to match directory

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(skills): rename review skill directory to code-review

Aligns the directory name with the manifest name (code-review) to prevent
incorrect override/dedup behavior in the bundled-skill loader. The name
stays "code-review" since other domains may also need review-type skills.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tools): add file edit guards — staleness detection, fuzzy matching, encoding preservation

Add file_edit_guard module with production-grade safeguards for file editing:
- ReadFileState tracks file reads with mtime for staleness detection
- 4-level fuzzy matching fallback (exact → whitespace-normalized → quote-normalized → both)
- UTF-16LE BOM detection and line ending style preservation (LF/CRLF/CR)
- Read-before-edit enforcement for ApplyPatch and WriteFile tools
- No-op edit rejection (old_string == new_string)
- Shared state injection via Arc<RwLock<>> across ReadFile, WriteFile, ApplyPatch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): address all PR review comments — session scoping, parallelism, security

- Session-scoped state: ReadFileState and FileHistory now keyed by job_id
  so concurrent sessions sharing the same registry don't leak state (#2025)
- Parallel metadata: grep files_with_matches uses JoinSet (max 64 concurrency)
  instead of sequential await per file for mtime sorting
- Shared env allowlist: grep_tool imports SAFE_ENV_VARS from shell.rs
  (made pub(crate)) instead of maintaining a divergent copy
- Glob traversal: uses Component::ParentDir check instead of substring ".."
  match, so patterns like "foo..bar" are no longer falsely rejected
- UTF-16LE in read_file: binary detection skips null-byte check for files
  with UTF-16LE BOM; read_file uses encoding-aware read path
- Partial flag: default 2000-line truncation now marks read as partial,
  preventing edits against unseen content
- write_file guard softened: staleness check logs warning instead of
  hard error (full-file replacement has lower risk than apply_patch)
- Updated e2e trace to include read_file before apply_patch
- Updated expected tool list in schema validation tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): use async metadata instead of blocking path.exists() in write_file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): fix false-positive panic detection for lifetimes in char lexer

The check_no_panics.py lexer misinterpreted Rust lifetimes ('static) as
char literal starts, causing in_char state to persist across lines and
hide all subsequent brace-delimited blocks — including #[cfg(test)] mod
tests. Reset in_char at line boundaries since Rust char literals cannot
span lines.

https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC

* test: verify MCP push works

* test

* chore: remove test file

* style: apply cargo fmt to file.rs

Collapse multi-line method chain to single line per rustfmt.

https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC

* style: apply cargo fmt to file.rs

Collapse multi-line method chain to single line per rustfmt.

https://claude.ai/code/session_012bJjER6L5zSAFd9BqYMUmC

* fix(file-tools): harden fuzzy patch matching and undo

* fix(ci): formatting + wasmtime 43 cache config compatibility

After merging latest staging, cargo fmt had diffs in file tools and the
wasmtime cache TOML format changed (v43 dropped the `enabled` field
under `[cache]`). Also removes accidental .fmt-test artifact.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(file-tools): simplify strip_trailing_whitespace

Remove redundant double-pass through .lines() — the first
collect+join was a no-op since .lines() already handles line endings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): address PR review comments — security, correctness, tests

- Add is_sensitive_path checks to GlobTool and GrepTool, matching the
  defense-in-depth posture of ReadFileTool/WriteFileTool/ListDirTool
- Fix UTF-8 panicking byte-index slice in apply_patch error preview
  (old_string[..200] → chars().take(200))
- Add 10MB size guard on file_history snapshots to prevent memory
  exhaustion from snapshotting large files
- Replace dead turn_number field with auto-incrementing sequence_number
  in FileHistory — callers no longer pass a hardcoded 0
- Fix glob mtime test flakiness by increasing sleep to 1100ms (above
  1s filesystem granularity)
- Fix emoji test to actually include emoji/non-ASCII content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Zaki Manian <zaki@iqlusion.io>
 2026-04-11 01:37:17 +09:00
Illia Polosukhin
cd8f3f24b6 feat(skills): commitments system — active intake for personal AI assistant (#1736) 
* v2 architecture phase 1

* feat(engine): Phase 2 — execution loop, capability system, thread runtime

Add the core execution engine to ironclaw_engine crate:

- CapabilityRegistry: register/get/list capabilities and actions
- LeaseManager: async lease lifecycle (grant, check, consume, revoke, expire)
- PolicyEngine: deterministic effect-level allow/deny/approve
- ThreadTree: parent-child relationship tracking
- ThreadSignal/ThreadOutcome: inter-thread messaging via mpsc
- ThreadManager: spawn threads as tokio tasks, stop, inject messages, join
- ExecutionLoop: core loop replacing run_agentic_loop() with signals,
  context building, LLM calls, action execution, and event recording
- Structured executor (Tier 0): lease lookup → policy check → effect execution
- Tool intent nudge detection
- MemoryStore + RetrievalEngine stubs for Phase 4
- Full 8-phase architecture plan in docs/plans/
- CLAUDE.md spec for the engine crate

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 3 — Monty Python executor with RLM pattern

Add CodeAct execution (Tier 1) using the Monty embedded Python
interpreter, following the Recursive Language Model (RLM) pattern
from arXiv:2512.24601.

Key additions:
- executor/scripting.rs: Monty integration with FunctionCall-based
  tool dispatch, catch_unwind panic safety, resource limits (30s,
  64MB, 1M allocs)
- LlmResponse::Code variant + ExecutionTier::Scripting
- Context-as-variables (RLM 3.4): thread messages, goal, step_number,
  previous_results injected as Python variables — LLM context stays
  lean while code accesses data selectively
- llm_query(prompt, context) (RLM 3.5): recursive subagent calls
  from within Python code — results stored as variables, not injected
  into parent's attention window (symbolic composition)
- Compact output metadata between code steps instead of full stdout
- MontyObject ↔ serde_json::Value bidirectional conversion
- Updated architecture plan with RLM design principles

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): RLM best-practices enhancements from cross-reference analysis

Cross-referenced our implementation against the official RLM (alexzhang13/rlm),
fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation.
Key enhancements:

- FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching
  all three reference implementations. Code can signal completion at any
  point, not just via return value.
- llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn,
  matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch.
- Output truncation increased to 8000 chars (from 120), matching Prime
  Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT].
- Step 0 orientation preamble: auto-injects context metadata (message count,
  total chars, goal, last user message preview) before first code step,
  matching fast-rlm's auto-print pattern.
- Error-to-LLM flow: Python parse errors, runtime errors, NameErrors,
  OS errors, and async errors now flow back as stdout content instead of
  terminating the step, enabling LLM self-correction on next iteration.
  Only VM panics (catch_unwind) terminate as EngineError.

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): update architecture plan with RLM cross-reference learnings

Comprehensive update after cross-referencing against official RLM
(alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), Prime Intellect
(verifiers/RLMEnv), rlm-rs (zircote/rlm-rs), and Google ADK RLM.

Changes:
- Mark Phases 1-3 as DONE with commit refs and test counts
- Add "Key Influences" section documenting all reference implementations
- Phase 3: full table of implemented RLM features with sources
- Phase 3: "Remaining gaps" table with which phase addresses each
- Phase 4: expanded with compaction (85% context), rlm_query() (full
  recursive sub-agent), dual model routing, budget controls (USD,
  timeout, tokens, consecutive errors), lazy loading, pass-by-reference
- Add "RLM Execution Model" cross-cutting section
- Add "Implementation Progress" tracking table
- Remove stale "TO IMPLEMENT" markers (all Phase 3 work is done)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 4 — budget controls, compaction, reflection pipeline

Budget enforcement in ExecutionLoop:
- max_tokens_total: cumulative token limit, checked before each iteration
- max_duration: wall-clock timeout for entire thread
- max_consecutive_errors: consecutive error steps threshold (resets on
  success, matching official RLM behavior)
- All produce ThreadOutcome::Failed with descriptive messages

Context compaction (from RLM paper, 85% threshold):
- estimate_tokens(): char-based estimation (chars/4, matching RLM)
- should_compact(): triggers when tokens >= threshold_pct * context_limit
- compact_messages(): asks LLM to summarize progress, replaces history
  with [system, summary, continuation_note], preserves intermediate results
- Configurable via ThreadConfig: model_context_limit, compaction_threshold

Dual model routing:
- LlmCallConfig gains depth field (0=root, 1+=sub-call)
- Implementations can route to cheaper models for sub-calls
- ExecutionLoop passes thread depth to every LLM call

Reflection pipeline (reflection/pipeline.rs):
- reflect(thread, llm): analyzes completed thread via LLM
- Produces Summary doc (always), Lesson doc (if errors), Issue doc (if failed)
- Builds transcript from thread messages + error events
- Returns ReflectionResult with docs + token usage

ThreadConfig extended with: max_tokens_total, max_consecutive_errors,
model_context_limit, enable_compaction, compaction_threshold, depth, max_depth.

78 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 5 — conversation surface separated from execution

Conversation is now a UI layer, not an execution boundary. Multiple
threads can run concurrently within one conversation; threads can
outlive their originating conversation.

New types (types/conversation.rs):
- ConversationSurface: channel + user + entries + active_threads
- ConversationEntry: sender (User/Agent/System) + content + origin_thread_id
- ConversationId, EntryId (UUID newtypes)
- EntrySender enum (User, Agent{thread_id}, System)

ConversationManager (runtime/conversation.rs):
- get_or_create_conversation(channel, user) — indexed by (channel, user)
- handle_user_message() — injects into active foreground thread or spawns new
- record_thread_outcome() — adds agent/system entries, untracks completed threads
- get_conversation(), list_conversations()

This enables the key architectural insight: a user can ask "what's the
weather?" while a deployment thread is still running. Both produce entries
in the same conversation.

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): simplify execution tiers — Monty-only for CodeAct/RLM

Restructure phases 6-8 to clarify execution model:

- Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker
  Python runtimes for LLM-generated code.
- WASM sandbox is for third-party tool isolation (existing infra, Phase 8)
- Docker containers are for thread-level isolation of high-risk work (Phase 8)
- Two-phase commit moves to Phase 6 (integration) at the adapter boundary

Phase renumbering:
- Old Phase 6 (Tier 2-3) → removed as separate phase
- Old Phase 7 (integration) → Phase 6
- Old Phase 8 (cleanup) → Phase 7
- New Phase 8: WASM tools + Docker thread isolation (infra integration)

Updated progress table: Phases 1-5 marked DONE with test counts and commits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 6 — bridge adapters for main crate integration

Strategy C parallel deployment: when ENGINE_V2=true env var is set,
user messages route through the engine instead of the existing agentic
loop. All existing behavior is unchanged when the flag is off.

Bridge module (src/bridge/):
- LlmBridgeAdapter: wraps LlmProvider as engine LlmBackend, converts
  ThreadMessage↔ChatMessage, ActionDef↔ToolDefinition, depth-based
  model routing (primary vs cheap_llm)
- EffectBridgeAdapter: wraps ToolRegistry+SafetyLayer as EffectExecutor,
  routes tool calls through existing execute_tool_with_safety pipeline
- InMemoryStore: HashMap-backed Store impl (no DB tables needed yet)
- EngineRouter: is_engine_v2_enabled() + handle_with_engine() that
  builds engine from Agent deps and processes messages end-to-end

Integration touchpoint (4 lines in agent_loop.rs):
  After hook processing, before session resolution, check ENGINE_V2
  flag and route UserInput through the engine path.

Accessor visibility widened: llm(), cheap_llm(), safety(), tools()
changed from pub(super) to pub(crate) for bridge access.

85 engine tests + main crate clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): add user message and system prompt to thread before execution

The ExecutionLoop was sending empty messages to the LLM because the
thread was spawned with the user's input as the goal but no messages.

Fixes:
- ThreadManager.spawn_thread() now adds the goal as an initial user
  message before starting the execution loop
- ExecutionLoop.run() injects a default system prompt if none exists

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): match existing LLM request format to prevent 400 errors

The LLM bridge was missing several defaults that the existing
Reasoning.respond_with_tools() sets:

- tool_choice: "auto" when tools are present (required by some providers)
- max_tokens: 4096 (default)
- temperature: 0.7 (default)
- When no tools (force_text): use plain complete() instead of
  complete_with_tools() with empty tools array — matches existing
  no-tools fallback path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): persist conversation context across messages

The engine was creating a fresh ThreadManager and InMemoryStore per
message, losing all context between turns. A follow-up question like
"what are the latest 10 issues?" had no memory of the prior "how many
issues" response.

Fixes:
- EngineState (ThreadManager, ConversationManager, InMemoryStore) now
  persists across messages via OnceLock, initialized on first use
- ConversationManager builds message history from prior conversation
  entries (user messages + agent responses) and passes it to new threads
- ThreadManager.spawn_thread_with_history() accepts initial_messages
  that are prepended before the current user message
- System notifications (thread started/completed) are filtered out of
  the history (not useful as LLM context)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): enable CodeAct/RLM mode with code block detection

The engine now operates in CodeAct/RLM mode:

System prompt (executor/prompt.rs):
- Instructs LLM to write Python in ```repl fenced blocks
- Documents available tools as callable Python functions
- Documents llm_query(), llm_query_batched(), FINAL()
- Documents context variables (context, goal, step_number, previous_results)
- Strategy guidance: examine context, break into steps, use tools, call FINAL()

Code block detection (bridge/llm_adapter.rs):
- extract_code_block() scans LLM text responses for ```repl or ```python blocks
- When detected, returns LlmResponse::Code instead of LlmResponse::Text
- The ExecutionLoop routes Code responses through Monty for execution

No structured tool definitions sent to LLM:
- Tools are described in the system prompt as Python functions
- The LLM call sends empty actions array, forcing text-mode responses
- This ensures the LLM writes code blocks (CodeAct) instead of
  structured tool calls (which would bypass the REPL)

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): add 8 CodeAct/RLM E2E tests with mock LLM

Comprehensive test coverage for the Monty Python execution path:

- codeact_simple_final: Python code calls FINAL('answer') → thread completes
- codeact_tool_call_then_final: code calls test_tool() → FunctionCall
  suspends VM → MockEffects returns result → code resumes → FINAL()
- codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15')
  with no tool calls — pure Python in Monty
- codeact_multi_step: first step prints output (no FINAL), second step
  sees output metadata and calls FINAL — tests iterative REPL flow
- codeact_error_recovery: first step has NameError → error flows to LLM
  as stdout → second step recovers with FINAL — tests error transparency
- codeact_context_variables_available: code accesses `goal` and `context`
  variables injected by the RLM context builder
- codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times
  → 3 FunctionCall suspensions → all results collected → FINAL
- codeact_llm_query_recursive: code calls llm_query('prompt') → VM
  suspends → MockLlm provides sub-agent response → result returned as
  Python string variable

93 tests passing (85 prior + 8 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): detect code blocks in plain completion path + multi-block support

Two bugs fixed:

1. The no-tools completion path (used by CodeAct since we send empty
   actions) returned LlmResponse::Text without checking for code blocks.
   Code blocks were rendered as markdown text instead of being executed.

2. extract_code_block now:
   - Handles bare ``` fences (skips non-Python languages)
   - Collects ALL code blocks in the response and concatenates them
     (models often split code across multiple blocks with explanation)
   - Tries markers in order: ```repl, ```python, ```py, then bare ```

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(bridge): add 11 regression tests for code block extraction

Covers the exact failure modes discovered during live testing:

- extract_repl_block: standard ```repl fenced block
- extract_python_block: ```python marker
- extract_py_block: ```py shorthand
- extract_bare_backtick_block: bare ``` with Python content
- skip_non_python_language: ```json should NOT be extracted
- no_code_blocks_returns_none: plain text, no fences
- multiple_code_blocks_concatenated: two ```repl blocks with
  explanation between them → concatenated with \n\n
- mixed_thinking_and_code: model outputs explanation + two
  ```python blocks (the Hyperliquid case) → both extracted
- repl_preferred_over_bare: ```repl takes priority over bare ```
- empty_code_block_skipped: empty fenced block returns None
- unclosed_block_returns_none: no closing ``` returns None

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): detect FINAL() in text responses + regression tests

Models sometimes write FINAL() outside code blocks — as plain text
after an explanation. The Hyperliquid case: model outputs a long
analysis then FINAL("""...""") at the end, not inside ```repl fences.

Fixes:
- extract_final_from_text(): regex-based FINAL detection in text
  responses, matching the official RLM's find_final_answer() fallback
- Handles: double-quoted, single-quoted, triple-quoted, unquoted,
  nested parens
- Checked in LlmResponse::Text handler BEFORE tool intent nudge
  (FINAL takes priority)

9 new tests:
- codeact_final_in_text_response: FINAL("answer") in plain text
- codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text
- final_double_quoted, final_single_quoted, final_triple_quoted,
  final_unquoted, final_with_nested_parens, final_after_long_text,
  no_final_returns_none

102 tests passing (93 + 9 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add crate extraction & cleanup roadmap

Documents architectural recommendations from the engine v2 design
process for future reference:

- Root directory consolidation (channels-src + tools-src → extensions/)
- Crate extraction tiers: zero-coupling (estimation, observability,
  tunnel), trivial-coupling (document_extraction, pairing, hooks),
  medium-coupling (secrets, MCP, db, workspace, llm, skills),
  heavy-coupling (web gateway, agent, extensions)
- src/ module reorganization into logical groups (core, persistence,
  infra, media, support)
- main.rs/app.rs slimming targets (100/500 lines after migration)
- WASM module candidates (document_extraction) and non-candidates
  (REPL, web gateway → separate crates instead)
- Priority ordering for extraction work
- Tracks completed items (ironclaw_safety, ironclaw_engine,
  transcription move)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): live progress status updates via event broadcast

Engine v2 now shows live progress in the CLI (and any channel):
- "Thinking..." when a step starts
- Tool name + success/error when actions execute
- "Processing results..." when a step completes

Implementation:
- ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256)
- ExecutionLoop.emit_event() writes to thread.events AND broadcasts
- ThreadManager.subscribe_events() returns a receiver
- Router uses tokio::select! to listen for events while waiting for
  thread completion, forwarding them as StatusUpdate to the channel

This replaces the polling approach with zero-latency event streaming.
Agent.channels visibility widened to pub(crate) for bridge access.

102 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): include tool results in code step output for LLM context

The LLM was ignoring tool results and answering from training data
because the compact output metadata didn't include what tools returned.
Tool results lived only as ActionResult messages (role: Tool) which
some providers flatten or the model ignores.

Now the code step output includes:
- stdout from Python print() statements
- [tool_name result] with the actual output (truncated to 4K per tool)
- [tool_name error] for failed tools
- [return] for the code's return value
- Total output truncated to 8K chars to prevent context bloat

This ensures the model sees web_search results, API responses, etc.
in the next iteration and can reason about them instead of hallucinating.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): add debug/trace logging for CodeAct execution

Three verbosity levels for debugging the engine:

RUST_LOG=ironclaw_engine=debug:
- LLM call: message count, iteration, force_text
- LLM response: type (text/code/action_calls), token usage
- Code execution: code length, action count, had_error, final_answer
- Text response: length, FINAL() detection

RUST_LOG=ironclaw_engine=trace:
- Full message list sent to LLM (role, length, first 200 chars each)
- Full code block being executed
- stdout preview (first 500 chars)
- Per-tool results (name, success, first 300 chars of output)
- Text response preview (first 500 chars)

Usage:
  ENGINE_V2=true RUST_LOG=ironclaw_engine=debug cargo run
  ENGINE_V2=true RUST_LOG=ironclaw_engine=trace cargo run

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): execution trace recording + retrospective analysis

Enable with ENGINE_V2_TRACE=1 to get full execution traces and
automatic issue detection after each thread completes.

Trace recording (executor/trace.rs):
- build_trace(): captures full thread state — messages (with full
  content), events, step count, token usage, detected issues
- write_trace(): writes JSON to engine_trace_{timestamp}.json
- log_trace_summary(): logs summary + issues at info/warn level

Retrospective analyzer detects 8 issue categories:
- thread_failure: thread ended in Failed state
- no_response: no assistant message generated
- tool_error: specific tool failures with error details
- code_error: Python errors (NameError, SyntaxError, etc.) in output
- missing_tool_output: tool results exist but not in system messages
- excessive_steps: >10 steps (may be stuck in loop)
- no_tools_used: single-step answer without tools (hallucination risk)
- mixed_mode: text responses without code blocks (prompt not followed)

Thread state now saved to store after execution completes (for trace
access after join_thread).

Usage:
  ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run
  # After each message: trace JSON + issue log in terminal

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): wire reflection pipeline + trace analysis into thread lifecycle

After every thread completes, ThreadManager now automatically runs:

1. Retrospective trace analysis (non-LLM, always):
   - Detects 8 issue categories (tool errors, code errors, missing
     outputs, excessive steps, hallucination risk, etc.)
   - Logs issues at warn level when found

2. Trace file recording (when ENGINE_V2_TRACE=1):
   - Writes full JSON trace to engine_trace_{timestamp}.json

3. LLM reflection (when enable_reflection=true):
   - Calls reflection pipeline to produce Summary, Lesson, Issue docs
   - Saves docs to store for future context retrieval
   - Enabled by default in the bridge router

All three run inside the spawned tokio task after exec.run() completes,
before saving the final thread state. No external wiring needed.

Removed duplicate trace recording from the router — it's now handled
by ThreadManager automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): convert tool name hyphens to underscores for Python compatibility

Root cause from trace analysis: the LLM writes `web_search()` (valid
Python identifier) but the tool registry has `web-search` (with hyphen).
The EffectBridgeAdapter couldn't find the tool → "Tool not found" error
→ model fabricated fake data instead.

Fixes:
- available_actions(): converts tool names from hyphens to underscores
  (web-search → web_search) so the system prompt lists valid Python names
- execute_action(): tries the original name first, then falls back to
  hyphenated form (web_search → web-search) for tool registry lookup
- Same conversion in router's capability registry builder

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): parse JSON tool output to prevent double-serialization

From trace analysis: web_search returned a JSON string, which was
wrapped as serde_json::json!(string) creating a Value::String containing
JSON. When Monty got this as MontyObject::String, the Python code
couldn't index it with result['title'] → TypeError.

Fix: try parsing the tool output string as JSON first. If valid, use the
parsed Value (becomes a Python dict/list). If not valid JSON, keep as
string. This means web_search results are directly indexable in Python:
  results = web_search(query="...")
  print(results["results"][0]["title"])  # works now

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): persist variables across code steps via `state` dict

Monty creates a fresh runtime per code step, so variables are lost
between steps. This caused the model to re-paste tool results from
system messages, wasting tokens.

Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that
accumulates across steps:
- Tool results stored by tool name: state["web_search"] = {results...}
- Return values stored: state["last_return"], state["step_0_return"]
- Injected as a `state` Python variable in each new MontyRun

Now the model can do:
  Step 1: results = web_search(query="...")  # tool result saved in state
  Step 2: data = state["web_search"]         # access previous result
          summary = llm_query("summarize", str(data))
          FINAL(summary)

System prompt updated to document the `state` variable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): add state hint on code errors + retrieval engine integration

When code fails with NameError/UnboundLocalError (model trying to
access variables from a previous step), the error output now includes:

  [HINT] Variables don't persist between code blocks. Use the `state`
  dict to access data from previous steps. Available keys: ["web_search",
  "last_return"]

This teaches the model to use `state["web_search"]` instead of `result`
after a NameError, reducing wasted steps from 3-4 to 1.

Also integrates RetrievalEngine into context building and ThreadManager:
- build_step_context() now accepts optional RetrievalEngine to inject
  relevant memory docs (Lessons, Specs, Playbooks) into LLM context
- RetrievalEngine uses keyword matching with doc-type priority scoring
- Memory docs from reflection (Phase 4) now feed back into future threads

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove trace files and add to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): replace web_fetch example with web_search in CodeAct prompt

The system prompt example used web_fetch(url="...") which doesn't exist
as a tool. The model learned from the example and tried web_fetch,
getting "Tool not found". Changed to web_search(query="...") which is
an actual registered tool.

Found via trace analysis — reflection pipeline correctly identified
this as a "Tool Name Correction" spec doc.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): extract prompt templates to markdown files

Prompt templates moved from inline Rust strings to plain markdown files
at crates/ironclaw_engine/prompts/ for easy inspection and iteration:

- prompts/codeact_preamble.md — main instructions, special functions,
  context variables, rules
- prompts/codeact_postamble.md — strategy section

Loaded at compile time via include_str!(), so no runtime file I/O.
Edit the .md files and rebuild to iterate on prompts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): replace byte-index slicing with char-safe truncation

Panic: 'byte index 80 is not a char boundary; it is inside ''' when
tool output contained multi-byte UTF-8 characters (smart quotes from
web search results).

Fixed 4 unsafe byte-index slices:
- thread.rs:281: message preview &content[..80] → chars().take(80)
- loop_engine.rs:556: tool output &str[..4000] → chars().take(4000)
- loop_engine.rs:579: output tail &str[len-8000..] → chars().skip()
- scripting.rs:82: stdout tail &str[len-N..] → chars().skip()

All now use .chars().take() or .chars().skip() which respect character
boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on
user-supplied or external strings."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): fix false positive missing_tool_output warning in trace analyzer

The check was looking for "[" + "result]" in System-role messages only,
but tool output metadata is added with patterns like "[shell result]"
and may appear in messages with any role. Changed to scan all messages
for " result]" or " error]" patterns regardless of role.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): update architecture plan with Phase 6 status and approval flow design

Phase 6 updated to reflect what was actually built:
- Bridge adapters (LLM, Effect, InMemoryStore, Router) — all done
- Integration touchpoint (4 lines in handle_message) — done
- Live progress via broadcast events — done
- Conversation persistence across messages — done
- Trace recording + retrospective analysis — done
- 8 bugs found and fixed via trace analysis — documented

Phase 6 remaining work documented:
- Approval flow: detailed 5-step design (send to channel, pause thread,
  route response, resume execution, always handling) with v1 reference
- Database persistence (InMemoryStore → real DB tables)
- Acceptance testing (TestRig + TraceLlm fixtures)
- Two-phase commit for high-stakes effects

Progress table updated: Phase 6 marked as DONE (partial), 134 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add self-improving engine design plan

Designs a system where the engine debugs and improves itself, based on
the pattern observed in the last session: 5 consecutive bug fixes all
followed trace → read → identify → edit → test, using tools the engine
already has access to.

Three levels of self-improvement:
- Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply.
- Level 2 (Config): adjust defaults/mappings. Branch + test + PR.
- Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR.

Architecture: Self-improvement Mission spawns a Reflection thread that
reads traces, reads source, proposes fixes, validates via cargo test,
and either auto-applies (Level 1) or creates a PR (Level 2-3).

Includes: fix pattern database (seeded from our 8 debugging session
fixes), feedback loop diagram, safety model, implementation phases
(A through D), and what exists vs what's new.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add engine v2 security model and audit

Comprehensive security analysis of engine v2 covering:

Threat model: 4 attacker profiles (malicious input, prompt injection
via tools, poisoned memory, supply chain).

Current state audit: 9 controls working (Monty sandbox, safety layer,
policy engine, leases, provenance, events) and 9 gaps identified.

Critical finding: ALL tools granted by default — CodeAct code can call
shell, write_file, apply_patch without approval. Proposed fix: 3-tier
tool classification (auto/approve-once/always-approve).

CodeAct-specific threats: tool call amplification, prompt injection via
search results, data exfiltration via tool chains, Monty escape.

Self-improvement security: poisoned trace attacks, memory poisoning via
reflection. Mitigations: edit validation, frequency caps, audit trail,
auto-rollback, reflection output scanning.

6-layer security architecture proposed: input validation, capability
gating, output sanitization, execution sandboxing, self-improvement
controls, observability.

Prioritized implementation plan with severity/effort ratings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(security): cross-reference v1 controls — use, don't reinvent

Updated security plan with detailed audit of ALL existing v1 security
controls and how they map to engine v2 bridge gaps:

Key finding: v1 already has solutions for every security gap identified.
The bridge just needs to wire them in:

- Tool::requires_approval() exists but bridge doesn't call it
- safety.wrap_for_llm() exists but tool results enter context unwrapped
- RateLimiter exists but bridge doesn't check rate limits
- BeforeToolCall hooks exist but bridge doesn't run them
- redact_params() exists but bridge doesn't redact sensitive params
- Shell risk classification (Low/Medium/High) is inherited but ignored

Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter,
not new security infrastructure. The bridge is the security boundary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): add missions, reliability tracker, reflection executor, and provenance-aware policy

- Add Mission type and MissionManager for recurring thread scheduling
- Add ReliabilityTracker for per-capability success/failure/latency tracking
- Add reflection executor that spawns CodeAct threads for post-completion reflection
- Extend PolicyEngine with provenance-aware taint checking (LLM-generated data
  requires approval for financial/external-write effects)
- Extend Store trait with mission CRUD methods
- Add conversation surface tracking, compaction token fix, context memory injection
- Wire new modules through lib.rs re-exports and bridge adapters

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): wire v1 security controls into engine v2 adapter

Zero engine crate changes. All security controls enforced at the bridge
boundary in EffectBridgeAdapter:

1. Tool approval (v1: Tool::requires_approval):
   - Checks each tool's approval requirement with actual params
   - Always → returns EngineError::LeaseDenied (blocks execution)
   - UnlessAutoApproved → checks auto_approved set, blocks if not approved
   - Never → proceeds
   - Per-session auto_approved HashSet (for future "always" handling)

2. Hook interception (v1: BeforeToolCall):
   - Runs HookEvent::ToolCall before every execution
   - HookOutcome::Reject → blocks with reason
   - HookError::Rejected → blocks with reason
   - Hook errors → fail-open (logged, execution continues)

3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm):
   - Leak detection: API keys in tool output are redacted
   - Policy enforcement: content policy rules applied
   - Length truncation: output capped at 100KB
   - XML boundary protection: prevents injection via tool output

4. Sensitive param redaction (v1: redact_params):
   - Tool's sensitive_params() consulted before hooks see parameters
   - Redacted params sent to hooks, original params used for execution

5. available_actions() now sets requires_approval based on each tool's
   default approval requirement, so the engine's PolicyEngine can
   gate tools it hasn't seen before.

6. Actual execution timing measured via Instant::now() (replaces
   placeholder Duration::from_millis(1)).

Accessor visibility: hooks() widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): implement tool approval flow for engine v2

Adds a complete approval flow that mirrors v1 behavior, using the
existing v1 security controls (Tool::requires_approval, auto-approve
sets, StatusUpdate::ApprovalNeeded).

## How it works

### Step 1: Tool blocked at execution
When the LLM's code calls a tool (e.g., `shell("ls")`):
1. EffectBridgeAdapter.execute_action() looks up the Tool object
2. Calls tool.requires_approval(&params) — returns ApprovalRequirement
3. If Always → EngineError::LeaseDenied (always blocks)
4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set,
   returns EngineError::LeaseDenied
5. If Never → proceeds to execution

### Step 2: Engine returns NeedApproval
The LeaseDenied error propagates through:
- CodeAct path: becomes Python RuntimeError, code halts, thread returns
  NeedApproval with action_name + parameters
- Structured path: same via ActionResult.is_error

### Step 3: Router stores pending approval
- PendingApproval { action_name, original_content } stored on EngineState
- StatusUpdate::ApprovalNeeded sent to channel (shows approval card in
  CLI/web with tool name, parameters, yes/always/no buttons)
- Returns text: "Tool 'shell' requires approval. Reply yes/always/no."

### Step 4: User responds
handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2:
- 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes
  original message (tool now passes the approval check on second run)
- 'always' → same + logs for session persistence
- 'no' → returns "Denied: tool was not executed."

### Key design choice
Instead of pausing/resuming mid-execution (which needs engine changes
to freeze/restore the Monty VM state), we auto-approve the tool and
re-run the full message. The EffectBridgeAdapter's auto_approved set
persists across runs, so the second execution passes immediately.

This trades one extra LLM call for zero engine modifications.

## Files changed
- src/bridge/router.rs: PendingApproval struct, handle_approval(),
  NeedApproval → StatusUpdate::ApprovalNeeded conversion
- src/bridge/mod.rs: export handle_approval
- src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2
- src/bridge/effect_adapter.rs: fmt fixes

151 tests passing, clippy + fmt clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): demote trace/reflection logging from info to debug

INFO-level log output from background tasks (trace analysis, reflection)
corrupts the REPL terminal UI. The trace summary, issue warnings, and
reflection doc previews were printing mid-approval-card, breaking the
interactive display.

Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!.
Trace analysis and reflection results now only show when
RUST_LOG=ironclaw_engine=debug is set.

Also added logging discipline rule to global CLAUDE.md:
- info! → user-facing status the REPL intentionally renders
- debug! → internal diagnostics (traces, reflection, engine internals)
- Background tasks must NEVER use info! — it breaks the TUI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): demote all router info! logging to debug!

"engine v2: initializing" and "engine v2: handling message" were
printing at INFO level, corrupting the REPL UI. All router logging
now uses debug! — only visible with RUST_LOG=ironclaw=debug.

Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(safety): demote leak detector warn-action logs from warn! to debug!

The leak detector's Warn-action matches (high_entropy_hex pattern on
web search results containing commit SHAs, CSS colors, URL hashes)
were logging at warn! level, corrupting the REPL UI with lines like:
  WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5

These are informational false positives — real leaks use LeakAction::Redact
which silently modifies the content. Warn-action matches only log for
debugging purposes and should not appear in production output.

Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): strengthen CodeAct prompt to prevent shallow text answers

The model was answering "Suggested 45 improvements" as a brief text
summary from training data without actually searching or listing them.
The trace showed: no code block, no tool calls, no FINAL().

Prompt changes:
- Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with
  plain text only." (was: "Always write code... plain text for brief
  explanations")
- Rule 2 (NEW): "NEVER answer from memory or training data alone.
  Always use tools to get real, current information before answering."
- Rule 3: FINAL answer "should be detailed and complete — not just a
  summary like 'found 45 items'"
- Rule 8 (NEW): "Include the actual content in your FINAL() answer,
  not just a count or summary. Users want to see the details."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): persist reflection docs to workspace for cross-session learning

Replaces InMemoryStore with HybridStore:
- Ephemeral data (threads, steps, events, leases) stays in-memory
- MemoryDocs (lessons, specs, playbooks from reflection) persist to
  the workspace at engine/docs/{type}/{id}.json

On engine init, load_docs_from_workspace() reads existing docs back
into the in-memory cache. This means:
- Lessons learned in session 1 are available in session 2
- The RetrievalEngine injects relevant past lessons into new threads
- The engine genuinely improves over time as reflection accumulates

Workspace paths:
  engine/docs/lessons/{uuid}.json
  engine/docs/specs/{uuid}.json
  engine/docs/playbooks/{uuid}.json
  engine/docs/summaries/{uuid}.json
  engine/docs/issues/{uuid}.json

No new database tables. Uses existing workspace write/read/list.
workspace() accessor widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): adapt to execute_tool_with_safety params-by-value change

Staging merge changed execute_tool_with_safety to take params by value
instead of by reference (perf optimization from PR #926). Updated
bridge adapter to clone params before passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): add web gateway integration plan to Phase 6

Documents three gaps between engine v2 and the web gateway:
1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent)
2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB)
3. No cross-channel visibility (REPL ↔ web messages invisible to each other)

Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1
conversation tables after thread completion. Prerequisite: AppEvent
extraction PR (in progress separately).

Also updated DB persistence status: HybridStore with workspace-backed
MemoryDocs is now implemented (partial persistence).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): document routine/job gap and SIGKILL crash scenario

Routines are entirely v1 — not hooked up to engine v2. When a user
asks "create a routine" as natural language, engine v2 tries to call
routine_create via CodeAct, but the tool needs RoutineEngine + Database
refs that the bridge's minimal JobContext doesn't provide. This caused
a SIGKILL crash during testing.

Options documented: block routine tools in v2 (short term), pass refs
through context (medium), replace with Mission system (long term).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract AppEvent to crates/ironclaw_common

SseEvent was defined in src/channels/web/types.rs but imported by 12+
modules across agent, orchestrator, worker, tools, and extensions — it
had become the application-wide event protocol, not a web transport
concern.

Create crates/ironclaw_common as a shared workspace crate and move the
enum there as AppEvent.  Also move the truncate_preview utility which
was similarly leaked from the web gateway into agent modules.

- New crate: crates/ironclaw_common (AppEvent, truncate_preview)
- Rename SseEvent → AppEvent, from_sse_event → from_app_event
- web/types.rs re-exports AppEvent for internal gateway use
- web/util.rs re-exports truncate_preview
- Wire format unchanged (serde renames are on variants, not the enum)

Aligned with the event bus direction on refactor/architectural-hardening
where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): integrate with web gateway via AppEvent + v1 conversation DB

Three changes to make engine v2 visible in the web gateway:

1. SSE event streaming (AppEvent broadcast):
   - ThreadEvent → AppEvent conversion via thread_event_to_app_event()
   - Events broadcast to SseManager during the poll loop
   - Covers: Thinking, ToolCompleted (success/error), Status, Response
   - Web gateway receives real-time progress without any gateway changes

2. Conversation persistence to v1 database:
   - After thread completes, writes user message + agent response to
     v1 ConversationStore via add_conversation_message()
   - Uses get_or_create_assistant_conversation() for per-user per-channel
   - Web gateway reads from DB as usual — chat history appears

3. Final response broadcast:
   - AppEvent::Response with full text + thread_id sent via SSE
   - Web gateway renders the response in the chat UI

New EngineState fields: sse (Option<Arc<SseManager>>),
db (Option<Arc<dyn Database>>). Both populated from Agent.deps.

Agent.deps visibility widened to pub(crate).

Depends on: ironclaw_common crate with AppEvent type (PR #1615).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): complete Phase 6 — v1-only tool blocking, rate limiting, call limits

Three security/stability improvements in EffectBridgeAdapter:

1. V1-only tool blocking:
   - routine_create, create_job, build_software (and hyphenated variants)
     return helpful error: "use the slash command instead"
   - Filtered out of available_actions() so system prompt doesn't list them
   - Prevents crash from tools needing RoutineEngine/Scheduler refs

2. Per-step tool call limit:
   - Max 50 tool calls per code block (AtomicU32 counter)
   - Prevents amplification: `for i in range(10000): shell(...)`
   - Returns "call limit reached, break into multiple steps"

3. Rate limiting:
   - Per-user per-tool sliding window via RateLimiter
   - Checks tool.rate_limit_config() before every execution
   - Returns "rate limited, try again in Ns"

Architecture plan updated:
- Gateway integration: DONE
- Routines: BLOCKED (gracefully, with slash command fallback)
- Rate limiting: DONE
- Call limit: DONE
- Phase 6 status: DONE (remaining: acceptance tests, two-phase commit)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Mission system design — goal-oriented autonomous threads

Missions replace routines with evolving, knowledge-accumulating
autonomous agents. Unlike routines (fixed prompt, stateless), Missions:

- Generate prompts from accumulated Project knowledge (lessons,
  playbooks, issues from prior threads)
- Adapt approach when something fails repeatedly
- Track progress toward a goal with success criteria
- Self-manage: pause when stuck, complete when goal achieved

Architecture: MissionManager with cron ticker spawns threads via
ThreadManager. Meta-prompt built from mission goal + Project MemoryDocs
via RetrievalEngine. Reflection feeds back automatically.

6-step implementation plan: cron trigger, meta-prompt builder, bridge
wiring, CodeAct tools, progress tracking, persistence.

Includes two worked examples: daily tech news briefing (ongoing) and
test coverage improvement (goal-driven, self-completing).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): extend Mission types with webhook/event triggers + evolving strategy

Mission types updated to support external activation sources:

MissionCadence expanded:
- Cron { expression, timezone } — timezone-aware scheduling
- OnEvent { event_pattern } — channel message pattern matching
- OnSystemEvent { source, event_type } — structured events from tools
- Webhook { path, secret } — external HTTP triggers (GitHub, email, etc.)
- Manual — explicit triggering only

The engine defines trigger TYPES. The bridge implements infrastructure
(cron ticker, webhook endpoints, event matchers). GitHub issues, PRs,
email, Slack events all use the generic Webhook cadence — no
special-casing in the engine. Webhook payload injected as
state["trigger_payload"] in the thread's Python context.

Mission struct extended:
- current_focus: what the next thread should work on (evolving)
- approach_history: what we've tried (for adaptation)
- max_threads_per_day / threads_today: daily budget
- last_trigger_payload: webhook/event data for thread context

Plan updated with trigger type table and webhook integration design.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): implement MissionManager execution with meta-prompts

The MissionManager now builds evolving meta-prompts and processes
thread outcomes for continuous learning:

fire_mission() upgraded:
- Loads Project MemoryDocs via RetrievalEngine for context
- Builds meta-prompt from: goal, current_focus, approach_history,
  project knowledge docs, trigger payload, thread count
- Spawns thread with meta-prompt as user message
- Background task waits for completion and processes outcome
- Daily thread budget enforcement (max_threads_per_day)

Meta-prompt structure:
  # Mission: {name}
  Goal: {goal}
  ## Current Focus (evolves between threads)
  ## Previous Approaches (what we've tried)
  ## Knowledge from Prior Threads (lessons, playbooks, issues)
  ## Trigger Payload (webhook/event data if applicable)
  ## Instructions (accomplish step, report next focus, check goal)

Outcome processing:
- Extracts "next focus:" from FINAL() response → updates current_focus
- Detects "goal achieved: yes" → completes mission
- Records accomplishment in approach_history
- Failed threads recorded as "FAILED: {error}"

Cron ticker:
- start_cron_ticker() spawns tokio task, ticks every 60s
- Checks active Cron missions, fires those past next_fire_at

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): wire MissionManager into engine v2 for CodeAct access

Missions are now callable from CodeAct Python code:

```python
# Create a daily briefing mission
result = mission_create(
    name="Tech News",
    goal="Daily AI/crypto/software news briefing",
    cadence="0 9 * * *"
)

# List all missions
missions = mission_list()

# Manually fire a mission
mission_fire(id="...")

# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```

Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
  lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
  "webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
  (avoids circular dependency)

System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): map routine_* calls to mission operations in v2

When the model calls routine_create, routine_list, routine_fire,
routine_pause, routine_resume, or routine_delete, the bridge now
routes them to the MissionManager instead of blocking with an error.

Mapping:
  routine_create → mission_create (with cadence parsing)
  routine_list   → mission_list
  routine_fire   → mission_fire
  routine_pause  → mission_pause
  routine_resume → mission_resume
  routine_update → mission_pause/resume (based on params)
  routine_delete → mission_complete (marks as done)

Routine tools removed from v1-only blocklist and restored in
available_actions(). The model can use either "routine" or "mission"
vocabulary — both work.

Still blocked: create_job, cancel_job, build_software (need v1
Scheduler/ContainerJobManager refs).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): add E2E mission flow tests — 7 new tests

Comprehensive mission lifecycle tests:

- fire_mission_builds_meta_prompt_with_goal: verifies thread spawned
  with project context and recorded in history
- outcome_processing_extracts_next_focus: "Next focus: X" in FINAL()
  response → mission.current_focus updated
- outcome_processing_detects_goal_achieved: "Goal achieved: yes" →
  mission status transitions to Completed
- mission_evolves_via_direct_outcome_processing: 3-step evolution:
  step 1 sets focus to "db module", step 2 evolves to "tools module",
  step 3 detects goal achieved → mission completes. Tests the full
  learning loop without background task timing dependencies.
- fire_with_trigger_payload: webhook payload stored on mission and
  threads_today counter incremented
- daily_budget_enforced: max_threads_per_day=1 → first fire succeeds,
  second returns None

157 tests passing (151 prior + 6 new mission E2E).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): self-improving engine via Mission system

Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.

Key changes:

Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
  matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
  prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
  async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading

Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add checkpoint-based engine thread recovery

* feat(engine): add Python orchestrator module and host functions

Add the orchestrator infrastructure for replacing the Rust execution
loop with versioned Python code. This commit adds the module and host
functions without switching over — the existing Rust loop is unchanged.

New files:
- orchestrator/default.py: v0 Python orchestrator (run_loop + helpers)
- executor/orchestrator.rs: host function dispatch, orchestrator
  loading from Store with version selection, OrchestratorResult parsing

Host functions exposed to orchestrator Python via Monty suspension:
  __llm_complete__, __execute_code_step__ (nested Monty VM),
  __execute_action__, __check_signals__, __emit_event__,
  __add_message__, __save_checkpoint__, __transition_to__,
  __retrieve_docs__, __check_budget__, __get_actions__

Also makes json_to_monty, monty_to_json, monty_to_string pub(crate)
in scripting.rs for cross-module use.

Design doc: docs/plans/2026-03-25-python-orchestrator.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): switch ExecutionLoop::run() to Python orchestrator

Replace the 900-line Rust execution loop with a ~80-line bootstrap
that loads and runs the versioned Python orchestrator via Monty VM.

The orchestrator Python code (orchestrator/default.py) is the v0
compiled-in version. Runtime versions can override it via MemoryDoc
storage (orchestrator:main with tag orchestrator_code).

Key fixes during switchover:
- Use ExtFunctionResult::NotFound for unknown functions so Monty
  falls through to Python-defined functions (extract_final, etc.)
- Move helper function definitions above run_loop for Monty scoping
- Use FINAL result value (not VM return value) in Complete handler
- Rename 'final' variable to 'final_answer' to avoid Python keyword

Status: 171/177 tests pass. 6 remaining failures are step_count and
token tracking bookkeeping — the orchestrator manages these internally
but doesn't yet update the thread's counters via host functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): all 177 tests pass with Python orchestrator

- Increment step_count and track tokens in __emit_event__("step_completed")
  so thread bookkeeping matches the old Rust loop behavior
- Remove double-counting of tokens in bootstrap (orchestrator handles it)
- Match nudge text to existing TOOL_INTENT_NUDGE constant
- Fix FINAL result propagation (use stored final_result, not VM return)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): orchestrator versioning, auto-rollback, and tests

Add version lifecycle for the Python orchestrator:
- Failure tracking via MemoryDoc (orchestrator:failures)
- Auto-rollback: after 3 consecutive failures, skip the latest version
  and fall back to previous (or compiled-in v0)
- Success resets the failure counter
- OrchestratorRollback event for observability

Update self-improvement Mission goal with Level 1.5 instructions for
orchestrator patches — the agent can now modify the execution loop
itself via memory_write with versioned orchestrator docs.

12 new tests: version selection (highest wins), rollback after failures,
rollback to default, failure counting/resetting, outcome parsing for
all 5 ThreadOutcome variants.

189 tests pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add engine v2 architecture, self-improvement, and dev history

Three new docs for contributors:

- engine-v2-architecture.md: Two-layer architecture (Rust kernel +
  Python orchestrator), five primitives, execution model with nested
  Monty VMs, bridge layer, memory/reflection, missions, capabilities

- self-improvement.md: Three improvement levels (prompt/orchestrator/
  config/code), autoresearch-inspired Mission loop, versioned
  orchestrator with auto-rollback, fix pattern database, safety model

- development-history.md: Summary of 6 Claude Code sessions that
  built the system, key design decisions and debugging moments,
  architecture evolution from 900-line Rust loop to Python orchestrator

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): complete v2 side-by-side integration with gateway API

Wire engine v2 into the full submission pipeline and expose threads,
projects, and missions through the web gateway REST API.

Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear
submissions to engine v2 when ENGINE_V2=true. Previously only UserInput
and ApprovalResponse were handled; all other control commands fell
through to disconnected v1 sessions.

Bridge query layer — add 11 read-only query functions and 6 DTO types
so gateway handlers can inspect engine state (threads, steps, events,
projects, missions) without direct access to the EngineState singleton.

Gateway endpoints — new /api/engine/* routes:
  GET  /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events
  GET  /projects, /projects/{id}
  GET  /missions, /missions/{id}
  POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume

SSE events — add ThreadStateChanged, ChildThreadSpawned, and
MissionThreadSpawned AppEvent variants. Expand the bridge event mapper
to forward StateChanged and ChildSpawned engine events to the browser.

Engine crate — add ConversationManager::clear_conversation() for /new
and /clear commands.

Code quality — replace 10 .expect() calls with proper error returns,
remove dead AgentConfig.engine_v2 field, log silent init errors, fix
duplicate doc comment, improve fallthrough documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): empty call_id on ActionResult and trace analyzer false positives

Fix structured executor not stamping call_id onto ActionResult — the
EffectExecutor trait doesn't receive call_id, so the structured executor
must copy it from the original ActionCall after execution. Empty call_id
caused OpenAI-compatible providers to reject the next LLM request with
"Invalid 'input[2].call_id': empty string".

Fix trace analyzer false positives:
- code_error check now only scans User-role code output messages
  (prefixed with [stdout]/[stderr]/[code ]/Traceback), not System
  prompt which contains example error text
- missing_tool_output check now recognizes ActionResult messages as
  valid tool output (Tier 0 structured path)
- Add NotImplementedError to detected code error patterns

New trace checks:
- empty_call_id: detect ActionResult messages with missing/empty
  call_id before they reach the LLM API (severity: Error)
- llm_error: extract LLM provider errors from Failed state reason
- orchestrator_error: extract orchestrator errors from Failed state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): add Missions tab to gateway UI

Add a full Missions page to the web gateway with list view, detail view,
and action buttons (Fire, Pause, Resume).

Backend: add /api/engine/missions/summary endpoint returning counts by
status (active/paused/completed/failed).

Frontend:
- New "Missions" tab between Jobs and Routines
- Summary cards showing mission counts by status
- Table with name, goal, cadence type, thread count, status, actions
- Detail view with goal, cadence, current focus, success criteria,
  approach history, spawned thread list, and action buttons
- Fire/Pause/Resume actions with toast notifications
- i18n support (English + Chinese)
- CSS following the existing routines/jobs patterns

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): eagerly initialize engine v2 at startup

The gateway API endpoints (/api/engine/missions, etc.) call bridge
query functions that return empty results when the engine state hasn't
been initialized yet. Previously, initialization only happened lazily
on the first chat message via handle_with_engine().

Now when ENGINE_V2=true, the engine is initialized in Agent::run()
before channels start, so the self-improvement mission and other
engine state is available to gateway API endpoints immediately.

Also rename get_or_init_engine → init_engine and make it public so
it can be called from agent_loop.rs at startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): improve mission detail with markdown goal and thread table

- Goal rendered as full-width markdown block instead of plain-text
  meta item (uses existing renderMarkdown/marked)
- Current focus and success criteria also rendered as markdown
- Spawned threads shown as a clickable table with goal, type, state,
  steps, tokens, and created date instead of a UUID list
- Clicking a thread row opens an inline thread detail view showing
  metadata grid and full message history with markdown rendering
- Back button returns to the mission detail view
- Backend: mission detail now returns full thread summaries (goal,
  state, step_count, tokens) instead of just thread IDs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): close SSE connections on page unload to prevent connection starvation

The browser limits concurrent HTTP/1.1 connections per origin to 6.
Without cleanup, SSE connections from prior page loads linger after
refresh/navigation, eating into the pool. After 2-3 refreshes, all 6
slots are consumed by stale SSE streams and new API fetch calls queue
indefinitely — the UI shows "connected" (SSE works) but data never
loads.

Add a beforeunload handler that closes both eventSource (chat events)
and logEventSource (log stream) so the browser can reuse connections
immediately on page reload.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): support multiple gateway tabs by reducing SSE connections

Each browser tab opened 2 SSE connections (chat events + log events).
With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the
pool and couldn't load any data.

Three changes:

1. Lazy log SSE — only connect when the logs tab is active, disconnect
   when switching away. Most users rarely view logs, so this saves a
   connection slot per tab.

2. Visibility API — close SSE when the browser tab goes to background
   (user switches to another tab), reconnect when it becomes visible.
   Background tabs don't need real-time events.

3. Combined with the existing beforeunload cleanup, this means:
   - Active foreground tab: 1 connection (chat SSE only, +1 if logs tab)
   - Background tabs: 0 connections
   - Closed/refreshed tabs: 0 connections (beforeunload cleanup)

This allows many gateway tabs to coexist within the 6-connection limit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): route messages to correct conversation by thread scope

Messages sent from a new conversation in the gateway always appeared in
the default assistant conversation because handle_with_engine ignored
the thread_id from the frontend.

Two fixes:

1. Engine conversation scoping — when the message carries a thread_id
   (from the frontend's conversation picker), use it as part of the
   engine conversation key: "gateway:<thread_id>" instead of just
   "gateway". This creates a distinct engine conversation per v1
   thread, so messages don't cross-contaminate.

2. V1 dual-write targeting — write user messages and assistant
   responses to the v1 conversation matching the thread_id (via
   ensure_conversation), not the hardcoded assistant conversation.
   Falls back to the assistant conversation when no thread_id is
   present (e.g., default chat).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): richer activity indicators for engine v2 execution

The gateway UI showed only generic "Thinking..." during engine v2
execution with no visibility into CodeAct code execution, tool calls,
or reflection. Now the event mapping produces detailed status updates:

Step lifecycle:
- "Calling LLM..." when a step starts (was "Thinking...")
- "Step complete — N in / M out tokens" when done (was "Processing...")

Tool execution:
- Emit ToolStarted + ToolCompleted SSE events so the frontend renders
  proper tool cards with spinner → checkmark/error transitions
- Duration shown in parameters field (e.g., "42ms")

CodeAct visibility:
- "Executing code..." when assistant produces a code block
- "Code executed" / "Code executed (no output)" for successful runs
- "Code error — retrying..." when Monty raises an exception

Reflection:
- "Reflecting on execution..." when post-thread analysis starts
- "Reflection complete — N insight(s) saved" when done

Also refactored thread_event_to_app_event → thread_event_to_app_events
(returns Vec<AppEvent>) to support emitting ToolStarted before
ToolCompleted in a single event handler pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): resolve tool names as callable stubs in CodeAct runtime

When LLM-generated code calls `mission_list()` or any tool function,
Monty's Python execution model first resolves the name (`mission_list`)
as a NameLookup before invoking it as a FunctionCall. The NameLookup
handler always returned Undefined, causing NameError before the function
call could dispatch to the effect executor.

Fix: before starting the Monty VM, collect all known tool names from
the effect executor's available_actions(). In the NameLookup handler,
if the name matches a known tool, return a MontyObject::Function stub
instead of Undefined. Monty then yields FunctionCall for the stub,
which dispatches to the normal tool execution pipeline.

This enables CodeAct code to call any registered tool as a Python
function: mission_list(), mission_create(), routine_list(), web_search(),
memory_search(), etc. — all without explicit imports or __execute_action__
boilerplate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): consolidate action execution, remove reflection, add learning missions

Three major changes to the v2 engine:

1. **Consolidated action execution** — `handle_execute_action` in Rust is now
   the single source of truth for lease lookup, policy check, lease consumption,
   action execution, event emission, and ActionResult message recording. The
   Python orchestrator no longer duplicates event/message logic. This fixes the
   empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant
   messages (Codex "No tool call found" error).

2. **Removed reflection system** — Deleted the per-thread reflection pipeline
   (pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection,
   enable_reflection config, and all 3 reflection event kinds. Learning is now
   handled entirely by event-driven missions that fire selectively.

3. **Three learning missions** replace reflection:
   - `self-improvement` — fires on trace issues (error diagnosis, prompt fixes)
   - `playbook-extraction` — fires on successful 5+ step threads (reusable procedures)
   - `conversation-insights` — fires every 5 threads per project (user preferences,
     domain knowledge, workflow patterns)

Additional fixes:
- llm_query()/llm_query_batched() always include system message (Codex compat)
- handle_llm_complete adds assistant message with structured action_calls for
  Tier 0 responses (prevents "No tool call found" errors)
- Gateway broadcasts without thread_id emit as Status events instead of being dropped
- Comprehensive tests for call_id propagation and trace analysis (17 new tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): extract ironclaw_skills crate and integrate with v2 engine

Extract the skills system into a standalone `ironclaw_skills` crate
(following the ironclaw_safety pattern) and wire it into the v2 engine
for deterministic skill selection, CodeAct code injection, and
confidence tracking.

**ironclaw_skills crate** (94 tests):
- Core types: SkillManifest, ActivationCriteria, LoadedSkill, SkillTrust
- V2 types: V2SkillMetadata, CodeSnippet, SkillMetrics, V2SkillSource
- Deterministic 4-phase selector (gating→scoring→budget→attenuation)
- apply_confidence_factor() for extracted skill scoring
- SKILL.md parser, validation/escaping, gating, registry, catalog
- Feature-gated: catalog (reqwest), registry (filesystem)

**Engine integration** (14 new tests):
- DocType::Skill with retrieval weight 0.45
- SkillSelector bridges MemoryDoc→LoadedSkill for shared scoring
- SkillTracker for usage/version/rollback confidence tracking
- System prompt injection via <skill> XML blocks
- CodeAct snippet injection via Monty NameLookup
- Skill extraction mission replaces playbook extraction
- ThreadManager.set_skill_selector() for runtime wiring

**Bridge + migration**:
- skill_migration.rs: v1 SKILL.md → v2 MemoryDoc (idempotent)
- init_engine() migrates v1 skills, builds SkillSelector
- src/skills/mod.rs → re-export shim

**E2E test** (tests/engine_v2_skill_codeact.rs):
- Full CodeAct loop: skill selected → LLM returns Python code →
  Monty executes http() → mock returns canned GitHub JSON →
  FINAL() terminates → thread completes with canned data
- GitHub SKILL.md in skills/github/ as reference implementation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Documenting research around how to extend to more integrations

* docs: update engine-v2-architecture for missions and skills

- Replace "Reflection Pipeline" with "Learning Missions" (self-improvement,
  skill-extraction, conversation-insights)
- Add "Skills System" section covering ironclaw_skills crate, deterministic
  selection pipeline, CodeAct integration, confidence tracking, v1 migration
- Update MemoryDoc types table (add Skill, remove Playbook as primary)
- Update Integration Scaling section: Skills replace Capabilities-as-knowledge
  as the concrete implementation
- Update example from Capability YAML to SKILL.md format with credentials
- Fix thread state machine (remove Reflecting state)
- Update key files table and test counts
- Add self-improvement feedback loop diagram

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: clean up legacy playbook references in engine crate

- Rename PLAYBOOK_MIN_STEPS/ACTIONS → SKILL_EXTRACTION_MIN_STEPS/ACTIONS
- Fix pattern DB uses DocType::Note instead of DocType::Playbook
- Update CLAUDE.md: skill-extraction mission, DocType list, module map

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): credential specs in skill frontmatter, HTTP tool hardening, mission leases

Skills can now declare API credentials in YAML frontmatter (SkillCredentialSpec,
SkillCredentialLocation, SkillOAuthConfig, ProviderRefreshStrategy). Valid specs
are registered into SharedCredentialRegistry at startup; the HttpTool auto-injects
credentials for matching hosts — same zero-exposure model as WASM tools.

HTTP tool security hardening:
- Block LLM-provided auth headers for hosts with registered credentials
- Return structured authentication_required error for missing credentials
- Strip sensitive response headers (Set-Cookie, WWW-Authenticate, Authorization)
- Scan response body through LeakDetector before returning to LLM

Mission capability leases: registered mission_create/list/fire/pause/resume/delete
as a "missions" capability so threads receive leases. Removed routine_* aliases
from effect adapter — descriptions mention "routine" for LLM intent mapping.

Includes 10 integration tests (tests/skill_credential_injection.rs) covering
the full pipeline: YAML parsing → validation → registry → HttpTool wiring →
per-user isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(engine): remove legacy Playbook doc type, superseded by Skill

Drop DocType::Playbook variant and all references — playbook extraction
mission was already renamed to skill extraction in the previous session.
Updates CLAUDE.md, architecture docs, context builder, retrieval weights,
mission comments, and store adapter path mapping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): move skill selection and injection to Python orchestrator

Skill selection was in Rust (SkillSelector in loop_engine.rs) — now it's
in the Python orchestrator where the self-improvement mission can evolve it.

Rust provides data access via two new host functions:
- __list_skills__() — loads DocType::Skill MemoryDocs from Store
- __record_skill_usage__(doc_id, success) — confidence tracking

Python orchestrator handles everything else:
- score_skill() — keyword/tag/confidence scoring (~40 lines)
- select_skills() — budget-aware top-N selection (~15 lines)
- format_skills() — XML block injection into system prompt (~20 lines)
- Injection at step 0 with active_skill_ids stored in state

Removed from Rust:
- SkillSelector field + builder on ExecutionLoop and ThreadManager
- format_skills_section() from prompt.rs
- Rust-side skill injection block in loop_engine.rs
- SkillSelector wiring in bridge/router.rs

E2E test updated: skills stored in TestStore, Python orchestrator
finds them via __list_skills__() and injects based on goal keywords.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: annotate v1-only code for removal after migration

Mark modules and functions that exist solely for the v1 agent with
"remove after v1 migration" notes:

- src/skills/mod.rs ��� shim, attenuation, credential registration
- src/skills/attenuation.rs — trust-based tool filtering (v1 only)
- ironclaw_skills: selector, gating, registry, catalog modules
- ironclaw_engine: skill_selector.rs (superseded by Python orchestrator)
- src/bridge/skill_migration.rs — one-time v1→v2 conversion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(engine): remove unused skill_selector.rs

Rust-side skill selection was moved to the Python orchestrator in
7f87d179. This module had no production callers — only its own tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): compile-time skill bundling infrastructure

Add support for embedding skills into the binary at compile time:

- build.rs: embed_skills() collects skills/*/SKILL.md into embedded_skills.json
- src/skills/bundled.rs: loads embedded skills via include_str!
- SkillRegistry: with_bundled_content(), load_from_content(), step 4 in discover_all()
- Bundled skills are Trusted (ship with binary), lowest discovery priority
- 4 new tests for bundled loading, user override, gating, and removal rejection
- Cargo.toml: add serde_json build-dependency

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): non-blocking auth signal, NeedAuthentication flow, timeout safety

When the HTTP tool detects a missing credential for a registered host:
1. EffectBridgeAdapter emits SSE AuthRequired event (best-effort, for
   connected frontends — silently dropped for missions/background threads)
2. Error flows back to LLM as normal ActionResult (non-blocking)
3. LLM tells the user to authenticate

This avoids the blocking interruption approach which would hang mission
threads and sub-threads that have no channel context.

Engine additions:
- EngineError::NeedAuthentication variant for structured auth failures
- ThreadOutcome::NeedAuthentication for batch interruption when needed
- structured.rs handles NeedAuthentication by interrupting the batch
  (stops subsequent calls, returns outcome to orchestrator)
- Auth callback on EffectBridgeAdapter (optional, set by router for SSE)
- extract_credential_name parser for HTTP tool error messages
- routine_* tools added to is_v1_only_tool blocklist

Safety: added 5-minute timeout to await_thread_outcome to prevent
infinite hangs (e.g. after denied tool approval where thread fails
to resume).

Tests: 3 structured executor tests (NeedAuthentication interrupts batch,
stops subsequent calls, regular errors don't interrupt) + 7 effect
adapter tests (credential extraction, callback firing, v1-only tools).

Also adds Linear API skill (skills/linear/SKILL.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): platform self-awareness, event pipeline fix, globals() builtin, prompt templates

Session 9 changes driven by live trace analysis:

- CodeAct event pipeline: handle_execute_code_step now transfers
  CodeExecutionResult events to thread.events and broadcasts via event_tx
  (fixes false-positive no_tools_used trace warnings)
- Monty globals()/locals() builtins: returns dict of available action names
  from capability leases, enabling "tool_name" in globals() probing
- PlatformInfo injection into system prompts (version, LLM backend, model,
  database, channels, owner, repo URL)
- Mission goal prompts moved to prompts/*.md files (include_str! pattern)
- /expected command for triggering self-improvement from user feedback
- Session 9 development history

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): auto-approve http calls with registered credentials in v2

The v1 approval flow (interactive yes/no prompt) doesn't exist in v2.
When the http tool returned UnlessAutoApproved for credentialed hosts,
the effect adapter blocked with LeaseDenied — making all skill-based
API calls fail.

Fix: credential-backed http calls bypass the v1 approval check. The
user authorized by storing the credential; the v1 interactive prompt
is redundant in v2's lease-based security model.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(ui): show activated skills in CLI and gateway

End-to-end skill activation display:

1. Python orchestrator emits __emit_event__("skill_activated", skill_names=...)
   after select_skills() picks skills for the conversation
2. Rust host function parses the comma-separated names into EventKind::SkillActivated
3. Router forwards to channels as StatusUpdate::SkillActivated
4. REPL renders: ◈ skills: github, linear (cyan)
5. Web gateway emits AppEvent::SkillActivated SSE event for frontend display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): show auth prompt in REPL when credential is missing

The AuthRequired SSE event was emitted but only reached the web gateway.
The REPL never saw it because it receives events through
forward_event_to_channel which converts ThreadEvents to StatusUpdates.

Fix: when forward_event_to_channel sees an ActionFailed with
"authentication_required" in the error, emit StatusUpdate::AuthRequired
to the channel. Also add AuthRequired/AuthCompleted rendering to the
REPL (was missing — fell through to unmatched arm).

CLI now shows:
  ⚿ Authentication required: github_token
    Store the credential with: ironclaw secret set <name> <value>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(ui): show tool arguments in CLI and gateway

Add params_summary to ActionExecuted/ActionFailed events so the CLI
and gateway can display what tools are doing:

  ● http(https://api.github.com/repos/nearai/ironclaw/issues)
  ● web_search(latest AI news)
  ● memory_read(HEARTBEAT.md)

The summarize_params() helper extracts the most relevant argument
per tool type (URL for http, query for search, path for memory, etc.)
and truncates to 80 chars. Sensitive params are not included.

Router forwards the summary in both StatusUpdate (CLI/REPL) and
AppEvent (web gateway SSE) display names.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle Python None params in http tool, add params_summary to CodeAct dispatch

Two fixes from live testing:

1. http tool: treat null headers/body as empty (Python's None becomes
   JSON null via Monty). Previously headers=None errored with
   "'headers' must be an object or array of {name, value}".

2. scripting.rs: compute params_summary before dispatching actions in
   the CodeAct path (was always None). Now http calls show their URL
   in the CLI: ● http(https://api.github.com/repos/.../issues)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove glob re-exports, fix clippy warnings, clean up duplicates

- Remove `pub use ironclaw_safety::*` from src/safety/mod.rs and migrate
  all 20+ call sites to import directly from `ironclaw_safety`
- Remove `pub use ironclaw_skills::*` from src/skills/mod.rs and migrate
  all 15+ call sites to import directly from `ironclaw_skills`
- Fix 4 clippy warnings: 2 shadow imports, 2 collapsible if-let chains
- Add missing SkillActivated arm to WASM channel StatusUpdate match
- Remove duplicate AuthRequired/AuthCompleted arms in repl.rs
- Update CLAUDE.md extracted crates guidance and prompt template rule
- Fix bench imports (safety_check, safety_pipeline)

46 files changed, zero warnings, 3836 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): guided credential flow — prompt for token and retry

When a thread completes with authentication_required, the router
enters "auth mode" for that user:

1. Detects credential_name from the error in the thread response
2. Looks up setup_instructions from the skill's credential spec
3. Emits AuthRequired to CLI/gateway with instructions
4. Stores PendingAuth — next user message is treated as a token
5. Stores the token in SecretsStore
6. Retries the original user request automatically

CLI flow:
  › create an issue in github
    ⚿ Authentication required: github_token
      Create a PAT at https://github.com/settings/tokens
    Paste your token below (or type 'cancel'):
  › ghp_abc123...
    ✓ github_token authenticated: Credential stored. Retrying...
    ● http(https://api.github.com/repos/.../issues)
    Issue created: https://github.com/...

Gateway flow: same but AuthRequired SSE event shows the auth modal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): skill-based OAuth flow tests

6 E2E tests covering the full skill credential lifecycle via the
gateway API:

- test_github_skill_loaded: github skill with credential spec loaded
- test_no_github_token_initially: no stored secrets before auth
- test_http_tool_returns_auth_required: http tool signals missing cred
- test_guided_auth_flow: request → auth prompt → paste token → retry
- test_auth_required_sse_event: SSE stream includes auth/skill events
- test_different_users_isolated: per-user credential scoping

Includes mock API server (aiohttp) requiring Bearer auth with token
tracking for assertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: document Monty runtime limitations in CodeAct prompt, fix new-thread read-only

- Add "Runtime environment" section to codeact_preamble.md documenting
  Monty's restrictions: no stdlib imports, single imports only, no classes/
  with/match/del/yield, available builtins and modules, workarounds
- Add MONTY.md tracking current pin, all limitations, upgrade process,
  and changelog for future Monty updates
- Fix gateway createNewThread() not resetting read-only state — new
  threads now eagerly enable chat input instead of waiting for async
  loadThreads() callback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): transition thread to Waiting on NeedApproval

The orchestrator Python returned {"outcome": "need_approval"} without
calling __transition_to__("waiting"), leaving the thread in Running
state. When the user later approved/denied, resume_thread rejected it
with "thread is not resumable from Running".

- Add __transition_to__("waiting", "approval needed") in both code-step
  and action-call approval paths in default.py
- Add Rust safety net in loop_engine.rs: if orchestrator returns
  NeedApproval but thread isn't Waiting, force the transition

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): restructure workspace storage for human readability

Rewrite HybridStore (src/bridge/store_adapter.rs) to produce a
developer-friendly workspace layout:

- Knowledge docs use frontmatter+markdown with slugified filenames
  instead of UUID.json with wrapped structs
- Orchestrator code, prompt overlays, and failure tracker grouped
  under engine/orchestrator/
- Missions nested under their project in named folders with room
  for working files alongside mission.json
- Runtime state (threads, leases, events) under engine/.runtime/
- Terminal threads archived to compact summaries, dead leases cleaned
  on startup
- Auto-generated engine/README.md with knowledge counts, mission
  status, and thread stats

Also includes: /expected command, approval state fix, platform
self-awareness, Monty limitations in preamble, prompt template
extraction. See docs/development-history.md Session 10 for details.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): add commitments system — active intake for personal AI assistant

Introduces 9 SKILL.md files that implement a complete commitments tracking
system using existing workspace storage, routines, and tools — no Rust code
changes or new database tables required.

Core skills:
- commitment-setup: one-time bootstrap (workspace structure + routines)
- commitment-triage: in-conversation signal extraction and commitment management
- commitment-digest: periodic summary composition and delivery

Supporting skills:
- decision-capture: detect and record decisions with rationale
- delegation-tracker: track delegated items and generate follow-ups
- idea-parking: park, resurface, and promote ideas

Persona bundles:
- ceo-assistant: delegation-heavy, meeting prep, 3x/day triage, responsibility-grouped digests
- content-creator-assistant: content pipeline stages, 6h trend expiry, cross-platform cascades
- trader-assistant: position-aware scoring, contradictory signal detection, trade journaling

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix cargo fmt in bridge/router.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): add companion skill references and explicit workspace setup to persona bundles

Each persona setup skill now:
- Lists companion skills (commitment-triage, decision-capture, etc.) with
  activation triggers so the agent knows what activates during conversation
- Warns if companion skills are missing from the skills/ directory
- Includes explicit workspace creation steps (README schema, directory
  placeholders) instead of vague "same as commitment-setup" references
- Bumps max_context_tokens for content-creator and trader to fit the
  additional setup instructions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): enrich commitments with plan parity and add requires.skills support

Skills updates (v0.2.0):
- Switch all routine_create calls to mission_create (v2 engine primitive)
- Add immediacy (realtime/prompt/batch) and expires_at to signal schema
- Add resolution_path (agent_can_handle/needs_reply/needs_decision/note_only)
  and stale_after/resolved_by to commitment schema
- Add outcome tracking fields to decision schema
- Add intelligence destination — informational signals write durable MemoryDocs
- Add trust calibration section to README schema (start conservative)
- Digest now ends with "Did I miss anything?" to catch false negatives
- Agent offers to handle agent_can_handle items with explicit user approval

Rust changes:
- Add requires.skills field to GatingRequirements for skill dependency
  declarations. Missing skills produce warnings (not failures) so the
  skill still loads but the agent knows to surface missing dependencies.
- Log skill dependency warnings during load_and_validate_skill and
  load_from_content
- Add test: test_skill_dependencies_produce_warnings

Persona bundles now declare their companion skill dependencies via
metadata.openclaw.requires.skills in YAML frontmatter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): lift requires to top-level field, implement chain installation

Moves skill gating requirements from `metadata.openclaw.requires` to a
top-level `requires` field on `SkillManifest`. The legacy nested path
still works via `effective_requires()` merge — existing skills with the
old format load without changes.

Chain installation in `skill_install` tool:
- After installing a skill, checks `requires.skills` for dependencies
- Missing dependencies are auto-fetched from the catalog and installed
- Reports chain_installed, chain_install_failed, and missing_dependencies
  in the tool output so the agent can inform the user

Changes:
- types.rs: Add `requires: GatingRequirements` to SkillManifest,
  add `effective_requires()` merge method
- registry.rs: Use effective_requires() instead of nested metadata path
- skill_tools.rs: Chain-install missing skill dependencies after install
- cli/skills.rs: Display requires.skills in skill info output
- All SKILL.md files: Use top-level `requires.skills` instead of
  `metadata.openclaw.requires.skills`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): explicit /skill-name activation in messages

Users can now write /github or /file-issues anywhere in their message
to force-activate a skill. The /skill-name is replaced with the skill's
description so the sentence reads naturally for the LLM:

  "fetch issues from /github" → "fetch issues from GitHub API"
  "please /file-issues for all bugs" → "please file detailed GitHub issues for all bugs"

Implementation:
- extract_skill_mentions() in selector.rs scans for /name patterns,
  matches against available skills, returns matched skills + rewritten
  message
- select_active_skills() returns (skills, rewritten_message) — explicit
  mentions merged with score-based selection
- dispatcher.rs rewrites the last user message in LLM context with
  expanded text
- 8 tests covering: basic mention, description expansion, hyphenated
  names, multiple mentions, unknown skills, URLs not matched

Also includes: seed_orchestrator_v0() for workspace visibility of
compiled-in orchestrator code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): wire NeedAuthentication and NeedApproval through v2 CodeAct path

Production traces revealed tool result desync on RequireApproval (no
ActionResult message → OpenAI 400), auth flow not triggering in CodeAct
(EffectAdapter returned Ok instead of Err(NeedAuthentication)), and HTTP
tool blocking unauthenticated requests.

Fixes:
- Add emit_and_record() to RequireApproval branch in handle_execute_action
- Wire NeedAuthentication through scripting.rs DispatchResult, orchestrator
  host functions, default.py, loop_engine safety net
- Add EngineError::NeedApproval variant; effect adapter returns it instead
  of LeaseDenied for tools needing approval
- HTTP tool: inject-if-available (proceed without auth, error only on 401)
- HTTP_ALLOW_LOCALHOST env flag for E2E testing with mock servers
- host_matches_pattern supports port in pattern (127.0.0.1:8080 matches
  host_str() output 127.0.0.1)
- CodeAct postamble: error recovery guidance
- Orchestrator user_id from thread.metadata instead of hardcoded "orchestrator"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): v1/v2 history, approval routing, cancel cleanup

Multiple v2 engine bridge fixes discovered by E2E tests:

- Write response to v1 DB for ALL thread outcomes (not just Completed),
  so history API shows NeedApproval/NeedAuthentication responses
- Remove v1 thread_id hint from pending_approval lookup (v1/v2 use
  different UUID spaces)
- Add has_pending_auth() check in agent_loop: route "cancel"/"no" through
  handle_with_engine when PendingAuth is active (SubmissionParser parsed
  "cancel" as ApprovalResponse, bypassing auth flow)
- Add engine_thread_id to PendingAuth; stop_thread on cancel
- Write cancel response to v1 DB
- NeedAuthentication handler enters guided auth flow with setup hints

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): comprehensive v2 engine test suite (12 tests, 5 files)

E2E tests for the v2 engine covering auth flow, approval lifecycle,
error handling, and edge cases. Uses mock API servers with strict token
validation, dedicated ironclaw server fixtures per module, and the mock
LLM's tool call pattern system.

Tests:
- Auth flow: skill activation, NeedAuthentication → token → retry,
  credential persistence across threads, cancel during auth, empty
  token treated as cancel, special character injection safety
- Approval: approve yes (text-based), deny, always (persists across
  threads), prompt mentions tool name
- Error handling: max iterations (30 step limit), tool intent nudge
  (LLM recovery after "let me search")

Infrastructure:
- mock_llm.py: runtime-configurable github_api_url, tool call patterns
  for issues/loop/drive, canned responses for intent nudge
- HTTP_ALLOW_LOCALHOST=true + SECRETS_MASTER_KEY in fixtures
- Separate server instances for cancel tests (cancel contaminates
  conversation state)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Session 11 — E2E test suite + engine hardening

Documents 14 bugs found across two production traces and E2E test
execution, the test infrastructure design (mock servers, dedicated
fixtures, HTTP_ALLOW_LOCALHOST), and the architecture evolution from
trace analysis → code fix → test to prevent regression.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address PR review — TOCTOU race, unbounded loop, lock thrashing

Fixes three issues from code review on #1736:

1. TOCTOU race: re-check has() under the write lock before
   commit_install to handle concurrent installs
2. Unbounded loop: cap chain dependencies at 10 to bound
   total fetch time
3. Lock thrashing: batch has() checks and install_target_dir()
   under a single read lock before entering the fetch loop

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): kernel-level pre-flight auth gate for engine v2

Transform authentication from a reactive post-execution error to a
proactive pre-flight check. The EffectBridgeAdapter now checks
credentials BEFORE executing tool calls, preventing wasted HTTP
requests and 401 errors from reaching the LLM.

Key changes:
- New AuthManager (src/bridge/auth_manager.rs) centralizes credential
  checking, setup instruction lookup, and tool readiness queries
- Pre-flight auth gate in execute_action() checks SharedCredentialRegistry
  + SecretsStore before tool execution
- Post-install auth pipeline: after tool_install, kernel auto-checks
  readiness and initiates auth flow or appends setup instructions
- tool_auth and tool_activate filtered from v2 LLM tool list and
  blocked in execute_action() — auth is kernel-level in v2
- Text-based auth detection kept as defense-in-depth fallback with
  tracing when it fires
- Setup instruction lookup deduplicated via AuthManager
- ExtensionManager gains check_tool_auth_status_pub() for auth queries

Also fixes pre-existing DocType::Plan exhaustiveness errors in the
engine crate and re-exports PlanStepDto from ironclaw_common.

Includes 10 unit tests (AuthManager + is_v1_auth_tool) and 5 E2E tests
covering pre-flight blocking, auth-then-retry, credential persistence,
v1 auth tools hidden, and auth cancellation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Session 12 — kernel-level auth rework decisions and rationale

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(plan): autonomous plan mode via v2 primitives (MemoryDoc, Mission, SSE)

Add plan mode for autonomous long-running task execution, composing
existing v2 engine primitives rather than new engine states. Inspired
by OpenAI Codex's update_plan checklist and Claude Code's file-based
plan mode — both enforce planning through prompts, not tool removal.

Engine: DocType::Plan variant for MemoryDoc (project-scoped, retrievable).
Events: PlanUpdate SSE event with PlanStepDto for live checklist rendering.
Tool: plan_update tool broadcasts structured plan progress via SSE.
Command: /plan (create/approve/status/revise/list) rewrites to UserInput
  with [PLAN MODE] prefix to activate the plan-mode skill.
Skill: skills/plan-mode/SKILL.md defines full plan protocol — creation
  (memory_write), approval (mission_create + mission_fire), execution
  (step-by-step with plan_update), and revision flows.
UI: Inline chat checklist widget with status badges, step icons
  (checkmark/spinner/circle), results, and progress summary.
Tests: 5 E2E scenarios + mock LLM patterns + helper selectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): add developer bundle, generalize workflow orchestrator, add tech debt tracker

New skills:
- developer-assistant: multi-repo setup, dev-tuned missions (morning brief,
  stale PR check, weekly retro, tech debt resurface, decision outcome check)
- github-workflow: generalized from ironclaw-workflow-orchestrator, works for
  any GitHub repo with parameterized templates
- project-setup: "add repo owner/repo" creates workspace project entity and
  installs per-repo workflow missions
- tech-debt-tracker: passive detection from conversation + PR review comment
  scanning, severity/category classification, promote-to-commitment flow

Removed:
- ironclaw-workflow-orchestrator: replaced entirely by github-workflow

Updated all persona bundles:
- Remove timezone questions (timezone comes from channel automatically)
- CEO, content-creator, trader skills updated

Projects are now first-class workspace entities under projects/<owner>-<repo>/
with metadata (maintainers, branches, AI agent authors, workflow status).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(skills): integrate v2-architecture features into commitment skills

Update skills to reference capabilities from the v2-architecture merge:

- /plan command: developer-assistant retro and tech-debt missions now
  suggest "/plan <description>" for complex items. Tech-debt-tracker
  promotion flow suggests /plan for multi-step refactors.
- /skill-name activation: developer-assistant confirm message lists
  /commitment-digest, /tech-debt-tracker as quick commands
- CEO confirm message mentions /plan for complex initiatives

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): add security, QA, review readiness, and product prioritization skills

Inspired by gstack's specialist role model and process completeness tracking.

New skills:
- security-review: OWASP-based security audit with fix-first model (auto-fix
  obvious issues, escalate ambiguous ones), health scoring, FP tracking
- qa-review: test coverage analysis, edge case identification, test plan
  generation, regression risk assessment with health scoring
- review-readiness: PR readiness dashboard tracking which reviews are complete
  per branch (code review, tests, security, QA, linting) with merge gating
- product-prioritization: evidence-based feature scoring with anti-sycophantic
  forcing questions, dual effort estimates (human vs AI-assisted time),
  demand×3 + impact×2 + alignment / effort scoring, user feedback analysis

Updated commitment schema:
- decision_type: mechanical (auto-act) | taste (auto-act, surface) | challenge (always ask)
- effort_human / effort_assisted: dual time estimates
- Autonomous resolution now respects decision_type classification

Updated developer-assistant bundle:
- Requires all 4 new skills
- Calibration includes decision classification framework, effort compression
  principle ("AI makes completeness cheap"), review readiness tracking
- Confirm message lists /security-review, /qa-review, /product-prioritization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(skills): flatten requires to top-level SkillManifest field

Remove the metadata.openclaw.requires nesting from staging in favor of
the flat manifest.requires path chosen on this branch. Drop SkillMetadata
and OpenClawMeta wrapper types. Update all call sites, test YAML, and
re-exports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address PR #1736 review feedback

- Delete dead-code duplicate capability/skill_tracker.rs (canonical
  version lives in memory/skill_tracker.rs; this one was never wired
  into capability/mod.rs)
- Delete dead-code executor/intent.rs (never wired into executor/mod.rs)
- Correct record_usage docstring to match actual behavior (returns Err
  on missing doc/invalid metadata; does not log-and-swallow)
- Remove unconditional companion-skill warnings from gating; the
  registry layer owns missing-skill detection since gating has no
  visibility into installed skills
- Chain-install: surface MAX_CHAIN_DEPS overflow as
  skipped_dependencies instead of silently dropping deps >10
- Chain-install: distinguish transient fetch errors (network/5xx) from
  genuine catalog misses (HTTP 404/410); transient errors now surface
  under chain_install_failed with the error message
- trader-assistant SKILL: renumber config question list (4→5, was 4→6)
- trader-assistant SKILL: correct trade-journal README path from
  bare decisions/ to commitments/decisions/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(test): live/replay tests for commitment persona bundles

Add three end-to-end persona tests (CEO, content creator, trader) that
exercise the full skill activation -> workspace setup flow against a
real LLM with recording, and replay deterministically from committed
trace fixtures.

### Engine fixes exposed by the tests

- **Engine v2 skill scoring missed regex patterns.** The Python
  orchestrator's `score_skill` only considered keywords and tags, while
  the v1 selector (and every SKILL.md in the repo) relies on regex
  patterns like `(?i)I'm a (CEO|manager|...)` for the strongest
  activation signal. Persona bundles that matched via patterns alone
  scored 0 and never activated. Added `__regex_match__` host function
  (backed by the `regex` crate) and regex pattern scoring (20 pts per
  match, cap 40) to `select_skills`, matching v1 parity.

### Test infrastructure

- `TestRigBuilder::with_skills_dir(dir)` to load real SKILL.md files
  from the repo's `skills/` directory instead of an empty tempdir. The
  rig now preserves AppBuilder's skill registry rather than blindly
  overwriting it with an empty one when skills are configured.
- Test channel user_id now matches `config.owner_id` so engine v2
  resolves the thread to the owner's default project (where skills
  are migrated) instead of creating a fresh per-user project with no
  skills.
- `TestRig::loaded_skill_names()` and `active_skill_names()` for
  asserting on the selection pipeline.
- `LiveTestHarnessBuilder::with_skills_dir()` wires the above into the
  live harness.
- `hydrate_llm_secrets_into_env()` in the live harness decrypts
  `llm_nearai_api_key` / `llm_anthropic_api_key` / `llm_openai_api_key`
  from the user's real secrets store and exports them as env vars
  before the provider chain is built, bypassing NEAR AI's interactive
  OAuth prompt in test mode.

### Committed fixtures

Each persona test has a recorded trace (`.json`) and a human-readable
session log (`.log`) under `tests/fixtures/llm_traces/live/`. Live mode
(`IRONCLAW_LIVE_TEST=1`) hits the configured LLM and refreshes these;
replay mode (default) plays them back deterministically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): repair stale unit tests against orchestrator-driven flow

Five engine unit tests had been broken since v2 landed but went unnoticed
because they exercise the legacy ExecutionLoop path. The tests encoded
assumptions that no longer hold:

- `loop_engine::action_then_text`, `tool_intent_nudge_injected`, and
  `codeact_multi_step` checked `thread.messages` for the orchestrator's
  working transcript. The Python orchestrator now persists working
  messages into `thread.internal_messages` (via `sync_runtime_state`),
  while `thread.messages` only carries the system prompt and the final
  assistant response. Tests now check `internal_messages`.

- `executor::trace::trace_serializes_approval_request_payload` indexed
  `trace.events[0]` for the manually-pushed `ApprovalRequested` event,
  but `Thread::add_message` now records a `MessageAdded` event for each
  call, so the approval lives further down the list. Find by kind
  instead. Also relax the parameter-map serialization assertion to
  check key presence rather than exact serialized form.

- `mission::system_mission_requires_system_user_to_manage` asserted the
  engine rejects non-owner pause/resume on shared missions. The current
  security model is intentionally permissive at the engine level — the
  web handler enforces admin role before forwarding the call (per the
  `pause_mission` doc comment). Test renamed and rewritten to pin down
  the contract: shared missions accept any user at the engine layer.

All 267 ironclaw_engine unit tests now pass. No production code change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(personas): multi-turn workflow tests + strict trace error checks

Replaces the shallow setup-only persona tests with multi-turn workflows
that exercise the actual commitment system: setup → real-world capture →
workspace verification. Each test reads the temp workspace via the rig
to confirm that meeting outcomes, content publications, and market
signals actually land as files in `commitments/` with the right names.

### New scenarios

- **CEO**: setup → "Sarah delivers Q2 budget by Friday, Bob drafts term
  sheet by Tuesday" → verify both commitments captured under
  commitments/open/ with assignee + deadline references.
- **Content creator**: setup → "Just published episode 47, need TikTok
  cuts and Twitter thread by tomorrow, also park an idea about
  debugging legacy code" → verify distribution commitments and parked
  idea files exist.
- **Trader**: setup with positions inline → "AAPL/TSMC partnership
  signal + closed SPY puts at $4.20" → verify signal file in
  signals/pending/ AND decision in decisions/.

### Strict trace error detection

The `LiveTestHarness` now exposes `finish_strict()` which scans the
captured status events and fails the test if any non-benign tool
failure is found. Three categories of "noise" are explicitly allowed
and bypass the strict check:

- **Workspace probing**: `Document not found` from memory_read on a
  file the agent is checking before writing.
- **Wrong tool selection**: write_file → memory_write redirects.
- **Wrong patch params**: memory_write missing new_string with old_string.

Engine bugs (Python SyntaxError, missing leases, registry 404s for
locally-loaded skills) still fail the test.

### Engine bugs caught and fixed by the strict check

1. **Effect adapters wrap tool errors as `Ok(ActionResult { is_error: true })`
   but the engine emitted `ActionExecuted` regardless.** Three call sites
   in `scripting.rs`, `orchestrator.rs`, and `structured.rs` now inspect
   `is_error` and emit `ActionFailed` accordingly so traces, observers,
   and approval flows see failures correctly.

2. **CodeAct SyntaxErrors were silent.** When `__execute_code_step__`
   returns `had_error: true` (e.g. Monty Python failed to parse the
   snippet), the orchestrator now emits an `ActionFailed` event with
   action_name `__codeact__` so the failure surfaces in traces.

3. **`extract_code_block` matched bare ``` blocks containing markdown.**
   The LLM emitted ``` blocks of markdown lists as examples, the
   bridge's heuristic extracted them as Python, and Monty crashed with
   SyntaxError on `- TICKER: SIZE, ...`. Added a `looks_like_python`
   heuristic that requires bare blocks to contain Python tokens
   (assignments, function calls, keywords, comments) — markdown lists,
   tables, blockquotes, prose, and numbered lists are now rejected.
   Five new unit tests pin the heuristic.

4. **`FINAL` emitted as a structured tool call has no lease.** The agent
   sometimes calls FINAL via tool_calls instead of inside a CodeAct
   block. The orchestrator's Python loop now intercepts any tool call
   named `FINAL` before dispatching to `__execute_actions_parallel__`
   and treats it as a completion signal with the answer pulled from
   `params.answer/result/content/text` (or the response content).

5. **`skill_install` for an already-loaded skill 404s against the
   registry.** Local SKILL.md files aren't published to ClawHub, so any
   agent that mis-interprets an active persona bundle as "needs to be
   installed" gets a 404. `skill_install` is now idempotent: if the
   registry already has a skill with the requested name, return success
   immediately without hitting the catalog.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Expand live persona commitment workflows

* fix skills install and CI regressions

* style live harness formatting

* fix(skills): address PR #1736 review feedback

- skill_tools chain install: check `guard.has(&dep_name)` before applying
  the MAX_CHAIN_DEPS cap so already-installed deps don't get reported under
  `skipped_dependencies`.
- skill_tools chain install: when a concurrent install wins the race after
  we've already written the SKILL.md to disk, clean up the orphan dir
  instead of leaving it on the filesystem. Reworked the commit block so the
  non-Send RwLockWriteGuard is dropped before the cleanup `.await`.
- gating: drop the always-empty `GatingResult.warnings` field — the
  registry layer is the right place to surface missing companion-skill
  warnings, and the field promised behavior that was never implemented.
- parser: detect the legacy `metadata.openclaw.requires` SKILL.md
  frontmatter shape and emit a `tracing::warn!` so authors don't silently
  lose their gating/dep config when serde drops the unknown nested keys.
- default.py orchestrator: filter FINAL out of `action_calls` passed to
  `append_message` so the message history doesn't record a FINAL action
  with no matching ActionResult (would confuse context replay on resume).
- default.py orchestrator FINAL handler: handle string `params`, add a
  `value` fallback (common LLM pattern), and emit a `final_fallback`
  trace event when falling back to `response.content` so the ambiguity
  is visible in traces.
- live_harness: fix misleading "read-only" docstring on the libsql DB
  open path — libsql doesn't expose RO mode here; document the actual
  guarantee (only `get_decrypted` is called).

Adds a regression test for legacy `metadata.openclaw.requires` detection
in `crates/ironclaw_skills/src/parser.rs`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(orchestrator): drop duplicate FINAL tool calls

Follow-up to a947fd0d for PR #1736 review comment 3053486977. The
previous draft would let a second FINAL call fall through into
`executable_calls` and try to run as a normal action (failing with a
lease error). Now any FINAL beyond the first is dropped via `continue`
so duplicate FINAL emissions degrade gracefully.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills,orchestrator): address PR #1736 follow-up review feedback

- skill_tools `already_installed` payload: include `trust: "installed"` so
  the response shape is consistent with the success / chain-install paths.
- skill_tools chain install: recover from `RwLock` poisoning via two
  helper wrappers (`registry_read` / `registry_write`) that log loudly
  but `into_inner()` instead of permanently bricking `skill_install`.
  The registry holds replace-on-success state, so recovery is safe.
- skill_tools `skipped_dependencies_message`: clarify that `MAX_CHAIN_DEPS`
  is an *attempt* cap (intentional bound on fetch time from large or
  malicious manifests), not a success cap, and tell the caller to retry
  via a follow-up `skill_install`.
- llm_adapter `looks_like_python`: tighten the function-call heuristic so
  it requires an identifier-style char immediately before `(`. Fixes the
  false positive where markdown links `[text](url)` and prose like
  "See (docs)" inside a bare ``` block were forwarded to Monty as code.
  Adds regression test `bare_backtick_markdown_link_is_rejected`.
- ironclaw_engine `__regex_match__`: document the ReDoS-safety reliance
  on the default `regex` crate's linear-time matching guarantee, with
  a top-of-crate comment warning future maintainers not to add
  `fancy-regex` to the dep tree without a wall-clock budget.
- default.py FINAL handling: emit `duplicate_final_dropped` event when
  more than one FINAL call is co-emitted, and truncate the
  `response.content` fallback to 500 chars (with an explanatory suffix
  and event metadata) so a model dumping its full reasoning into a
  paramless FINAL doesn't ship thousands of tokens as the answer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): address PR #1736 review (gating fast-path, skills cap, round-trip test)

- gating::check_requirements: fast-path return when `bins`/`env`/`config`
  are all empty, avoiding a `spawn_blocking` + `which` subprocess call
  per skill load for skills with no subprocess-checkable requirements
  (the common case).
- types: cap `requires.skills` at `MAX_REQUIRED_SKILLS_PER_MANIFEST = 10`
  via a new `GatingRequirements::enforce_limits()` method, mirroring the
  host-side `MAX_CHAIN_DEPS` so a hostile manifest can't cause unbounded
  queue growth in the chain installer before the downstream cap kicks in.
  Parser calls `enforce_limits()` alongside `activation.enforce_limits()`.
  Regression test: `test_requires_skills_is_capped_at_parse_time`.
- skill_tools: add a round-trip integration test exercising the
  `install_dependencies=false` → `install_dependencies=true` flow via
  `install_missing_skill_dependencies`, verifying the pending deps are
  actually picked up on the second call (PR #1736 review 3058525543).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills,orchestrator): address PR #1736 security review

- skill_tools chain install: stop enqueueing nested deps once
  `attempted >= MAX_CHAIN_DEPS`, plus a hard `MAX_CHAIN_QUEUE = 100`
  belt-and-braces cap. Prevents unbounded BFS queue growth from a
  hostile manifest with deep `requires.skills` fan-out.
- skill_tools: override `SkillInstallTool::requires_approval` to return
  `Always` when `install_dependencies=true`, forcing a per-call
  approval prompt for chain installs. Single-skill installs retain
  `UnlessAutoApproved`.
- skill_tools main install path: close the TOCTOU window between the
  pre-write `guard.has()` check (released read lock + async
  `prepare_install_to_disk`) and the `commit_install` write. Re-check
  under the write lock and return the idempotent `already_installed`
  response on the race (cleaning up the orphan on-disk copy first).
- skill_tools chain installer: dependency-confusion guard — reject
  (and clean up) when `prepare_install_to_disk` returns a manifest
  name that does not match the requested `dep_name`. Blocks a
  hostile catalog entry from publishing a skill named "dep-a" whose
  manifest declares `name: evil-skill`.
- orchestrator `handle_regex_match`: add `dfa_size_limit(MAX_REGEX_SIZE)`
  alongside `size_limit`. Fixes the doc comment that referenced a
  `compile_error!` guard I'd removed in a previous commit.
- orchestrator tests: new `regex_match_host_function_is_callable_from_monty`
  regression test confirming Monty's NameLookup + FunctionCall dispatch
  actually reaches `handle_regex_match`. Teaches `eval_python_bool` to
  route `__regex_match__` calls through the real handler instead of
  stubbing to `None`.
- ironclaw_skills registry: after `discover_all()` completes, walk
  every loaded skill and emit a `tracing::warn!` for each
  `requires.skills` entry not present in the registry. Workspace-
  discovered bundle skills (e.g., `ceo-assistant`) no longer silently
  degrade when their companion skills aren't installed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills,orchestrator,tests): address PR #1736 review (limits, ids, hydration)

- skills/commitment-triage: trim activation lists to fit parser caps
  (`MAX_KEYWORDS_PER_SKILL = 20`, `MAX_PATTERNS_PER_SKILL = 5`). Previously
  declared 24 keywords + 11 patterns; the overflow was silently dropped at
  parse time, breaking some intended activations. Added a comment naming
  the limits so future edits stay in budget.
- skills/developer-assistant: trim `requires.skills` from 13 to the cap
  of 10 (`MAX_REQUIRED_SKILLS_PER_MANIFEST`). Drops `qa-review`,
  `review-readiness`, and `product-prioritization` — they remain
  available via manual `skill_install`.
- orchestrator CodeAct ActionFailed event: stop emitting empty `call_id`,
  which trips the `loop_engine.rs:1277` non-empty assertion. Use a
  synthetic `format!("codeact-step-{}", step_id.0)` so trace correlation
  still works for snippet failures that have no LLM-provided call_id.
- live_harness `hydrate_llm_secrets_into_env`: stop hardcoding
  `owner_id = "default"`. Read `IRONCLAW_OWNER_ID` first, fall back to
  the legacy `"default"` scope only when the env var is unset/empty.
  Users with a non-default owner scope no longer get forced into
  interactive auth despite having seeded secrets in the real DB.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(orchestrator,skills): address PR #1736 paranoid-architect review

- default.py: hoist `CHARS_PER_TOKEN` and `MESSAGE_OVERHEAD_CHARS` above
  the `estimate_context_tokens` definition (and the `FINAL(result)`
  entry-point call). The constants were previously defined at lines
  846-847 — *after* the entry point — so any execution path that ran
  `compact_if_needed → estimate_context_tokens` would NameError. The
  bug was latent because `enable_compaction` defaults to false in
  CI; this commit removes the dead-zone landmine.
- ironclaw_skills validation: add `validate_skill_version` enforcing a
  semver-ish character class (`[a-zA-Z0-9._\-+~]{1,32}`) and reject
  hostile values at parse time. Wired into `parser.rs` via a new
  `SkillParseError::InvalidVersion` variant. Closes the XML attribute
  injection vector through `format_skills` in default.py, which
  interpolates `version` directly into `<skill version="...">` and
  was the only field without character-class validation.
  Regression test: `test_parser_rejects_xml_breakout_in_version`.
- orchestrator handle_execute_action + handle_execute_actions_parallel:
  replace the non-atomic `find_lease_for_action` + `consume_use` pair
  with the atomic `find_and_consume` after the policy check passes.
  Mirrors `structured.rs::execute_action_batch_with_results` and
  closes the TOCTOU window where two concurrent calls could each
  observe a one-use lease and both proceed to execute.
- skill_credential_injection: add the three missing security tests
  (#6 LLM auth header rejection on credentialed host, #7 non-auth
  header passthrough on credentialed host, #8 LLM auth header
  passthrough on unregistered host) that were listed in the file
  doc comment but never implemented. Tests drive `HttpTool::execute`
  directly so they exercise the actual rejection branch in
  `http.rs:503-520`, not just the upstream `requires_approval` gate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 00:06:11 +09:00
Illia Polosukhin
4c9a985bac feat(engine): Unified Thread-Capability-CodeAct execution engine (v2 architecture) (#1557) 
* v2 architecture phase 1

* feat(engine): Phase 2 — execution loop, capability system, thread runtime

Add the core execution engine to ironclaw_engine crate:

- CapabilityRegistry: register/get/list capabilities and actions
- LeaseManager: async lease lifecycle (grant, check, consume, revoke, expire)
- PolicyEngine: deterministic effect-level allow/deny/approve
- ThreadTree: parent-child relationship tracking
- ThreadSignal/ThreadOutcome: inter-thread messaging via mpsc
- ThreadManager: spawn threads as tokio tasks, stop, inject messages, join
- ExecutionLoop: core loop replacing run_agentic_loop() with signals,
  context building, LLM calls, action execution, and event recording
- Structured executor (Tier 0): lease lookup → policy check → effect execution
- Tool intent nudge detection
- MemoryStore + RetrievalEngine stubs for Phase 4
- Full 8-phase architecture plan in docs/plans/
- CLAUDE.md spec for the engine crate

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 3 — Monty Python executor with RLM pattern

Add CodeAct execution (Tier 1) using the Monty embedded Python
interpreter, following the Recursive Language Model (RLM) pattern
from arXiv:2512.24601.

Key additions:
- executor/scripting.rs: Monty integration with FunctionCall-based
  tool dispatch, catch_unwind panic safety, resource limits (30s,
  64MB, 1M allocs)
- LlmResponse::Code variant + ExecutionTier::Scripting
- Context-as-variables (RLM 3.4): thread messages, goal, step_number,
  previous_results injected as Python variables — LLM context stays
  lean while code accesses data selectively
- llm_query(prompt, context) (RLM 3.5): recursive subagent calls
  from within Python code — results stored as variables, not injected
  into parent's attention window (symbolic composition)
- Compact output metadata between code steps instead of full stdout
- MontyObject ↔ serde_json::Value bidirectional conversion
- Updated architecture plan with RLM design principles

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): RLM best-practices enhancements from cross-reference analysis

Cross-referenced our implementation against the official RLM (alexzhang13/rlm),
fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation.
Key enhancements:

- FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching
  all three reference implementations. Code can signal completion at any
  point, not just via return value.
- llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn,
  matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch.
- Output truncation increased to 8000 chars (from 120), matching Prime
  Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT].
- Step 0 orientation preamble: auto-injects context metadata (message count,
  total chars, goal, last user message preview) before first code step,
  matching fast-rlm's auto-print pattern.
- Error-to-LLM flow: Python parse errors, runtime errors, NameErrors,
  OS errors, and async errors now flow back as stdout content instead of
  terminating the step, enabling LLM self-correction on next iteration.
  Only VM panics (catch_unwind) terminate as EngineError.

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): update architecture plan with RLM cross-reference learnings

Comprehensive update after cross-referencing against official RLM
(alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), Prime Intellect
(verifiers/RLMEnv), rlm-rs (zircote/rlm-rs), and Google ADK RLM.

Changes:
- Mark Phases 1-3 as DONE with commit refs and test counts
- Add "Key Influences" section documenting all reference implementations
- Phase 3: full table of implemented RLM features with sources
- Phase 3: "Remaining gaps" table with which phase addresses each
- Phase 4: expanded with compaction (85% context), rlm_query() (full
  recursive sub-agent), dual model routing, budget controls (USD,
  timeout, tokens, consecutive errors), lazy loading, pass-by-reference
- Add "RLM Execution Model" cross-cutting section
- Add "Implementation Progress" tracking table
- Remove stale "TO IMPLEMENT" markers (all Phase 3 work is done)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 4 — budget controls, compaction, reflection pipeline

Budget enforcement in ExecutionLoop:
- max_tokens_total: cumulative token limit, checked before each iteration
- max_duration: wall-clock timeout for entire thread
- max_consecutive_errors: consecutive error steps threshold (resets on
  success, matching official RLM behavior)
- All produce ThreadOutcome::Failed with descriptive messages

Context compaction (from RLM paper, 85% threshold):
- estimate_tokens(): char-based estimation (chars/4, matching RLM)
- should_compact(): triggers when tokens >= threshold_pct * context_limit
- compact_messages(): asks LLM to summarize progress, replaces history
  with [system, summary, continuation_note], preserves intermediate results
- Configurable via ThreadConfig: model_context_limit, compaction_threshold

Dual model routing:
- LlmCallConfig gains depth field (0=root, 1+=sub-call)
- Implementations can route to cheaper models for sub-calls
- ExecutionLoop passes thread depth to every LLM call

Reflection pipeline (reflection/pipeline.rs):
- reflect(thread, llm): analyzes completed thread via LLM
- Produces Summary doc (always), Lesson doc (if errors), Issue doc (if failed)
- Builds transcript from thread messages + error events
- Returns ReflectionResult with docs + token usage

ThreadConfig extended with: max_tokens_total, max_consecutive_errors,
model_context_limit, enable_compaction, compaction_threshold, depth, max_depth.

78 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 5 — conversation surface separated from execution

Conversation is now a UI layer, not an execution boundary. Multiple
threads can run concurrently within one conversation; threads can
outlive their originating conversation.

New types (types/conversation.rs):
- ConversationSurface: channel + user + entries + active_threads
- ConversationEntry: sender (User/Agent/System) + content + origin_thread_id
- ConversationId, EntryId (UUID newtypes)
- EntrySender enum (User, Agent{thread_id}, System)

ConversationManager (runtime/conversation.rs):
- get_or_create_conversation(channel, user) — indexed by (channel, user)
- handle_user_message() — injects into active foreground thread or spawns new
- record_thread_outcome() — adds agent/system entries, untracks completed threads
- get_conversation(), list_conversations()

This enables the key architectural insight: a user can ask "what's the
weather?" while a deployment thread is still running. Both produce entries
in the same conversation.

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): simplify execution tiers — Monty-only for CodeAct/RLM

Restructure phases 6-8 to clarify execution model:

- Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker
  Python runtimes for LLM-generated code.
- WASM sandbox is for third-party tool isolation (existing infra, Phase 8)
- Docker containers are for thread-level isolation of high-risk work (Phase 8)
- Two-phase commit moves to Phase 6 (integration) at the adapter boundary

Phase renumbering:
- Old Phase 6 (Tier 2-3) → removed as separate phase
- Old Phase 7 (integration) → Phase 6
- Old Phase 8 (cleanup) → Phase 7
- New Phase 8: WASM tools + Docker thread isolation (infra integration)

Updated progress table: Phases 1-5 marked DONE with test counts and commits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): Phase 6 — bridge adapters for main crate integration

Strategy C parallel deployment: when ENGINE_V2=true env var is set,
user messages route through the engine instead of the existing agentic
loop. All existing behavior is unchanged when the flag is off.

Bridge module (src/bridge/):
- LlmBridgeAdapter: wraps LlmProvider as engine LlmBackend, converts
  ThreadMessage↔ChatMessage, ActionDef↔ToolDefinition, depth-based
  model routing (primary vs cheap_llm)
- EffectBridgeAdapter: wraps ToolRegistry+SafetyLayer as EffectExecutor,
  routes tool calls through existing execute_tool_with_safety pipeline
- InMemoryStore: HashMap-backed Store impl (no DB tables needed yet)
- EngineRouter: is_engine_v2_enabled() + handle_with_engine() that
  builds engine from Agent deps and processes messages end-to-end

Integration touchpoint (4 lines in agent_loop.rs):
  After hook processing, before session resolution, check ENGINE_V2
  flag and route UserInput through the engine path.

Accessor visibility widened: llm(), cheap_llm(), safety(), tools()
changed from pub(super) to pub(crate) for bridge access.

85 engine tests + main crate clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): add user message and system prompt to thread before execution

The ExecutionLoop was sending empty messages to the LLM because the
thread was spawned with the user's input as the goal but no messages.

Fixes:
- ThreadManager.spawn_thread() now adds the goal as an initial user
  message before starting the execution loop
- ExecutionLoop.run() injects a default system prompt if none exists

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): match existing LLM request format to prevent 400 errors

The LLM bridge was missing several defaults that the existing
Reasoning.respond_with_tools() sets:

- tool_choice: "auto" when tools are present (required by some providers)
- max_tokens: 4096 (default)
- temperature: 0.7 (default)
- When no tools (force_text): use plain complete() instead of
  complete_with_tools() with empty tools array — matches existing
  no-tools fallback path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): persist conversation context across messages

The engine was creating a fresh ThreadManager and InMemoryStore per
message, losing all context between turns. A follow-up question like
"what are the latest 10 issues?" had no memory of the prior "how many
issues" response.

Fixes:
- EngineState (ThreadManager, ConversationManager, InMemoryStore) now
  persists across messages via OnceLock, initialized on first use
- ConversationManager builds message history from prior conversation
  entries (user messages + agent responses) and passes it to new threads
- ThreadManager.spawn_thread_with_history() accepts initial_messages
  that are prepended before the current user message
- System notifications (thread started/completed) are filtered out of
  the history (not useful as LLM context)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): enable CodeAct/RLM mode with code block detection

The engine now operates in CodeAct/RLM mode:

System prompt (executor/prompt.rs):
- Instructs LLM to write Python in ```repl fenced blocks
- Documents available tools as callable Python functions
- Documents llm_query(), llm_query_batched(), FINAL()
- Documents context variables (context, goal, step_number, previous_results)
- Strategy guidance: examine context, break into steps, use tools, call FINAL()

Code block detection (bridge/llm_adapter.rs):
- extract_code_block() scans LLM text responses for ```repl or ```python blocks
- When detected, returns LlmResponse::Code instead of LlmResponse::Text
- The ExecutionLoop routes Code responses through Monty for execution

No structured tool definitions sent to LLM:
- Tools are described in the system prompt as Python functions
- The LLM call sends empty actions array, forcing text-mode responses
- This ensures the LLM writes code blocks (CodeAct) instead of
  structured tool calls (which would bypass the REPL)

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): add 8 CodeAct/RLM E2E tests with mock LLM

Comprehensive test coverage for the Monty Python execution path:

- codeact_simple_final: Python code calls FINAL('answer') → thread completes
- codeact_tool_call_then_final: code calls test_tool() → FunctionCall
  suspends VM → MockEffects returns result → code resumes → FINAL()
- codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15')
  with no tool calls — pure Python in Monty
- codeact_multi_step: first step prints output (no FINAL), second step
  sees output metadata and calls FINAL — tests iterative REPL flow
- codeact_error_recovery: first step has NameError → error flows to LLM
  as stdout → second step recovers with FINAL — tests error transparency
- codeact_context_variables_available: code accesses `goal` and `context`
  variables injected by the RLM context builder
- codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times
  → 3 FunctionCall suspensions → all results collected → FINAL
- codeact_llm_query_recursive: code calls llm_query('prompt') → VM
  suspends → MockLlm provides sub-agent response → result returned as
  Python string variable

93 tests passing (85 prior + 8 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): detect code blocks in plain completion path + multi-block support

Two bugs fixed:

1. The no-tools completion path (used by CodeAct since we send empty
   actions) returned LlmResponse::Text without checking for code blocks.
   Code blocks were rendered as markdown text instead of being executed.

2. extract_code_block now:
   - Handles bare ``` fences (skips non-Python languages)
   - Collects ALL code blocks in the response and concatenates them
     (models often split code across multiple blocks with explanation)
   - Tries markers in order: ```repl, ```python, ```py, then bare ```

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(bridge): add 11 regression tests for code block extraction

Covers the exact failure modes discovered during live testing:

- extract_repl_block: standard ```repl fenced block
- extract_python_block: ```python marker
- extract_py_block: ```py shorthand
- extract_bare_backtick_block: bare ``` with Python content
- skip_non_python_language: ```json should NOT be extracted
- no_code_blocks_returns_none: plain text, no fences
- multiple_code_blocks_concatenated: two ```repl blocks with
  explanation between them → concatenated with \n\n
- mixed_thinking_and_code: model outputs explanation + two
  ```python blocks (the Hyperliquid case) → both extracted
- repl_preferred_over_bare: ```repl takes priority over bare ```
- empty_code_block_skipped: empty fenced block returns None
- unclosed_block_returns_none: no closing ``` returns None

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): detect FINAL() in text responses + regression tests

Models sometimes write FINAL() outside code blocks — as plain text
after an explanation. The Hyperliquid case: model outputs a long
analysis then FINAL("""...""") at the end, not inside ```repl fences.

Fixes:
- extract_final_from_text(): regex-based FINAL detection in text
  responses, matching the official RLM's find_final_answer() fallback
- Handles: double-quoted, single-quoted, triple-quoted, unquoted,
  nested parens
- Checked in LlmResponse::Text handler BEFORE tool intent nudge
  (FINAL takes priority)

9 new tests:
- codeact_final_in_text_response: FINAL("answer") in plain text
- codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text
- final_double_quoted, final_single_quoted, final_triple_quoted,
  final_unquoted, final_with_nested_parens, final_after_long_text,
  no_final_returns_none

102 tests passing (93 + 9 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add crate extraction & cleanup roadmap

Documents architectural recommendations from the engine v2 design
process for future reference:

- Root directory consolidation (channels-src + tools-src → extensions/)
- Crate extraction tiers: zero-coupling (estimation, observability,
  tunnel), trivial-coupling (document_extraction, pairing, hooks),
  medium-coupling (secrets, MCP, db, workspace, llm, skills),
  heavy-coupling (web gateway, agent, extensions)
- src/ module reorganization into logical groups (core, persistence,
  infra, media, support)
- main.rs/app.rs slimming targets (100/500 lines after migration)
- WASM module candidates (document_extraction) and non-candidates
  (REPL, web gateway → separate crates instead)
- Priority ordering for extraction work
- Tracks completed items (ironclaw_safety, ironclaw_engine,
  transcription move)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): live progress status updates via event broadcast

Engine v2 now shows live progress in the CLI (and any channel):
- "Thinking..." when a step starts
- Tool name + success/error when actions execute
- "Processing results..." when a step completes

Implementation:
- ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256)
- ExecutionLoop.emit_event() writes to thread.events AND broadcasts
- ThreadManager.subscribe_events() returns a receiver
- Router uses tokio::select! to listen for events while waiting for
  thread completion, forwarding them as StatusUpdate to the channel

This replaces the polling approach with zero-latency event streaming.
Agent.channels visibility widened to pub(crate) for bridge access.

102 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): include tool results in code step output for LLM context

The LLM was ignoring tool results and answering from training data
because the compact output metadata didn't include what tools returned.
Tool results lived only as ActionResult messages (role: Tool) which
some providers flatten or the model ignores.

Now the code step output includes:
- stdout from Python print() statements
- [tool_name result] with the actual output (truncated to 4K per tool)
- [tool_name error] for failed tools
- [return] for the code's return value
- Total output truncated to 8K chars to prevent context bloat

This ensures the model sees web_search results, API responses, etc.
in the next iteration and can reason about them instead of hallucinating.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): add debug/trace logging for CodeAct execution

Three verbosity levels for debugging the engine:

RUST_LOG=ironclaw_engine=debug:
- LLM call: message count, iteration, force_text
- LLM response: type (text/code/action_calls), token usage
- Code execution: code length, action count, had_error, final_answer
- Text response: length, FINAL() detection

RUST_LOG=ironclaw_engine=trace:
- Full message list sent to LLM (role, length, first 200 chars each)
- Full code block being executed
- stdout preview (first 500 chars)
- Per-tool results (name, success, first 300 chars of output)
- Text response preview (first 500 chars)

Usage:
  ENGINE_V2=true RUST_LOG=ironclaw_engine=debug cargo run
  ENGINE_V2=true RUST_LOG=ironclaw_engine=trace cargo run

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): execution trace recording + retrospective analysis

Enable with ENGINE_V2_TRACE=1 to get full execution traces and
automatic issue detection after each thread completes.

Trace recording (executor/trace.rs):
- build_trace(): captures full thread state — messages (with full
  content), events, step count, token usage, detected issues
- write_trace(): writes JSON to engine_trace_{timestamp}.json
- log_trace_summary(): logs summary + issues at info/warn level

Retrospective analyzer detects 8 issue categories:
- thread_failure: thread ended in Failed state
- no_response: no assistant message generated
- tool_error: specific tool failures with error details
- code_error: Python errors (NameError, SyntaxError, etc.) in output
- missing_tool_output: tool results exist but not in system messages
- excessive_steps: >10 steps (may be stuck in loop)
- no_tools_used: single-step answer without tools (hallucination risk)
- mixed_mode: text responses without code blocks (prompt not followed)

Thread state now saved to store after execution completes (for trace
access after join_thread).

Usage:
  ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run
  # After each message: trace JSON + issue log in terminal

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): wire reflection pipeline + trace analysis into thread lifecycle

After every thread completes, ThreadManager now automatically runs:

1. Retrospective trace analysis (non-LLM, always):
   - Detects 8 issue categories (tool errors, code errors, missing
     outputs, excessive steps, hallucination risk, etc.)
   - Logs issues at warn level when found

2. Trace file recording (when ENGINE_V2_TRACE=1):
   - Writes full JSON trace to engine_trace_{timestamp}.json

3. LLM reflection (when enable_reflection=true):
   - Calls reflection pipeline to produce Summary, Lesson, Issue docs
   - Saves docs to store for future context retrieval
   - Enabled by default in the bridge router

All three run inside the spawned tokio task after exec.run() completes,
before saving the final thread state. No external wiring needed.

Removed duplicate trace recording from the router — it's now handled
by ThreadManager automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): convert tool name hyphens to underscores for Python compatibility

Root cause from trace analysis: the LLM writes `web_search()` (valid
Python identifier) but the tool registry has `web-search` (with hyphen).
The EffectBridgeAdapter couldn't find the tool → "Tool not found" error
→ model fabricated fake data instead.

Fixes:
- available_actions(): converts tool names from hyphens to underscores
  (web-search → web_search) so the system prompt lists valid Python names
- execute_action(): tries the original name first, then falls back to
  hyphenated form (web_search → web-search) for tool registry lookup
- Same conversion in router's capability registry builder

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): parse JSON tool output to prevent double-serialization

From trace analysis: web_search returned a JSON string, which was
wrapped as serde_json::json!(string) creating a Value::String containing
JSON. When Monty got this as MontyObject::String, the Python code
couldn't index it with result['title'] → TypeError.

Fix: try parsing the tool output string as JSON first. If valid, use the
parsed Value (becomes a Python dict/list). If not valid JSON, keep as
string. This means web_search results are directly indexable in Python:
  results = web_search(query="...")
  print(results["results"][0]["title"])  # works now

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): persist variables across code steps via `state` dict

Monty creates a fresh runtime per code step, so variables are lost
between steps. This caused the model to re-paste tool results from
system messages, wasting tokens.

Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that
accumulates across steps:
- Tool results stored by tool name: state["web_search"] = {results...}
- Return values stored: state["last_return"], state["step_0_return"]
- Injected as a `state` Python variable in each new MontyRun

Now the model can do:
  Step 1: results = web_search(query="...")  # tool result saved in state
  Step 2: data = state["web_search"]         # access previous result
          summary = llm_query("summarize", str(data))
          FINAL(summary)

System prompt updated to document the `state` variable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): add state hint on code errors + retrieval engine integration

When code fails with NameError/UnboundLocalError (model trying to
access variables from a previous step), the error output now includes:

  [HINT] Variables don't persist between code blocks. Use the `state`
  dict to access data from previous steps. Available keys: ["web_search",
  "last_return"]

This teaches the model to use `state["web_search"]` instead of `result`
after a NameError, reducing wasted steps from 3-4 to 1.

Also integrates RetrievalEngine into context building and ThreadManager:
- build_step_context() now accepts optional RetrievalEngine to inject
  relevant memory docs (Lessons, Specs, Playbooks) into LLM context
- RetrievalEngine uses keyword matching with doc-type priority scoring
- Memory docs from reflection (Phase 4) now feed back into future threads

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove trace files and add to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): replace web_fetch example with web_search in CodeAct prompt

The system prompt example used web_fetch(url="...") which doesn't exist
as a tool. The model learned from the example and tried web_fetch,
getting "Tool not found". Changed to web_search(query="...") which is
an actual registered tool.

Found via trace analysis — reflection pipeline correctly identified
this as a "Tool Name Correction" spec doc.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): extract prompt templates to markdown files

Prompt templates moved from inline Rust strings to plain markdown files
at crates/ironclaw_engine/prompts/ for easy inspection and iteration:

- prompts/codeact_preamble.md — main instructions, special functions,
  context variables, rules
- prompts/codeact_postamble.md — strategy section

Loaded at compile time via include_str!(), so no runtime file I/O.
Edit the .md files and rebuild to iterate on prompts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): replace byte-index slicing with char-safe truncation

Panic: 'byte index 80 is not a char boundary; it is inside ''' when
tool output contained multi-byte UTF-8 characters (smart quotes from
web search results).

Fixed 4 unsafe byte-index slices:
- thread.rs:281: message preview &content[..80] → chars().take(80)
- loop_engine.rs:556: tool output &str[..4000] → chars().take(4000)
- loop_engine.rs:579: output tail &str[len-8000..] → chars().skip()
- scripting.rs:82: stdout tail &str[len-N..] → chars().skip()

All now use .chars().take() or .chars().skip() which respect character
boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on
user-supplied or external strings."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): fix false positive missing_tool_output warning in trace analyzer

The check was looking for "[" + "result]" in System-role messages only,
but tool output metadata is added with patterns like "[shell result]"
and may appear in messages with any role. Changed to scan all messages
for " result]" or " error]" patterns regardless of role.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): update architecture plan with Phase 6 status and approval flow design

Phase 6 updated to reflect what was actually built:
- Bridge adapters (LLM, Effect, InMemoryStore, Router) — all done
- Integration touchpoint (4 lines in handle_message) — done
- Live progress via broadcast events — done
- Conversation persistence across messages — done
- Trace recording + retrospective analysis — done
- 8 bugs found and fixed via trace analysis — documented

Phase 6 remaining work documented:
- Approval flow: detailed 5-step design (send to channel, pause thread,
  route response, resume execution, always handling) with v1 reference
- Database persistence (InMemoryStore → real DB tables)
- Acceptance testing (TestRig + TraceLlm fixtures)
- Two-phase commit for high-stakes effects

Progress table updated: Phase 6 marked as DONE (partial), 134 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add self-improving engine design plan

Designs a system where the engine debugs and improves itself, based on
the pattern observed in the last session: 5 consecutive bug fixes all
followed trace → read → identify → edit → test, using tools the engine
already has access to.

Three levels of self-improvement:
- Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply.
- Level 2 (Config): adjust defaults/mappings. Branch + test + PR.
- Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR.

Architecture: Self-improvement Mission spawns a Reflection thread that
reads traces, reads source, proposes fixes, validates via cargo test,
and either auto-applies (Level 1) or creates a PR (Level 2-3).

Includes: fix pattern database (seeded from our 8 debugging session
fixes), feedback loop diagram, safety model, implementation phases
(A through D), and what exists vs what's new.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add engine v2 security model and audit

Comprehensive security analysis of engine v2 covering:

Threat model: 4 attacker profiles (malicious input, prompt injection
via tools, poisoned memory, supply chain).

Current state audit: 9 controls working (Monty sandbox, safety layer,
policy engine, leases, provenance, events) and 9 gaps identified.

Critical finding: ALL tools granted by default — CodeAct code can call
shell, write_file, apply_patch without approval. Proposed fix: 3-tier
tool classification (auto/approve-once/always-approve).

CodeAct-specific threats: tool call amplification, prompt injection via
search results, data exfiltration via tool chains, Monty escape.

Self-improvement security: poisoned trace attacks, memory poisoning via
reflection. Mitigations: edit validation, frequency caps, audit trail,
auto-rollback, reflection output scanning.

6-layer security architecture proposed: input validation, capability
gating, output sanitization, execution sandboxing, self-improvement
controls, observability.

Prioritized implementation plan with severity/effort ratings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(security): cross-reference v1 controls — use, don't reinvent

Updated security plan with detailed audit of ALL existing v1 security
controls and how they map to engine v2 bridge gaps:

Key finding: v1 already has solutions for every security gap identified.
The bridge just needs to wire them in:

- Tool::requires_approval() exists but bridge doesn't call it
- safety.wrap_for_llm() exists but tool results enter context unwrapped
- RateLimiter exists but bridge doesn't check rate limits
- BeforeToolCall hooks exist but bridge doesn't run them
- redact_params() exists but bridge doesn't redact sensitive params
- Shell risk classification (Low/Medium/High) is inherited but ignored

Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter,
not new security infrastructure. The bridge is the security boundary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): add missions, reliability tracker, reflection executor, and provenance-aware policy

- Add Mission type and MissionManager for recurring thread scheduling
- Add ReliabilityTracker for per-capability success/failure/latency tracking
- Add reflection executor that spawns CodeAct threads for post-completion reflection
- Extend PolicyEngine with provenance-aware taint checking (LLM-generated data
  requires approval for financial/external-write effects)
- Extend Store trait with mission CRUD methods
- Add conversation surface tracking, compaction token fix, context memory injection
- Wire new modules through lib.rs re-exports and bridge adapters

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): wire v1 security controls into engine v2 adapter

Zero engine crate changes. All security controls enforced at the bridge
boundary in EffectBridgeAdapter:

1. Tool approval (v1: Tool::requires_approval):
   - Checks each tool's approval requirement with actual params
   - Always → returns EngineError::LeaseDenied (blocks execution)
   - UnlessAutoApproved → checks auto_approved set, blocks if not approved
   - Never → proceeds
   - Per-session auto_approved HashSet (for future "always" handling)

2. Hook interception (v1: BeforeToolCall):
   - Runs HookEvent::ToolCall before every execution
   - HookOutcome::Reject → blocks with reason
   - HookError::Rejected → blocks with reason
   - Hook errors → fail-open (logged, execution continues)

3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm):
   - Leak detection: API keys in tool output are redacted
   - Policy enforcement: content policy rules applied
   - Length truncation: output capped at 100KB
   - XML boundary protection: prevents injection via tool output

4. Sensitive param redaction (v1: redact_params):
   - Tool's sensitive_params() consulted before hooks see parameters
   - Redacted params sent to hooks, original params used for execution

5. available_actions() now sets requires_approval based on each tool's
   default approval requirement, so the engine's PolicyEngine can
   gate tools it hasn't seen before.

6. Actual execution timing measured via Instant::now() (replaces
   placeholder Duration::from_millis(1)).

Accessor visibility: hooks() widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): implement tool approval flow for engine v2

Adds a complete approval flow that mirrors v1 behavior, using the
existing v1 security controls (Tool::requires_approval, auto-approve
sets, StatusUpdate::ApprovalNeeded).

## How it works

### Step 1: Tool blocked at execution
When the LLM's code calls a tool (e.g., `shell("ls")`):
1. EffectBridgeAdapter.execute_action() looks up the Tool object
2. Calls tool.requires_approval(&params) — returns ApprovalRequirement
3. If Always → EngineError::LeaseDenied (always blocks)
4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set,
   returns EngineError::LeaseDenied
5. If Never → proceeds to execution

### Step 2: Engine returns NeedApproval
The LeaseDenied error propagates through:
- CodeAct path: becomes Python RuntimeError, code halts, thread returns
  NeedApproval with action_name + parameters
- Structured path: same via ActionResult.is_error

### Step 3: Router stores pending approval
- PendingApproval { action_name, original_content } stored on EngineState
- StatusUpdate::ApprovalNeeded sent to channel (shows approval card in
  CLI/web with tool name, parameters, yes/always/no buttons)
- Returns text: "Tool 'shell' requires approval. Reply yes/always/no."

### Step 4: User responds
handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2:
- 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes
  original message (tool now passes the approval check on second run)
- 'always' → same + logs for session persistence
- 'no' → returns "Denied: tool was not executed."

### Key design choice
Instead of pausing/resuming mid-execution (which needs engine changes
to freeze/restore the Monty VM state), we auto-approve the tool and
re-run the full message. The EffectBridgeAdapter's auto_approved set
persists across runs, so the second execution passes immediately.

This trades one extra LLM call for zero engine modifications.

## Files changed
- src/bridge/router.rs: PendingApproval struct, handle_approval(),
  NeedApproval → StatusUpdate::ApprovalNeeded conversion
- src/bridge/mod.rs: export handle_approval
- src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2
- src/bridge/effect_adapter.rs: fmt fixes

151 tests passing, clippy + fmt clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): demote trace/reflection logging from info to debug

INFO-level log output from background tasks (trace analysis, reflection)
corrupts the REPL terminal UI. The trace summary, issue warnings, and
reflection doc previews were printing mid-approval-card, breaking the
interactive display.

Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!.
Trace analysis and reflection results now only show when
RUST_LOG=ironclaw_engine=debug is set.

Also added logging discipline rule to global CLAUDE.md:
- info! → user-facing status the REPL intentionally renders
- debug! → internal diagnostics (traces, reflection, engine internals)
- Background tasks must NEVER use info! — it breaks the TUI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): demote all router info! logging to debug!

"engine v2: initializing" and "engine v2: handling message" were
printing at INFO level, corrupting the REPL UI. All router logging
now uses debug! — only visible with RUST_LOG=ironclaw=debug.

Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(safety): demote leak detector warn-action logs from warn! to debug!

The leak detector's Warn-action matches (high_entropy_hex pattern on
web search results containing commit SHAs, CSS colors, URL hashes)
were logging at warn! level, corrupting the REPL UI with lines like:
  WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5

These are informational false positives — real leaks use LeakAction::Redact
which silently modifies the content. Warn-action matches only log for
debugging purposes and should not appear in production output.

Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): strengthen CodeAct prompt to prevent shallow text answers

The model was answering "Suggested 45 improvements" as a brief text
summary from training data without actually searching or listing them.
The trace showed: no code block, no tool calls, no FINAL().

Prompt changes:
- Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with
  plain text only." (was: "Always write code... plain text for brief
  explanations")
- Rule 2 (NEW): "NEVER answer from memory or training data alone.
  Always use tools to get real, current information before answering."
- Rule 3: FINAL answer "should be detailed and complete — not just a
  summary like 'found 45 items'"
- Rule 8 (NEW): "Include the actual content in your FINAL() answer,
  not just a count or summary. Users want to see the details."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): persist reflection docs to workspace for cross-session learning

Replaces InMemoryStore with HybridStore:
- Ephemeral data (threads, steps, events, leases) stays in-memory
- MemoryDocs (lessons, specs, playbooks from reflection) persist to
  the workspace at engine/docs/{type}/{id}.json

On engine init, load_docs_from_workspace() reads existing docs back
into the in-memory cache. This means:
- Lessons learned in session 1 are available in session 2
- The RetrievalEngine injects relevant past lessons into new threads
- The engine genuinely improves over time as reflection accumulates

Workspace paths:
  engine/docs/lessons/{uuid}.json
  engine/docs/specs/{uuid}.json
  engine/docs/playbooks/{uuid}.json
  engine/docs/summaries/{uuid}.json
  engine/docs/issues/{uuid}.json

No new database tables. Uses existing workspace write/read/list.
workspace() accessor widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): adapt to execute_tool_with_safety params-by-value change

Staging merge changed execute_tool_with_safety to take params by value
instead of by reference (perf optimization from PR #926). Updated
bridge adapter to clone params before passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): add web gateway integration plan to Phase 6

Documents three gaps between engine v2 and the web gateway:
1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent)
2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB)
3. No cross-channel visibility (REPL ↔ web messages invisible to each other)

Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1
conversation tables after thread completion. Prerequisite: AppEvent
extraction PR (in progress separately).

Also updated DB persistence status: HybridStore with workspace-backed
MemoryDocs is now implemented (partial persistence).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(engine): document routine/job gap and SIGKILL crash scenario

Routines are entirely v1 — not hooked up to engine v2. When a user
asks "create a routine" as natural language, engine v2 tries to call
routine_create via CodeAct, but the tool needs RoutineEngine + Database
refs that the bridge's minimal JobContext doesn't provide. This caused
a SIGKILL crash during testing.

Options documented: block routine tools in v2 (short term), pass refs
through context (medium), replace with Mission system (long term).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract AppEvent to crates/ironclaw_common

SseEvent was defined in src/channels/web/types.rs but imported by 12+
modules across agent, orchestrator, worker, tools, and extensions — it
had become the application-wide event protocol, not a web transport
concern.

Create crates/ironclaw_common as a shared workspace crate and move the
enum there as AppEvent.  Also move the truncate_preview utility which
was similarly leaked from the web gateway into agent modules.

- New crate: crates/ironclaw_common (AppEvent, truncate_preview)
- Rename SseEvent → AppEvent, from_sse_event → from_app_event
- web/types.rs re-exports AppEvent for internal gateway use
- web/util.rs re-exports truncate_preview
- Wire format unchanged (serde renames are on variants, not the enum)

Aligned with the event bus direction on refactor/architectural-hardening
where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): integrate with web gateway via AppEvent + v1 conversation DB

Three changes to make engine v2 visible in the web gateway:

1. SSE event streaming (AppEvent broadcast):
   - ThreadEvent → AppEvent conversion via thread_event_to_app_event()
   - Events broadcast to SseManager during the poll loop
   - Covers: Thinking, ToolCompleted (success/error), Status, Response
   - Web gateway receives real-time progress without any gateway changes

2. Conversation persistence to v1 database:
   - After thread completes, writes user message + agent response to
     v1 ConversationStore via add_conversation_message()
   - Uses get_or_create_assistant_conversation() for per-user per-channel
   - Web gateway reads from DB as usual — chat history appears

3. Final response broadcast:
   - AppEvent::Response with full text + thread_id sent via SSE
   - Web gateway renders the response in the chat UI

New EngineState fields: sse (Option<Arc<SseManager>>),
db (Option<Arc<dyn Database>>). Both populated from Agent.deps.

Agent.deps visibility widened to pub(crate).

Depends on: ironclaw_common crate with AppEvent type (PR #1615).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): complete Phase 6 — v1-only tool blocking, rate limiting, call limits

Three security/stability improvements in EffectBridgeAdapter:

1. V1-only tool blocking:
   - routine_create, create_job, build_software (and hyphenated variants)
     return helpful error: "use the slash command instead"
   - Filtered out of available_actions() so system prompt doesn't list them
   - Prevents crash from tools needing RoutineEngine/Scheduler refs

2. Per-step tool call limit:
   - Max 50 tool calls per code block (AtomicU32 counter)
   - Prevents amplification: `for i in range(10000): shell(...)`
   - Returns "call limit reached, break into multiple steps"

3. Rate limiting:
   - Per-user per-tool sliding window via RateLimiter
   - Checks tool.rate_limit_config() before every execution
   - Returns "rate limited, try again in Ns"

Architecture plan updated:
- Gateway integration: DONE
- Routines: BLOCKED (gracefully, with slash command fallback)
- Rate limiting: DONE
- Call limit: DONE
- Phase 6 status: DONE (remaining: acceptance tests, two-phase commit)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Mission system design — goal-oriented autonomous threads

Missions replace routines with evolving, knowledge-accumulating
autonomous agents. Unlike routines (fixed prompt, stateless), Missions:

- Generate prompts from accumulated Project knowledge (lessons,
  playbooks, issues from prior threads)
- Adapt approach when something fails repeatedly
- Track progress toward a goal with success criteria
- Self-manage: pause when stuck, complete when goal achieved

Architecture: MissionManager with cron ticker spawns threads via
ThreadManager. Meta-prompt built from mission goal + Project MemoryDocs
via RetrievalEngine. Reflection feeds back automatically.

6-step implementation plan: cron trigger, meta-prompt builder, bridge
wiring, CodeAct tools, progress tracking, persistence.

Includes two worked examples: daily tech news briefing (ongoing) and
test coverage improvement (goal-driven, self-completing).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): extend Mission types with webhook/event triggers + evolving strategy

Mission types updated to support external activation sources:

MissionCadence expanded:
- Cron { expression, timezone } — timezone-aware scheduling
- OnEvent { event_pattern } — channel message pattern matching
- OnSystemEvent { source, event_type } — structured events from tools
- Webhook { path, secret } — external HTTP triggers (GitHub, email, etc.)
- Manual — explicit triggering only

The engine defines trigger TYPES. The bridge implements infrastructure
(cron ticker, webhook endpoints, event matchers). GitHub issues, PRs,
email, Slack events all use the generic Webhook cadence — no
special-casing in the engine. Webhook payload injected as
state["trigger_payload"] in the thread's Python context.

Mission struct extended:
- current_focus: what the next thread should work on (evolving)
- approach_history: what we've tried (for adaptation)
- max_threads_per_day / threads_today: daily budget
- last_trigger_payload: webhook/event data for thread context

Plan updated with trigger type table and webhook integration design.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): implement MissionManager execution with meta-prompts

The MissionManager now builds evolving meta-prompts and processes
thread outcomes for continuous learning:

fire_mission() upgraded:
- Loads Project MemoryDocs via RetrievalEngine for context
- Builds meta-prompt from: goal, current_focus, approach_history,
  project knowledge docs, trigger payload, thread count
- Spawns thread with meta-prompt as user message
- Background task waits for completion and processes outcome
- Daily thread budget enforcement (max_threads_per_day)

Meta-prompt structure:
  # Mission: {name}
  Goal: {goal}
  ## Current Focus (evolves between threads)
  ## Previous Approaches (what we've tried)
  ## Knowledge from Prior Threads (lessons, playbooks, issues)
  ## Trigger Payload (webhook/event data if applicable)
  ## Instructions (accomplish step, report next focus, check goal)

Outcome processing:
- Extracts "next focus:" from FINAL() response → updates current_focus
- Detects "goal achieved: yes" → completes mission
- Records accomplishment in approach_history
- Failed threads recorded as "FAILED: {error}"

Cron ticker:
- start_cron_ticker() spawns tokio task, ticks every 60s
- Checks active Cron missions, fires those past next_fire_at

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): wire MissionManager into engine v2 for CodeAct access

Missions are now callable from CodeAct Python code:

```python
# Create a daily briefing mission
result = mission_create(
    name="Tech News",
    goal="Daily AI/crypto/software news briefing",
    cadence="0 9 * * *"
)

# List all missions
missions = mission_list()

# Manually fire a mission
mission_fire(id="...")

# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```

Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
  lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
  "webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
  (avoids circular dependency)

System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(bridge): map routine_* calls to mission operations in v2

When the model calls routine_create, routine_list, routine_fire,
routine_pause, routine_resume, or routine_delete, the bridge now
routes them to the MissionManager instead of blocking with an error.

Mapping:
  routine_create → mission_create (with cadence parsing)
  routine_list   → mission_list
  routine_fire   → mission_fire
  routine_pause  → mission_pause
  routine_resume → mission_resume
  routine_update → mission_pause/resume (based on params)
  routine_delete → mission_complete (marks as done)

Routine tools removed from v1-only blocklist and restored in
available_actions(). The model can use either "routine" or "mission"
vocabulary — both work.

Still blocked: create_job, cancel_job, build_software (need v1
Scheduler/ContainerJobManager refs).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): add E2E mission flow tests — 7 new tests

Comprehensive mission lifecycle tests:

- fire_mission_builds_meta_prompt_with_goal: verifies thread spawned
  with project context and recorded in history
- outcome_processing_extracts_next_focus: "Next focus: X" in FINAL()
  response → mission.current_focus updated
- outcome_processing_detects_goal_achieved: "Goal achieved: yes" →
  mission status transitions to Completed
- mission_evolves_via_direct_outcome_processing: 3-step evolution:
  step 1 sets focus to "db module", step 2 evolves to "tools module",
  step 3 detects goal achieved → mission completes. Tests the full
  learning loop without background task timing dependencies.
- fire_with_trigger_payload: webhook payload stored on mission and
  threads_today counter incremented
- daily_budget_enforced: max_threads_per_day=1 → first fire succeeds,
  second returns None

157 tests passing (151 prior + 6 new mission E2E).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): self-improving engine via Mission system

Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.

Key changes:

Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
  matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
  prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
  async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading

Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add checkpoint-based engine thread recovery

* feat(engine): add Python orchestrator module and host functions

Add the orchestrator infrastructure for replacing the Rust execution
loop with versioned Python code. This commit adds the module and host
functions without switching over — the existing Rust loop is unchanged.

New files:
- orchestrator/default.py: v0 Python orchestrator (run_loop + helpers)
- executor/orchestrator.rs: host function dispatch, orchestrator
  loading from Store with version selection, OrchestratorResult parsing

Host functions exposed to orchestrator Python via Monty suspension:
  __llm_complete__, __execute_code_step__ (nested Monty VM),
  __execute_action__, __check_signals__, __emit_event__,
  __add_message__, __save_checkpoint__, __transition_to__,
  __retrieve_docs__, __check_budget__, __get_actions__

Also makes json_to_monty, monty_to_json, monty_to_string pub(crate)
in scripting.rs for cross-module use.

Design doc: docs/plans/2026-03-25-python-orchestrator.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): switch ExecutionLoop::run() to Python orchestrator

Replace the 900-line Rust execution loop with a ~80-line bootstrap
that loads and runs the versioned Python orchestrator via Monty VM.

The orchestrator Python code (orchestrator/default.py) is the v0
compiled-in version. Runtime versions can override it via MemoryDoc
storage (orchestrator:main with tag orchestrator_code).

Key fixes during switchover:
- Use ExtFunctionResult::NotFound for unknown functions so Monty
  falls through to Python-defined functions (extract_final, etc.)
- Move helper function definitions above run_loop for Monty scoping
- Use FINAL result value (not VM return value) in Complete handler
- Rename 'final' variable to 'final_answer' to avoid Python keyword

Status: 171/177 tests pass. 6 remaining failures are step_count and
token tracking bookkeeping — the orchestrator manages these internally
but doesn't yet update the thread's counters via host functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): all 177 tests pass with Python orchestrator

- Increment step_count and track tokens in __emit_event__("step_completed")
  so thread bookkeeping matches the old Rust loop behavior
- Remove double-counting of tokens in bootstrap (orchestrator handles it)
- Match nudge text to existing TOOL_INTENT_NUDGE constant
- Fix FINAL result propagation (use stored final_result, not VM return)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): orchestrator versioning, auto-rollback, and tests

Add version lifecycle for the Python orchestrator:
- Failure tracking via MemoryDoc (orchestrator:failures)
- Auto-rollback: after 3 consecutive failures, skip the latest version
  and fall back to previous (or compiled-in v0)
- Success resets the failure counter
- OrchestratorRollback event for observability

Update self-improvement Mission goal with Level 1.5 instructions for
orchestrator patches — the agent can now modify the execution loop
itself via memory_write with versioned orchestrator docs.

12 new tests: version selection (highest wins), rollback after failures,
rollback to default, failure counting/resetting, outcome parsing for
all 5 ThreadOutcome variants.

189 tests pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add engine v2 architecture, self-improvement, and dev history

Three new docs for contributors:

- engine-v2-architecture.md: Two-layer architecture (Rust kernel +
  Python orchestrator), five primitives, execution model with nested
  Monty VMs, bridge layer, memory/reflection, missions, capabilities

- self-improvement.md: Three improvement levels (prompt/orchestrator/
  config/code), autoresearch-inspired Mission loop, versioned
  orchestrator with auto-rollback, fix pattern database, safety model

- development-history.md: Summary of 6 Claude Code sessions that
  built the system, key design decisions and debugging moments,
  architecture evolution from 900-line Rust loop to Python orchestrator

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): complete v2 side-by-side integration with gateway API

Wire engine v2 into the full submission pipeline and expose threads,
projects, and missions through the web gateway REST API.

Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear
submissions to engine v2 when ENGINE_V2=true. Previously only UserInput
and ApprovalResponse were handled; all other control commands fell
through to disconnected v1 sessions.

Bridge query layer — add 11 read-only query functions and 6 DTO types
so gateway handlers can inspect engine state (threads, steps, events,
projects, missions) without direct access to the EngineState singleton.

Gateway endpoints — new /api/engine/* routes:
  GET  /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events
  GET  /projects, /projects/{id}
  GET  /missions, /missions/{id}
  POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume

SSE events — add ThreadStateChanged, ChildThreadSpawned, and
MissionThreadSpawned AppEvent variants. Expand the bridge event mapper
to forward StateChanged and ChildSpawned engine events to the browser.

Engine crate — add ConversationManager::clear_conversation() for /new
and /clear commands.

Code quality — replace 10 .expect() calls with proper error returns,
remove dead AgentConfig.engine_v2 field, log silent init errors, fix
duplicate doc comment, improve fallthrough documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): empty call_id on ActionResult and trace analyzer false positives

Fix structured executor not stamping call_id onto ActionResult — the
EffectExecutor trait doesn't receive call_id, so the structured executor
must copy it from the original ActionCall after execution. Empty call_id
caused OpenAI-compatible providers to reject the next LLM request with
"Invalid 'input[2].call_id': empty string".

Fix trace analyzer false positives:
- code_error check now only scans User-role code output messages
  (prefixed with [stdout]/[stderr]/[code ]/Traceback), not System
  prompt which contains example error text
- missing_tool_output check now recognizes ActionResult messages as
  valid tool output (Tier 0 structured path)
- Add NotImplementedError to detected code error patterns

New trace checks:
- empty_call_id: detect ActionResult messages with missing/empty
  call_id before they reach the LLM API (severity: Error)
- llm_error: extract LLM provider errors from Failed state reason
- orchestrator_error: extract orchestrator errors from Failed state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): add Missions tab to gateway UI

Add a full Missions page to the web gateway with list view, detail view,
and action buttons (Fire, Pause, Resume).

Backend: add /api/engine/missions/summary endpoint returning counts by
status (active/paused/completed/failed).

Frontend:
- New "Missions" tab between Jobs and Routines
- Summary cards showing mission counts by status
- Table with name, goal, cadence type, thread count, status, actions
- Detail view with goal, cadence, current focus, success criteria,
  approach history, spawned thread list, and action buttons
- Fire/Pause/Resume actions with toast notifications
- i18n support (English + Chinese)
- CSS following the existing routines/jobs patterns

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): eagerly initialize engine v2 at startup

The gateway API endpoints (/api/engine/missions, etc.) call bridge
query functions that return empty results when the engine state hasn't
been initialized yet. Previously, initialization only happened lazily
on the first chat message via handle_with_engine().

Now when ENGINE_V2=true, the engine is initialized in Agent::run()
before channels start, so the self-improvement mission and other
engine state is available to gateway API endpoints immediately.

Also rename get_or_init_engine → init_engine and make it public so
it can be called from agent_loop.rs at startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): improve mission detail with markdown goal and thread table

- Goal rendered as full-width markdown block instead of plain-text
  meta item (uses existing renderMarkdown/marked)
- Current focus and success criteria also rendered as markdown
- Spawned threads shown as a clickable table with goal, type, state,
  steps, tokens, and created date instead of a UUID list
- Clicking a thread row opens an inline thread detail view showing
  metadata grid and full message history with markdown rendering
- Back button returns to the mission detail view
- Backend: mission detail now returns full thread summaries (goal,
  state, step_count, tokens) instead of just thread IDs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): close SSE connections on page unload to prevent connection starvation

The browser limits concurrent HTTP/1.1 connections per origin to 6.
Without cleanup, SSE connections from prior page loads linger after
refresh/navigation, eating into the pool. After 2-3 refreshes, all 6
slots are consumed by stale SSE streams and new API fetch calls queue
indefinitely — the UI shows "connected" (SSE works) but data never
loads.

Add a beforeunload handler that closes both eventSource (chat events)
and logEventSource (log stream) so the browser can reuse connections
immediately on page reload.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): support multiple gateway tabs by reducing SSE connections

Each browser tab opened 2 SSE connections (chat events + log events).
With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the
pool and couldn't load any data.

Three changes:

1. Lazy log SSE — only connect when the logs tab is active, disconnect
   when switching away. Most users rarely view logs, so this saves a
   connection slot per tab.

2. Visibility API — close SSE when the browser tab goes to background
   (user switches to another tab), reconnect when it becomes visible.
   Background tabs don't need real-time events.

3. Combined with the existing beforeunload cleanup, this means:
   - Active foreground tab: 1 connection (chat SSE only, +1 if logs tab)
   - Background tabs: 0 connections
   - Closed/refreshed tabs: 0 connections (beforeunload cleanup)

This allows many gateway tabs to coexist within the 6-connection limit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): route messages to correct conversation by thread scope

Messages sent from a new conversation in the gateway always appeared in
the default assistant conversation because handle_with_engine ignored
the thread_id from the frontend.

Two fixes:

1. Engine conversation scoping — when the message carries a thread_id
   (from the frontend's conversation picker), use it as part of the
   engine conversation key: "gateway:<thread_id>" instead of just
   "gateway". This creates a distinct engine conversation per v1
   thread, so messages don't cross-contaminate.

2. V1 dual-write targeting — write user messages and assistant
   responses to the v1 conversation matching the thread_id (via
   ensure_conversation), not the hardcoded assistant conversation.
   Falls back to the assistant conversation when no thread_id is
   present (e.g., default chat).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): richer activity indicators for engine v2 execution

The gateway UI showed only generic "Thinking..." during engine v2
execution with no visibility into CodeAct code execution, tool calls,
or reflection. Now the event mapping produces detailed status updates:

Step lifecycle:
- "Calling LLM..." when a step starts (was "Thinking...")
- "Step complete — N in / M out tokens" when done (was "Processing...")

Tool execution:
- Emit ToolStarted + ToolCompleted SSE events so the frontend renders
  proper tool cards with spinner → checkmark/error transitions
- Duration shown in parameters field (e.g., "42ms")

CodeAct visibility:
- "Executing code..." when assistant produces a code block
- "Code executed" / "Code executed (no output)" for successful runs
- "Code error — retrying..." when Monty raises an exception

Reflection:
- "Reflecting on execution..." when post-thread analysis starts
- "Reflection complete — N insight(s) saved" when done

Also refactored thread_event_to_app_event → thread_event_to_app_events
(returns Vec<AppEvent>) to support emitting ToolStarted before
ToolCompleted in a single event handler pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): resolve tool names as callable stubs in CodeAct runtime

When LLM-generated code calls `mission_list()` or any tool function,
Monty's Python execution model first resolves the name (`mission_list`)
as a NameLookup before invoking it as a FunctionCall. The NameLookup
handler always returned Undefined, causing NameError before the function
call could dispatch to the effect executor.

Fix: before starting the Monty VM, collect all known tool names from
the effect executor's available_actions(). In the NameLookup handler,
if the name matches a known tool, return a MontyObject::Function stub
instead of Undefined. Monty then yields FunctionCall for the stub,
which dispatches to the normal tool execution pipeline.

This enables CodeAct code to call any registered tool as a Python
function: mission_list(), mission_create(), routine_list(), web_search(),
memory_search(), etc. — all without explicit imports or __execute_action__
boilerplate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): consolidate action execution, remove reflection, add learning missions

Three major changes to the v2 engine:

1. **Consolidated action execution** — `handle_execute_action` in Rust is now
   the single source of truth for lease lookup, policy check, lease consumption,
   action execution, event emission, and ActionResult message recording. The
   Python orchestrator no longer duplicates event/message logic. This fixes the
   empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant
   messages (Codex "No tool call found" error).

2. **Removed reflection system** — Deleted the per-thread reflection pipeline
   (pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection,
   enable_reflection config, and all 3 reflection event kinds. Learning is now
   handled entirely by event-driven missions that fire selectively.

3. **Three learning missions** replace reflection:
   - `self-improvement` — fires on trace issues (error diagnosis, prompt fixes)
   - `playbook-extraction` — fires on successful 5+ step threads (reusable procedures)
   - `conversation-insights` — fires every 5 threads per project (user preferences,
     domain knowledge, workflow patterns)

Additional fixes:
- llm_query()/llm_query_batched() always include system message (Codex compat)
- handle_llm_complete adds assistant message with structured action_calls for
  Tier 0 responses (prevents "No tool call found" errors)
- Gateway broadcasts without thread_id emit as Status events instead of being dropped
- Comprehensive tests for call_id propagation and trace analysis (17 new tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): extract ironclaw_skills crate and integrate with v2 engine

Extract the skills system into a standalone `ironclaw_skills` crate
(following the ironclaw_safety pattern) and wire it into the v2 engine
for deterministic skill selection, CodeAct code injection, and
confidence tracking.

**ironclaw_skills crate** (94 tests):
- Core types: SkillManifest, ActivationCriteria, LoadedSkill, SkillTrust
- V2 types: V2SkillMetadata, CodeSnippet, SkillMetrics, V2SkillSource
- Deterministic 4-phase selector (gating→scoring→budget→attenuation)
- apply_confidence_factor() for extracted skill scoring
- SKILL.md parser, validation/escaping, gating, registry, catalog
- Feature-gated: catalog (reqwest), registry (filesystem)

**Engine integration** (14 new tests):
- DocType::Skill with retrieval weight 0.45
- SkillSelector bridges MemoryDoc→LoadedSkill for shared scoring
- SkillTracker for usage/version/rollback confidence tracking
- System prompt injection via <skill> XML blocks
- CodeAct snippet injection via Monty NameLookup
- Skill extraction mission replaces playbook extraction
- ThreadManager.set_skill_selector() for runtime wiring

**Bridge + migration**:
- skill_migration.rs: v1 SKILL.md → v2 MemoryDoc (idempotent)
- init_engine() migrates v1 skills, builds SkillSelector
- src/skills/mod.rs → re-export shim

**E2E test** (tests/engine_v2_skill_codeact.rs):
- Full CodeAct loop: skill selected → LLM returns Python code →
  Monty executes http() → mock returns canned GitHub JSON →
  FINAL() terminates → thread completes with canned data
- GitHub SKILL.md in skills/github/ as reference implementation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Documenting research around how to extend to more integrations

* docs: update engine-v2-architecture for missions and skills

- Replace "Reflection Pipeline" with "Learning Missions" (self-improvement,
  skill-extraction, conversation-insights)
- Add "Skills System" section covering ironclaw_skills crate, deterministic
  selection pipeline, CodeAct integration, confidence tracking, v1 migration
- Update MemoryDoc types table (add Skill, remove Playbook as primary)
- Update Integration Scaling section: Skills replace Capabilities-as-knowledge
  as the concrete implementation
- Update example from Capability YAML to SKILL.md format with credentials
- Fix thread state machine (remove Reflecting state)
- Update key files table and test counts
- Add self-improvement feedback loop diagram

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: clean up legacy playbook references in engine crate

- Rename PLAYBOOK_MIN_STEPS/ACTIONS → SKILL_EXTRACTION_MIN_STEPS/ACTIONS
- Fix pattern DB uses DocType::Note instead of DocType::Playbook
- Update CLAUDE.md: skill-extraction mission, DocType list, module map

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): credential specs in skill frontmatter, HTTP tool hardening, mission leases

Skills can now declare API credentials in YAML frontmatter (SkillCredentialSpec,
SkillCredentialLocation, SkillOAuthConfig, ProviderRefreshStrategy). Valid specs
are registered into SharedCredentialRegistry at startup; the HttpTool auto-injects
credentials for matching hosts — same zero-exposure model as WASM tools.

HTTP tool security hardening:
- Block LLM-provided auth headers for hosts with registered credentials
- Return structured authentication_required error for missing credentials
- Strip sensitive response headers (Set-Cookie, WWW-Authenticate, Authorization)
- Scan response body through LeakDetector before returning to LLM

Mission capability leases: registered mission_create/list/fire/pause/resume/delete
as a "missions" capability so threads receive leases. Removed routine_* aliases
from effect adapter — descriptions mention "routine" for LLM intent mapping.

Includes 10 integration tests (tests/skill_credential_injection.rs) covering
the full pipeline: YAML parsing → validation → registry → HttpTool wiring →
per-user isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(engine): remove legacy Playbook doc type, superseded by Skill

Drop DocType::Playbook variant and all references — playbook extraction
mission was already renamed to skill extraction in the previous session.
Updates CLAUDE.md, architecture docs, context builder, retrieval weights,
mission comments, and store adapter path mapping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): move skill selection and injection to Python orchestrator

Skill selection was in Rust (SkillSelector in loop_engine.rs) — now it's
in the Python orchestrator where the self-improvement mission can evolve it.

Rust provides data access via two new host functions:
- __list_skills__() — loads DocType::Skill MemoryDocs from Store
- __record_skill_usage__(doc_id, success) — confidence tracking

Python orchestrator handles everything else:
- score_skill() — keyword/tag/confidence scoring (~40 lines)
- select_skills() — budget-aware top-N selection (~15 lines)
- format_skills() — XML block injection into system prompt (~20 lines)
- Injection at step 0 with active_skill_ids stored in state

Removed from Rust:
- SkillSelector field + builder on ExecutionLoop and ThreadManager
- format_skills_section() from prompt.rs
- Rust-side skill injection block in loop_engine.rs
- SkillSelector wiring in bridge/router.rs

E2E test updated: skills stored in TestStore, Python orchestrator
finds them via __list_skills__() and injects based on goal keywords.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: annotate v1-only code for removal after migration

Mark modules and functions that exist solely for the v1 agent with
"remove after v1 migration" notes:

- src/skills/mod.rs ��� shim, attenuation, credential registration
- src/skills/attenuation.rs — trust-based tool filtering (v1 only)
- ironclaw_skills: selector, gating, registry, catalog modules
- ironclaw_engine: skill_selector.rs (superseded by Python orchestrator)
- src/bridge/skill_migration.rs — one-time v1→v2 conversion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(engine): remove unused skill_selector.rs

Rust-side skill selection was moved to the Python orchestrator in
7f87d179. This module had no production callers — only its own tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): compile-time skill bundling infrastructure

Add support for embedding skills into the binary at compile time:

- build.rs: embed_skills() collects skills/*/SKILL.md into embedded_skills.json
- src/skills/bundled.rs: loads embedded skills via include_str!
- SkillRegistry: with_bundled_content(), load_from_content(), step 4 in discover_all()
- Bundled skills are Trusted (ship with binary), lowest discovery priority
- 4 new tests for bundled loading, user override, gating, and removal rejection
- Cargo.toml: add serde_json build-dependency

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): non-blocking auth signal, NeedAuthentication flow, timeout safety

When the HTTP tool detects a missing credential for a registered host:
1. EffectBridgeAdapter emits SSE AuthRequired event (best-effort, for
   connected frontends — silently dropped for missions/background threads)
2. Error flows back to LLM as normal ActionResult (non-blocking)
3. LLM tells the user to authenticate

This avoids the blocking interruption approach which would hang mission
threads and sub-threads that have no channel context.

Engine additions:
- EngineError::NeedAuthentication variant for structured auth failures
- ThreadOutcome::NeedAuthentication for batch interruption when needed
- structured.rs handles NeedAuthentication by interrupting the batch
  (stops subsequent calls, returns outcome to orchestrator)
- Auth callback on EffectBridgeAdapter (optional, set by router for SSE)
- extract_credential_name parser for HTTP tool error messages
- routine_* tools added to is_v1_only_tool blocklist

Safety: added 5-minute timeout to await_thread_outcome to prevent
infinite hangs (e.g. after denied tool approval where thread fails
to resume).

Tests: 3 structured executor tests (NeedAuthentication interrupts batch,
stops subsequent calls, regular errors don't interrupt) + 7 effect
adapter tests (credential extraction, callback firing, v1-only tools).

Also adds Linear API skill (skills/linear/SKILL.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): platform self-awareness, event pipeline fix, globals() builtin, prompt templates

Session 9 changes driven by live trace analysis:

- CodeAct event pipeline: handle_execute_code_step now transfers
  CodeExecutionResult events to thread.events and broadcasts via event_tx
  (fixes false-positive no_tools_used trace warnings)
- Monty globals()/locals() builtins: returns dict of available action names
  from capability leases, enabling "tool_name" in globals() probing
- PlatformInfo injection into system prompts (version, LLM backend, model,
  database, channels, owner, repo URL)
- Mission goal prompts moved to prompts/*.md files (include_str! pattern)
- /expected command for triggering self-improvement from user feedback
- Session 9 development history

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): auto-approve http calls with registered credentials in v2

The v1 approval flow (interactive yes/no prompt) doesn't exist in v2.
When the http tool returned UnlessAutoApproved for credentialed hosts,
the effect adapter blocked with LeaseDenied — making all skill-based
API calls fail.

Fix: credential-backed http calls bypass the v1 approval check. The
user authorized by storing the credential; the v1 interactive prompt
is redundant in v2's lease-based security model.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(ui): show activated skills in CLI and gateway

End-to-end skill activation display:

1. Python orchestrator emits __emit_event__("skill_activated", skill_names=...)
   after select_skills() picks skills for the conversation
2. Rust host function parses the comma-separated names into EventKind::SkillActivated
3. Router forwards to channels as StatusUpdate::SkillActivated
4. REPL renders: ◈ skills: github, linear (cyan)
5. Web gateway emits AppEvent::SkillActivated SSE event for frontend display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): show auth prompt in REPL when credential is missing

The AuthRequired SSE event was emitted but only reached the web gateway.
The REPL never saw it because it receives events through
forward_event_to_channel which converts ThreadEvents to StatusUpdates.

Fix: when forward_event_to_channel sees an ActionFailed with
"authentication_required" in the error, emit StatusUpdate::AuthRequired
to the channel. Also add AuthRequired/AuthCompleted rendering to the
REPL (was missing — fell through to unmatched arm).

CLI now shows:
  ⚿ Authentication required: github_token
    Store the credential with: ironclaw secret set <name> <value>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(ui): show tool arguments in CLI and gateway

Add params_summary to ActionExecuted/ActionFailed events so the CLI
and gateway can display what tools are doing:

  ● http(https://api.github.com/repos/nearai/ironclaw/issues)
  ● web_search(latest AI news)
  ● memory_read(HEARTBEAT.md)

The summarize_params() helper extracts the most relevant argument
per tool type (URL for http, query for search, path for memory, etc.)
and truncates to 80 chars. Sensitive params are not included.

Router forwards the summary in both StatusUpdate (CLI/REPL) and
AppEvent (web gateway SSE) display names.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle Python None params in http tool, add params_summary to CodeAct dispatch

Two fixes from live testing:

1. http tool: treat null headers/body as empty (Python's None becomes
   JSON null via Monty). Previously headers=None errored with
   "'headers' must be an object or array of {name, value}".

2. scripting.rs: compute params_summary before dispatching actions in
   the CodeAct path (was always None). Now http calls show their URL
   in the CLI: ● http(https://api.github.com/repos/.../issues)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove glob re-exports, fix clippy warnings, clean up duplicates

- Remove `pub use ironclaw_safety::*` from src/safety/mod.rs and migrate
  all 20+ call sites to import directly from `ironclaw_safety`
- Remove `pub use ironclaw_skills::*` from src/skills/mod.rs and migrate
  all 15+ call sites to import directly from `ironclaw_skills`
- Fix 4 clippy warnings: 2 shadow imports, 2 collapsible if-let chains
- Add missing SkillActivated arm to WASM channel StatusUpdate match
- Remove duplicate AuthRequired/AuthCompleted arms in repl.rs
- Update CLAUDE.md extracted crates guidance and prompt template rule
- Fix bench imports (safety_check, safety_pipeline)

46 files changed, zero warnings, 3836 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): guided credential flow — prompt for token and retry

When a thread completes with authentication_required, the router
enters "auth mode" for that user:

1. Detects credential_name from the error in the thread response
2. Looks up setup_instructions from the skill's credential spec
3. Emits AuthRequired to CLI/gateway with instructions
4. Stores PendingAuth — next user message is treated as a token
5. Stores the token in SecretsStore
6. Retries the original user request automatically

CLI flow:
  › create an issue in github
    ⚿ Authentication required: github_token
      Create a PAT at https://github.com/settings/tokens
    Paste your token below (or type 'cancel'):
  › ghp_abc123...
    ✓ github_token authenticated: Credential stored. Retrying...
    ● http(https://api.github.com/repos/.../issues)
    Issue created: https://github.com/...

Gateway flow: same but AuthRequired SSE event shows the auth modal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): skill-based OAuth flow tests

6 E2E tests covering the full skill credential lifecycle via the
gateway API:

- test_github_skill_loaded: github skill with credential spec loaded
- test_no_github_token_initially: no stored secrets before auth
- test_http_tool_returns_auth_required: http tool signals missing cred
- test_guided_auth_flow: request → auth prompt → paste token → retry
- test_auth_required_sse_event: SSE stream includes auth/skill events
- test_different_users_isolated: per-user credential scoping

Includes mock API server (aiohttp) requiring Bearer auth with token
tracking for assertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: document Monty runtime limitations in CodeAct prompt, fix new-thread read-only

- Add "Runtime environment" section to codeact_preamble.md documenting
  Monty's restrictions: no stdlib imports, single imports only, no classes/
  with/match/del/yield, available builtins and modules, workarounds
- Add MONTY.md tracking current pin, all limitations, upgrade process,
  and changelog for future Monty updates
- Fix gateway createNewThread() not resetting read-only state — new
  threads now eagerly enable chat input instead of waiting for async
  loadThreads() callback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): transition thread to Waiting on NeedApproval

The orchestrator Python returned {"outcome": "need_approval"} without
calling __transition_to__("waiting"), leaving the thread in Running
state. When the user later approved/denied, resume_thread rejected it
with "thread is not resumable from Running".

- Add __transition_to__("waiting", "approval needed") in both code-step
  and action-call approval paths in default.py
- Add Rust safety net in loop_engine.rs: if orchestrator returns
  NeedApproval but thread isn't Waiting, force the transition

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): restructure workspace storage for human readability

Rewrite HybridStore (src/bridge/store_adapter.rs) to produce a
developer-friendly workspace layout:

- Knowledge docs use frontmatter+markdown with slugified filenames
  instead of UUID.json with wrapped structs
- Orchestrator code, prompt overlays, and failure tracker grouped
  under engine/orchestrator/
- Missions nested under their project in named folders with room
  for working files alongside mission.json
- Runtime state (threads, leases, events) under engine/.runtime/
- Terminal threads archived to compact summaries, dead leases cleaned
  on startup
- Auto-generated engine/README.md with knowledge counts, mission
  status, and thread stats

Also includes: /expected command, approval state fix, platform
self-awareness, Monty limitations in preamble, prompt template
extraction. See docs/development-history.md Session 10 for details.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(skills): explicit /skill-name activation in messages

Users can now write /github or /file-issues anywhere in their message
to force-activate a skill. The /skill-name is replaced with the skill's
description so the sentence reads naturally for the LLM:

  "fetch issues from /github" → "fetch issues from GitHub API"
  "please /file-issues for all bugs" → "please file detailed GitHub issues for all bugs"

Implementation:
- extract_skill_mentions() in selector.rs scans for /name patterns,
  matches against available skills, returns matched skills + rewritten
  message
- select_active_skills() returns (skills, rewritten_message) — explicit
  mentions merged with score-based selection
- dispatcher.rs rewrites the last user message in LLM context with
  expanded text
- 8 tests covering: basic mention, description expansion, hyphenated
  names, multiple mentions, unknown skills, URLs not matched

Also includes: seed_orchestrator_v0() for workspace visibility of
compiled-in orchestrator code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): wire NeedAuthentication and NeedApproval through v2 CodeAct path

Production traces revealed tool result desync on RequireApproval (no
ActionResult message → OpenAI 400), auth flow not triggering in CodeAct
(EffectAdapter returned Ok instead of Err(NeedAuthentication)), and HTTP
tool blocking unauthenticated requests.

Fixes:
- Add emit_and_record() to RequireApproval branch in handle_execute_action
- Wire NeedAuthentication through scripting.rs DispatchResult, orchestrator
  host functions, default.py, loop_engine safety net
- Add EngineError::NeedApproval variant; effect adapter returns it instead
  of LeaseDenied for tools needing approval
- HTTP tool: inject-if-available (proceed without auth, error only on 401)
- HTTP_ALLOW_LOCALHOST env flag for E2E testing with mock servers
- host_matches_pattern supports port in pattern (127.0.0.1:8080 matches
  host_str() output 127.0.0.1)
- CodeAct postamble: error recovery guidance
- Orchestrator user_id from thread.metadata instead of hardcoded "orchestrator"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): v1/v2 history, approval routing, cancel cleanup

Multiple v2 engine bridge fixes discovered by E2E tests:

- Write response to v1 DB for ALL thread outcomes (not just Completed),
  so history API shows NeedApproval/NeedAuthentication responses
- Remove v1 thread_id hint from pending_approval lookup (v1/v2 use
  different UUID spaces)
- Add has_pending_auth() check in agent_loop: route "cancel"/"no" through
  handle_with_engine when PendingAuth is active (SubmissionParser parsed
  "cancel" as ApprovalResponse, bypassing auth flow)
- Add engine_thread_id to PendingAuth; stop_thread on cancel
- Write cancel response to v1 DB
- NeedAuthentication handler enters guided auth flow with setup hints

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): comprehensive v2 engine test suite (12 tests, 5 files)

E2E tests for the v2 engine covering auth flow, approval lifecycle,
error handling, and edge cases. Uses mock API servers with strict token
validation, dedicated ironclaw server fixtures per module, and the mock
LLM's tool call pattern system.

Tests:
- Auth flow: skill activation, NeedAuthentication → token → retry,
  credential persistence across threads, cancel during auth, empty
  token treated as cancel, special character injection safety
- Approval: approve yes (text-based), deny, always (persists across
  threads), prompt mentions tool name
- Error handling: max iterations (30 step limit), tool intent nudge
  (LLM recovery after "let me search")

Infrastructure:
- mock_llm.py: runtime-configurable github_api_url, tool call patterns
  for issues/loop/drive, canned responses for intent nudge
- HTTP_ALLOW_LOCALHOST=true + SECRETS_MASTER_KEY in fixtures
- Separate server instances for cancel tests (cancel contaminates
  conversation state)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Session 11 — E2E test suite + engine hardening

Documents 14 bugs found across two production traces and E2E test
execution, the test infrastructure design (mock servers, dedicated
fixtures, HTTP_ALLOW_LOCALHOST), and the architecture evolution from
trace analysis → code fix → test to prevent regression.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): kernel-level pre-flight auth gate for engine v2

Transform authentication from a reactive post-execution error to a
proactive pre-flight check. The EffectBridgeAdapter now checks
credentials BEFORE executing tool calls, preventing wasted HTTP
requests and 401 errors from reaching the LLM.

Key changes:
- New AuthManager (src/bridge/auth_manager.rs) centralizes credential
  checking, setup instruction lookup, and tool readiness queries
- Pre-flight auth gate in execute_action() checks SharedCredentialRegistry
  + SecretsStore before tool execution
- Post-install auth pipeline: after tool_install, kernel auto-checks
  readiness and initiates auth flow or appends setup instructions
- tool_auth and tool_activate filtered from v2 LLM tool list and
  blocked in execute_action() — auth is kernel-level in v2
- Text-based auth detection kept as defense-in-depth fallback with
  tracing when it fires
- Setup instruction lookup deduplicated via AuthManager
- ExtensionManager gains check_tool_auth_status_pub() for auth queries

Also fixes pre-existing DocType::Plan exhaustiveness errors in the
engine crate and re-exports PlanStepDto from ironclaw_common.

Includes 10 unit tests (AuthManager + is_v1_auth_tool) and 5 E2E tests
covering pre-flight blocking, auth-then-retry, credential persistence,
v1 auth tools hidden, and auth cancellation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add Session 12 — kernel-level auth rework decisions and rationale

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(plan): autonomous plan mode via v2 primitives (MemoryDoc, Mission, SSE)

Add plan mode for autonomous long-running task execution, composing
existing v2 engine primitives rather than new engine states. Inspired
by OpenAI Codex's update_plan checklist and Claude Code's file-based
plan mode — both enforce planning through prompts, not tool removal.

Engine: DocType::Plan variant for MemoryDoc (project-scoped, retrievable).
Events: PlanUpdate SSE event with PlanStepDto for live checklist rendering.
Tool: plan_update tool broadcasts structured plan progress via SSE.
Command: /plan (create/approve/status/revise/list) rewrites to UserInput
  with [PLAN MODE] prefix to activate the plan-mode skill.
Skill: skills/plan-mode/SKILL.md defines full plan protocol — creation
  (memory_write), approval (mission_create + mission_fire), execution
  (step-by-step with plan_update), and revision flows.
UI: Inline chat checklist widget with status badges, step icons
  (checkmark/spinner/circle), results, and progress summary.
Tests: 5 E2E scenarios + mock LLM patterns + helper selectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): scope pending approvals by thread to prevent cross-thread leakage

The `engine_pending_approval()` handler was ignoring the v1 thread_id
parameter, passing `None` to the resolver. This caused two bugs:

1. An approval pending on thread A would appear in thread B's history
2. Multiple concurrent approvals for the same user returned Ambiguous

Fix: pass the v1 thread_id as a hint and update
`resolve_pending_approval_for_thread()` to match against both engine
thread UUIDs (direct match) and v1 session UUIDs embedded in the
conversation channel key ("web:{v1_uuid}").

This eliminates the unused `thread_id` variable warning in chat.rs:293.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): fix gateway auth card for skill credentials + fallback token storage

Two bugs in the gateway auth flow for v2 engine skill credentials:

1. **Frontend: auth card not shown for skill credentials**
   When `auth_url` is None but `instructions` ARE present (skill-based
   credential like `github_token`), the frontend incorrectly called
   `showConfigureModal()` (extension setup UI) instead of `showAuthCard()`
   (token paste UI). The configure modal fails for skill credentials
   (they're not extensions), permanently blocking the chat input.

   Fix: show `showAuthCard()` when instructions are present, regardless
   of `auth_url`. The configure modal is now only used when neither
   `auth_url` nor `instructions` are provided (pure extension setup).

2. **Backend: /api/chat/auth-token doesn't handle skill credentials**
   The auth-token endpoint calls `ext_mgr.configure_token()` which
   fails for skill credentials ("extension not installed"). The token
   is never stored, leaving the user stuck.

   Fix: when `configure_token()` fails with "not installed"/"not found",
   fall back to storing the token directly in SecretsStore via the
   tool registry. This bridges the frontend auth card and the v2
   engine's skill credential system.

Includes 3 E2E tests (test_v2_kernel_auth_gateway_flow.py) covering
the auth-token API path, chat-message token path, and cancel flow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve pre-existing clippy warnings and E2E test failures

Clippy fixes:
- Collapse nested `if let` into `&&` chains (effect_adapter.rs, http.rs)
- Replace `match` with `if let` for single-pattern destructure (store_adapter.rs)

E2E test fixes (test_v2_engine_oauth_google.py):
- Add `HTTP_ALLOW_LOCALHOST` and `SECRETS_MASTER_KEY` to test env (mock API
  runs on localhost — without this the HTTP tool silently blocks the request)
- Skip `test_oauth_redirect_flow` when extension returns "not installed"
  (was only checking for HTTP 404, but the endpoint returns 200 with
  success:false for missing extensions)
- Fix NoneType crash: `turns[-1].get("response", "")` returns None when
  key exists with None value — use `(... or "")` pattern instead
- Skip `test_invalid_token_paste` when credentials already stored from
  prior test (test ordering dependency)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): add logging to skill credential fallback in auth-token endpoint

Add debug/warn logging when the skill credential fallback path fires
in chat_auth_token_handler, making it easier to diagnose when the
secrets store is unavailable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(auth): strict pre-flight gate E2E test + unit integration test

Add strict assertion: mock API must receive ZERO requests when pre-flight
gate blocks (was previously lenient). The test was failing because the
E2E conftest built the binary to `target/debug/` while `cargo build`
with shared-target outputs to `~/.cargo/shared-target/debug/`. Fixed
via symlink.

Also adds `preflight_gate_blocks_missing_credential` unit test that
exercises `execute_action()` directly with a mocked ToolRegistry
containing credential mappings — verifies NeedAuthentication is returned
without executing the tool.

Diagnostic logging: warn-level log when pre-flight gate is skipped due
to missing auth_manager or credential_registry dependency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): harden auth flow — clear v2 pending state, fix binary path, SSE broadcast

Three hardening fixes from fragility audit:

1. **Clear v2 pending_auth from API path** (#5/#6): The /api/chat/auth-token
   endpoint now calls `clear_engine_pending_auth()` after storing credentials.
   Without this, the next chat message would be intercepted as a token retry
   even though auth was completed via the API endpoint.

2. **Fix E2E binary path resolution** (#1): conftest.py now resolves the
   actual cargo target-dir from ~/.cargo/config.toml instead of hardcoding
   `target/debug/`. Also adds `crates/` to the mtime check inputs so
   engine crate changes trigger rebuilds.

3. **Send AuthCompleted SSE from chat message path** (#7): The chat-message
   token submission path now broadcasts AuthCompleted via SSE (same as the
   API path), so the frontend dismisses the auth card immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): survive SSE reconnect — include pending_auth in history response

When SSE drops during an auth flow and the frontend reconnects, the auth
card was lost (DOM cleared by loadHistory) but authFlowPending remained
true, permanently blocking the chat input.

Fix: include `pending_auth` in the `/api/chat/history` response (same
pattern as `pending_approval`). The frontend's `loadHistory()` now
re-shows the auth card when `pending_auth` is present, and clears stale
auth UI state when it's absent.

Backend:
- Add `PendingAuthInfo` type to gateway types
- Add `get_engine_pending_auth()` to router (queries v2 pending_auth)
- Include `pending_auth` in all HistoryResponse constructions

Frontend:
- `loadHistory()` calls `handleAuthRequired()` when `pending_auth` present
- Clears `authFlowPending` when `pending_auth` is absent (cleanup stale state)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): robust secrets store fallback in auth-token endpoint

The skill credential fallback in /api/chat/auth-token was silently
failing when tool_registry.secrets_store() returned None.

Fix: try tool_registry first, then fall back to extension_manager.secrets().
If neither is available, return an explicit error instead of falling
through to the "Extension not installed" message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): add credential fallback to ACTUAL auth-token handler in server.rs

Root cause: there are TWO `chat_auth_token_handler` functions — one in
handlers/chat.rs (dead code, never called) and one in server.rs (the
real one registered on the route). All previous fallback fixes went to
the wrong file.

Fix: add the skill credential fallback (store directly in SecretsStore
when extension manager returns NotInstalled) to the REAL handler in
server.rs. Uses extension_manager.secrets() as fallback when
tool_registry.secrets_store() is None.

Also strengthens the E2E test to assert `success: true` in the response
body, not just HTTP 200 status (which masked this bug for weeks).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(engine): update Monty to v0.0.9 (7a0d4b7)

Removes three runtime limitations: multi-module imports now work,
datetime and json modules now available as builtins. Updates CodeAct
preamble and MONTY.md tracking doc accordingly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(gateway): remove 504 lines of dead chat handlers from handlers/chat.rs

Five handler functions in handlers/chat.rs were dead code — identical
copies existed in server.rs where the routes are actually registered:
- chat_send_handler
- chat_approval_handler
- chat_auth_token_handler (the root cause of the auth-token fallback bug)
- chat_auth_cancel_handler
- chat_history_handler (+ engine_pending_approval/auth helpers)

These duplicates caused the auth-token fallback bug: fixes were applied
to the dead copy in handlers/chat.rs while the real handler in server.rs
remained unchanged. Removing them prevents this class of bug entirely.

Kept: clear_auth_mode (shared helper), chat_events_handler,
chat_ws_handler, chat_threads_handler, chat_new_thread_handler,
and unit tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): strengthen assertions to prevent false confidence

Audit found 18 weak assertions across 7 E2E test files that could pass
even when features are broken. Key patterns fixed:

1. **Require token in mock API** (auth_flow, oauth_google):
   Replace `token_received OR "improve" in response OR "http" in response`
   with `assert token in mock_api_tokens` — must verify the mechanism,
   not just that "something happened"

2. **Remove passive assertions** (auth_flow):
   Replace `if tokens: assert True; else: pass` with
   `assert len(tokens) > 0` — silent passes hide failures

3. **Remove generic keyword matches** (approval_flow):
   Replace `"tool" in response` (matches anything) with specific
   `pending_approval is None` (verifies state change)

4. **Verify mock API received requests** (preflight, auth_flow):
   Add `assert request_count > 0` after credential storage to prove
   credential injection actually worked

5. **Poll for state change, not just text** (approval_flow):
   Wait for `pending_approval` to be cleared rather than checking
   for specific keywords in response text

6. **Add negative auth checks after cancel** (auth_cancel):
   Verify "paste your token" not in response after cancel flow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): fix skipped OAuth tests — reorder for isolation, add fake WASM extension

Two tests were skipped due to test ordering and missing infrastructure:

1. **test_invalid_token_paste**: Was skipped because credentials stored by
   prior test_api_key_then_api_call prevented auth prompt from triggering.
   Fix: reorder to run BEFORE api_key test. Now runs without skip.

2. **test_oauth_redirect_flow**: Was skipped because google_drive isn't a
   real WASM extension. Added fake WASM extension with OAuth capabilities
   (empty .wasm + capabilities.json). Still skips because wasmtime can't
   load the empty binary, but now has clear infrastructure for when a real
   test binary is available.

Also made test_api_key_then_api_call resilient to prior bad-token state
from test_invalid_token_paste (graceful fallback if no auth prompt).

Result: 24 passed, 1 skipped (down from 2 skipped).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): use real google-drive WASM binary for OAuth redirect test

Replace the fake empty WASM binary with the real google-drive tool built
from tools-src/google-drive/. The extension manager can now activate it
via wasmtime, generate a real OAuth URL, and complete the redirect flow.

Result: 25 passed, 0 skipped (was 24 passed, 1 skipped).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): full multi-tenant isolation for v2 engine

Add user_id as a first-class field to Thread, Mission, MemoryDoc, and
Project types. Update Store trait with user-scoped list methods and
admin cross-tenant methods. Enforce ownership validation throughout:

- ThreadManager: stop/inject/resume require user_id, validate ownership
- MissionManager: fire/pause/resume validate ownership, per-user learning
  missions (self-improvement, skill-extraction, conversation-insights)
- ConversationManager: validate conversation ownership on message/clear
- Bridge router: all public functions take user_id, web handlers pass
  AuthenticatedUser identity through

Shared space model for system resources:
- list_memory_docs_with_shared / list_missions_with_shared merge user's
  own docs/missions with system-owned ones (admin-installed skills,
  shared knowledge)
- System missions require admin role to manage (403 for non-admins)
- Learning missions are per-user: pause/resume is independent per user

Legacy migration: on startup, stamps owner_id onto pre-existing records
that deserialized with user_id="legacy" (serde default).

Event listener fires learning missions with the completed thread's
user_id (not a hardcoded owner_id), ensuring artifacts stay user-scoped.

8 new multi-tenancy tests covering isolation, cross-user denial,
shared visibility, admin-only management, and per-user event scoping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): audit fixes — dead code, tautological check, dedup, formatting

- Fix tautological trace check in executor/trace.rs that could never fire:
  the "missing_tool_output" diagnostic now correctly checks User-role
  messages instead of re-checking ActionResult role
- Remove dead code in loop_engine.rs: save_runtime_checkpoint,
  check_signals, SignalAction (replaced by Python orchestrator);
  simplify RuntimeCheckpoint to just persisted_state; move
  extract_final_from_text to #[cfg(test)]
- Deduplicate default_user_id() — single definition in types/mod.rs
  used by thread, memory, project, and mission types
- Add PartialEq derive to Provenance enum
- Remove phantom skill_selector.rs from CLAUDE.md module map
- Fix cargo fmt violations in test code

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(skills): audit fixes — CRLF parser bug, logging levels, dedup, SkillSource::Installed

- Fix parse_skill_md to normalize \r\n internally so callers don't need
  to pre-normalize (find_closing_delimiter byte offset was wrong on CRLF)
- Change tracing::info! to tracing::debug! in registry (3 sites) per
  CLAUDE.md logging policy — info! corrupts REPL/TUI
- Extract shared build_loaded_skill() helper, eliminating ~40 duplicated
  lines between load_and_validate_skill and load_from_content
- Add SkillSource::Installed variant so installed-dir skills have correct
  provenance metadata (was incorrectly using SkillSource::User)
- Fix misleading dedup log labels in discover_all override source strings
- Log warning on reqwest::Client builder failure instead of silent fallback
- Add regression tests for CRLF and mixed line endings in parser
- Auto-fix 155 uninlined_format_args clippy warnings in test code

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(engine): auto-fix clippy uninlined_format_args warnings

Formatting-only changes applied by cargo clippy --fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): deduplicate test Store mocks with shared InMemoryStore

Expand the shared InMemoryStore in lib.rs to support all entity types
(threads, steps, events, projects, docs, leases, missions) with proper
CRUD semantics and user_id/project_id filtering.

Replace 3 duplicate mock Store implementations (~350 lines removed):
- executor/context.rs: DocStore → InMemoryStore
- memory/retrieval.rs: DocStore → InMemoryStore
- memory/store.rs: InMemoryDocStore → InMemoryStore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): audit fixes — UTF-8 panic, missing Plan type, dedup, logging

- Fix UTF-8 panic in truncate_for_readme: use char-based truncation
  instead of byte-index slicing on user content (thread goals, messages)
- Add missing "Plan" arm to deserialize_knowledge_doc — was silently
  falling through to Note, losing doc type on workspace reload
- Extract shared event display helpers (format_action_display_name,
  interpret_message_event) to deduplicate logic between
  forward_event_to_channel and thread_event_to_app_events
- Change info! to debug! in skill_migration to avoid corrupting REPL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): move SkillTracker from capability/ to memory/

SkillTracker does MemoryDoc CRUD (load skill → update metrics → save),
not capability/lease/policy operations. It belongs with the memory
persistence layer, not the access-control layer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(engine): add v2 acceptance tests with per-agent ENGINE_V2 toggle

Add engine v2 acceptance test infrastructure and 8 initial tests proving
the v2 pipeline works end-to-end through the real agent loop.

Infrastructure:
- Add `engine_v2: bool` to AgentConfig (resolved from ENGINE_V2 env var)
- Replace process-global `is_engine_v2_enabled()` checks in agent_loop.rs
  with per-agent `self.config.engine_v2` — safe for parallel test execution
- Add `reset_engine_state()` to clear the OnceLock singleton between tests
- Add `.with_engine_v2()` to TestRigBuilder and `run_recorded_trace_v2()`

Tests (tests/e2e_engine_v2.rs):
- v2_smoke_text_response: basic text routing through engine v2
- v2_single_tool_call: echo tool dispatch via EffectBridgeAdapter
- v2_multi_tool_chain: sequential echo + time tool execution
- v2_tool_error_recovery: JSON parse error propagation and LLM recovery
- v2_multi_turn_conversation: context persistence across ConversationManager
- v2_status_events: ToolStarted/ToolCompleted event emission
- v2_recorded_telegram_check: v1 parity — replay recorded trace through v2
- v2_recorded_weather_sf: v1 parity — HTTP tool with large response

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(bridge): auto-format effect_adapter, store_adapter, codeact test

Formatting-only changes applied by cargo fmt.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(engine): remove executor/intent.rs — duplicated by Python orchestrator

signals_tool_intent() and TOOL_INTENT_NUDGE were from the Rust-native
loop path. The Python orchestrator (default.py) has its own
signals_tool_intent() implementation. No Rust code referenced the
module — safe to delete.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* merge: integrate origin/staging — fix ensure_conversation arity

Merge staging to pick up the 5th `source_channel` parameter added to
`ensure_conversation()` in V15 migration. Fix the v2 bridge call site
at router.rs:1390 to pass `Some(&message.channel)`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): formatting and no-panics check in merged router tests

Fix two CI issues from the staging merge:
- Reformat make_expected_test_state signature (single-line args)
- Add inline // safety: comment on test-only assert! to suppress
  the no-panics production code check

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): cargo-deny wildcard and git source errors

- Pin monty to rev 7a0d4b7 instead of branch=main (deterministic builds)
- Add version = "0.1.0" to ironclaw_engine and ironclaw_skills path deps
  (fixes wildcard dependency errors)
- Allow git sources for pydantic/monty and astral-sh/ruff in deny.toml
- Set allow-wildcard-paths = true (monty is git-only, no crates.io version)
- Add // safety: comments on unwrap() calls guarded by len()==1 checks
- Auto-fix clippy warnings and formatting in merged engine files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): port V1 tool-intent nudge to Python orchestrator

The V2 Python orchestrator's signals_tool_intent() was too aggressive —
matching "I can" + "call"/"fetch" anywhere in text caused false positives
on news content and past-tense summaries, creating a nudge loop that
burned 1.6M tokens on a simple query.

Ported V1's approach: strip code blocks and quoted strings, check 15
exclusion phrases, then require a future-tense prefix ("let me",
"I'll", "I will", "I'm going to") immediately followed by an action
verb. Also fixed the nudge counter to use V1's consecutive semantics —
it no longer resets on action/code responses, only on non-intent text.

Added 11 Monty-based unit tests covering true positives, true negatives,
exclusions, code blocks, quoted strings, and 3 regression tests from the
trace that triggered this fix.

Also includes: parallel store persistence, pre-fetched system docs for
orchestrator loading, and parallel action execution in scripting.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): async-first CodeAct tool dispatch via Monty ResolveFutures

Tool calls in CodeAct now use Monty's async suspension model: each tool
FunctionCall does preflight (lease/policy) synchronously, then spawns a
tokio task and calls resume_pending() to return an ExternalFuture to
Python. When Python awaits the future (or gathers multiple via
asyncio.gather), Monty yields ResolveFutures and the host resolves all
pending tools — which ran concurrently as tokio tasks.

This replaces the old synchronous dispatch_action + execute_parallel
approach with native Python async/await semantics. The LLM writes
natural Python: `await tool()` for sequential, `asyncio.gather()` for
parallel — no special API needed.

Changes:
- scripting.rs: async tool dispatch via resume_pending + ResolveFutures
  handler, preflight_action for lease/policy, PendingTool tracking
- Removed: dispatch_action, DispatchResult, handle_execute_parallel,
  execute_parallel NameLookup entry
- Builtins (FINAL, llm_query, etc.) remain synchronous
- 9 new tests: single await, 2/3-way gather, sequential chains, error
  propagation, denied tools, empty/single gather, globals, FINAL sync

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(security): address PR review findings — 11 fixes across engine and bridge

Critical:
- C3: Fix UTF-8 byte-slice panic in summarize_params (event.rs) — use
  truncate() helper instead of raw &u[..77]

High:
- H1: Stop leaking internal errors to HTTP clients — all 12 engine API
  handlers now return generic "Internal engine error" instead of e.to_string()
- H3: Add MAX_AUTH_RETRY_DEPTH=2 recursion limit to auth retry in router
- H9: Change default_trust() from Trusted to Installed (fail-closed)
- H6: Demote all warn!() to debug!() in engine crate (~25 locations) to
  prevent REPL/TUI corruption per CLAUDE.md logging policy
- H14: Remove no-op test_approval_prompt_contains_tool_name (was just `pass`)
- H2/M5: Add credential name validation (alphanumeric+underscore, max 64 chars)

Medium:
- M15: Replace raw byte-slicing with .get() in deserialize_knowledge_doc
- M14: Prevent pending_auth overwrite — check for existing entry before
  insert in text-fallback path
- M13: Log swallowed store errors in record_orchestrator_failure instead
  of silent unwrap_or_default
- LeaseNotFound: Add distinct EngineError::LeaseNotFound variant instead
  of reusing LeaseExpired for missing leases

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(docker): add python3-dev build dependency for monty/pyo3

The monty crate (embedded Python interpreter) uses pyo3-build-config
with the resolve-config feature, which probes for Python 3 headers at
compile time. Without python3-dev in the builder stage, the Docker
build fails.

Added python3-dev to the chef stage's apt-get install. This is a
build-only dependency — the runtime image (debian:bookworm-slim) is
unchanged and does not include Python.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(engine): make llm_query and llm_query_batched async via ResolveFutures

llm_query() and llm_query_batched() now use the same async dispatch as
tool calls: spawn tokio task, resume_pending(), resolve in
ResolveFutures handler. This enables:

  import asyncio
  summary, results = await asyncio.gather(
      llm_query("summarize this", context=data),
      web_search(query="latest news"),
  )

The LLM call and tool call run concurrently — saving 1-3s per step
when both are needed.

rlm_query() stays synchronous because it spawns a child Monty VM which
isn't Send (can't cross tokio::spawn boundary).

Refactored PendingTool → PendingFuture enum with Tool and Llm variants.
Extracted resolve_tool_future() and resolve_llm_future() helpers for
clean resolution in the ResolveFutures handler. Token usage from async
LLM calls is accumulated via the (ExtFunctionResult, TokenUsage) return
type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): auto-approve tool after single approval to prevent infinite loop

When a user approves a tool call with "yes" (not "always"), the engine
resumes the thread and the LLM issues a NEW tool_install call — which
triggers another approval prompt, creating an infinite approve loop.

Fix: auto-approve the tool for the session on any "yes" approval, not
just on "always". The user already consented to this tool — asking again
is a UX bug. "always" still works the same (persistent across threads).

Discovered via trace analysis: engine_trace_20260331T222859.json showed
the GitHub tool_install stuck in a Waiting→approve→Waiting loop.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(http): move leak detection before credential injection

The leak detector was scanning outbound HTTP headers AFTER the
credential registry injected Authorization headers, causing false
positives — legitimate system-injected GitHub tokens were blocked
as "secret leaks."

Fix: scan the LLM-controlled headers/URL/body first (catches actual
exfiltration attempts), THEN inject system credentials (trusted,
not LLM-controlled). This preserves leak detection for LLM-crafted
headers while allowing the credential injection system to work.

Discovered via trace: engine_trace_20260331T225126.json showed
http(api.github.com) blocked with "Secret leak blocked: pattern
'header:Authorization' matched 'github_fine_grained_pat'".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(bridge): 4 integration fixes — auth cancel, history, per-user projects, plan scope

P1: Clear engine pending auth on /api/chat/auth-cancel
  The cancel endpoint only cleared v1 session state, leaving
  bridge::pending_auth active. Next message was consumed as
  a token value instead of normal input.

P2: Populate pending_auth in chat history response
  All 4 HistoryResponse code paths hardcoded pending_auth: None.
  On SSE reconnect/refresh during an auth flow, the UI cleared
  the auth card while the bridge was still waiting for a token.
  Now surfaces engine pending-auth state via get_engine_pending_auth().

P1: Per-user default project instead of global owner project
  Engine init created one project under owner_id and used it for
  all users. In multi-user gateway deployments, non-owner threads
  and missions were created inside the owner's project, making
  /api/engine/projects return nothing for non-owner accounts.
  Added resolve_user_project() that creates per-user projects.

P2: Include thread_id in plan_update SSE events
  plan_update events had thread_id: None, so plan checklists from
  background threads rendered in whichever chat was open. Now
  carries ctx.conversation_id so clients can scope plan rendering.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): address review feedback — lease audit, policy logging, char count

From ilblackdragon's review:

1. Lease revocation now stores reason for audit trail — added
   revoked_reason: Option<String> to CapabilityLease, logged at
   debug! level on revocation (was silently discarding _reason param)

2. Policy denial decisions now logged at debug! level with action
   name, capability, and reason — enables incident investigation
   for privilege escalation attempts

3. Fixed byte/char count mismatch in compact_output_metadata —
   stdout.len() (bytes) was displayed as "chars" but
   stdout.chars().count() was used for truncation. Now consistent.

4. Store trait splitting (H3) acknowledged as follow-up — documenting
   that default impls are stubs, not real behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(engine): address zmanian review — orchestrator gate, sandbox tests, TOCTOU, matching

C1: Add ORCHESTRATOR_SELF_MODIFY disable flag (default: off)
  Runtime orchestrator loading is now disabled by default. Only the
  compiled-in v0 runs unless explicitly opted in. Prevents unreviewed
  self-improvement patches from executing with full tool access.

C2: Add 3 Monty sandbox security negative tests (224 total)
  - sandbox_denies_os_operations: os.system() blocked
  - sandbox_enforces_resource_limits: infinite loop terminated
  - sandbox_restricts_imports: subprocess import blocked

H3: Fix TOCTOU race in lease find+consume
  Added LeaseManager::find_and_consume() that atomically finds a lease
  and consumes a use under a single write lock. structured.rs now uses
  this instead of separate find (read lock) + consume (write lock).

M3: Fix ActionCondition::ActionMatches substring → exact match
  "delete" no longer matches "undelete_restore". Changed contains()
  to == for exact action name matching.

Also: log store errors in loop_engine.rs instead of silent
unwrap_or_default().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(security): block orchestrator/prompt writes when self-modify disabled

Defense-in-depth: three layers now enforce ORCHESTRATOR_SELF_MODIFY:

1. memory_write tool: blocks writes to orchestrator:* and prompt:*
   paths with a clear error message when the flag is off

2. HybridStore adapter: save_memory_doc rejects protected docs
   (except system-internal v0 seeding and failure tracking) with
   EngineError::AccessDenied

3. Mission system: process_self_improvement_output skips prompt
   additions when the flag is off, logging the skip at debug level

Previously, ORCHESTRATOR_SELF_MODIFY only controlled loading — the
LLM could still write malicious orchestrator/prompt MemoryDocs via
memory_write, which would take effect once self-modify was enabled.
Now writes are blocked at all layers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(gate): unified ExecutionGate abstraction for approval + auth flows (#1818)

* feat(gate): unified ExecutionGate abstraction for approval + auth flows

Introduce a composable gate pipeline that structurally prevents the 6
recurring bug categories found across ~50 approval/auth fixes:
TOCTOU races, cross-channel hijacking, privilege escalation via
composition, silent error swallowing, state loss on restart, and
execution path mismatch between approval and authentication.

Engine crate (ironclaw_engine::gate):
- ExecutionGate trait with priority-ordered GatePipeline (fail-closed)
- GateDecision (Allow/Pause/Deny) — no None variant by construction
- ResumeKind (Approval/Authentication/External) — unified pause type
- GateResolution with Cancelled variant (fixes cancel-as-approval misrouting)
- ToolTier classification (ReadOnly < Stateful < Privileged < Administrative)
- LeaseGate — deny if no valid capability lease (priority 10)
- ThreadOutcome::GatePaused + EngineError::GatePaused variants

Lease system:
- LeasePlanner now thread-type-aware (was grant-everything):
  Foreground=all, Research=read+stateful, Mission=no-admin
- derive_child_leases() with intersection semantics for child threads
- Children never exceed parent expiry or budget

Bridge layer (src/gate + src/bridge):
- PendingGateStore: Mutex-based (not RwLock), keyed by (user_id, thread_id)
- take_verified(): atomic request_id + channel + expiry check — single lock
- GatePersistence trait for restart recovery
- TRUSTED_GATE_CHANNELS and RESERVED_CHANNEL_NAMES constants
- resolve_gate() public API with auto-approve rollback on resume failure
- Concrete gates: ApprovalGate, AuthenticationGate, HookGate,
  RateLimitGate, RelayChannelGate

Python orchestrator:
- gate_paused outcome handling for both Tier 0 and Tier 1 paths
- Rust-side GatePaused error → {"gate_paused": true} JSON mapping
- loop_engine.rs safety net includes GatePaused in Waiting transition

46 new tests across both crates, including regression tests for:
74cbe5c2, 52d935d7, 5d1d504e, 427f908e, 92138b8c, aa151d9f,
e3b66f69, 09e1c97a, 0e5f1b12, e75fa8c4, 49b4c398

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(gate): add integration tests for unified gate lifecycle

17 integration tests covering the full gate abstraction:

Engine-level (ThreadManager → EffectExecutor → GatePaused → Waiting):
- gate_paused_transitions_thread_to_waiting: Tier 0 tool call → GatePaused
  outcome → thread state == Waiting (regression: 67d5a473)
- gate_paused_authentication_carries_credential_name: auth gate carries
  credential info through the outcome

PendingGateStore lifecycle:
- pending_gate_full_lifecycle: insert → peek → take_verified → removed
- cross_channel_approval_blocked: telegram gate rejected from slack (5d1d504e)
- trusted_channel_can_resolve_any_gate: web/gateway bypass channel check
- gate_scoped_to_thread_no_leakage: thread A gate invisible to B (e3b66f69)
- expired_gate_cannot_be_resolved: TTL enforcement
- wrong_request_id_does_not_consume_gate: stale ID doesn't eat gate (74cbe5c2)
- concurrent_resolution_exactly_one_succeeds: TOCTOU prevention (52d935d7)
- persistence_round_trip_survives_restart: GatePersistence → restore

Lease system:
- lease_planner_research_excludes_privileged: Research = ReadOnly+Stateful
- lease_planner_mission_excludes_denylisted: Mission excludes Administrative
- child_lease_inherits_subset_of_parent: intersection semantics
- expired_parent_yields_no_child_leases: fail-closed
- lease_gate_denies_without_lease / allows_with_valid_lease
- pipeline_first_deny_wins: GatePipeline composition

Also wires GatePaused through structured.rs and scripting.rs executors
so EffectExecutor::execute_action() can return the new error variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gate): address review findings — redaction, wildcard lease, TOCTOU, rollback

Critical fixes:
- C1: Replace .expect() with .ok_or() in take_verified() (production panic)
- C2: Redact sensitive params via redact_params() before SSE broadcast and
  PendingGate storage. Add tools() accessor to EffectBridgeAdapter.
- C3: Fix wildcard parent lease (granted_actions=[]) producing wildcard child
  instead of requested subset. Add regression test
  wildcard_parent_lease_gives_requested_subset_not_wildcard.

Major fixes:
- M1: Remove false panic safety documentation from GatePipeline (async
  catch_unwind impractical with borrowed context). Docs now accurately state
  gate implementations must not panic.
- M2: Batch child lease insertion under single write lock instead of
  per-iteration locking in derive_child_leases().
- M3: Auto-approve rollback on resume failure now revokes both underscore
  and hyphenated tool name variants.
- M4: Log persistence.remove() failures at debug level instead of silently
  discarding with let _.
- M5: expire_stale() now calls persistence.remove() for each expired gate,
  preventing indefinite storage accumulation.

Minor fixes:
- m3: Downgrade gate insert failure from warn! to debug! (AlreadyExists
  is a normal race condition).
- m5: Fix GateContext doc claiming "all fields are borrowed" — ThreadId and
  ExecutionMode are Copy/inline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(gate): add InteractiveAutoApprove execution mode

Add ExecutionMode::InteractiveAutoApprove for foreground threads with
AGENT_AUTO_APPROVE_TOOLS=true. In this mode:

- Never tools: allowed (same as all modes)
- UnlessAutoApproved tools (shell, file_write, http, etc.): auto-approved
  without prompting — no approval pause
- Always tools (destructive operations): still pause for explicit approval

All other safeguards remain active: leases, rate limits, hooks, relay
channel checks, authentication gates, parameter redaction.

This maps the existing v1 auto_approve_tools config flag into the v2
gate abstraction, providing a "power user" mode where experienced users
skip repetitive approval prompts while retaining defense-in-depth.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add --auto-approve flag for autonomous foreground mode

ironclaw run --auto-approve

Wires the existing AGENT_AUTO_APPROVE_TOOLS config into a CLI flag
via set_runtime_env() (thread-safe override, no unsafe set_var).
Activates InteractiveAutoApprove execution mode where:
- shell, file_write, http, etc. execute without prompting
- Always-gated destructive operations still pause for approval
- All other safeguards remain active (leases, rate limits, hooks, auth)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): suppress false positive in check_no_panics for test assert!

The CI script's brace-depth tracker loses #[cfg(test)] mod tests {}
context when the sanitizer state carries over from earlier string
processing. Add // safety: test-only annotation to the affected assert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Harden engine gate recovery and thread-scoped auth

* fix(engine): never delete LLM output data, fix mission thread visibility

Mission detail pages showed no threads because cleanup_terminal_state()
deleted thread/event/step data from the database. LLM execution data is
the most valuable information — it must never be deleted.

Changes:
- cleanup_terminal_state() now only evicts from in-memory caches, never
  deletes database rows (threads, events, steps all preserved)
- load_thread/load_steps/load_events fall back to database on cache miss
- backfill_archived_threads() recovers mission threads on startup from
  both active DB path and legacy archive summaries
- Document "never delete LLM output" principle in CLAUDE.md,
  engine CLAUDE.md, and database rules
- Fix pre-existing compile error in orchestrator (params use-after-move)
- Add ApprovalRequested event fields (parameters, description,
  allow_always, gate_name, params_summary) for richer gate UX

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Refactoring approval gates

* Working on improving authentication/approval flows

* Updating the implementaton plan

* Stabilize engine v2 transcripts and extension tests

* fix(engine): address PR #1557 review feedback — security, types, validation

- Remove credential-backed HTTP auto-approval bypass (zmanian H2): credential
  presence no longer skips the approval gate in EffectBridgeAdapter
- Add GrantedActions enum (zmanian M1, ilblackdragon #6): replace implicit
  empty-vec-means-wildcard with explicit All/Specific variants, backward-
  compatible serde
- Validate lease duration/max_uses at grant time (zmanian M2, ilblackdragon #5):
  reject non-positive durations and zero max_uses
- Fix byte/char label mismatch in compact_output_metadata (ilblackdragon #4,
  zmanian #5): use chars().count() consistently
- Add 6 Monty sandbox security negative tests (zmanian C2): OS call denial,
  file access, socket access, resource limits, lease enforcement, syntax errors
- Fix pre-existing clippy warnings in structured.rs and scripting.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve CI failures from staging merge (clippy, no-panics)

- router.rs: Box PendingGateResolution::Resolved to fix large_enum_variant,
  add safety comments for unwrap() calls, simplify Option::map
- auth_manager.rs: allow await_holding_lock in tests (env guard must span test)
- selector.rs: remove unnecessary double parentheses

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix remaining fmt diff in router.rs from staging merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: include unstaged merge changes (server.rs 2-arg calls, test updates, snapshots)

- server.rs: pass thread_id to clear_engine_pending_auth (2-arg signature)
- server.rs: seed workspace on resolve, add test
- effect_adapter.rs: update test expectation for approval-before-auth ordering
- cli snapshots: add --auto-approve flag

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 19:22:27 -07:00
Illia Polosukhin
806d402876 feat: chat onboarding and routine advisor (#927) 
* feat: port NPA psychographic profiling system into IronClaw

Port the complete psychographic profiling system from NPA into IronClaw,
including enriched profile schema, conversational onboarding, profile
evolution, and three-tier prompt augmentation.

Personal onboarding moved from wizard Step 9 to first assistant
interaction per maintainer feedback — the First Contact system prompt
block now instructs the LLM to conduct a natural onboarding conversation
that builds the psychographic profile via memory_write.

Changes:
- Enrich profile.rs with 5 new structs, 9-dimension analysis framework,
  custom deserializers for backward compatibility, and rendering methods
- Add conversational onboarding engine with one-step-removed questioning
  technique, personality framework, and confidence-scored profile generation
- Add profile evolution with confidence gating, analysis metadata tracking,
  and weekly update routine
- Replace thin interaction style injection with three-tier system gated on
  confidence > 0.6 and profile recency
- Replace wizard Step 9 with First Contact system prompt block that drives
  conversational onboarding during the user's first interaction
- Add autonomy progression to SOUL.md seed and personality framework to
  AGENTS.md seed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: replace chat-based onboarding with bootstrap greeting and workspace seeds

Remove the interactive onboarding_chat.rs engine in favor of a simpler
bootstrap flow: fresh workspaces get a proactive LLM greeting that
naturally profiles the user. Identity files are now seeded from
src/workspace/seeds/ instead of being hardcoded. Also removes the
identity-file write protection (seeds are now managed), adds routine
advisor integration, and includes an e2e trace for bootstrap greeting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(safety): sanitize identity file writes via Sanitizer to prevent prompt injection

Identity files (SOUL.md, AGENTS.md, USER.md, IDENTITY.md) are injected into
every system prompt. Rather than hard-blocking writes (which broke onboarding),
scan content through the existing Sanitizer and reject writes with High/Critical
severity injection patterns. Medium/Low warnings are logged but allowed.

Also clarifies AGENTS.md identity file roles (USER.md = user info, IDENTITY.md =
agent identity) and adds IDENTITY.md setup as an explicit bootstrap step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update profile_onboarding_completed comment to reflect current wiring

The field is now actively used by the agent loop to suppress BOOTSTRAP.md
injection — remove the stale "not yet wired" TODO.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): use env_or_override for NEARAI_API_KEY in model fetch config

When the user authenticates via NEAR AI Cloud API key (option 4),
api_key_login() stores the key via set_runtime_env(). But
build_nearai_model_fetch_config() was using std::env::var() which
doesn't check the runtime overlay — so model listing fell back to
session-token auth and re-triggered the interactive NEAR AI
authentication menu.

Switch to env_or_override() which checks both real env vars and the
runtime overlay.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(agent): correct channel/user_id in bootstrap greeting persist call

persist_assistant_response was called with channel="default",
user_id="system" but the assistant thread was created via
get_or_create_assistant_conversation("default", "gateway") which owns
the conversation as user_id="default", channel="gateway". The mismatch
caused ensure_writable_conversation to reject the write with:

  WARN Rejected write for unavailable thread id user=system channel=default

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web): remove all inline event handlers for CSP compliance

The Content-Security-Policy header (added in f48fe95) blocks inline JS
via script-src 'self'. All onclick/onchange attributes in index.html
are replaced with getElementById().addEventListener() calls. Dynamic
inline handlers in app.js (jobs, routines, memory breadcrumb, code
blocks, TEE report) are replaced with data-action attributes and a
single delegated click handler on document.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(agent): align bootstrap message user/channel and update fixture schema field

- Bootstrap IncomingMessage now uses ("default", "gateway") consistently
  with persist and session registration calls
- Update bootstrap_greeting.json fixture: schema_version → version to
  match current PROFILE_JSON_SCHEMA

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: cargo fmt

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(safety): address PR review — expand injection scanning and harden profile sync

- BOOTSTRAP.md: fix target "profile" → "context/profile.json" so the
  write hits the correct path and triggers profile sync
- IDENTITY_FILES: add context/assistant-directives.md to the scanned
  set since it is also injected into the system prompt
- sync_profile_documents(): scan derived USER.md and assistant-directives
  content through Sanitizer before writing, rejecting High/Critical
  injection patterns
- profile_evolution_prompt(): wrap recent_messages_summary in <user_data>
  delimiters with untrusted-data instruction to mitigate indirect
  prompt injection
- routine-advisor skill: update cron examples from 6-field to standard
  5-field format for consistency with routine_create tool docs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: cargo fmt

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): detect env-provided LLM keys during quick-mode onboarding

Quick-mode wizard now checks LLM_BACKEND, NEARAI_API_KEY,
ANTHROPIC_API_KEY, and OPENAI_API_KEY env vars to pre-populate
the provider setting, so users aren't re-prompted for credentials
they already supplied. Also teaches setup_nearai() to recognize
NEARAI_API_KEY from env (previously only checked session tokens).

Includes web UI cleanup (remove duplicate event listeners) and
e2e test response count adjustment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): update routine_create_list to expect 7-field normalized cron

The cron normalizer now always expands to 7-field format, so the
stored schedule is "0 0 9 * * * *" not "0 0 9 * * *".

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(setup): skip LLM provider prompts when NEARAI_API_KEY is present

In quick mode, if NEARAI_API_KEY is set in the environment and the
backend was auto-detected as nearai, skip the interactive inference
provider and model selection steps. The API key is persisted to the
secrets store and a default model is set automatically.

Also simplify the static fallback model list for nearai to a single
default entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: unify default model, static bootstrap greeting, and web UI cleanup

- Add DEFAULT_MODEL const and default_models() fallback list in
  llm/nearai_chat.rs; use from config, wizard, and .env.example so the
  default model is defined in one place
- Restore multi-model fallback list in setup wizard (was reduced to 1)
- Move BOOTSTRAP_GREETING to module-level const (out of run() body)
- Replace LLM-based bootstrap with static greeting (persist to DB before
  channels start, then broadcast — eliminates startup LLM call and race)
- Fix double env::var read for NEARAI_API_KEY in quick setup path
- Move thread sidebar buttons into threads-section-header (web UI)
- Remove orphaned .thread-sidebar-header CSS and fix double blank line
- Update bootstrap e2e test for static greeting (no LLM trace needed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(safety): move prompt injection scanning into Workspace write/append

Addresses PR #927 review comments (#1, #3) — identity file write
protection and unsanitized profile fields in system prompt.

Instead of scanning at the tool layer (memory.rs) or the sync layer
(sync_profile_documents), injection scanning now lives in
Workspace::write() and Workspace::append() for all files that are
injected into the system prompt. This ensures every code path that
writes to these files is protected, including future ones.

- Add SYSTEM_PROMPT_FILES const and reject_if_injected() in workspace
- Add WorkspaceError::InjectionRejected variant
- Add map_write_err() in memory.rs to convert InjectionRejected to
  ToolError::NotAuthorized
- Remove redundant IDENTITY_FILES/Sanitizer from memory.rs
- Remove redundant sanitizer calls from sync_profile_documents()
- Move sanitization tests to workspace::tests
- Existing integration test (test_memory_write_rejects_injection)
  continues to pass through the new path

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot review — merge marker order, orphan thread, stale fixture

- merge_profile_section: search for END marker after BEGIN position to
  avoid matching a stray END earlier in the file
- Bootstrap phase 2: use get_or_create_session + Thread::with_id instead
  of resolve_thread(None) to avoid creating an orphan thread
- setup_nearai: use env_or_override for NEARAI_API_KEY consistency with
  runtime overlay
- Delete orphaned bootstrap_greeting.json fixture (no test references it)
- Add test_merge_end_marker_must_follow_begin regression test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fmt agent_loop.rs (CI stable rustfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: lazy-init sanitizer, check profile non-empty before skipping bootstrap

Address Copilot review:
- Use LazyLock<Sanitizer> to avoid rebuilding Aho-Corasick + regexes
  on every workspace write
- has_profile check now requires non-empty content, not just file
  existence, to prevent empty profile.json from suppressing onboarding
- Add seed_tests integration tests (libsql-backed) verifying:
  - Empty profile.json does not suppress BOOTSTRAP.md seeding
  - Non-empty profile.json correctly suppresses bootstrap for upgrades

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: duplicate language handler, empty LLM_BACKEND, test_rig style

Address Copilot review on PR #927:
- Remove duplicate language-option click listeners (delegated
  data-action handler already covers them)
- Guard LLM_BACKEND env prefill against empty string to prevent
  suppressing API-key-based auto-detection
- Use destructured local `keep_bootstrap` instead of `self.keep_bootstrap`
  in test_rig for consistency after destructure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update stale BOOTSTRAP.md write-protection comment [skip-regression-check]

BOOTSTRAP.md is now in SYSTEM_PROMPT_FILES and gets injection scanning
on write. The old comment incorrectly stated it was not write-protected.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace debug_assert panics with graceful error returns [skip-regression-check]

debug_assert! in execute_tool_with_safety and JobContext::transition_to
panicked in test builds before the graceful error path could run.
Existing tests (test_cancel_job_completed, test_execute_empty_tool_name_returns_not_found)
already cover these paths — they were the ones failing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot review — schema label, env var check, path normalization, profile validation

1. Label ANALYSIS_FRAMEWORK and PROFILE_JSON_SCHEMA sections separately
   in bootstrap prompt so the LLM knows which blob is the target structure.

2. Wizard quick-mode backend auto-detection now rejects empty env vars
   (std::env::var().is_ok_and(|v| !v.is_empty())) to avoid selecting the
   wrong backend when e.g. NEARAI_API_KEY="" is set.

3. Normalize the target path before comparing with paths::PROFILE in
   memory_write so non-canonical variants like "context//profile.json"
   still trigger profile sync.

4. seed_if_empty now requires valid JSON parse of context/profile.json
   before treating it as a populated profile. Corrupted content no longer
   permanently suppresses bootstrap seeding.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt

* fix: address Copilot review — append scan, profile validation, env_or_override

1. Workspace::append() now scans the combined content (existing + new)
   for prompt injection, not just the appended chunk. Prevents split-
   injection evasion across multiple appends.

2. seed_if_empty() now deserializes into PsychographicProfile instead of
   serde_json::Value for profile validation. Stray/legacy JSON that
   doesn't match the expected schema no longer suppresses bootstrap.

3. Wizard quick-mode backend auto-detection now uses env_or_override()
   to honor runtime overlays and injected secrets. LLM_BACKEND value
   is trimmed before storage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add bootstrap_onboarding_clears_bootstrap E2E trace test

Exercises the full onboarding flow end-to-end:
1. Bootstrap greeting fires automatically on fresh workspace
2. User converses for 3 turns (name, tools, work style)
3. Agent writes psychographic profile to context/profile.json
4. Profile sync generates USER.md and assistant-directives.md
5. Agent writes IDENTITY.md (chosen persona)
6. Agent clears BOOTSTRAP.md via memory_write(target: "bootstrap")

Verifies:
- BOOTSTRAP.md is non-empty before onboarding, empty after
- bootstrap_completed flag is set
- Profile contains expected user data (name, profession, interests)
- USER.md contains profile-derived content (name, tone, profession)
- Assistant-directives.md references user and communication style
- IDENTITY.md contains agent's chosen persona name
- All memory_write calls succeed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot review — slash collapse, env_or_override, cron trim [skip-regression-check]

1. memory.rs path normalization now uses the same char-by-char loop as
   Workspace::normalize_path() to fully collapse consecutive slashes
   (e.g. "context///profile.json" → "context/profile.json").

2. Quick-mode NEARAI_API_KEY check (line 239) now uses env_or_override()
   consistently with the backend auto-detection block above it.

3. normalize_cron_expression() trims input before field counting so the
   passthrough branch (7+ fields) also strips whitespace.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jay Zalowitz <jayzalowitz@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 22:20:34 -07:00
Henry Park
428303af11 Redesign routine create requests for LLMs (#1147) 
* Redesign routine create requests for LLMs

* Fix panic-check false positives in routine tests

* Tighten routine schema requirements

* Tighten routine schema tests

* Mark test assertions safe for CI scan

* Align test assertions with panic scan

* Polish routine schema metadata

* Simplify routine test assertions

* Improve tool discovery guidance

* Clarify lightweight routine delivery prompts

* Fix routine delivery target defaults
 2026-03-18 09:04:00 -07:00
Illia Polosukhin
f05896fe6a Migrate GitHub webhook normalization into github tool (#758) 
* Add event-triggered routines and workflow skill templates

* Add generic host-verified webhook ingress for tools

* Migrate GitHub webhook normalization into github tool

* Bump github tool registry version

* Stabilize trace E2E test rig and approval behavior

* Add reusable gateway workflow harness with mock LLM server (#762)

* Add reusable gateway workflow test harness with mock LLM server

* Fix clippy issues in workflow harness

* Stabilize trace E2E test rig and approval behavior

* Address PR review feedback on gateway workflow harness

- Extract shared TestChannelHandle into test_channel.rs with name override
  support, eliminating ~55 lines of duplication between test_rig.rs and
  gateway_workflow_harness.rs
- Remove redundant RoutineEngine creation that was immediately overwritten
  by Agent::run()
- Replace flaky sleep(500ms) with polling loop for routine run count check
- Use components.context_manager instead of creating a fresh ContextManager
  for job tools, ensuring agent and tools share the same instance

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix import ordering in gateway_workflow_harness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR #758 review feedback

- Fix header_value to use fully case-insensitive lookup (iterate with
  to_ascii_lowercase) instead of checking only exact/lower/upper variants
- Change comment_id from u32 to u64 to handle GitHub's billion-range IDs
- Remove handle_webhook from LLM-facing JSON schema to prevent direct
  invocation bypassing HMAC verification
- Rename enrichment keys from repository/sender to repository_name/
  sender_login to preserve original JSON objects in webhook payloads
- Remove put_string_normalized helper (no longer needed)
- Replace no-op tests (test_validate_event_in_create_pr_review,
  test_validate_merge_method) with test_header_value_case_insensitive
- Add README docs for 6 undocumented actions (list_issue_comments,
  create_issue_comment, list_pull_request_comments,
  reply_pull_request_comment, get_pull_request_reviews,
  get_combined_status)
- Add comment explaining max_tool_calls <= 8 bound in e2e test
- Fix gateway workflow harness: add webhook_capability with secret auth
  to MockGithubWebhookTool, matching staging's hardened webhook security
- Fix merge artifacts: remove duplicate test function, orphaned code
  fragment in e2e_routine_heartbeat

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix formatting in gateway workflow harness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address Copilot review: filter keys, pr_number fallback, feature gate, version alignment

- Update SKILL.md and workflow-routines.md templates to use `repository_name`
  and `sender_login` (matching enriched payload field names)
- Mark webhook HMAC secret as required in SKILL.md prerequisites
- Fall back to `/issue/number` for `pr_number` on issue_comment PR webhooks
- Gate `gateway_workflow_harness` module behind `#[cfg(feature = "libsql")]`
- Align tool version to 0.2.1 in Cargo.toml and capabilities.json

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-12 01:52:47 +00:00
Illia Polosukhin
6e1ed939cc Add event-triggered routines and workflow skill templates (#756) 
* Add event-triggered routines and workflow skill templates

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for event_emit security and quality

Security fixes:
- Require approval (UnlessAutoApproved) for event_emit, matching routine_fire
- Enable sanitization on event_emit payload (external JSON reaches LLM)
- Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id

Correctness fixes:
- Rename source → event_source in event_emit for consistency with routine_create
- Use json_value_as_filter_string for filter parsing (handles numbers/booleans)
- Case-insensitive matching for event source and event_type
- Add debug logging for missing filter keys in payload
- Fix skill_install_routine_webhook_sim test missing .with_skills()
- Fix schema_validator test for event_emit payload properties

Code quality:
- Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout)
- Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs
- Add test section headers in e2e_routine_heartbeat.rs
- Clarify event_emit description to specify system_event routines only

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: make routine_system_event_emit test create routine before emitting

- Add routine_create step to trace fixture so event_emit has a matching
  routine to fire
- Assert fired_routines > 0, not just key presence (Copilot review)
- Add .with_auto_approve_tools(true) since event_emit now requires approval

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: renumber test headers after system_event test insertion

Test 4 was duplicated (routine_cooldown and heartbeat_findings).
Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge staging and add missing RoutineEngine args in test

RoutineEngine::new on staging requires `tools` and `safety` params.
Update system_event_trigger_matches_and_filters test to pass them.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address new Copilot review comments

- Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim
  test so event_emit doesn't block on approval
- Fix module-level doc comment for event_emit to specify system_event trigger

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deduplicate json_value_as_string helper

Remove private `json_value_as_string` from routine_engine.rs and use
the identical public `json_value_as_filter_string` from routine.rs,
eliminating divergence risk. (Copilot review)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-10 11:08:04 -07:00
Illia Polosukhin
3b57d5bec9 chore: add reviewer-feedback guardrails (CLAUDE.md, pre-commit hook, skill) (#665) 
* chore: add reviewer-feedback guardrails (CLAUDE.md, pre-commit hook, skill)

Analysis of ~50 PRs from the past week identified 10 recurring themes
in Copilot and Gemini code review comments. This change addresses them
at development time through three layers:

1. CLAUDE.md additions (7 new rules):
   - Transaction safety for multi-step DB operations
   - UTF-8 string safety (no byte-index slicing)
   - Case-insensitive comparisons for paths/media types
   - Decorator/wrapper trait method delegation
   - Sensitive data redaction in logs/SSE
   - tempfile crate for test temporary files
   - Trust boundaries for worker container data

2. Pre-commit hook (scripts/pre-commit-safety.sh):
   Mechanical checks for unsafe byte slicing, case-sensitive
   extension comparisons, hardcoded /tmp paths, unredacted
   tool parameter logging, and non-transactional DB operations.
   Installed via dev-setup.sh alongside existing commit-msg hook.

3. Review checklist skill (skills/review-checklist/SKILL.md):
   Activates on "review"/"merge" keywords. Covers the judgment-based
   items that can't be linted: transaction safety, SSRF validation,
   approval checks, decorator delegation, test quality, and doc accuracy.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback on pre-commit-safety.sh

- Cache diff output in variable to avoid ~10 redundant git diff calls (Gemini)
- Add early exit when no .rs files are changed (Gemini)
- Fix header comment: list all 5 checks, not just 4 (Copilot)
- Fix check 2 comment: only mentions file extensions, not media types (Copilot)
- Add resolve_base_ref() with fallback candidates instead of hardcoded
  origin/main for standalone mode (Copilot)
- TX check: use -W (function context) to reduce false positives, honor
  // safety: suppression, print triggering lines (Copilot)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-07 21:20:37 +00:00
Pierre LE GUEN
bf2a08be94 feat: add local-test skill and Dockerfile.test for web gateway testing (#524) 
Add Dockerfile.test as reusable infrastructure for spinning up local
test instances with libsql (no PostgreSQL dependency). Defaults to
port 3003 to avoid conflict with dev server.

Add local-test workspace skill that teaches the agent how to build,
run, and test against local Docker containers using Chrome MCP browser
automation tools. Covers LLM backend configuration, multi-instance
testing, cleanup, and troubleshooting.
 2026-03-04 05:53:35 +00:00
Zaki Manian
2544df1c4a feat: add web UI test skill for Chrome extension (#302) 
* feat: add web UI test skill for Chrome extension testing

Add a SKILL.md checklist for manually testing the IronClaw web gateway
UI using the Claude for Chrome browser extension. Covers connection,
chat, skills tab (search, install by search, install by URL, remove),
and smoke tests for other tabs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use placeholder token and correct cleanup path per review

- Replace hardcoded test123 token with <your-token> placeholder
- Fix cleanup path: ~/.ironclaw/installed_skills/ (not skills/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 08:49:35 +00:00