lywsvip/ironclaw
1
0
Fork 0
mirror of https://github.com/nearai/ironclaw.git synced 2026-09-03 08:06:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
4dea5dd5dabe3e078876b3fa8f9d23f01d596433
ironclaw/Cargo.toml
Illia Polosukhin ff119531d4 test(replay): promote engine traces to insta-backed snapshot gate (#2621) 
* test(replay): promote engine replay traces to insta-backed snapshot gate

Adds a ReplayOutcome snapshot type, a replay-gate CI workflow, and a
developer script wrapper for cargo-insta. Replaces unreviewable 3,000-line
JSON diffs on engine changes with a YAML snapshot of the observable run
shape (tool sequence, final state, retrospective analyzer issues).

Why: engine v2 live-fixture traces had grown past reviewability. A single
prompt-wording change could move the whole fixture, and reviewers had no
way to see which behaviour actually changed. Splitting the fixture into a
"replay driver" (JSON stays in tests/fixtures/) and a "regression
snapshot" (YAML in tests/snapshots/) gives reviewers a narrow, stable diff
to approve, while keeping the full recorded context for deterministic
replay.

Changes:
- `tests/support/replay_outcome.rs` — ReplayOutcome + assert_replay_snapshot!
  macro; snapshots include retrospective analyzer output (TraceIssue
  severity/category) via a new `ironclaw::bridge::engine_retrospectives_for_test()`
  helper that runs `build_trace()` over engine threads
- `tests/e2e_engine_v2.rs` — three POC snapshot tests
  (single_tool_echo, tool_error_recovery, zizmor_scan_v2)
- `tests/e2e_bug_bash_snapshots.rs` + `tests/fixtures/llm_traces/bug_bash/`
  — bug-regression fixture template, mapped to open issues in the README
- `.github/workflows/replay-gate.yml` — cargo insta test --check on
  engine/agent/LLM/tools/bridge path changes; rejects committed .snap.new
- `scripts/replay-snap.sh` — review/accept/test/record wrappers around
  cargo-insta and IRONCLAW_RECORD_TRACE
- `scripts/trace-coverage.sh` — reports EventKind variants with
  snapshot coverage; `--strict` mode for future CI promotion
- `tests/e2e_live.rs` — `#[ignore]` swapped for
  `cfg_attr(not(feature="replay"), ignore)` so the replay CI job can
  run the scenarios without `-- --ignored`
- `Cargo.toml` — new `replay = ["libsql"]` feature; insta gains
  the `yaml` feature
- `tests/fixtures/llm_traces/README.md` — documents the two-role
  driver/snapshot split

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(replay): address PR #2621 review + swap cargo-insta installer

Review fixes:

- Replay gate was missing the bug-bash snapshot suite. Adds
  `tests/e2e_bug_bash_snapshots.rs` to the workflow paths trigger and the
  `cargo insta test --check` invocation so bug-regression snapshots are
  actually gated. (copilot-pull-request-reviewer)

- `cargo install cargo-insta --locked` added ~40s of cold-cache compile
  to the gate. Swapped for `taiki-e/install-action@v2`, which downloads
  a precompiled binary in a few seconds. Also updated
  `scripts/replay-snap.sh` to *fail closed* when cargo-insta is missing
  instead of silently auto-installing it. (gemini-code-assist)

- `engine_retrospectives_for_test` was `pub` and re-exported under the
  default-enabled `libsql` feature, contradicting its "not part of any
  public API" doc. Split the re-export, kept `reset_engine_state` as a
  plain `pub use`, and hid `engine_retrospectives_for_test` behind
  `#[doc(hidden)]` — it still needs to cross the crate boundary for
  integration tests (which live in a separate crate, so `#[cfg(test)]`
  doesn't reach them), but no longer appears in published docs.
  (copilot-pull-request-reviewer)

- Added an explicit "caller must serialize" note on
  `engine_retrospectives_for_test` explaining the `ENGINE_STATE`
  singleton and pointing new callers at `engine_v2_test_lock()` /
  `reset_engine_state()`. Matches what the existing snapshot tests
  already do. (gemini-code-assist)

Doc corrections:

- `snapshot_zizmor_scan_v2` doc claimed the snapshot pinned
  `ApprovalNeeded` events and response wording — it doesn't. Rewrote to
  describe what the snapshot actually asserts (tool order, step count,
  retrospective issues, final state). (copilot-pull-request-reviewer)

- `llm_call_count` was documented as "bucketed" but passed through
  verbatim. Updated the field doc to reflect the raw value. Bucketing
  wasn't needed because fixtures are deterministic. (copilot-pull-request-reviewer)

- `src/bridge/router.rs` doc referenced a non-existent
  `ReplayOutcome.trace_issues` field — the struct uses `engine_threads`.
  Fixed the reference. (copilot-pull-request-reviewer)

- `scripts/trace-coverage.sh` header claimed CI runs it with `--strict`;
  the workflow runs it in advisory mode. Rewrote the header to match,
  with a pointer for when to promote to strict. (copilot-pull-request-reviewer)

No-change replies (rationale commented in the code):

- `event_kind_name` uses an exhaustive `match` on `EventKind` rather
  than `Debug` or a `strum` derive. The compile-time exhaustiveness
  check is the point — adding a new engine event should force a
  conscious decision about how the snapshot represents it, not a silent
  fallthrough. Added a comment making that intent explicit.

- `trace-coverage.sh` awk parser of `event.rs` is fragile — agreed, but
  the script is advisory and its failure mode is false negatives
  (uncovered variants simply aren't gated). Documented the tradeoff and
  the rewrite-in-Rust escape hatch in the script header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(replay-gate): prime cache on staging, restrict PR runs to read-only

The second run on PR #2621 missed the cache ("No cache found" in the
rust-cache restore step) even though the workflow is wired correctly.
Root cause: the repo sits close to GitHub's 10 GB per-repo cache quota
(~59 entries, many >500 MB), and the LRU policy evicts PR-scoped caches
before they get reused.

Fix:
- Add `push: [staging, main]` so the gate runs (and saves a ~1.2 GB
  cache under the `replay-gate` key) on every merge to the branches
  PRs actually target. Subsequent PRs restore from that base-branch
  cache — GitHub Actions permits cross-ref restore when the restoring
  ref's base matches the saved ref.
- Set `save-if: ${{ github.event_name == 'push' }}` so PR runs only
  *read* the cache. Without this gate, each PR push would save its
  own copy and crowd out the primed base-branch cache, putting us
  right back in the eviction loop.

Expected effect: cold-cache 9m → warm ~2-3m once staging has a run with
the new workflow. Base-branch prime run still pays 9m (no regression).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(replay): drop bug-bash fixture scaffolding

Replay fixtures can't reproduce the Phase 3 target bugs because the
fixture *is* the LLM's output — handwriting a trace where the LLM
emits a tool call doesn't test whether the real LLM would have emitted
that call, only that the harness dispatches a scripted one. What
`summarization_uses_tools.json` actually pinned was the happy path,
not the #2541 bug.

Of the 7 open bug-bash issues, only #2544 ("plans and delegates but
never executes") is catchable by replay, and only via a live-recorded
fixture. The other six are LLM-behavior or infra-timing bugs outside
replay's reach. Rather than ship regression theater, tear out the
scaffolding.

Removed:
- tests/e2e_bug_bash_snapshots.rs
- tests/fixtures/llm_traces/bug_bash/
- tests/snapshots/replay__bug_bash_summarization_uses_tools.snap

Unwired:
- Replay-gate workflow paths + test list no longer mention bug_bash
- scripts/replay-snap.sh test command drops the extra --test flag

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: switch to cargo-nextest with per-test timeouts

Nextest runs each integration test in its own process and runs test
binaries in parallel, which is a big unlock for this repo:

- Engine v2 tests share a process-global `ENGINE_STATE` singleton
  (OnceLock), which the current test lock serialises inside a single
  test binary. Nextest's process-per-test model gives each test a
  clean state automatically, so the 16 engine_v2 tests stop running
  one-by-one.

- Cross-binary parallelism: `cargo test --test A --test B` runs
  binaries in sequence; nextest runs them concurrently.

Measured locally: the replay-gate test set (3 binaries, 21 tests)
went from ~30s sequential to **2.7s parallel**.

Adds `.config/nextest.toml` with:
- `slow-timeout = 60s / terminate-after 3` in the default profile so
  a hung test fails fast instead of blocking the workflow-level 25-
  minute cap.
- A `ci` profile with `fail-fast = false` (one flake shouldn't mask
  other failures), `failure-output = immediate-final`,
  `success-output = never` for readable Actions logs.
- Per-test 300s override for the handful of genuinely slow scenarios
  (zizmor scan, e2e_thread_scheduling).

Workflows updated:
- `replay-gate.yml`: installs cargo-nextest via taiki-e/install-action
  alongside cargo-insta (one step), runs `cargo insta test
  --test-runner nextest` with `NEXTEST_PROFILE=ci`.
- `test.yml`: all five `cargo test` invocations swapped for
  `cargo nextest run --profile ci`. Nextest doesn't execute doctests,
  so every nextest step is paired with a `cargo test --doc` follow-up
  to preserve coverage.

Local dev is unchanged — `cargo test` still works; nextest is only
required in CI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: re-trigger replay-gate workflow after nextest migration

Previous push only modified workflow files and `.config/nextest.toml`;
GitHub skipped the `pull_request` workflow events for that sync, so
the nextest migration didn't actually get exercised in CI. Empty
commit forces re-evaluation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(replay): note nextest wiring in the fixtures README

Also forces a CI re-run: the previous empty commit had no matching
paths, so the `pull_request.paths` filters skipped every workflow
including replay-gate. Touching a file under
`tests/fixtures/llm_traces/**` re-matches the filter and runs the
nextest-based gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(test): defer test.yml nextest migration

Staging restructured test.yml significantly while this PR was open
(matrix-config dynamic matrix, `changes` code-detection job,
composite install-cargo-component action, save-if restricted to
base-branch pushes). The merge into staging had heavy conflicts for
every nextest-swap hunk.

Rather than force a re-layering of the new staging structure on top
of the nextest migration in this PR, revert test.yml to staging's
current version. This PR now scopes the nextest change to just the
replay-gate workflow (where it cleanly demonstrates the value) plus
the shared `.config/nextest.toml` profile. Migrating the rest of
test.yml to nextest is a follow-up that can rebase on the new
structure without the heavy conflict surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Henry Park <henrypark133@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 17:34:01 +09:00

317 lines
9.9 KiB
TOML
Raw Blame History

[workspace]
members = [".", "crates/ironclaw_common", "crates/ironclaw_safety", "crates/ironclaw_skills", "crates/ironclaw_engine", "crates/ironclaw_gateway", "crates/ironclaw_tui"]
exclude = [
"channels-src/discord",
"channels-src/telegram",
"channels-src/slack",
"channels-src/whatsapp",
"tools-src/composio",
"tools-src/github",
"tools-src/gmail",
"tools-src/google-calendar",
"tools-src/google-docs",
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/slack",
"tools-src/telegram",
"fuzz",
"crates/ironclaw_safety/fuzz",
]
[package]
name = "ironclaw"
version = "0.25.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
authors = ["NEAR AI <support@near.ai>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
[package.metadata.wix]
upgrade-guid = "D0156E61-BA37-451E-8AB9-1A2ECCCFA48F"
path-guid = "F90B6EA6-87F7-499B-BB19-CF55DE1EB339"
license = false
eula = false
[dependencies]
# Async runtime
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
tokio-util = { version = "0.7", features = ["compat"] }
futures = "0.3"
tokio-tungstenite = { version = "0.26", features = ["rustls-tls-native-roots"] }
eventsource-stream = "0.2"
# Agent Client Protocol (ACP) — standard communication with coding agents
agent-client-protocol = "0.10"
# HTTP client
reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls-native-roots", "stream"] }
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Database - PostgreSQL (default, feature-gated)
deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
webpki-roots = { version = "0.26", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
# Error handling
thiserror = "2"
anyhow = "1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
# Configuration
dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = "0.10"
iana-time-zone = "0.1"
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
# Async traits
async-trait = "0.1"
# CLI
clap = { version = "4", features = ["derive", "env"] }
# Terminal
crossterm = "0.29"
rustyline = { version = "17", features = ["custom-bindings", "derive", "with-file-history"] }
termimad = "0.34"
# Channel integrations
axum = { version = "0.8", features = ["ws"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["trace", "cors", "set-header", "catch-panic"] }
# Cron scheduling for routines
cron = "0.13"
# JSON Schema validation (workspace document metadata).
# default-features disabled to avoid pulling a second `reqwest` major version
# for remote $ref resolution (we only validate against in-memory schemas).
jsonschema = { version = "0.45", default-features = false }
# Shared types
ironclaw_common = { path = "crates/ironclaw_common", version = "0.2.0" }
# Safety/sanitization
ironclaw_engine = { path = "crates/ironclaw_engine", version = "0.1.0" }
ironclaw_gateway = { path = "crates/ironclaw_gateway", version = "0.1.0" }
ironclaw_safety = { path = "crates/ironclaw_safety", version = "0.2.1" }
ironclaw_skills = { path = "crates/ironclaw_skills", version = "0.1.0" }
ironclaw_tui = { path = "crates/ironclaw_tui", optional = true, version = "0.1.0" }
regex = "1"
aho-corasick = "1"
# YAML parsing for SKILL.md frontmatter
serde_yml = "0.0.12"
# Filesystem paths
dirs = "6"
fs4 = "0.6"
glob = "0.3"
# Semantic versioning
semver = "1"
# Secrecy for sensitive values
secrecy = { version = "0.10", features = ["serde"] }
# URL parsing and encoding
url = "2"
urlencoding = "2"
# Open URLs in browser
open = "5"
# Vector embeddings for semantic search
# The postgres feature provides ToSql/FromSql for postgres-types (shared by tokio-postgres)
pgvector = { version = "0.4", features = ["postgres"], optional = true }
# WASM sandbox for untrusted tool execution
wasmtime = { version = "43.0.1", features = ["component-model"] }
wasmtime-wasi = "43.0.1" # WASI support for component model
wasmparser = "0.245.1" # WASM binary parsing for validation
# Cryptography for secrets management
aes-gcm = "0.10"
hkdf = "0.12"
hmac = "0.12"
sha2 = "0.10"
blake3 = "1"
rand = "0.8"
subtle = "2" # Constant-time comparisons for token validation
# Multi-provider LLM support
rig-core = { version = "0.30", default-features = false, features = ["reqwest-rustls"] }
# AWS Bedrock (native Converse API, opt-in via --features bedrock)
aws-config = { version = "1", features = ["behavior-version-latest"], optional = true }
aws-sdk-bedrockruntime = { version = "1", optional = true }
aws-smithy-types = { version = "1", optional = true }
# Docker sandbox
bollard = "0.18"
# Archive extraction for WASM extension bundles
flate2 = "1"
tar = "0.4"
# Document text extraction
pdf-extract = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
# HTTP proxy for sandboxed network access
hyper = { version = "1.5", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "tokio", "http1", "http2"] }
http-body-util = "0.1"
bytes = "1"
base64 = "0.22.1"
cookie = "0.18"
jsonwebtoken = "9"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
bs58 = "0.5"
hex = "0.4.3"
# OpenClaw import (feature gated)
json5 = { version = "0.4", optional = true }
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
security-framework = "3"
# PTY allocation for Claude CLI stdout buffering fix (Unix only)
[target.'cfg(unix)'.dependencies]
pty-process = { version = "0.5", features = ["async"] }
# Linux secret-service (GNOME Keyring, KWallet)
[target.'cfg(target_os = "linux")'.dependencies]
secret-service = { version = "4", features = ["rt-tokio-crypto-rust"] }
zbus = "4"
[build-dependencies]
serde_json = "1"
[dev-dependencies]
tokio-test = "0.4"
tracing-test = "0.2"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = { version = "1.46.3", features = ["yaml"] }
[features]
default = ["postgres", "libsql", "html-to-markdown", "tui"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:webpki-roots",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
"rust_decimal/db-tokio-postgres",
]
libsql = ["dep:libsql"]
# Opt-in feature for especially heavy integration-test targets that run in a
# dedicated CI job instead of the default Rust test matrix.
integration = []
# Replay-gate snapshot tests promoted out of `--ignored` into a dedicated CI
# job. Enabled by `.github/workflows/replay-gate.yml`; implies `libsql` so the
# test rig can spin up its embedded database.
replay = ["libsql"]
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
tui = ["dep:ironclaw_tui"]
import = ["dep:json5", "libsql"]
[[test]]
name = "e2e_thread_scheduling"
required-features = ["libsql", "integration"]
[[test]]
name = "html_to_markdown"
required-features = ["html-to-markdown"]
[profile.release]
strip = true # Remove debug symbols from release binaries
# The profile that 'cargo dist' will build with
[profile.dist]
inherits = "release"
lto = "thin"
# Config for 'dist'
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
installers = ["shell", "powershell", "npm", "msi"]
# Publish jobs to run in CI
publish-jobs = []
# Target platforms to build apps for (Rust target-triple syntax)
targets = [
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"x86_64-apple-darwin",
"x86_64-unknown-linux-gnu",
"x86_64-unknown-linux-musl",
"x86_64-pc-windows-msvc",
]
# The archive format to use for windows builds (defaults .zip)
windows-archive = ".tar.gz"
# The archive format to use for non-windows builds (defaults .tar.xz)
unix-archive = ".tar.gz"
# Which actions to run on pull requests
pr-run-mode = "skip"
# Path that installers should place binaries in
install-path = "CARGO_HOME"
# Whether to install an updater program
install-updater = true
# Cache intermediate build artifacts to speed up the release pipelines
cache-builds = true
[workspace.metadata.dist.github-custom-runners]
aarch64-unknown-linux-gnu = "ubuntu-24.04-arm"
aarch64-unknown-linux-musl = "ubuntu-24.04-arm"
x86_64-unknown-linux-gnu = "ubuntu-22.04"
x86_64-unknown-linux-musl = "ubuntu-22.04"
x86_64-pc-windows-msvc = "windows-2022"
x86_64-apple-darwin = "macos-15-intel"
aarch64-apple-darwin = "macos-14"
Reference in New Issue View Git Blame Copy Permalink