mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
8a6cbcf717e2267fb1fea8951e7f80c1e6e6b856
19 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Henry Park
|
8a6cbcf717 | test: update approval e2e expectations (#3054) | ||
|
Illia Polosukhin
|
e3df3ec4ae |
feat(skills): setup-marker lifecycle, chain-loading, and live GitHub workflow test (#2268)
* chore: gitignore live test fixture containing recorded credentials
The github_dev_workflow live test records HTTP exchanges including
the github_token Bearer header. GitHub push protection correctly
blocks this. The fixture is only useful locally for replay; the
test skips gracefully without it.
* test: add live test for github developer workflow
Adds tests/e2e_github_dev_workflow.rs — a multi-turn live/replay test that
drives the developer-assistant + github-workflow skills end-to-end against
a synthetic nearai/ironclaw repository:
1. Setup — installs the wf-* mission set (excluding
wf-staging-review per the implement-but-don't-
auto-merge autonomy contract)
2. Issue opened — synthetic github.issue.opened webhook payload
3. Maintainer LGTM — pr.comment.created from a maintainer
4. PR review — non-maintainer review comment
5. CI failure — failing check_run
6. Approval — maintainer approval; asserts NO merge call ever
fires across the whole session
7. Digest — status report referencing the issue/PR
Webhook payloads are injected via TestRig::send_message with a
[GITHUB WEBHOOK] frame that matches what a real webhook→channel
adapter would emit. The mission OnSystemEvent firing path is covered
separately by mission.rs unit tests; this test exercises skill
behavior given the right inputs.
Adds two helpers to tests/support/live_harness.rs:
- trace_contains_tool_call(name, needle)
- assert_trace_contains_tool_call(name, needle, ctx)
Both scan ToolStarted.detail and ToolResult.preview for case-insensitive
substring matches, so behavior tests can assert *what the agent
actually called* without scraping the recorded trace JSON.
Drive-by cleanups from the extension-lifecycle merge:
- thread_ops.rs: drop orphaned RecordingStatusChannel + helper that
came from a dropped extension-lifecycle test variant
- bridge/router.rs: clippy needless_borrow on PendingGate args
- skills/mod.rs: SkillManifest no longer has metadata field; add
requires: GatingRequirements::default() to test fixture
- cargo fmt fallout in recording.rs / live_mission.rs / trace_llm.rs
The test is #[ignore]-tagged (live tier) and skips gracefully in replay
mode until tests/fixtures/llm_traces/live/github_dev_workflow_full_loop.json
is recorded with IRONCLAW_LIVE_TEST=1. Compile coverage is automatic
via the existing test matrix; live execution follows the same pattern
as e2e_live_personas.rs (manual recording + commit fixture).
cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --test e2e_github_dev_workflow -- -D warnings: clean
cargo test --features libsql --test e2e_github_dev_workflow -- --ignored: passes (skips, fixture missing)
* test(harness): add pre-seed secrets + diagnostic activity dump
Three additions to make the github_dev_workflow live test runnable:
1. **TestRigBuilder::with_secret(name, value)** — pre-seed credentials
in the SecretsStore before the agent starts. The kernel pre-flight
auth gate fires when a skill with a credential spec activates (e.g.
the github skill needs github_token); without a stored credential
the agent gets stuck in 'Authentication required' mode and can't
make progress. Tests inject a fake/dummy value so the gate is
satisfied — the test isn't actually hitting the credentialed API.
Implementation: AppComponents.secrets_store is captured during
build_all() and any pre-seeded (name, value) pairs are written via
secrets_store.create() with user_id = config.owner_id. Already-exists
errors are silenced so the helper is idempotent on seeded DBs.
2. **LiveTestHarnessBuilder::with_secret** — forwards to
TestRigBuilder::with_secret. Plumbed through both build_live and
build_replay so the same fixture works in both modes.
3. **dump_activity helper in e2e_github_dev_workflow.rs** — formats
captured StatusUpdate stream (skill activations + every tool
started/completed/result) to stderr. Used as a pre-assertion
diagnostic so failing live runs surface the agent's actual tool
sequence instead of an opaque panic on a workspace check.
Test relaxations from running this against the real LLM:
- verify_setup_landed accepts either developer-assistant OR
github-workflow as the active skill (the deterministic selector
picks based on keyword scoring + token budget; both routes are
valid since github-workflow owns the mission templates)
- final required-skills check drops developer-assistant in favor of
github-workflow + github (the orchestrator persona is optional)
- setup turn now pre-seeds github_token via with_secret
cargo check --features libsql --tests: clean
* test: rewrite github_dev_workflow as fully real live integration
Pivots the test from synthetic webhook simulation to a real end-to-end
integration test against the real nearai/ironclaw repo. Per project
owner: 'fully real live tests doing useful work on github repo... test
everything like it's live while recording all interactions to debug
what doesn't work and improve that'.
## Why the rewrite
The previous synthetic-event version injected fake GitHub payloads as
channel messages. With a real github_token in scope, the agent
attempted to fetch the fake issue 99001, got a 404, and helpfully
created 3 real issues + 3 real comments on nearai/ironclaw to
"reconcile" the discrepancy. The synthetic approach didn't surface
realistic failure modes anyway (auth gates, payload format mismatches,
rate limits), so we go all-in on real artifacts.
## New flow (2 turns + real artifact lifecycle)
1. Setup turn — agent installs the wf-* mission set for nearai/ironclaw
2. Test (NOT the agent) creates a real issue via direct REST API with
the title "[live-test {timestamp}] Add /metrics Prometheus endpoint"
and a real feature-request body.
3. Triage turn — test asks agent to triage issue #N. Agent reads via
github skill, generates a plan, posts a real comment back.
4. Verification — test polls api.github.com/issues/N/comments and
asserts at least one new comment exists since baseline. Comment
bodies are logged to stderr for human review (the most useful
debug output for iterating on skill quality).
5. Cleanup — std::panic::catch_unwind wraps the body so cleanup runs
regardless of pass/fail. Closes the issue with a final "live test
complete" comment. If cleanup itself fails, the issue URL is
printed for manual recovery.
## Test infrastructure additions
- TestRig.get_secret(name) — read decrypted secrets back from the
rig's SecretsStore. Required so the test can read the github_token
the harness pre-seeded via with_secrets(["github_token"]).
- TestRig captures secrets_store + owner_id from AppComponents during
build (needed for get_secret).
- github_api submodule inside the test file — direct REST helpers for
create_issue, list_issue_comments, post_issue_comment, close_issue.
Uses reqwest directly so the test has guaranteed GitHub access
regardless of skill selection / tool gating.
## Recording
- LLM trace fixture: tests/fixtures/llm_traces/live/github_dev_workflow_full_loop.json (65K)
- Session log: github_dev_workflow_full_loop.log (5.9K)
- Both committed so future runs can replay deterministically without
hitting real GitHub.
## What's NOT covered yet
Dropped from the previous version (can be added back as follow-ups):
- PR creation flow (agent opens a real PR with a real branch + real
code change)
- CI failure simulation (would need a real failing CI run)
- Mission OnSystemEvent firing via real webhooks (needs an HTTP
server registered as a GitHub webhook)
- Maintainer approval flow
This first version validates the most valuable slice: setup → react
to real issue → produce real comment → cleanup. If the agent's
comment quality is good, we expand from here.
cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --test e2e_github_dev_workflow -- -D warnings: clean
Live recording: passed in 85.9s
- Created issue #2185
- Agent posted 2 comments (full plan + follow-up)
- Closed issue #2185
* feat(skills): one-time setup-marker exclusion + rename persona skills to *-setup
The persona orchestrator skills (developer-assistant, ceo-assistant,
trader-assistant, content-creator-assistant) are pure first-time
onboarding flows — their entire body is Steps 1-N of workspace setup,
mission registration, and calibration memory writes. After those steps
run successfully, there is nothing left for the skill to do, but the
deterministic selector kept evaluating them on every conversation
turn, burning ~3000 tokens of activation budget for work already
completed and risking partial re-runs of setup steps.
This commit makes setup skills opt-in to one-time activation:
## Mechanism: setup_marker exclusion
New optional field on ActivationCriteria:
activation:
setup_marker: commitments/.developer-setup-complete
Before scoring, the selector caller (Agent::select_active_skills)
collects every distinct setup_marker referenced by loaded skills,
checks the workspace for each via Workspace::exists(), and passes
the set of satisfied markers into prefilter_skills. Any skill whose
marker is in the satisfied set is excluded from scoring entirely
(returns None from the filter map, skipping the score_skill call).
The selector check is opt-in: skills without a setup_marker are
unaffected. Reactive operational skills (commitment-triage,
decision-capture, github, github-workflow, etc.) keep activating
on every matching message as before.
Tests:
- 4 unit tests in crates/ironclaw_skills/src/selector.rs covering
marker present/absent, marker mismatch, and skill-without-marker
unaffected paths
- All 152 ironclaw_skills tests pass
- Live e2e_github_dev_workflow run on real nearai/ironclaw passes
(issue #2186 created, comment posted, closed) in 88s
## Rename: *-assistant → *-setup
Per project owner: 'rename persona skills to -setup skills to make
it explicit they are called once'. The -assistant suffix obscured
the lifecycle — these are not always-on assistants, they are
one-time onboarding wizards.
Renamed directories (via git mv) and updated SKILL.md `name:`
fields:
- skills/ceo-assistant → skills/ceo-setup
- skills/content-creator-assistant → skills/content-creator-setup
- skills/developer-assistant → skills/developer-setup
- skills/trader-assistant → skills/trader-setup
All four now declare `setup_marker: commitments/.<name>-setup-complete`
and have a new final 'Step N: Mark setup complete' instructing the
agent to write the marker via memory_write after confirming setup
with the user. Different personas have different markers so they
remain independently triggerable in separate workspaces.
Cross-references updated:
- tests/e2e_live_personas.rs (4 persona test invocations)
- tests/e2e_github_dev_workflow.rs (doc comments)
- tests/e2e/LIVE_TOOL_FAILURES.md (1 reference)
- crates/ironclaw_skills/src/types.rs (doc comment example)
## Bump: SKILLS_MAX_CONTEXT_TOKENS default 4000 → 6000
The previous default was so tight that a setup skill (3000 tokens)
plus its companion github-workflow (2000) plus github (2000) would
overflow at 7000. Reactive operational skills like
commitment-triage, decision-capture, tech-debt-tracker often got
budget-evicted. With setup skills now excluded after onboarding,
the freed budget plus the bump to 6000 lets the most useful
combinations fit comfortably (e.g. github-workflow + github +
product-prioritization is now active in the live recording, where
previously product-prioritization would have been evicted).
## Plumbing changes
- ActivationCriteria gains pub setup_marker: Option<String>
(#[serde(default)], so existing skills are unaffected)
- prefilter_skills signature gains
&satisfied_setup_markers: &HashSet<String> (caller passes empty
set to disable filtering — used by all existing tests via the
prefilter_no_markers wrapper)
- Agent::select_active_skills is now async — it needs to
Workspace::exists() each marker. dispatcher.rs caller updated
to .await. Snapshots the skill list under the read lock then
drops the guard before any await to avoid holding a poisonable
RwLock across an await point.
cargo check --features libsql --tests: clean
cargo clippy --features libsql --tests --all-targets -- -D warnings: clean
cargo test -p ironclaw_skills: 152 passed
Live e2e_github_dev_workflow run: passes (88s)
* feat(skills): chain-load companions + v2 marker exclusion + commitment-setup marker
Three orthogonal follow-ups to the skill lifecycle work.
## 1. Chain-loading via requires.skills (v1 Rust + v2 Python)
When a parent skill is selected by the scorer, its requires.skills
companions are now automatically loaded, bypassing the score filter.
Persona/bundle skills like developer-setup can finally work as
designed: the orchestrator declares which operational skills it
delegates to, and selecting the orchestrator pulls them all in.
- **v1 Rust** (crates/ironclaw_skills/src/selector.rs): extracted
skill_token_cost() and try_select() helpers used by both the
scored-selection loop and the new chain-loading pass. Companions
consume the same budget and respect max_candidates. Non-transitive
(depth 1 only) to keep behavior predictable.
- **v2 Python** (crates/ironclaw_engine/orchestrator/default.py):
select_skills() gains an inline chain-loading pass that mirrors
the Rust logic. Uses a name-indexed lookup built from the skill
list passed in by handle_list_skills. No closure-over-outer-var
tricks that Monty would reject — the inner try-add is inlined.
7 chain-load unit tests in selector.rs covering: pulls in
companions, skipped when parent not selected, respects budget,
skips companion with satisfied marker, non-transitive (depth 2
not pulled), missing companion silent, dedup across parents.
## 2. v2 setup_marker exclusion
The v2 engine's Python orchestrator handles skill selection via
handle_list_skills (Rust) -> select_skills (Python). Since
handle_list_skills already has the full project doc list in scope,
we filter there: any skill whose metadata.activation.setup_marker
is in the set of existing doc titles gets excluded before the
Python orchestrator ever sees it. Zero extra store calls — we
reuse the existing list_memory_docs_with_shared result to build
an O(1) title set.
This is the v2 parity of the v1 satisfied_setup_markers parameter
threaded through prefilter_skills. Both paths now implement the
same rule: a one-time setup skill whose marker file has been
written has finished its job and should not keep burning
activation budget.
## 3. commitment-setup gets a setup_marker
commitment-setup writes commitments/README.md as its first step,
so the marker is automatically set after a successful first run.
Added:
activation:
setup_marker: commitments/README.md
To re-trigger (e.g. migrate to a new schema), delete README.md
first. project-setup was NOT given a marker — it's per-repo,
invoked repeatedly, not a singleton (each call creates a new
projects/<owner>-<repo>/project.md).
## 4. Lifecycle integration test
tests/skill_setup_marker_lifecycle.rs drives a real agent turn
through the v1 selector pipeline (Agent::select_active_skills ->
Workspace::exists -> prefilter_skills) to verify that a setup
skill:
Phase 1: activates on the first matching message (marker absent)
Phase 2: marker file is written via workspace.write()
Phase 3: is excluded on the second matching message
The test asserts on the captured LLM system prompt content (via
rig.captured_llm_requests) rather than on StatusUpdate events so
it's agnostic to v1/v2 path differences in how skill activations
are announced. The skill's body contains a distinctive marker
string (LIFECYCLE-TEST-SKILL-BODY-MARKER-Z7Q) — if the skill was
selected, that string appears in the system prompt; if excluded,
it doesn't.
Cover matrix after this commit:
- v1 selector: 35 unit tests + 4 setup-marker tests + 7 chain-load tests
- v2 handle_list_skills marker exclusion: 1 integration test (lifecycle)
plus structural verification via cargo check (the filter uses the
existing list_memory_docs API, no new store calls to test)
- v2 Python select_skills chain-load: covered by the v1 unit tests
through shared semantic contract (both paths mirror the same
algorithm); a direct Python-level test would require spinning up
the Monty interpreter which is out of scope for this session.
Verification:
cargo test -p ironclaw_skills --lib: 159 passed
cargo test -p ironclaw_engine: 304 passed
cargo test --features libsql --test skill_setup_marker_lifecycle: 1 passed
cargo clippy --features libsql --tests --all-targets -- -D warnings: clean
* feat(skills): carry requires through v1→v2 migration + chain-load test
V2SkillMetadata was missing the `requires` field entirely, so the
v1→v2 skill migration silently dropped `requires.skills` and the
chain-loading code I added to the v2 Python orchestrator in the
previous commit was effectively dead code — it always read an empty
companion list.
This was caught while writing an end-to-end chain-load test: the v1
test (through the Rust selector) passes, the v2 test (through the
Python orchestrator) was failing in a way that only made sense if
the companion metadata never reached Python. Inspection confirmed
`V2SkillMetadata` had no `requires` field, only `activation`.
## Fix
1. `V2SkillMetadata` gains `pub requires: GatingRequirements` with
`#[serde(default)]` for backwards compatibility (legacy
MemoryDocs in existing databases deserialize with an empty
`requires`).
2. `src/bridge/skill_migration.rs::v1_skill_to_memory_doc` now
copies `skill.manifest.requires.clone()` into the new field.
3. Four other explicit `V2SkillMetadata { ... }` literal
constructions updated with `requires: Default::default()`:
- `crates/ironclaw_engine/src/memory/skill_tracker.rs` (test helper)
- `crates/ironclaw_engine/src/runtime/mission.rs` (test helper)
- `crates/ironclaw_skills/src/v2.rs` (serde roundtrip test)
- `tests/engine_v2_skill_codeact.rs` (test fixture)
## New test: tests/skill_chain_load_lifecycle.rs
End-to-end lifecycle test for chain-loading. Writes three skills to
a tempdir:
- `parent-setup-test` — scored by a distinctive keyword, declares
two companions via `requires.skills`
- `companion-one-test` / `companion-two-test` — zero-scoring on
their own (keywords deliberately don't match)
Each skill body carries a distinctive marker string
(`CHAIN-LOAD-PARENT-BODY-J4V`, `CHAIN-LOAD-COMPANION-ONE-K5W`,
`CHAIN-LOAD-COMPANION-TWO-L6X`) that the test greps for in the
captured LLM system prompt via `rig.captured_llm_requests()`. If a
marker is present, the skill was injected into the prompt; if
absent, it wasn't.
Two test variants:
- **v1** (default rig, Rust selector path): **PASSES**. Proves the
chain-loading pass in `prefilter_skills` correctly pulls in both
companions despite their zero individual scores.
- **v2** (with_engine_v2, Python orchestrator path):
**`#[ignore]`d** with a detailed explanation. The v2 engine runs
a Python orchestrator that makes multiple LLM calls per user
message, but the default TestRig uses a single-turn TraceLlm that
exhausts after the first call — observing skill injection through
the v2 path needs a multi-turn TraceLlm harness or a dedicated v2
skill test rig. The structural wiring for v2 chain-loading
(V2SkillMetadata.requires + skill_migration copy + Python
select_skills chain-load pass) compiles and passes the 304-test
engine suite, so this is a test-harness gap, not a code gap.
When the multi-turn harness exists, flipping `#[ignore]` on the v2
test will exercise the full path.
Verification:
cargo test -p ironclaw_skills --lib: 159 passed
cargo test -p ironclaw_engine --lib: 304 passed
cargo test --features libsql --test skill_chain_load_lifecycle
-- --test-threads=1: 1 passed, 1 ignored
cargo test --features libsql --test skill_setup_marker_lifecycle
-- --test-threads=1: 1 passed
cargo clippy --features libsql --tests --all-targets -- -D warnings: clean
Also includes an updated fixture recording from the last live
`e2e_github_dev_workflow` run (issue #2204, agent posted 2 comments,
cleanup closed it). No functional difference; committed for
completeness since the fixture was modified on disk by the live run
and the test is hermetic in replay mode.
* fix: adapt thread_ops test to staging's test helper API
Use make_test_agent_with_status_channel instead of removed
make_thread_ops_test_agent, StdMutex instead of TokioMutex,
and fix String comparison direction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: cargo fmt
* fix: remove dead try_add function and stale comments in Python orchestrator
Addresses PR #2268 review feedback: the try_add closure was defined but
never called since the logic was inlined for Monty compatibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reconcile test harness after staging merge
Restore our branch's test helpers (SessionTurn, finish_turns_strict,
with_skills_dir, loaded_skill_names, active_skill_names, etc.) that
staging removed, while incorporating staging's new features
(record_trace, with_no_trace_recording, secrets_store/owner_id
accessors). Bridge the API gap with finish_turns_simple for tests
using staging's (String, Vec<String>) tuple convention.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR #2268 review feedback
- live_harness: replace panic with graceful TestMode::Skipped when
record_trace=false in replay mode; update e2e_live callers to check
mode() != Live instead of == Replay
- test_rig: match SecretError::NotFound explicitly in get_secret(),
return None silently instead of logging expected misses
- test_rig: replace brittle "already exists" string matching in
pre-seed loop with get_decrypted existence check before create
- default.py: align max_context_tokens fallback from 1000 to 2000
to match Rust ActivationCriteria default (both parent and companion)
- e2e_builtin_tool_coverage: fix routine_create_list using hardcoded
"test-user" instead of rig.owner_id() (broke when .with_skills()
changed channel user to config owner_id)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address PR #2268 review feedback (round 2)
1. Fix memory_write `path:` → `target:` in all 4 setup skill completion
markers (developer, ceo, content-creator, trader). The `memory_write`
tool reads `target`, not `path`, so markers were never written to the
correct location.
2. Add setup_marker validation in enforce_limits(): max 256 chars, reject
`..` path traversal. Prevents untrusted skills from abusing markers.
3. Fix v2 Python skill budget: default 4000 → 6000 to match v1 Rust
config. Also port the approx_tokens > declared * 2 sanity check from
Rust to prevent budget bypass via low max_context_tokens declarations.
4. Reorder developer-setup companion skills to put github/github-workflow
first (critical for setup) and fix misleading budget comment in config.
5. Move AssertUnwindSafe cleanup guard in e2e GitHub test to wrap
everything after create_issue, preventing orphaned issues on panic.
6. Scope workspace in select_active_skills to the requesting user_id so
multi-user channels check the correct user's setup marker state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove duplicate skills_dir field from LiveTestHarnessBuilder
Both sides of the merge added the same field, resulting in a duplicate
declaration that failed compilation in test targets.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CI failures and Copilot review feedback
1. Fix formatting (cargo fmt).
2. Filter existing_titles to non-Skill docs in v2 orchestrator so setup
markers don't collide with skill doc titles of the same name.
3. Fix stale doc comment in types.rs (commitments/README.md →
commitments/.developer-setup-complete).
4. Fix misleading comment on v2 requires field — the full
GatingRequirements struct is preserved, not just the companion list.
5. Match SecretError::NotFound explicitly in test_rig pre-seed loop
instead of catching all errors — other errors (DB, crypto) now
surface instead of triggering a blind create.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Illia Polosukhin
|
6a28a4c861 |
fix(test): case-insensitive tool_search description assertion (#2608)
* fix(test): case-insensitive assertion in tool_search description The e2e assertion at tests/e2e_builtin_tool_coverage.rs:1230 checked for a lowercase "use the `message` tool ..." substring, but #2515 capitalized the first word in src/tools/builtin/extension_tools.rs:110. The local unit test in that file was updated; this e2e test was missed, breaking the Run Tests job on main and blocking release-plz PR #2606. Normalize to lowercase before substring match so a future copy-edit doesn't silently break CI again. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update tests/e2e_builtin_tool_coverage.rs Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> |
||
|
Henry Park
|
4f277c91be |
Handle empty tool completions in autonomous jobs (#1720)
* Handle empty tool completions in autonomous jobs * Address malformed tool recovery review comments * style: apply rustfmt to reasoning tests --------- Co-authored-by: Firat Sertgoz <f@nuff.tech> |
||
|
Henry Park
|
d97c0145cf |
Clarify message tool vs channel setup guidance (#1715)
* Clarify message tool and channel setup guidance * Add target format hints to proactive messaging prompt * Clarify search and message tool edge cases * Fix stale tool_search e2e assertion * Update src/llm/reasoning.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Tighten prompt guidance for message replies * Format prompt guidance assertions --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Nige
|
dd0a0e10ab |
fix(routines): recover delete name after failed update fallback (#1108)
Co-authored-by: ilblackdragon@gmail.com <ilblackdragon@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Henry Park
|
ab0ad948f3 |
Normalize cron schedules on routine create (#1648)
* Fix REPL single-message hang and cap CI test duration * Fix Clippy nested-if lint in REPL startup * Fix single-message approval flow * Handle empty single-message REPL exits * Wait for one-shot event routines before exit * Fix MCP lifecycle trace user scope * Normalize cron schedules on routine create |
||
|
Henry Park
|
dea789cca9 |
Default new lightweight routines to tools-enabled (#1573)
* Default new lightweight routines to tools-enabled * Fix fmt and clippy on lightweight routine PR * Use grouped execution field in routine no-tools fixture * Align CLI routine defaults with tools-enabled lightweight mode |
||
|
Illia Polosukhin
|
a09c023642 |
feat(ux): complete UX overhaul — design system, onboarding, web polish (#1277)
* feat(ux): complete UX overhaul — design system, boot screen, onboarding, web polish Shared design system: CSS custom properties for spacing, typography, transitions, and color tokens used across web UI and boot screen. Boot screen: compact feature-tags line showing enabled subsystems (db, tools, routines, heartbeat, skills, sandbox, embeddings) at a glance. Downgrade startup info logs (libSQL, webhook, workspace seed) to debug level since the boot screen now covers this. Onboarding wizard: model picker with live API fetch, provider-aware auth flow, improved error recovery and progress display. Web UI: ARIA attributes, welcome card, streaming debounce, connection status banner, skeleton loaders, send cooldown. CLI: doctor command enhancements, status command cleanup, REPL banner consolidation, shared fmt module. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(ux): Apple-level design refinements — spring physics, glass morphism, chat polish Merge staging theme support (dark/light/system toggle) and layer UX polish on top: spring-physics motion, glass morphism depth, chat experience improvements, and responsive mobile refinements. Design system: - Restore and extend design token system (spacing, typography, timing, easing) with legacy aliases for theme compatibility - Add shadow tiers, accent glow, glass morphism, spring easing tokens - Tokens defined in both dark (:root) and light ([data-theme="light"]) Micro-interactions (Phase 2): - Spring-overshoot message entry animation (slideUp) - Spring-scale button press on all interactive buttons - Tab crossfade animation, tool card smooth accordion (max-height) - Modal scale(0.95) + blur(8px) entry, toast spring slide - Sidebar width crossfade, card hover lift Visual depth (Phase 3): - Tab bar glass morphism + surface highlight + sliding indicator - Active tab accent background pill - Assistant message accent left border, user message bubble tail - Floating input area (rounded + shadow + margin) Chat polish (Phase 4): - Smooth streaming cursor (cursorPulse), message hover timestamps - Time separators (Today/Yesterday/date) - Textarea smooth auto-expand, send button glow Settings & forms (Phase 5): - iOS-style toggle switches for boolean settings - Input focus glow, save feedback spring animation - Welcome card with gradient background + proper spacing - Sticky settings group headers with glass backdrop Accessibility & mobile (Phase 6): - Animated focus ring, prefers-reduced-motion global kill-switch - Touch target audit (44px min), mobile bottom-sheet modals - Mobile bottom tab bar, toast redesign (icon + border + countdown) - Thread hover translateX, badge in_progress pulse Bug fixes: - Gateway/TEE popover z-index (tab-bar z-index: 200, popovers 500) - Connection lost banner as fixed top bar instead of flex child - Sidebar collapse keeps toggle + new thread buttons visible - Downgrade noisy startup logs (db, webhook, vector) to debug - Remove green dot pulse animation on connected status - Deduplicate confirm-modal in HTML, add tab-indicator div Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): mobile layout improvements — sidebar toggle, settings drill-down, tab bar polish - Fix mobile sidebar toggle: use expanded-mobile class instead of collapsed, add backdrop overlay, auto-close on thread select, outside-click dismiss - Settings: replace cramped horizontal tabs with drill-down navigation (category list → detail view → back button) - Bottom tab bar: add glass morphism, hide theme toggle, flip tab indicator to top edge - Keep thread toggle button visible in collapsed 36px sidebar strip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(repl): interactive approval selector and transient status lines - Replace ASCII-art approval box with clean horizontal rule card - Add inquire-based interactive selector for tool approvals (↑↓ + Enter) - Selector runs directly from send_status via spawn_blocking, with stdin_locked flag to prevent readline from competing for stdin - Transient thinking/tool-started lines: each replaces the previous, all erased before final output (no clutter left in scrollback) - Esc in selector sends denial so agent never gets stuck Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: widen TurnCost token fields to u64 and remove unused variable - Change input_tokens/output_tokens from u32 to u64 in StatusUpdate::TurnCost, SseEvent::TurnCost, and the thread_ops emit site to avoid truncation on large conversations - Remove unused _routine_engine_for_loop binding in agent_loop.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: reduce startup log noise — demote info to debug Demote routine startup messages (builder, WASM tools, tunnel, WASM channels) from info to debug so the default log output stays clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow CDN scripts in CSP connect-src directive Add cdn.jsdelivr.net and cdnjs.cloudflare.com to connect-src so the browser can fetch marked.js and DOMPurify without CSP violations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix cargo fmt in repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): gate turn_cost SSE handler on current thread Prevents cost badge from attaching to the wrong message when switching threads or receiving events from background threads. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: retrigger CI * fix: add missing extension_manager to webhook EngineContext The webhook trigger path added in #736 was missing the extension_manager field introduced by #1453. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: ignore RUSTSEC-2026-0049 rustls-webpki CRL advisory Low impact — requires compromised CA to exploit. Tracked for upstream rustls-webpki upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(routines): use fields.join for cron normalization Use split_whitespace fields instead of re-trimming the original string to avoid preserving extra internal whitespace in cron expressions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(repl): Apple-style approval card — clean vertical flow - Drop verbose tool description (the command IS the decision surface) - Unified vertical pipe layout: ◆ header → │ params → │ selector - Selector options show keyboard shortcuts inline: Approve (y) - Compact help message, answered state uses └ to close the flow - No horizontal rules, no blank-line padding — just breathing room Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(repl): replace inquire with crossterm for approval selector Drop the inquire dependency (which pulled in crossterm 0.25, duplicating the existing 0.28). The 3-option approval selector is now built directly with crossterm raw mode — same UX, zero new dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(deps): upgrade crossterm 0.28 → 0.29, eliminate duplication termimad (via crokey) uses crossterm 0.29. Upgrading our direct dependency from 0.28 to 0.29 collapses to a single crossterm version in the dependency tree. Also migrated termimad::crossterm:: references to the direct crossterm import. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review comments — box_top off-by-one, smart_truncate overflow, mobile theme toggle - Fix box_top() fill calculation: was off-by-one, producing boxes 1 char too wide (fmt.rs) - Fix smart_truncate(): account for "..." in the budget so output never exceeds max_chars (repl.rs) - Move theme toggle to settings sidebar on mobile instead of display:none, so mobile users can still switch themes (style.css, index.html, app.js) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review — retry duplication, CSP connect-src, deny color - Remove failed message before retry to prevent duplicate user messages - Revert connect-src to 'self' — CDN hosts only need script-src - Use red for Deny confirmation in REPL approval selector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Henry Park
|
ee6f5cd62a |
Use live owner tool scope for autonomous routines and jobs (#1453)
* Use live owner tool scope for autonomous runs * Address autonomous tool scope review feedback * Normalize routine context paths again |
||
|
Henry Park
|
cac6f4013c |
Add owner-scoped permissions for full-job routines (#1440)
* docs: add comments explaining CLI_ENABLED=false in service templates (#990) Clarify that CLI_ENABLED=false is needed in daemon mode (launchd/systemd) to prevent blocking on stdin when running as a background service. Closes #990 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add owner-scoped full-job routine permissions * Address PR review feedback * Fix owner gate test timing --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Henry Park
|
428303af11 |
Redesign routine create requests for LLMs (#1147)
* Redesign routine create requests for LLMs * Fix panic-check false positives in routine tests * Tighten routine schema requirements * Tighten routine schema tests * Mark test assertions safe for CI scan * Align test assertions with panic scan * Polish routine schema metadata * Simplify routine test assertions * Improve tool discovery guidance * Clarify lightweight routine delivery prompts * Fix routine delivery target defaults |
||
|
Henry Park
|
878a67cdb6 |
Refactor owner scope across channels and fix default routing fallback (#1151)
* refactor: add explicit owner scope across channels * fix: tighten routine owner target routing * fix: address owner scope review feedback * Fix owner-scope onboarding and event trigger isolation * Tighten routing fallback and wizard owner validation * fix: address owner-scope follow-up review * fix: tighten owner-scope follow-up details * fix: import Channel trait in telegram test * fix: normalize http webhook sender ids * fix: address remaining owner-scope review issues * fix: reconcile config rebase fallout * fix: reconcile extension manager rebase drift * fix: address current copilot review regressions * fix: restore clippy matrix after rebase |
||
|
Henry Park
|
7d745d5479 | tools: improve routine schema guidance (#1089) | ||
|
Henry Park
|
8a60fa2d37 |
fix: add tool_info schema discovery for WASM tools (#1086)
* fix: add tool_info schema discovery for WASM tools * refactor: simplify WASM schema and hint state * refactor: store tool_info registry reference as Weak |
||
|
Illia Polosukhin
|
369741fc60 |
Add generic host-verified /webhook/tools/{tool} ingress (#757)
* Add generic host-verified webhook ingress for tools * Stabilize trace E2E test rig and approval behavior * Fix webhook security issues from review feedback - Reject tools without webhook_capability() (was unauthenticated RCE) - Remove secret-in-query-string fallback (leak via logs/referrers) - Require approval for event_emit tool (escalation via routine triggers) - Simplify header_value() (HeaderMap already case-insensitive) - Redact internal errors from webhook HTTP responses - Remove unused hmac_timestamp_tolerance_secs field - Add regression test for tool without webhook capability [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Harden webhook ingress: require auth mechanism, body limit layer, health check - Reject webhook capabilities that declare no auth mechanism (empty WebhookCapability would previously allow unauthenticated access) - Add DefaultBodyLimit layer to reject oversized payloads before buffering - Health check (GET) now verifies tool has webhook_capability(), not just existence - Add regression tests for all three fixes [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix auto_approve_tools inconsistency between dispatcher and thread_ops dispatcher.rs skips all approval checks (including Always) when auto_approve_tools is true, but thread_ops.rs still required approval for Always tools. This caused deferred tool calls to unexpectedly halt in test rigs and auto-approve configurations. Match dispatcher behavior: short-circuit all approval when auto_approve_tools is enabled. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Illia Polosukhin
|
6e1ed939cc |
Add event-triggered routines and workflow skill templates (#756)
* Add event-triggered routines and workflow skill templates * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback for event_emit security and quality Security fixes: - Require approval (UnlessAutoApproved) for event_emit, matching routine_fire - Enable sanitization on event_emit payload (external JSON reaches LLM) - Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id Correctness fixes: - Rename source → event_source in event_emit for consistency with routine_create - Use json_value_as_filter_string for filter parsing (handles numbers/booleans) - Case-insensitive matching for event source and event_type - Add debug logging for missing filter keys in payload - Fix skill_install_routine_webhook_sim test missing .with_skills() - Fix schema_validator test for event_emit payload properties Code quality: - Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout) - Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs - Add test section headers in e2e_routine_heartbeat.rs - Clarify event_emit description to specify system_event routines only Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix: make routine_system_event_emit test create routine before emitting - Add routine_create step to trace fixture so event_emit has a matching routine to fire - Assert fired_routines > 0, not just key presence (Copilot review) - Add .with_auto_approve_tools(true) since event_emit now requires approval Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: renumber test headers after system_event test insertion Test 4 was duplicated (routine_cooldown and heartbeat_findings). Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: merge staging and add missing RoutineEngine args in test RoutineEngine::new on staging requires `tools` and `safety` params. Update system_event_trigger_matches_and_filters test to pass them. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address new Copilot review comments - Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim test so event_emit doesn't block on approval - Fix module-level doc comment for event_emit to specify system_event trigger [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: deduplicate json_value_as_string helper Remove private `json_value_as_string` from routine_engine.rs and use the identical public `json_value_as_filter_string` from routine.rs, eliminating divergence risk. (Copilot review) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Henry Park <henrypark133@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Protocol Zero
|
732b3ecfeb |
test(agent): wire TestRig job tools through the scheduler (#716)
Align TestRig with the production agent wiring so create_job exercises the real scheduler path instead of silently falling back to an unscheduled context-only job. Tighten the e2e assertion to lock in the in-progress scheduler behavior for future refactors. Made-with: Cursor Co-authored-by: Zaki Manian <zaki@iqlusion.io> |
||
|
Illia Polosukhin
|
37bba72397 |
test: add 29 E2E trace tests for issues #571-575 (#593)
* test: add 29 E2E trace tests for worker, threading, tools, workspace, and routines (#571-575) Add comprehensive E2E test coverage across five test files: - e2e_worker_coverage (7 tests): parallel tool calls, error feedback, unknown tools, invalid params, rate limiting, iteration limits, planning mode - e2e_thread_scheduling (3 tests + 2 deferred): multi-turn state, undo/redo, concurrent dispatch - e2e_builtin_tool_coverage (8 tests): time parse/diff/invalid, routine CRUD/history, job create/status/list/cancel, HTTP replay - e2e_workspace_coverage (6 tests): chunked search, multi-doc search, hybrid search, directory tree, document lifecycle, identity in system prompt - e2e_routine_heartbeat (5 tests): cron triggers, event matching, cooldown enforcement, heartbeat findings, empty checklist skip Infrastructure: extend TestRig with database/workspace/trace_llm accessors, register job and routine tools by default, add with_extra_tools() for custom stub tools. Includes 24 JSON trace fixtures across worker/, threading/, tools/, and workspace/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use 6-field cron format in routine_create_list fixture The cron 0.13 crate accepts both 6 and 7 fields, but the routine_create tool documents 6-field format. Align the fixture to match. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: eliminate vacuous passes and silently-skipped assertions in E2E tests - job_create_status: replace job_status (needs dynamic UUID) with list_jobs, assert both succeed via completed() not just started() - job_list_cancel: keep cancel_job but explicitly assert it fails with invalid canned job_id "latest", verify create_job + list_jobs succeed - unknown_tool_name: add !is_empty() guard before .all() to prevent vacuous pass on empty iterator - workspace tests: change `if let Some(ws)` to `.expect()` so assertions are never silently skipped when workspace/trace_llm is available [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add template substitution to TraceLlm for dynamic tool result forwarding Add {{call_id.json_path}} template syntax to trace fixtures, enabling tool results from one step to flow into subsequent steps' arguments. TraceLlm extracts variables from Role::Tool messages (stripping the safety layer's <tool_output> XML wrapper and unescaping entities) and substitutes them in canned tool_call arguments before returning. This fixes job_create_status and job_list_cancel tests to properly test job_status and cancel_job with real dynamic UUIDs from create_job, instead of using invalid canned IDs that silently failed. Also adds tool result content assertions to job_create_status to verify the actual tool output contains expected data (job_id, title). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback on E2E tests - undo_redo_cycle: assert exactly 3 turns instead of >= 2 - tool_error_feedback: use tempfile::tempdir() instead of hardcoded /tmp path, patch fixture path at runtime for CI portability - worker_timeout → iteration_limit: rename to accurately describe what's tested - post_plan_work_remaining → simple_echo_flow: rename, test doesn't exercise planning - identity_in_system_prompt: seed IDENTITY.md before test, assert system prompt contains the seeded content instead of just checking Role::System exists [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: strengthen workspace E2E test assertions per PR review - write_chunk_search: assert memory_search was called and returned payment/architecture-related results - multi_document_search: assert memory_search was called for cross-document search - hybrid_search_with_embeddings: assert both memory_write and memory_search were called to confirm write-then-search pipeline - directory_tree: assert tree output contains expected alpha/beta project paths [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |