mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
* feat(reborn): WASM tool install from zip + env-provisioned tenant-shared credentials (#5459) Squashed branch for rebase onto main (12 commits: the feature, two review-response rounds for serrrfirat's reviews on #5499, and CI/test hardening). Feature: POST /api/webchat/v2/extensions/import uploads a standalone WASM tool bundle (zip: manifest.toml + wasm/ + schemas/ + prompts/), validated as ManifestSource::InstalledLocal (never first-party trust, wasm-component runtime only, all manifest-declared assets required, duplicate/unsafe zip entries rejected, zip-bomb caps), materialized under /system/extensions/<id>/ and added to the catalog; the WebUI Registry tab gains an admin-only Import button. Keyless-networked tools may declare network_targets without credentials. IRONCLAW_REBORN_DEV_SECRET__<handle> env vars seed tenant-shared admin-managed secrets at serve startup so keyed tools resolve InjectSecretOnce for every tenant user from one operator-set key. Review fixes (serrrfirat, #5499): - import route classified operator-only: in is_webui_v2_operator_webui_config_route_id and mounted inside the mount_operator_routes block, so composition strips it from deployments without an operator surface and pre-gates non-operator callers before body buffering - filesystem discovery stamps InstalledLocal (HostBundled reserved for binary-compiled extensions), so import -> restart -> install cannot launder an upload into first-party trust; first-party-claiming disk manifests now fail discovery loudly (fail-closed) - import decode runs in spawn_blocking behind a 2-permit semaphore acquired before any lifecycle lock (bounds decode memory at 128 MiB, keeps zip work off the async runtime), bundle passed by value - caller-level route coverage: import_extension_requires_operator_ webui_config, import_extension_forwards_zip_bytes_to_facade_call, import_extension_is_stripped_alongside_operator_routes, operator predicate contract pin - serve dev-secret bridge extracted to dev_secret_seeds_from_env (env as iterator param) with tests for prefix selection, empty-value skip, invalid-handle startup error, tenant-shared scope targeting Regression tests include imported_bundle_reloads_as_installed_local_after_restart, concurrent_imports_of_distinct_ids_all_succeed, and the route/bridge tests above; WASI-component enforcement and test-tools fixture builds included. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reborn): introduce private installs of tools (#5459 part 2) (#5525) * feat(reborn): per-user private tool installs (#5459 P1, #5525) Introduce per-user private extension installs with a typed installation owner, slot rules, and admin-wins eviction, plus the design plan under docs/plans. A member's install is private (invisible and non-dispatchable to other users and the tenant operator); a tenant-operator install is shared. One installation slot per extension id per tenant, with the owner deciding who may claim an occupied slot; a tenant install evicts a private one (admin-wins) and self-heals the "two users want it privately" case. Incorporates serrrfirat's review on #5525: - Command-path lifecycle actions derive the caller from the verified command auth claim (`lifecycle_caller`) instead of rejecting every non-surface context, so `/extension_*` commands work and stay owner-attributed. - `ensure_caller_may_operate` masks every non-owner, the tenant operator included; admin power over a foreign slot is limited to eviction. - A shared install that fails after eviction restores the evicted private install (pre-eviction snapshot + compensation on every failure arm), so a failed shared install never strands the user's tool. - The ownership check runs before the activation credential preflight in both the capability and WebUI paths, so a non-owner gets the masked "is not installed" denial instead of a credential-requirement leak. - The settings tool catalog fails closed when installation owner data is missing or unreadable, hiding install-backed registry tools. - The private-install owner/slot policy and its tests live in install_policy.rs and tests/private_install_tests.rs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(reborn): membership replaces install slots — any number of users can install the same tool (#5459 P1) Manual testing killed the slot model: with a tool held privately by one member, every other user's install died with a masked "unavailable", surfaced in the WebUI as a bare "Validation" banner on a tool that looked installable. New contract (Emil, 2026-07-08): a tenant install makes a tool available to everyone; import makes it installable by anyone who wants it — two people or any number. - InstallationOwner::Users { user_ids: BTreeSet } replaces the single-user variant; rows written by the slot iteration ({"kind":"user"}) load as a singleton member set, pre-#5459 rows still default to Tenant, and empty member sets are unconstructable and rejected on the wire. - Install on an existing id is a single row rewrite: a member JOINS the member set; the operator EVICTS every member's private installation by replacing the set with Tenant (admin-wins survives as a semantic; nobody loses the tool and there is nothing to compensate). - Remove leaves the member set while co-holders remain; the last holder's remove runs the full compensated teardown. Tenant rows stay operator-only to remove. - The eviction snapshot/restore machinery (EvictedPrivateInstall, ensure_slot_available, evict_private_installation, restore_evicted_private_install, fail_install_restoring_evicted) is deleted. The #5525 review fixes that outlive the slot iteration (non-member masking incl. the operator, command-path caller derivation, credential-preflight ordering, fail-closed owner catalog, policy extraction) carry over unchanged. - Grant minting and list masking key on set membership (visible_to); the "unavailable" denial is gone — install outcome no longer depends on whether other users hold the tool, which is strictly stronger anti-enumeration. - Plan doc rewritten with the dated pivot (docs/plans/2026-07-01-private-tool-installs.md). Regression tests (facade-level plus the grant-minting choke point): two members install the same tool independently and both hold it; member remove leaves the co-holder intact; last-member remove tears down and frees the id; operator install evicts to tenant-shared with activation preserved; installing an already-shared tool reports already installed; command-path caller derivation and non-member masking; member-set grants and provider trust; wire compat for slot-iteration owner rows and empty-set rejection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(reborn): e2e for per-user private tool installs (#5459 P1) Full-path Playwright/httpx scenario driving the real `ironclaw-reborn serve` binary: operator imports the three test-tools/ fixture bundles, installs+activates ascii-renderer tenant-wide, creates alice and bob via the admin API; alice privately installs hacker-news and dispatches both her shared and private tools; bob privately installs market-data, cannot see alice's hacker-news in his extension list (membership-scoped visibility, checked without prompting), and dispatches ascii-renderer + market-data in one turn. Supporting harness: - conftest `test_tool_zips` session fixture rebuilds stale test-tools/*.zip via scripts/build-test-tools.sh; skips locally / fails in CI when wasm32-wasip2 is missing. - reborn_webui_harness `reborn_v2_private_installs_yolo_server` seeds the market-data tenant-shared dev secret at boot (read once at serve start); capability-preview helpers promoted here and reused by the legacy tool-execution scenario. - mock_llm dispatches the three fixtures by their `__`-encoded provider tool names (the gateway's provider_tool_name validator rejects a raw dot in the capability id). Registered in reborn_coverage_tests.txt so it runs in the post-merge Reborn E2E coverage job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(reborn): address PR 5499 review feedback Protect extension import and lifecycle paths from lock-held I/O, silent asset loss, and unauthorized shared activation. Align WebUI upload contracts and fixture coverage. Refs #5499 * docs(composition): correct host api facade docs * fix(reborn): close remaining import review gaps Bound ZIP entry/path and secret handoff races, split import policy from lifecycle/catalog orchestration, and add production-wired WebUI import coverage. * fix(reborn): harden lifecycle follow-up paths Keep import permits through materialization, simplify policy projections, and reduce internal recheck logs. * fix: canonicalize legacy extension installs * fix: redact extension state write errors * fix(reborn): derive command lifecycle scope from auth claim + review cleanup - lifecycle_resource_scope now resolves Command contexts through the verified auth-claim caller instead of rejecting them as non-surface; tenant claim overrides the local default scope. - from_persisted_parts takes ExtensionInstallationPersistedParts (clippy too_many_arguments) and canonicalization threads it through. - write_snapshot logs the underlying filesystem error at debug before returning the redacted installation-state error. - New integration test pins private-install membership through the production webui facade: member join, masked non-member visibility and mutation (400, no identity leak), operator eviction to tenant scope, member remove denied on tenant rows. - Align QA binary-e2e extension-lifecycle profile to the turn's real binding subject user (install wrote ownership under a fixed profile user while remove read it under the authenticated actor). - Bless golden payload snapshots for main-side surface-hash drift (network_targets in the capability descriptor hash; prompt text unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reborn): skip catalog-absent installation rows on lifecycle restore restore_extension_lifecycle_state aborted the entire runtime build when any persisted installation resolved to no available-catalog package (e.g. placeholder rows written by the v1 migration tool, which does not materialize packages yet). Every subsequent serve start failed until the operator hand-edited state.json. Skip and warn instead: the row is preserved untouched and restores normally once the catalog gains the package. Pinned crate-tier (restore_skips_installation_absent_from_catalog_and_restores_valid_installation) and integration-tier via runtime restart over a hand-edited state.json (production_runtime_restart_skips_installation_row_absent_from_catalog). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Henry Park <henrypark133@gmail.com>
75 lines
3.3 KiB
Bash
Executable File
75 lines
3.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# build-test-tools.sh — build the test-tools/ WASM fixture bundles.
|
|
#
|
|
# Each test-tools/<tool>/ directory is a standalone uploadable extension
|
|
# bundle (manifest.toml + wasm/ + schemas/ + prompts/) used to exercise the
|
|
# WebUI v2 "Import Tool" flow (`POST /api/webchat/v2/extensions/import`)
|
|
# during live QA. See test-tools/README.md for the tool matrix.
|
|
#
|
|
# For each tool this script:
|
|
# 1. builds wasm-src/ with `cargo build --release --target wasm32-wasip2`
|
|
# (wasip2 emits a WASI COMPONENT directly — the runtime loads tools via
|
|
# `wasmtime::component::Component::new`, so a wasip1 core module fails at
|
|
# dispatch with the redacted "the tool manifest is invalid")
|
|
# 2. verifies the artifact really is a component (layer bytes), not a core module
|
|
# 3. copies the artifact into <tool>/wasm/ (the path the manifest declares)
|
|
# 4. zips manifest.toml + wasm/ + schemas/ + prompts/ into test-tools/<tool>.zip
|
|
#
|
|
# The .zip files and wasm-src/target/ are git-ignored build artifacts.
|
|
#
|
|
# Usage: bash scripts/build-test-tools.sh [tool ...]
|
|
# (no args = all tools)
|
|
set -euo pipefail
|
|
|
|
repo_root="$(git rev-parse --show-toplevel)"
|
|
tools_root="$repo_root/test-tools"
|
|
|
|
tools=("$@")
|
|
if [ ${#tools[@]} -eq 0 ]; then
|
|
for dir in "$tools_root"/*/; do
|
|
[ -f "$dir/manifest.toml" ] && tools+=("$(basename "$dir")")
|
|
done
|
|
fi
|
|
|
|
rustup target list --installed | grep -q '^wasm32-wasip2$' \
|
|
|| { echo "missing target: run 'rustup target add wasm32-wasip2'" >&2; exit 1; }
|
|
|
|
# WASI component vs core module: bytes 4-7 after the `\0asm` magic are
|
|
# version+layer — `0d 00 01 00` marks a component, `01 00 00 00` a core module.
|
|
require_component() {
|
|
local file="$1" label="$2"
|
|
local header
|
|
header="$(head -c 8 "$file" | od -An -tx1 | tr -d ' \n')"
|
|
# `\0asm` magic + layer bytes `01 00` at offsets 6-7 mark a component
|
|
# (version may move, the layer marker is the discriminator).
|
|
case "$header" in
|
|
0061736d????0100) ;;
|
|
*) echo "$label: not a WASI component (header: $header) — the runtime requires a component" >&2
|
|
exit 1 ;;
|
|
esac
|
|
}
|
|
|
|
for tool in "${tools[@]}"; do
|
|
tool_dir="$tools_root/$tool"
|
|
[ -f "$tool_dir/manifest.toml" ] || { echo "no such tool: $tool" >&2; exit 1; }
|
|
|
|
# 1. Build the WASM component (wasip2 componentizes via wasm-component-ld).
|
|
(cd "$tool_dir/wasm-src" && cargo build --release --target wasm32-wasip2)
|
|
|
|
# 2. Verify + copy the artifact to the manifest's [runtime].module path.
|
|
crate_name="$(sed -n 's/^name = "\(.*\)"/\1/p' "$tool_dir/wasm-src/Cargo.toml" | head -1)"
|
|
artifact="$tool_dir/wasm-src/target/wasm32-wasip2/release/${crate_name//-/_}.wasm"
|
|
module_rel="$(sed -n 's/^module = "\(.*\)"/\1/p' "$tool_dir/manifest.toml" | head -1)"
|
|
[ -f "$artifact" ] || { echo "$tool: build artifact not found: $artifact" >&2; exit 1; }
|
|
[ -n "$module_rel" ] || { echo "$tool: manifest declares no [runtime].module" >&2; exit 1; }
|
|
require_component "$artifact" "$tool"
|
|
mkdir -p "$tool_dir/$(dirname "$module_rel")"
|
|
cp "$artifact" "$tool_dir/$module_rel"
|
|
|
|
# 3. Zip the uploadable bundle.
|
|
rm -f "$tools_root/$tool.zip"
|
|
(cd "$tool_dir" && zip -q -r "../$tool.zip" manifest.toml wasm schemas prompts -x "*.DS_Store")
|
|
echo "$tool: built $module_rel (component) and $tool.zip"
|
|
done
|