Files
ironclaw/scripts/build-test-tools.sh
Emil Bogomolov 8625c71360 feat(reborn): WASM tool install from zip + env-provisioned tenant-shared credentials (#5459 part 1) (#5499)
* feat(reborn): WASM tool install from zip + env-provisioned tenant-shared credentials (#5459)

Squashed branch for rebase onto main (12 commits: the feature, two
review-response rounds for serrrfirat's reviews on #5499, and CI/test
hardening).

Feature: POST /api/webchat/v2/extensions/import uploads a standalone
WASM tool bundle (zip: manifest.toml + wasm/ + schemas/ + prompts/),
validated as ManifestSource::InstalledLocal (never first-party trust,
wasm-component runtime only, all manifest-declared assets required,
duplicate/unsafe zip entries rejected, zip-bomb caps), materialized
under /system/extensions/<id>/ and added to the catalog; the WebUI
Registry tab gains an admin-only Import button. Keyless-networked
tools may declare network_targets without credentials.
IRONCLAW_REBORN_DEV_SECRET__<handle> env vars seed tenant-shared
admin-managed secrets at serve startup so keyed tools resolve
InjectSecretOnce for every tenant user from one operator-set key.

Review fixes (serrrfirat, #5499):
- import route classified operator-only: in
  is_webui_v2_operator_webui_config_route_id and mounted inside the
  mount_operator_routes block, so composition strips it from
  deployments without an operator surface and pre-gates non-operator
  callers before body buffering
- filesystem discovery stamps InstalledLocal (HostBundled reserved for
  binary-compiled extensions), so import -> restart -> install cannot
  launder an upload into first-party trust; first-party-claiming disk
  manifests now fail discovery loudly (fail-closed)
- import decode runs in spawn_blocking behind a 2-permit semaphore
  acquired before any lifecycle lock (bounds decode memory at 128 MiB,
  keeps zip work off the async runtime), bundle passed by value
- caller-level route coverage: import_extension_requires_operator_
  webui_config, import_extension_forwards_zip_bytes_to_facade_call,
  import_extension_is_stripped_alongside_operator_routes, operator
  predicate contract pin
- serve dev-secret bridge extracted to dev_secret_seeds_from_env (env
  as iterator param) with tests for prefix selection, empty-value
  skip, invalid-handle startup error, tenant-shared scope targeting

Regression tests include
imported_bundle_reloads_as_installed_local_after_restart,
concurrent_imports_of_distinct_ids_all_succeed, and the route/bridge
tests above; WASI-component enforcement and test-tools fixture builds
included.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(reborn): introduce private installs of tools (#5459 part 2) (#5525)

* feat(reborn): per-user private tool installs (#5459 P1, #5525)

Introduce per-user private extension installs with a typed installation
owner, slot rules, and admin-wins eviction, plus the design plan under
docs/plans. A member's install is private (invisible and non-dispatchable
to other users and the tenant operator); a tenant-operator install is
shared. One installation slot per extension id per tenant, with the owner
deciding who may claim an occupied slot; a tenant install evicts a private
one (admin-wins) and self-heals the "two users want it privately" case.

Incorporates serrrfirat's review on #5525:

- Command-path lifecycle actions derive the caller from the verified
  command auth claim (`lifecycle_caller`) instead of rejecting every
  non-surface context, so `/extension_*` commands work and stay
  owner-attributed.
- `ensure_caller_may_operate` masks every non-owner, the tenant operator
  included; admin power over a foreign slot is limited to eviction.
- A shared install that fails after eviction restores the evicted private
  install (pre-eviction snapshot + compensation on every failure arm), so
  a failed shared install never strands the user's tool.
- The ownership check runs before the activation credential preflight in
  both the capability and WebUI paths, so a non-owner gets the masked
  "is not installed" denial instead of a credential-requirement leak.
- The settings tool catalog fails closed when installation owner data is
  missing or unreadable, hiding install-backed registry tools.
- The private-install owner/slot policy and its tests live in
  install_policy.rs and tests/private_install_tests.rs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(reborn): membership replaces install slots — any number of users can install the same tool (#5459 P1)

Manual testing killed the slot model: with a tool held privately by
one member, every other user's install died with a masked
"unavailable", surfaced in the WebUI as a bare "Validation" banner on
a tool that looked installable. New contract (Emil, 2026-07-08): a
tenant install makes a tool available to everyone; import makes it
installable by anyone who wants it — two people or any number.

- InstallationOwner::Users { user_ids: BTreeSet } replaces the
  single-user variant; rows written by the slot iteration
  ({"kind":"user"}) load as a singleton member set, pre-#5459 rows
  still default to Tenant, and empty member sets are unconstructable
  and rejected on the wire.
- Install on an existing id is a single row rewrite: a member JOINS
  the member set; the operator EVICTS every member's private
  installation by replacing the set with Tenant (admin-wins survives
  as a semantic; nobody loses the tool and there is nothing to
  compensate).
- Remove leaves the member set while co-holders remain; the last
  holder's remove runs the full compensated teardown. Tenant rows
  stay operator-only to remove.
- The eviction snapshot/restore machinery (EvictedPrivateInstall,
  ensure_slot_available, evict_private_installation,
  restore_evicted_private_install, fail_install_restoring_evicted)
  is deleted. The #5525 review fixes that outlive the slot iteration
  (non-member masking incl. the operator, command-path caller
  derivation, credential-preflight ordering, fail-closed owner
  catalog, policy extraction) carry over unchanged.
- Grant minting and list masking key on set membership (visible_to);
  the "unavailable" denial is gone — install outcome no longer
  depends on whether other users hold the tool, which is strictly
  stronger anti-enumeration.
- Plan doc rewritten with the dated pivot
  (docs/plans/2026-07-01-private-tool-installs.md).

Regression tests (facade-level plus the grant-minting choke point):
two members install the same tool independently and both hold it;
member remove leaves the co-holder intact; last-member remove tears
down and frees the id; operator install evicts to tenant-shared with
activation preserved; installing an already-shared tool reports
already installed; command-path caller derivation and non-member
masking; member-set grants and provider trust; wire compat for
slot-iteration owner rows and empty-set rejection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(reborn): e2e for per-user private tool installs (#5459 P1)

Full-path Playwright/httpx scenario driving the real `ironclaw-reborn
serve` binary: operator imports the three test-tools/ fixture bundles,
installs+activates ascii-renderer tenant-wide, creates alice and bob via
the admin API; alice privately installs hacker-news and dispatches both
her shared and private tools; bob privately installs market-data, cannot
see alice's hacker-news in his extension list (membership-scoped
visibility, checked without prompting), and dispatches ascii-renderer +
market-data in one turn.

Supporting harness:
- conftest `test_tool_zips` session fixture rebuilds stale
  test-tools/*.zip via scripts/build-test-tools.sh; skips locally /
  fails in CI when wasm32-wasip2 is missing.
- reborn_webui_harness `reborn_v2_private_installs_yolo_server` seeds the
  market-data tenant-shared dev secret at boot (read once at serve
  start); capability-preview helpers promoted here and reused by the
  legacy tool-execution scenario.
- mock_llm dispatches the three fixtures by their `__`-encoded provider
  tool names (the gateway's provider_tool_name validator rejects a raw
  dot in the capability id).

Registered in reborn_coverage_tests.txt so it runs in the post-merge
Reborn E2E coverage job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(reborn): address PR 5499 review feedback

Protect extension import and lifecycle paths from lock-held I/O, silent asset loss, and unauthorized shared activation. Align WebUI upload contracts and fixture coverage.

Refs #5499

* docs(composition): correct host api facade docs

* fix(reborn): close remaining import review gaps

Bound ZIP entry/path and secret handoff races, split import policy from lifecycle/catalog orchestration, and add production-wired WebUI import coverage.

* fix(reborn): harden lifecycle follow-up paths

Keep import permits through materialization, simplify policy projections, and reduce internal recheck logs.

* fix: canonicalize legacy extension installs

* fix: redact extension state write errors

* fix(reborn): derive command lifecycle scope from auth claim + review cleanup

- lifecycle_resource_scope now resolves Command contexts through the
  verified auth-claim caller instead of rejecting them as non-surface;
  tenant claim overrides the local default scope.
- from_persisted_parts takes ExtensionInstallationPersistedParts
  (clippy too_many_arguments) and canonicalization threads it through.
- write_snapshot logs the underlying filesystem error at debug before
  returning the redacted installation-state error.
- New integration test pins private-install membership through the
  production webui facade: member join, masked non-member visibility
  and mutation (400, no identity leak), operator eviction to tenant
  scope, member remove denied on tenant rows.
- Align QA binary-e2e extension-lifecycle profile to the turn's real
  binding subject user (install wrote ownership under a fixed profile
  user while remove read it under the authenticated actor).
- Bless golden payload snapshots for main-side surface-hash drift
  (network_targets in the capability descriptor hash; prompt text
  unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(reborn): skip catalog-absent installation rows on lifecycle restore

restore_extension_lifecycle_state aborted the entire runtime build when
any persisted installation resolved to no available-catalog package
(e.g. placeholder rows written by the v1 migration tool, which does not
materialize packages yet). Every subsequent serve start failed until
the operator hand-edited state.json.

Skip and warn instead: the row is preserved untouched and restores
normally once the catalog gains the package. Pinned crate-tier
(restore_skips_installation_absent_from_catalog_and_restores_valid_installation)
and integration-tier via runtime restart over a hand-edited state.json
(production_runtime_restart_skips_installation_row_absent_from_catalog).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Henry Park <henrypark133@gmail.com>
2026-07-10 21:29:03 +00:00

75 lines
3.3 KiB
Bash
Executable File

#!/usr/bin/env bash
#
# build-test-tools.sh — build the test-tools/ WASM fixture bundles.
#
# Each test-tools/<tool>/ directory is a standalone uploadable extension
# bundle (manifest.toml + wasm/ + schemas/ + prompts/) used to exercise the
# WebUI v2 "Import Tool" flow (`POST /api/webchat/v2/extensions/import`)
# during live QA. See test-tools/README.md for the tool matrix.
#
# For each tool this script:
# 1. builds wasm-src/ with `cargo build --release --target wasm32-wasip2`
# (wasip2 emits a WASI COMPONENT directly — the runtime loads tools via
# `wasmtime::component::Component::new`, so a wasip1 core module fails at
# dispatch with the redacted "the tool manifest is invalid")
# 2. verifies the artifact really is a component (layer bytes), not a core module
# 3. copies the artifact into <tool>/wasm/ (the path the manifest declares)
# 4. zips manifest.toml + wasm/ + schemas/ + prompts/ into test-tools/<tool>.zip
#
# The .zip files and wasm-src/target/ are git-ignored build artifacts.
#
# Usage: bash scripts/build-test-tools.sh [tool ...]
# (no args = all tools)
set -euo pipefail
repo_root="$(git rev-parse --show-toplevel)"
tools_root="$repo_root/test-tools"
tools=("$@")
if [ ${#tools[@]} -eq 0 ]; then
for dir in "$tools_root"/*/; do
[ -f "$dir/manifest.toml" ] && tools+=("$(basename "$dir")")
done
fi
rustup target list --installed | grep -q '^wasm32-wasip2$' \
|| { echo "missing target: run 'rustup target add wasm32-wasip2'" >&2; exit 1; }
# WASI component vs core module: bytes 4-7 after the `\0asm` magic are
# version+layer — `0d 00 01 00` marks a component, `01 00 00 00` a core module.
require_component() {
local file="$1" label="$2"
local header
header="$(head -c 8 "$file" | od -An -tx1 | tr -d ' \n')"
# `\0asm` magic + layer bytes `01 00` at offsets 6-7 mark a component
# (version may move, the layer marker is the discriminator).
case "$header" in
0061736d????0100) ;;
*) echo "$label: not a WASI component (header: $header) — the runtime requires a component" >&2
exit 1 ;;
esac
}
for tool in "${tools[@]}"; do
tool_dir="$tools_root/$tool"
[ -f "$tool_dir/manifest.toml" ] || { echo "no such tool: $tool" >&2; exit 1; }
# 1. Build the WASM component (wasip2 componentizes via wasm-component-ld).
(cd "$tool_dir/wasm-src" && cargo build --release --target wasm32-wasip2)
# 2. Verify + copy the artifact to the manifest's [runtime].module path.
crate_name="$(sed -n 's/^name = "\(.*\)"/\1/p' "$tool_dir/wasm-src/Cargo.toml" | head -1)"
artifact="$tool_dir/wasm-src/target/wasm32-wasip2/release/${crate_name//-/_}.wasm"
module_rel="$(sed -n 's/^module = "\(.*\)"/\1/p' "$tool_dir/manifest.toml" | head -1)"
[ -f "$artifact" ] || { echo "$tool: build artifact not found: $artifact" >&2; exit 1; }
[ -n "$module_rel" ] || { echo "$tool: manifest declares no [runtime].module" >&2; exit 1; }
require_component "$artifact" "$tool"
mkdir -p "$tool_dir/$(dirname "$module_rel")"
cp "$artifact" "$tool_dir/$module_rel"
# 3. Zip the uploadable bundle.
rm -f "$tools_root/$tool.zip"
(cd "$tool_dir" && zip -q -r "../$tool.zip" manifest.toml wasm schemas prompts -x "*.DS_Store")
echo "$tool: built $module_rel (component) and $tool.zip"
done