mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
bb9351a5ab
* feat(channels): add attachment transfer vocabulary to egress descriptors Re-applied from codex/telegram-slack-attachments: ironclaw_attachments materialized-file/budget/workspace-ref types + ChannelEgressDescriptor paths/path_prefixes/body-limit bounds with fail-closed validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(channels): transfer inbound/outbound channel attachments through restricted egress Re-applied from codex/telegram-slack-attachments onto the restructured tree: ChannelAdapter::fetch_attachment seam + AttachmentTransfer error in ironclaw_host_api's product_adapter contract; envelope-transient channel_attachment_refs; ChannelInboundProductSurface transfer door with fail-closed default; post-policy fetch/validate/land orchestration in ironclaw_product's inbound turn service; workspace-file materialization in the delivery coordinator; Telegram getFile/download + sendDocument via the manifest's path-constrained egress; Slack fails closed both directions; composition wiring for the per-request policy-enforced channel egress. InboundAttachment/MaterializedFile moved down into ironclaw_host_api (the trait contract owner) because ironclaw_attachments depends on host_api. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(reborn): prove channel attachment journeys on the production mount Relocates the composition-resident attachment journey coverage to the extension-delivery integration lane per the tests/integration-first rule: the telegram delivery scenario now drives a document update through the production ingress mount — transient getFile failure releases the ledger attempt (503), the vendor redelivery refetches through the manifest's path-constrained egress with the token injected host-side, bytes land at the canonical /workspace/attachments ref exactly once, duplicate replay does no vendor I/O, and a follow-up conversation's final reply referencing the landed file is materialized through the real project-scoped reader and delivered natively via sendDocument. The envelope's transient-refs serde(skip) contract is pinned beside the type in ironclaw_host_api; sink-level door routing and inherited fail-closed transfer stay as local contract tests beside the moved code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(reborn): quality-gate fixes for the attachment re-application Bundle the delivery coordinator's materialization inputs (clippy arg budget), reuse the VendorResponseRouter alias, drop a dead test accessor, update the ingress contract fixture to the current manifest schema (admin_configuration-declared verification handle), and annotate large-file growth per the arch-sprawl gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(telegram): move channel test modules under src/tests/ The no-panics production gate exempts src/**/tests/*.rs; the flat channel_*_tests.rs siblings were test-only (cfg(test) #[path] mounts) but not recognizable as such by the path heuristic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(channels): correct attachment transfer bugs and retire the large-file exempts Audit follow-ups on the attachment transfer path. Functional fixes, each with a regression test that fails without it: - The fetched/declared MIME check normalized only the fetched side, so a descriptor carrying a parameter (`text/plain; charset=utf-8` — what Telegram clients routinely report for text documents) never matched and rejected the whole message, caption included. Compare canonical forms on both sides so the check catches a real provider mismatch instead. - The descriptor filename overwrote the fetched one unconditionally, discarding the name the adapter recovers from the `getFile` path for payloads that carry none (photos, voice notes, stickers). Keep it when the descriptor has no filename. - The multipart boundary was derived from `bytes.len()` and searched in a `0..=u32::MAX` loop with a full payload scan per iteration. The payload is attacker-authored (an inbound attachment can be landed and later referenced by a reply), so a sender could pad a file with collisions and force unbounded rescans. Use a v4 UUID and scan once. - The manifest declared 5 MiB transfer caps while the code enforced the 10 MiB host budget, so a file in between passed every code check and was then refused at egress — inbound reported as "denied" rather than "too large", outbound after the coordinator had committed to the send. Derive one bound (TELEGRAM_MAX_TRANSFER_BYTES), pre-flight the assembled multipart body against the declared request cap, map ResponseTooLarge to a size error, and pin constants to the manifest with a test. - The Bot API target's 64 KiB response cap also covered sendMessage, whose response echoes `reply_to_message` now that replies are threaded; an oversize echo failed a send the user had already received. Keep the host default there and document why. - The two fail-closed defaults for "this layer cannot transfer attachments" disagreed: the product surface said retryable, the inbound turn service said permanent. Retryable left the vendor redelivering forever while the user got nothing at all. Both are permanent now; a missing deployment egress transport stays retryable as an operator-fixable condition, and a permanent Invalid outcome is logged rather than settling silently. Security and hygiene: - `path_prefixes` matched by raw byte prefix, so a declared `/file/bot{token}` also authorized `/file/bot{token}Evil/…`. Require a trailing `/` at descriptor validation and match on a segment boundary. - `InboundAttachment` (new in this PR) derived `Debug` over raw bytes while its sibling in the same file hand-writes a redacting one with a leak test. Both now redact, and `ProjectFsFile` — the wire type — does too. - `MaterializedFile<P>` had exactly one instantiation; collapsed to `WorkspaceFile`. - Removed 39 stale committed frontend build artifacts (3.5 MB) under `crates/ironclaw_webui_v2/`, a crate folded into `ironclaw_webui`; nothing reads the path. Closed the `.gitignore` gap that admitted them. - Dropped all four `arch-exempt: large_file` annotations by shrinking the files instead: two were spurious (one on a 1,242-line file the 1,500-line gate never fires on, one licensing a semantically no-op edit), and the exempt grep scans the whole file body, so each would have disabled the gate for that file permanently. Test modules carved into their own files per the composition budget's own carve-out guidance. - Deleted a sink test whose distinguishing assertion read the test double's own payload construction; the telegram journey covers door selection. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * refactor(attachments): close the egress prefix bypass and collapse duplicated vocabulary Security: - `path_prefixes` matched by raw byte prefix, so a declared `/file/bot{token}` also authorized `/file/bot{token}Evil/…` — a sibling path on the same pinned host and credential that the manifest author never allowed. Descriptor validation now requires the prefix to end on a segment boundary, and the matcher enforces the boundary independently so a prefix reaching policy by another route still cannot authorize a sibling. Duplicated vocabulary, per .claude/rules/type-placement.md: - `mime_hint` was written in two adapters as a verbatim copy of `descriptor.mime_type` and read in zero production locations. Deleted with its writers; the one new production consumer already reads the descriptor. - `AttachmentRef` named two different concepts — the durable byte-free transcript reference and this transient vendor fetch reference — which forced an `as ChannelAttachmentRef` import alias where both appeared. The channel one is now `ChannelAttachmentRef` at its definition and the alias is gone. - `ProductAttachmentCapabilities` re-declared `AttachmentBudgets`' three fields and hand-copied each. Embedded with `#[serde(flatten)]`; the JSON shape is unchanged (the wire assertions still read the same keys) and a new budget field now reaches the browser with no intermediate edit. - The `/workspace` prefix was defined independently in `ironclaw_attachments` (deciding which model-text refs become egress attachments) and in composition (deciding which paths are readable at all). Divergence would be a silently undelivered file or an extraction/confinement mismatch, so `ironclaw_attachments` owns it, composition imports it, and a test pins the prefix to the alias. - `NoProjectFilesystem` was defined verbatim in three crates. One inert double now lives beside the trait it implements, under `test-support`. Also corrects two comments that asserted guarantees the code no longer made: the reader's "same 25 MiB limit" (the delivery instance is deliberately tighter) and `ProjectFsEntryKind`'s "without depending on that crate" (the dependency exists; it is a wire projection, which is the real reason). Retains the cause of a provider parse failure server-side via `tracing::debug` while keeping the user-facing reason a fixed literal, and constructs the declared bot-token handle in one place. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * refactor(product): move the scoped project-filesystem adapters out of composition Continues the direction of #6615/#6616/#6619: composition is assembly, and these files were not assembly. Backend selection already happened upstream — composition hands them a `ScopedFilesystem`. What is left is contract policy owned by the port they implement: alias confinement with the explicit sibling-prefix guard, sensitive-filename omission from listings, the TOCTOU-hardened two-stage size guard, extension→MIME derivation that must match the download `Content-Type`, and the substrate→port error sanitization table (including the deliberate MountNotFound→503 vs Contract→400 split). `ProjectScopedFilesystemReader` and the attachment lander/reader therefore move to `ironclaw_product::scoped_fs`, beside the `ProjectFilesystemReader` / `InboundAttachmentLander` traits they implement. Every import they need was already a production dependency of that crate, and the in-crate precedent is `filesystem_ledger.rs`, which likewise hosts a generic `ScopedFilesystem` implementation of a product-owned port. `mount_filesystem_reader.rs` deliberately stays in composition: its `alias_for(FsMount)` table is the "which mounts does this deployment serve" decision, which is composition's job. It now consumes the shared scoped-path helpers from the owner crate by name. Composition src: 66,998 → 66,208 LOC (10.39% → 10.27% of production). Two gates caught this change and were fixed rather than silenced: the struct ratchet entry follows the file to its new path, and the extension-specificity gate rejected a comment naming a concrete vendor in generic code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * refactor(channels): invert the pairing-outcome observer to a host-owned trait The sink is generic channel machinery, but its pairing-outcome observer was an enum naming a concrete composition type (`RunDeliveryPostAdmissionObserver`) plus a `#[cfg(test)]` `Recording` variant compiled into the production type. That is the coupling that keeps the generic ingress sink pinned to composition. It is now a trait: the delivery observer implements it, and tests supply an ordinary double instead of a variant. This is also the seam that has to invert before the sink itself can move to `ironclaw_extension_host`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * refactor(extension-host): move the generic channel ingress sink out of composition `ironclaw_extension_host` already owned the ingress *port* (`InboundSink`, `InboundAdmission`, the router and verifier); composition owned the *implementation*. The boundary test's own inventory shows the neighbourhood was already evacuated — `reply_contexts`, `channel_delivery`, `channel_dm_targets` and `channel_lifecycle` are all listed as externalized generic modules, and `extension_ingress` appeared on neither that list nor the internal one. It was the holdout. Moved to `ironclaw_extension_host::ingress::sink`: the registration table behind the router's ports, the trusted-evidence mint, the pairing pre-admission gate, `GenericChannelInboundSink`, `StaticIngressSecrets`, and the `build_extension_ingress` factory — module-owned initialization, as the composition guide requires. The pairing outcome vocabulary moves with it to `ingress::pairing`; the pairing *service* (CAS claim, identity bind, completion fan-out) stays in composition and implements the host trait. Composition keeps `mod serve_mount`: `ingress/mod.rs` states the crate is deliberately transport-neutral, so the axum `PublicRouteMount` stays on the composition side. Its public re-export is preserved, so downstream binaries and tests are unaffected — only the source crate changed, which is what the pub-use snapshot update records. `host-auth-mint` is enabled on the host crate's `ironclaw_product` dependency with the rationale the feature rule requires: it is a privilege boundary, and this crate is one of the host runtimes entitled to mint verified evidence after the router has executed the manifest's verification recipe. Composition src: 66,205 → 65,560 LOC (10.27% → 10.17%). Across this branch: 66,998 → 65,560, with the file itself going 1,242 → 186 lines. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * fix(llm): point the fault-injection doc example at its own crate The example imported `ironclaw::testing::fault_injection`, a path that died with the v1 monolith, so the doc-test failed to compile. It only runs under workspace-wide feature unification — the root dev-dependency enables `ironclaw_llm/test-support`, which compiles the `testing` module — so `cargo test -p ironclaw_llm --doc` alone reports zero tests and never surfaced it. `cargo test --workspace` has been failing on it. The doc-test is its own regression test: it now compiles, where before it could not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5yb6tF3rwMq8KSvuMDhyK * fix(cli): pin channel authority in native ingress test * fix(egress): enforce body limits after secret injection * fix(product): preserve delivery failure semantics * fix(telegram): allow attachments without size hints * docs: design generic cross-channel attachments * docs: plan generic cross-channel attachments * feat(filesystem): add atomic subtree creation * fix(attachments): land inbound batches atomically * test(attachments): isolate channel lander seam * feat(outbound): persist reply attachment intents * feat(attachments): complete cross-channel reply delivery * feat(attachments): durably assemble provider batches * fix(slack): verify batched attachment delivery * docs: record cross-channel attachment verification * fix(attachments): harden replay and provider boundaries * fix(attachments): repair merge-blocking CI coverage * fix(attachments): retain test reply intent store * fix(attachments): reuse outbound test store seam * fix(attachments): render durable file references cleanly * docs(attachments): define structured multimodal replies * feat(attachments): complete multimodal kind vocabulary * feat(attachments): add opaque reply attachment handles * feat(attachments): return opaque handles to the model * refactor(attachments): make structured replies canonical * fix(ci): close attachment coverage gates * fix mixed attachment cleanup snapshots * fix(attachments): harden cross-channel reply delivery * fix(ci): qualify attachment integration type --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>