Files
ironclaw/scripts
Benjamin Kurrek 2d64363101 fix(qa): stop the agent asserting unverified state — automation status, per-caller extension auth, recalled memory (#7246, #7247, #7294) (#7474)
* fix(assistant): ground automation-status claims in actual state checks (#7246)

The agent confidently reported a BTC-news-digest automation as running and
delivering to Telegram while the Automations page showed "No automations
yet" — status fabricated from conversation history instead of checked.

The read path already exists: builtin.trigger_list is model-callable
(PermissionMode::Allow), core-tier always-advertised, granted in the
interactive policy, and deliberately retained for scheduled fires. The
failure is grounding: its description said only "List scheduled triggers
owned by the current caller scope" — no bridge from the user vocabulary
("automation", the Automations page; "routine") to the trigger capability,
and no instruction to consult it before asserting status.

Mechanism (mirrors the proven builtin__outbound_delivery_targets_list
grounding pattern — description-level positive rule tied to the exact
assertion the model must not fabricate):

- trigger_management.rs: TRIGGER_LIST_DESCRIPTION now names the surface
  ("the automations shown on the Automations page"), declares the listing
  the authoritative current state, instructs calling it before answering
  which routines/automations exist or saying one is running, paused,
  already set up, delivering, or missing, forbids reporting status from
  conversation history or memory, and grounds the empty result as "the
  caller has no routines".
- schemas.rs: trigger_list.input.v1.json gains a root description carrying
  the same authoritative-state framing into the model-visible schema.

Regression test (red on the old description, green now):
builtin_trigger_list_surface_grounds_automation_status_claims in
first_party_builtin_tools.rs, driven through the production
visible_capabilities surface assembly — not the constant — asserting the
vocabulary bridge, the check-before-assert rule, the memory ban, the
empty-state grounding, and the schema root description.

Validated: ironclaw_host_runtime suite + clippy -D warnings green;
ironclaw_loop_host, ironclaw_turn_runner, ironclaw_composition green;
reborn_group_triggers integration group green, proving the new description
survives the prompt-build descriptor validation chain (VerifiedCatalog
surface, 4096-byte cap) under production wiring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(memory): frame recalled memory as recollection, not live state; pin cross-thread transcript containment (#7294)

Investigation verdict: the reported "agent remembers a Telegram routine
from another scope or thread" is NOT a retrieval leak. Every isolation
seam holds: providers scope-filter (native retains only scope-equal
results and excludes threads/ scratch from the long-term lane; mem0
partitions by the composed user namespace), and the host re-applies the
ExpectedScope drop filter in ironclaw_host_runtime::memory_context.
Durable memory crossing conversations for the same user is the
contract's design. The defect is presentation: recalled snippets entered
the prompt as bare "Untrusted memory content: ..." system messages, so
the model read a recollection ("user asked for a BTC news routine") as
verified current state ("you already have this set up").

Fix, at the prompt-assembly seam (InstructionBundleBuilder): whenever at
least one memory snippet is admitted, the memory section now opens with
a recall-framing system message (prompts/memory_recall_framing.md,
include_str!) telling the model these are recollections from earlier
turns/conversations that describe past state and must be verified with a
tool before being asserted as currently configured. No rows are
filtered, deleted, or re-scoped.

Regression coverage making the scoping contract explicit (all green
today; sabotage-verified to arm):
- shared conformance suite (ironclaw_memory::test_support, runs for
  native + mem0): a recorded conversation transcript is invisible to
  another thread's short-term AND long-term lanes and to thread-less
  (trigger-shaped) long-term retrieval; durable memory written during
  one conversation stays retrievable from another (cross-thread recall
  is by design, not a leak).
- instruction-bundle unit tests: framing precedes the snippets; absent
  when no snippets are admitted.
- scenario_proactive_prompt_recall_libsql (production wiring, libSQL):
  the writer conversation's after-turn transcript (proven recorded via
  its own short-term lane) never surfaces in the reader conversation's
  prompt, and recalled durable memory arrives behind the framing.

Consumer contract tests in ironclaw_turns / ironclaw_loop_host updated
for the new memory-section header; tests/CLAUDE.md coverage row updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(assistant): communication context reports per-caller credential truth (#7247)

The model-facing communication runtime context asserted connection state it
never verified: RuntimeCommunicationContextProvider hard-coded
`authenticated: true` for every host-Active channel-surface extension, and
carried no per-caller credential truth at all for credentialed tools-only
extensions (the GitHub repro — the model saw 49 github.* tools plus
"installed and active" catalog state and told the user no further connection
was required, right before the next call raised Authentication required).

Mechanism:

- ironclaw_assistant: the provider now takes the existing readiness ports —
  ExtensionCredentialSetupService (the same scope-gated credential_status the
  extensions card and the runtime auth gate resolve through) and
  ironclaw_auth::ChannelConnectionService — and classifies each Active
  installed extension with a new shared caller_extension_auth verdict in
  reborn_services/extensions.rs. The extensions card's channel-unconnected
  computation is extracted into caller_channel_connection and reused by both
  paths, so the card and the prompt can never diverge on "connected for this
  caller".
- Channels: `authenticated` is now the per-caller truth. A channel needing a
  personal OAuth/pairing binding with no proof for this caller renders
  "unauthenticated"; a genuinely paired/connected channel (or one requiring
  no personal binding, e.g. admin-managed) still renders "authenticated" —
  the #6478 truthful positive is preserved and pinned.
- ironclaw_loop_contracts: CommunicationRuntimeContext gains
  PendingExtensionAuthState; a bounded, sanitized render line names installed
  extensions the calling user has NOT authenticated and forbids the
  "already connected" claim. Unknown/empty render nothing.
- Fail-closed: when a needed verdict is unknowable (ports unwired, lookup
  failed, budget expired) both states degrade to Unknown — the slice claims
  nothing in either direction. Tools stay visible; the auth gate still owns
  enforcement at dispatch.
- ironclaw_composition: wires ProductAuthExtensionCredentialSetup and the
  generic channel-connection facade (assembly extracted into
  build_generic_channel_connection_facade, shared with the product surface)
  into the provider.
- Architecture: ironclaw_loop_contracts size ceiling raised 13112 -> 13172
  for the declaration/render vocabulary (reason recorded at the ceiling).

Regression tests: communication_context::tests
{active_channel_requiring_connection_is_not_claimed_authenticated_without_proof,
credentialed_tool_extension_without_caller_credential_is_pending_auth,
credentialed_tool_extension_with_expired_credential_is_pending_auth,
oauth_channel_not_connected_by_caller_reads_unauthenticated,
oauth_channel_connected_by_caller_reads_authenticated,
credentialed_extension_without_credential_port_degrades_to_unknown} plus
render pins in ironclaw_loop_contracts runtime_context/tests.rs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(arch): re-measure loop_contracts size ceiling for the #7247+#7294 union

Each fix measured the ceiling alone against main (#7247 raised it to
13,172 for its additions); batching both crate-growing commits onto one
branch requires the union measurement, 13,306 — read from the gate's own
failure message, pinned exactly per the #7147 no-untracked-slack lesson.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(ci): raise composition arc_dyn ceiling 814 -> 816 for the #7247 context-provider ports

Two genuine dyn seams (ExtensionCredentialSetupService + the channel-
connection facade) wired into communication-context assembly; observed
831 sites, effective ceiling re-pinned to exactly that — no slack.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* review: apply the 8 CodeRabbit findings (#7474)

- memory_recall_framing: tool results are authoritative for tool-queryable
  current state; conversation text no longer outranks a fresh tool result.
- trigger_list: limit 0 rejected (schema minimum: 1) so an empty result is
  always proof of absence; regression tests at the handler and the
  boundary suite (the old boundary test pinned the buggy empty-success).
- runtime_context: worst-case SafeSummary fixture now saturates the
  pending-auth arm (fits within 4 KiB); byte-budget truncation test added.
- agent_loop_host_contract: recall-framing filter keyed by content_ref.
- communication_context: per-extension credential lookups run under
  bounded concurrency (8, matching the extensions card) instead of
  serially inside the 500 ms budget; account-backed unconnected test
  (expired / refresh-failed rows read unauthenticated).
- composition-budget: arc_dyn_observed re-measured to 831 with the
  rationale corrected (observed == effective ceiling exactly).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 02:02:55 +00:00
..