lywsvip/ironclaw
1
0
Fork 0
mirror of https://github.com/nearai/ironclaw.git synced 2026-09-03 08:06:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

prompt: tighten evidence rule to FINAL() claims only, not tool use

Browse Source 
Live-test validation of the "Evidence before claiming side effects"
section (added in the prior commit) showed it inhibited legitimate
tool use. With the original wording, `zizmor_scan_v2` live-recording
timed out at 302s with zero responses; reverting the postamble
restored healthy behavior (88s run, 8 shell calls including
`cargo install zizmor` and full workflow analysis).

The original phrasing conflated two things: what the agent should
claim and what tools it should call. The rule is only about the
claim. Re-tunes the section to:

- Open with an explicit "this does not restrict tool calls" scope.
- Drop the "<1ms = failure" heuristic (too broad — normal tools like
  `tool_info(schema)` are legitimately fast).
- Drop the full enumeration of forbidden side-effect verbs; keep the
  rule narrower and clearer.
- Shorten the code example (remove redundant early-return).

Re-tuned run: agent is active (shell calls, real reasoning), live
recording completes in ~9s. The remaining test failure is a
pre-existing assertion bug (exact `t == "shell"` match against tool
strings that now carry arguments like `"shell(cmd)"`) — reproduces
with the old postamble too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Illia Polosukhin
2026-04-20 06:52:46 +00:00
parent 9d7b43a9f3
commit 50d851758c
1 changed files with 11 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
crates/ironclaw_engine/prompts/codeact_postamble.md
View File

@@ -77,29 +77,21 @@ string with real data from tool results (`proposal["rationale"]`,
`proposal["projected_annual_gain_usd"]`, etc.), then call `FINAL()` once
with the complete Markdown.
## Evidence before claiming side effects
## Claims in FINAL() need tool evidence
If the user asked for a side effect — send, save, install, schedule, post,
write, delete — your `FINAL()` answer must either:
This rule is only about what your `FINAL()` answer asserts — it does not
restrict tool calls. Call as many tools as the task needs.
1. Cite the specific result of a successful tool call (the `message_id`,
`bytes_written`, `external_id`, `job_id`, `created_at` the tool
returned), or
2. Say plainly that the action was not performed, and why.
Never narrate "I've sent your message" / "I attached the file" / "I
installed the tool" without the corresponding tool result in the same
response. Your text is narration, not proof. A user who reads "I sent
it" while nothing arrived is the worst failure mode this system has.
If a tool call returned empty output or completed in under a millisecond,
treat that as failure — the side effect did not happen. Retry or report
the failure, do not claim success.
If `FINAL()` says you did something — "sent", "saved", "installed",
"posted", "scheduled", "wrote", "deleted" — the same answer must cite
the tool result that proves it (e.g. `message_id`, `bytes_written`,
`external_id`, `job_id`). If no tool produced that evidence, say what
actually happened instead: "Tried to install X, cargo returned error Y."
```repl
result = await telegram_send(chat_id=chat, text=body)
if not result or not result.get("message_id"):
FINAL(f"Tried to send, but Telegram didn't confirm delivery: {result}")
else:
if result and result.get("message_id"):
FINAL(f"Sent (message_id={result['message_id']}).")
else:
FINAL(f"Tried to send but Telegram did not confirm delivery: {result}")
```