mirror of https://github.com/Hmbown/DeepSeek-TUI.git synced 2026-09-03 06:50:13 +08:00

Files

Hmbown f2ba5d15e4 docs: make the tool-surface, subagent, and config references match the code

docs/TOOL_SURFACE.md carried four claims the runtime's own tests contradict:

1. "The default-active policy contains exactly these ten names" listing
   `update_plan`. `DEFAULT_ACTIVE_NATIVE_TOOLS`
   (crates/tui/src/core/engine/tool_catalog.rs:44-58) has eight entries and
   `update_plan` is not among them — it appears nowhere in tool_catalog.rs. The
   policy is nine (those eight plus synthetic `tool_search`), eight with memory
   disabled. `update_plan` is registered (crates/tui/src/tools/plan.rs:401) but
   reachable only through `tool_search`; the tool table now says so.
2. "A memory-disabled or Moraine-fallback runtime". There is no Moraine
   fallback — docs/MEMORY.md:11-13 records the removal, and
   crates/tui/src/prompts.rs:2445-2449 is a test asserting MEMORY_GUIDANCE must
   not contain the word.
3. A "Replay-only aliases" table promising "saved transcripts, sessions, and
   recorded automation replay without migration" for 23 names, 16 of which are
   asserted REMOVED at crates/tui/src/tools/registry.rs:2066-2088 ("{retired}
   must stay removed") and 6 more at :2290-2304 ("{alias} must be removed").
   Split into a "Removed spellings" section (with the registry.rs:313-316 note
   that resolve has no fuzzy step, so those calls fail rather than dispatch) and
   a "Replay-only aliases" section holding only what is still registered:
   apply_patch, task_*, github_*, automation_*, rlm_*, checklist_*/todo_*.
4. A "Release verification" block whose three cargo filters name tests that do
   not exist (`rg` finds those three strings only in that doc). `cargo test`
   exits 0 with "0 passed; N filtered out" on a filter that matches nothing, so
   a release engineer following it got three green checkmarks having verified
   nothing. Replaced with the real names —
   `shell_surface_contains_only_the_canonical_bash_tool` (registry.rs:2290) and
   `runtime_task_families_expose_only_canonical_tools` (registry.rs:2333) — plus
   the receipt test, and a warning about the silent-pass failure mode.

docs/RUNTIME_SIMPLIFICATION_DESIGN.md repeats errors 1 and 3 and is designated
authoritative by docs/TOOL_LIFECYCLE.md:3-7, but carries no status marker. Given
a status banner naming both divergences and pointing at TOOL_SURFACE.md; the
"Rejected alternatives" provenance is worth keeping, so not deleted.

docs/SUBAGENTS.md:
- "a bounded queue of up to 200 running plus queued sub-agents by default" —
  `MAX_SUBAGENT_ADMISSION` is 1024 (crates/tui/src/config/subagent_limits.rs:21),
  which is what docs/TOOL_SURFACE.md:182 already said. The 64/128 concurrency
  figures on the same page were correct and are untouched.
- The memory section described a `memory.md` that does not exist and omitted the
  `scope` parameter. crates/tui/src/tools/remember.rs:165 states the legacy
  single-file path was removed in v0.9.4; writes go through
  `NativeMemoryStore::remember(scope, workspace_id, note)` (remember.rs:77-108).

config.example.toml documented two key sets that do not exist. Neither struct has
`deny_unknown_fields`, so both were silently discarded rather than rejected:
- `[advisor] max_tool_pairs` / `system_prompt`. `AdvisorConfigToml`
  (crates/config/src/lib.rs:2369-2394) has enabled, max_tool_calls (default 10,
  clamped 1-50 — the doc said 8, max 32), rate_limit_secs, dedup_window_secs,
  and model. `model` was undocumented; now it is.
- `[fleet.profiles.*.permissions] allow_tools` / `deny_tools`.
  `FleetProfilePermissions` (lib.rs:1966-1977) has allow_shell, trust,
  approval_required. `rg 'allow_tools|deny_tools' crates/` finds nothing. The
  example value was `"exec_shell"`, itself a removed tool name.

docs/CONFIGURATION.md: deleted the "Parsed but currently unused" section. Its one
entry, `tools_file`, is not parsed by anything — the field was removed in
346bfe3b6 and the doc bullet was orphaned. Repo-wide `rg` finds the string only
in that section, and nothing links a #parsed-but-currently-unused anchor.

docs/TTC_DESIGN.md said implementation "is deferred beyond v0.9.0". The `verify`
tool shipped and is default-on (crates/tui/src/tools/verify.rs,
features.rs:262, registry.rs:1040-1041 with verify_tool_enabled defaulted true).
Retitled as landed-in-part; capability (B) is still genuinely deferred, so the
doc stays. Its interface line said `with_verify(critic)`; the real signature is
`with_verify_tool(client, model)` (registry.rs:886).

docs/skills/README.md advertised `gh-plan-issues`, deleted in 18de2ebc0, and
credited these skills to "the v0.8.61 release" at a 0.9.4 release.

docs/architecture/provider-model-settings-v091.md pinned
`provider_is_configured` to config.rs:8625-8669; it is at :10160 and that region
is now unrelated code. Replaced with the symbol name, since config.rs is under
active edit.

docs/architecture/command-dispatch.md:133 claimed EPIC-002 was "ready for PR".
The PR (#3706) merged and #2870 closed 2026-08-01. Line 145 was an empty
"Current Evidence (Draft)" heading with no content; removed.

.gitignore: `git check-ignore -v` attributes .claude/settings.json,
scheduled_tasks.lock, worktrees/, and *.local.* to the blanket `.claude/` at
line 126, not to the specific rules above them. Dropped the redundant ones and
annotated why the HANDOFF_/CODEMAP_ patterns are deliberately kept.

2026-08-03 22:51:59 -07:00

8.6 KiB

Raw Blame History

Test-Time Compute (TTC) in CodeWhale — design

Status: landed in part. Capability (A), the verify tool, shipped and is default-on as of v0.9.4 — crates/tui/src/tools/verify.rs, feature key verify_tool (crates/tui/src/features.rs:262), registered at crates/tui/src/tools/registry.rs:1040-1041 with verify_tool_enabled defaulting to true (registry.rs:546-561). Capability (B) (sub-agent reasoning-effort escalation) and the #3982/#4013 triggers remain deferred. Treat the (A) sections below as a design record of what shipped, not as a plan; the deferred-work notes at the end still stand for (B).

Synthesized from three independent reviews — the verify-tool implementation contributor, GLM 5.2, and an internal analysis — which all converged.

What TTC means here

Spend more inference at decision time for a better answer, on the agent's own judgment — not an always-on tax. Two capabilities:

(A) An agent-invoked verify / critic pass — the model chooses to adversarially review its own recent work before claiming done, to catch "green-but-wrong" (motivating case: a fix passed 16/16 CI but only covered a CLI path, not the interactive TUI; a critic pass caught it, deterministic CI did not).
(B) Reasoning-effort escalation for sub-agents — un-cap the hard Low clamp so a Fleet role can think at the tier its job needs.

Core principle: one `CriticEngine`, three triggers

Factor a single CriticEngine that owns: the target-context snapshot (recent tool calls + the claimed-done state + a diff/evidence gather), the prompt-template family (adversarial "refute it"), the reasoning effort (Max), the tools-disabled flag, and the structured verdict schema (verdict: pass|fail|uncertain, findings [{severity, issue, evidence, suggested_fix}], unresolved_risk).

Then three distinct entry points share that engine but keep their own invocation contracts — do NOT merge the triggers:

Trigger	Contract	Issue
`verify` tool	sync, model-chosen, default-on	#4196 (MVP in PR #4199)
advisor watcher	async, rate-limited, off-by-default	#3982
verification gates	post-turn, deterministic (compile/test/lint/review)	#4013

Unify the engine, never the trigger. Merging sync/model-chosen + async/throttled + post-turn/deterministic produces a Frankenstein. Sharing the engine keeps drift at zero while each entry point keeps its character.

(A) The `verify` tool

Why a tool, not a critic sub-agent: a verify tool is structurally isomorphic to the existing review tool — same ToolSpec trait, same ToolRegistryBuilder path, same Feature gate, same MessageRequest reasoning normalization. It inherits every existing guarantee for almost no new surface. A critic sub-agent would be a second runtime with a second policy surface (spawn-depth, allowlist, sub-agent tier resolution) — the textbook bolted-on smell. (A sub-agent critic that autonomously explores may return as an opt-in follow-up behind the same tool contract once #4193's spawn work has settled — but it is NOT the default.)

Interface: registered via ToolRegistryBuilder::with_verify_tool(client, model) (registry.rs:886 — the design said with_verify(critic); the shipped signature takes an optional client and a model name), gated by a Feature flag. Input: claim (required) + optional requirement, scope (diff|staged|none), base, files[], focus. It snapshots evidence deterministically (diff by scope — including uncommitted working-tree changes when a base is given, per PR #4199 fix — plus named files), builds ONE MessageRequest at ReasoningEffort::Max with tools disabled, and returns the structured verdict as a tool result.

Where it plugs in: the standard tool loop. No new control plane. The model invokes it like read/edit/review.

How the model decides (and abuse is bounded):

Constitution rule (harness-enforced, not prose): verify before claiming done when debugging, on multi-file changes, security-sensitive edits, or changes touching divergent surfaces (CLI vs TUI, sync vs async). The green-CI-but-wrong case is the canonical trigger.
Engine-level rate limit / TtcBudget: per-turn (one verify) and per-session budget, held in the engine, not in MessageRequest.
Feedback loop: the verdict returns to the model; several consecutive clean verdicts should, via the Constitution, discourage further calls that session. The model sees its own hit rate and self-corrects.

Verdict semantics (PR #4199, already hardened): any finding at medium severity or above forces unresolved_risk = true and downgrades an upheld verdict to uncertain; only low nits are exempt. Advisory by default; a Constitution rule may make a fail/unresolved_risk verdict something the agent must address before claiming done (soft-block in the harness, not hard-coded in the tool).

Recursion / cost bounding:

By construction: tools disabled inside the critic call ⇒ no further tool calls.
By registry: verify is structurally refused when building a SubAgentRuntime allowlist — via a Feature::CriticProducerOnly (or equivalent) the ToolRegistryBuilder checks. The spawn-depth guard is only the backup line, not the primary.
By budget: the per-session TtcBudget consulted by the engine.

(B) Sub-agent reasoning — replace the clamp with a floor

The bug in auto_reasoning.rs was never "Low is wrong" — it's that Low is a ceiling for sub-agents. Fix: Low stays the default floor; remove the ceiling.

Tier resolution order: Profile (#4137) → explicit task override → session default → Low.

SubAgentRuntime.reasoning_effort continues to be forwarded verbatim.
Auto inside a sub-agent resolves through a Fleet-role-aware resolver (a review-role profile pins High, a search-role pins Low, a planner pins Max) — NOT the global keyword resolver.
The agent tool's reasoning_effort becomes "inherit from the Fleet profile unless explicitly overridden at spawn."

Do not change the default floor from Low — sub-agent traffic is majority search/lookup, and raising the floor silently raises cost on every existing fleet. Non-surprising > clever. This composes with #4137 (profile carries the tier alongside provider/model) rather than competing with it.

Anti-patterns (what would read as bolted-on in CodeWhale specifically)

A second critic implementation — if verify/review/#3982/#4013 each roll their own prompt+call+parse, four paths diverge on the first bug. The single CriticEngine is the whole game.
A non-tool control plane for reasoning escalation — CodeWhale's model contract is tool-shaped; a side-channel breaks symmetry and bleeds into every provider adapter. The verify call is the escalation (Max internally). One vocabulary.
Recursion policy in Constitution prose — enforce it in the registry builder; the depth guard is secondary.
Cost accounting leaking into MessageRequest — budget belongs in the engine + a session TtcBudget. Don't make every tool cost-aware.
auto_reasoning.rs becoming TTC-aware — Auto resolves effort per turn; the model decides verify. Keep them separate or you get non-determinism the user can't reason about.
Gate-ordering ambiguity between #4013 and verify — different lifecycle points (mid-turn/model-chosen vs post-turn/deterministic). Document in the Constitution so contributors don't merge them.
Conflating the watcher's contract with verify's — #3982 is async/throttled/off; verify is sync/chosen/on. Share the engine; never the trigger.

Issue map & sequencing

#4196 — verify tool. MVP in PR #4199 (direct-critic, Max, tools-disabled, recursion-guarded, verdict-hardened). Refactor to sit on the extracted CriticEngine before merge. Config-disjoint (crates/tui/src/tools/).
CriticEngine extraction — new; refactor review's call/parse into the shared engine, then have verify consume it. Prereq for wiring #3982/#4013 to it later.
#4137 — Fleet profile carries a reasoning tier; drives (B). Touches crates/config — sequence with the config work + after #4136 (canonical AgentProfile) and #4193 (landed).
(B) resolver — auto_reasoning.rs clamp→floor + Fleet-role-aware Auto resolution.
#3982 / #4013 — rebase onto the CriticEngine as additional triggers (later).

All of the above is post-v0.9.0 work. PR #4199 remains a useful, green implementation candidate, but it is too large to join the release at the final gate without the shared CriticEngine refactor and fresh end-to-end review.

8.6 KiB Raw Blame History