* Add hosted single-tenant Postgres profile
* Persist hosted extension state under tenant storage
* fix(reborn): bound filesystem event tail reads
* fix(reborn): reduce hosted postgres read amplification
* fix(reborn): batch filesystem thread history reads
* fix(filesystem): avoid postgres prefix scans
* fix(reborn): harden hosted postgres bootstrap
* fix(webui): advance empty projection cursors
* fix(reborn): batch llm secret metadata reads
* perf(webui): back off idle stream polling
* perf(turns): cache fresh filesystem snapshots for reads
* fix(reborn): authorize extension lifecycle catalog mounts
* fix(reborn): time out wedged turn drivers
* fix(reborn): bound stuck heartbeat calls
* refactor: split postgres PR hot path helpers
* reborn: address hosted postgres review feedback
* turns: move test module after production items
* reborn: fix hosted postgres review feedback
* reborn: fix projection ci regressions
* tests: relax reborn harness heartbeat
* tests: widen reborn harness heartbeat
* ci: harden cargo network fetches
* reborn: fix hosted runtime gate diagnostic
* tests: wait for budget gate materialization
* reborn: fix hosted postgres review and ci issues
* reborn: tighten hosted postgres review fixes
* reborn: keep nearai bootstrap outcome local
* reborn: clarify hosted local-runtime seams
* fix(reborn): address CodeRabbit review feedback (#5081)
* test(reborn): stabilize runtime no-gateway failure check
* fix(ci): update test secret metadata expiry plumbing
* fix(threads): make postgres message acceptance atomic
* fix(filesystem): retry postgres migration connect
* fix(reborn): lower default postgres pool size
* fix(reborn): allow postgres pool cap override
* fix(filesystem): extend postgres startup retry window
* fix(railway): disable reborn deployment overlap
* fix(reborn): serve startup health before postgres runtime
* fix(reborn): harden startup and inbound accept paths
* fix(ci): restore reborn bootstrap build
* fix(ci): stabilize reborn harness heartbeat
* fix(reborn): collapse repeated auto-approve lookups
* fix(reborn): suppress noisy debug logs in hosted serve
* fix(reborn): raise hosted-single-tenant postgres pool to 16
The hosted profile shared a single deadpool across every Postgres-backed
subsystem (turns, threads, messages, events, secrets), and every filesystem
op checks out a connection. With pool_max_size=2 a single turn's reads plus
an open transaction monopolize both connections, so the runner heartbeat and
webui block indefinitely on pool.get() (no get() timeout), the 90s lease
expires, and the turn wedges with failure_category=lease_expired.
The cap was lowered to fit a managed session-pool limit during blue-green
deploy overlap; that overlap is resolved, so restore a healthy pool (16, the
prior default). Runtime override via IRONCLAW_REBORN_POSTGRES_POOL_MAX_SIZE
is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* perf(reborn): cache prepared statements + bound postgres pool checkout
Two latency/deadlock fixes for the Postgres-backed root filesystem, which
backs every Reborn subsystem (turns, threads, messages, events, secrets) and
checks out a pooled connection per op.
Caching: every fixed-SQL read/write went through `client.query_opt(sql, ..)`
with a string, so tokio_postgres issued a fresh `Parse` (prepare) on every
call — ~2.77ms RTT to remote Postgres measured from production logs, ~48% of
each read, and a monotonically growing set of server-side prepared statements
(s1625, s1626, ... never reused). Route the deadpool `Object` (not the
deref'd `Client`) through new `cached_query_opt`/`cached_query`/
`cached_query_one`/`cached_execute` helpers that use deadpool's per-connection
`prepare_cached`. The Parse round-trip is now paid once per connection per
distinct statement; the pooled connection is held for ~half as long per op,
which is what relieves the pool contention behind the lease-expiry wedge.
Dynamic SQL (filter `query`, index DDL, create_dir_all's txn) stays uncached
to keep the cache bounded. Helpers return `tokio_postgres::Error` so existing
`db_error` mapping at call sites is unchanged.
Deadlock guard: the pool was built with no checkout timeout, so `Pool::get()`
blocks forever once all connections are busy — an unbounded wait wedges the
runner heartbeat and webui until the 90s lease expires. Add 30s
wait/create/recycle timeouts (well under the lease) so a saturated pool
surfaces a retryable error instead of hanging the process.
Verified: `cargo clippy` clean and `cargo check` pass for ironclaw_filesystem,
ironclaw_reborn_event_store, and ironclaw_reborn_composition with
postgres[,webui-v2-beta]. Live Postgres behavior + latency delta verified via
PR CI and Railway (no local live-pg test / Docker).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(reborn): size hosted postgres pool for the Supabase pooler (10)
The reference hosted deployment fronts Postgres with the Supabase Supavisor
session pooler (default pool size ~15), so the shared app pool must stay under
that cap. 16 could exceed it; 10 covers runtime concurrency (heartbeat, webui,
trigger poller, turn driver reads + open txn) with headroom for migrations and
admin sessions. Runtime override IRONCLAW_REBORN_POSTGRES_POOL_MAX_SIZE still
wins over this file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Document Reborn serve/WebUI testing flow in reborn-binary.md
The doc still listed "web gateway/UI startup" under "does not yet
support", but `serve` now starts the WebChat v2 web UI (behind
`--features webui-v2-beta`). It also omitted several shipped commands and
gave no provider-agnostic setup path for testers.
- Correct the stale "does not yet support" claim; note serve's WebUI is a
feature-gated beta surface.
- Add a "Running with the WebUI (serve)" walkthrough: provider-neutral
quick start, the required IRONCLAW_REBORN_WEBUI_* auth env vars, the /v2
UI path, a table of common startup errors (missing token/user-id,
identity-owner mismatch, workspace/skill-root overlap) with fixes, and a
curl API smoke test.
- Add a "Choose your model provider" table (NEAR AI, OpenAI, Anthropic,
Ollama) with each provider's set-provider id, key env var, and default
model, plus how to discover any provider's env var via `models status`
(default.api_key_env) and `models list --verbose`.
- Document the workspace-root-overlap gotcha: serve/run/repl use the cwd
as the workspace root, so IRONCLAW_REBORN_HOME must live outside it.
- Add command sections for onboard, repl, serve, and models set-provider;
correct the models status field list to match actual output; list the
new commands (and the contributor-only traces tree) in current status.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Clarify local-dev skill-root paths in overlap gotcha
Address review feedback on the workspace-overlap note. The overlap
validation guards the flat/default storage roots (/skills ->
<reborn-home>/local-dev/skills, /tenant-shared/skills, /system/skills,
/system/extensions), which is what the error message names. Make that
precise, and add that resolved per-user skills actually live under the
tenant-scoped path (<reborn-home>/local-dev/tenants/default/users/<owner>/skills),
into which the legacy flat root is backfilled.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address review: required-key startup failure, shell-safe quick start
Incorporate Codex and Copilot review feedback on reborn-binary.md:
- Required-key providers (openai/anthropic, api_key_required=true) fail at
startup during LLM resolution when the key env var is missing
("llm provider '<id>' requires API key env var '<VAR>'"), not "later at
turn time". Document this; keyless providers (ollama) and NEAR AI's
session flow (api_key_required=false) boot without the key. (Codex P2)
- Replace shell-breaking <provider>/<PROVIDER_API_KEY> placeholders in the
quick-start block with a concrete nearai example; unquoted angle brackets
are parsed as redirections and break copy/paste. (Codex P3)
- Note set-provider writes no api_key_env for keyless providers like ollama,
in both the provider table intro and the command section. (Copilot)
- Clarify that the home-outside-cwd rule is specific to serve/run/repl,
which use the cwd as the local-dev workspace root. (Copilot)
- Note the WEBUI auth var NAMES are defaults, overridable via
[webui].env_token_var / env_user_id_var. (Copilot)
- Note models status field names are the text output; --json nests under
default with raw struct fields (provider_id, not provider). (Copilot)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add scripts/run-reborn-webui.sh launcher
A one-command wrapper around the serve/WebUI quick start that handles the
setup footguns documented in reborn-binary.md:
- keeps IRONCLAW_REBORN_HOME outside the repo and refuses an in-repo home
(which would trip the workspace/skill-root overlap validation);
- runs `models set-provider` for the chosen provider (PROVIDER/MODEL env);
- discovers the credential env var via `models status` and warns if unset;
- generates the WebUI bearer token and matches the WebUI user to the
identity owner so serve doesn't refuse to start;
- prints the login token + /v2 URL, then execs serve.
PROVIDER/MODEL/HOST/PORT/IRONCLAW_REBORN_HOME are overridable. Referenced
from the "Running with the WebUI (serve)" section as the shortcut path.
Verified end to end: the script configures the route, binds the WebChat v2
listener on 127.0.0.1:3000, and /v2 + /api/health respond.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Reject PORT=0 in run-reborn-webui.sh launcher
The launcher prints a login URL for a browser, so a kernel-assigned port
(PORT=0) produces an unusable http://HOST:0/v2. Reject PORT=0 early with a
clear message pointing at the raw `serve --port 0` test-harness form,
rather than advertising an onboarding URL that can't be opened.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address review: onboard flag, port-0 config, status base_url
Second Copilot review pass on reborn-binary.md:
- Add `onboard --import-history` to the current-status command list; it is
a real flag (onboard.rs) documented later but omitted up top.
- Clarify that `--port 0` (ephemeral) is the CLI flag only; serve rejects
`[webui].listen_port = 0` from config (serve.rs).
- Add `default.base_url` to the `models status` text field list; it is
printed conditionally when the route configures a base URL (models.rs).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address review: derive WebUI owner from config, clarify async timeline
- run-reborn-webui.sh now reads [identity].default_owner from the home's
config.toml (after set-provider seeds it) and uses it for the WebUI user,
falling back to reborn-cli. Previously it only defaulted to reborn-cli, so
a home with a custom owner would still hit the owner-mismatch error the
header comment claimed to prevent. Comment reworded to match. (Copilot)
- Smoke-test snippet: clarify the single timeline GET is not a loop — turn
execution is async, so re-run until an assistant message reaches
status "finalized". (Copilot)
The third comment (${!key_env:-} tripping set -u) was a false positive:
verified on bash 3.2 that the :- form returns empty for an unset referenced
var and does not abort; only the form without :- aborts. Code unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Address review: HOST/zsh collision, path canonicalization, doc clarifications
Incorporate Firat and Copilot review feedback.
run-reborn-webui.sh:
- Rename HOST/PORT -> REBORN_HOST/REBORN_PORT. A bare HOST collides with
zsh's auto-set $HOST (machine hostname); a zsh user without an explicit
HOST would bind serve to a non-loopback interface and expose the bearer
token over plain HTTP. (Firat, Security/Medium)
- Canonicalize the home and repo root (resolve `..`/symlinks via the parent
dir's `pwd -P`) before the overlap prefix check, so a valid sibling home
like `../reborn-home` is no longer falsely rejected. (Copilot)
- Escape the literal dot in the sed key-env extraction. (Firat, Bug/Low)
docs/reborn-binary.md:
- Annotate `onboard --import-history` in the status command list as parsed
but not-yet-wired, and note `serve` is only compiled with the feature.
(Firat, Conventions/Low)
- State explicitly that the `serve` subcommand is compiled behind
`--features webui-v2-beta` and is absent from `--help` without it
(verified: #[cfg(feature = "webui-v2-beta")] on the Serve variant). (Copilot)
- Update launcher override vars to REBORN_HOST/REBORN_PORT.
Verified: launcher binds on a custom REBORN_PORT, overlap guard accepts
`../reborn-home` and rejects in-repo homes, REBORN_PORT=0 rejected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Wire Reborn production Postgres storage config
* Fix Reborn production secret master key config
* Make Reborn production runtime launchable
* fix(reborn): fail closed when production hooks are enabled
* style(reborn): tidy production hooks regression test attributes
* fix(reborn): address production runtime review feedback
* fix(reborn): update runtime test harness wiring
* test(reborn): align production hook fixture with sandbox policy
---------
Co-authored-by: serrrfirat <f@nuff.tech>
* arch(ws-17): prove product live planned-runtime cutover
Squash of #3653 (8 commits) onto reborn-integration stack.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* arch(ws-17): address PR review (debug log, cancel-path proof, fail-closed gates)
Resolve reviewer feedback on #3653:
- zmanian: add `debug!` on the default `is_product_cancellation_observed`
Ok(false) so on-call has a breadcrumb when a factory is not product-live.
- zmanian: replace remaining `.unwrap()` with typed `.expect("planned
default profile resolver")` in WS-14/WS-16/WS-17 reborn tests.
- serrrfirat #1: remove the manual `request_product_cancellation`
backdoor from the product-live cancellation proof. Wire a
`CompositeTurnRunWakeNotifier` in `build_default_planned_runtime` so
`coordinator.cancel_run` fans out to both the worker wake channel and
`RunCancellationFactory::notify_run_wake`. The cancellation contract
test now drives observation purely from `cancel_run` and polls until
the retained run handle flips.
- henrypark133 #1-3: extend the product-live readiness gate with
fail-closed checks for `ModelPolicyGuard`, `ModelBudgetAccountant`,
and `SafetyContext`. Adds matching `DefaultPlannedRuntimeParts`
fields, three new `ProductLiveRuntimeReadinessComponent` variants,
builder wiring on `RebornLoopDriverHostFactory`, and three new
regression tests asserting each missing component is rejected.
Item #4 (`production_readiness` gate invocation from a startup entry
point) and serrrfirat #2 (tool-use canary) remain deferred per the
PR description — both are part of the composition-root flip, which
zmanian's review tagged for a separate PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* arch(ws-17): tidy review nits (typed probe error, probe lifetime doc, test helpers)
- runtime: re-type ProductLiveRuntimeBuildError::Probe.reason
String -> source: AgentLoopHostError. Strongly-typed per
.claude/rules/types.md, carries kind + diagnostic_ref, and Error::source
now returns the underlying probe failure for chain inspectors.
- cancellation_port: doc-comment RunCancellationFactory::notify_run_wake
with the sync/non-blocking contract that CompositeTurnRunWakeNotifier
relies on; doc-comment ProductLiveCancellationProbe with the
ephemeral-handle contract.
- inbound_turn_contract / loop_driver_host tests: make the test
ReadyRunCancellationProbe own its RunCancellationHandle directly
(was leaking one entry into the factory handles map on every
readiness verify); add local turn_state_store_dyn() and
test_safety_context() helpers and route the duplicated cast +
InstructionSafetyContext::new("policy:test", ...) call sites
through them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
