fix(docker): repair home ownership so a root-written file cannot brick boot (#7924)

* fix(docker): repair home ownership so a root-written file cannot brick boot

A hosted instance crash-looped on every restart with:

    Error: could not resolve LLM environment fallback: Provider
    provider_registry request failed: failed to read provider registry
    overlay `/data/ironclaw-reborn/providers.json`: Permission denied (os
    error 13)

while `config.toml` in the same directory loaded fine — so this was one
file's ownership, not directory traversal. The provider-registry overlay is
fail-closed by design (an unreadable explicit overlay must not silently fall
back to compiled-in defaults), so a single root-owned file ends the boot
rather than degrading it.

Root cause is the interaction between two deliberate choices. #7723 changed
the runtime image from `USER ironclaw` to `USER root`, because sshd must
start as root before the entrypoint drops via `gosu` — so for the first time
a root process can write to the persistent volume. And the root pass's
`chown` is non-recursive, because start-sshd.sh refuses to run unless it owns
`$IRONCLAW_REBORN_HOME/ssh`. Together they mean the entrypoint fixes the home
DIRECTORY and leaves its CONTENTS untouched, and a volume outlives the image
that wrote it.

Repair ownership of the home's contents in the root pass, excluding `ssh`.
Recursing into `ssh` would trade this crash loop for a broken SSH listener,
which is why the exclusion is explicit rather than incidental.

Reproduced before fixing, in debian:bookworm-slim, seeding a volume the way
the failing instance looks:

    BEFORE  -rw-r--r-- ironclaw ironclaw  config.toml
            -rw------- root     root      providers.json
    AFTER entrypoint (root pass ran):  unchanged
    uid 1000 reading providers.json:   Permission denied

With the fix, providers.json and nested files become ironclaw-owned and
readable, while `ssh/` and its host key stay root:root.

Regression test `home_contents_ownership` asserts both halves — that every
top-level home entry is handed to chown, and that `ssh` is NOT. It fails
without the fix and passes with it. The chown stub records argv rather than
executing, so the test pins what the root pass asks for.

Scope note: this is a robustness fix, not a diagnosis of one box. I could not
inspect the instance, so I have not proven how its providers.json became
root-owned — only that this mechanism produces exactly the logged error and
that `USER root` is what makes root-owned files possible at all. A persistent
volume shared across image versions with differing runtime users should not be
able to brick boot regardless of which write created the file.

Also applies to the 1.3 line, which has carried `USER root` since #7723.

Verified: entrypoint self-tests pass on debian:bookworm-slim as a non-root
user (as CI runs them); the real-sshd regression harness still reports 16/16
against the release/1.3.1 baseline; a full entrypoint -> start-sshd -> real
SSH login still succeeds as agent:1000 with the ssh state dir root-owned;
shellcheck clean apart from two pre-existing SC2016 notices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(docker): follow a symlinked home and pin the repair's real properties

Review of the first cut found three defects, two of which meant the fix or its
test did not do what they claimed.

**The repair silently did nothing when the home is a symlink.** `find` defaults
to a physical walk, so `find "$IRONCLAW_REBORN_HOME" -mindepth 1` matches
nothing when the start path is a symlink -- no error, no repair, and the boot
loop persists with no diagnostic. Verified in debian:bookworm-slim:
`find /link -mindepth 1 -maxdepth 1` returns 0 entries, `find -H /link` returns
2. Now uses `-H`, which follows the start path only, so symlinks planted
*inside* the home are still never followed; combined with `chown -h`, a
planted symlink cannot redirect ownership outside the home. Confirmed: a
`planted -> /etc` symlink leaves /etc and /etc/passwd root-owned.

**The test did not pin recursion.** It asserted the top-level `nested` entry
appeared in the recorded argv, which stays true even with a non-recursive
chown -- while `nested/deep.db` would keep its old ownership and still brick
startup. It now asserts the nested file itself.

**The test swallowed the entrypoint's exit status** with `) || true`, so a
run that failed after the ownership commands could still pass on argv alone.
The run is now required to succeed.

Also dropped a `! -user ironclaw` filter added in the first cut as a
steady-state optimization. Its own tests caught why it was wrong: the
predicate resolves the name at find time and errors with
`find: 'ironclaw' is not the name of a known user` wherever it does not
resolve, which under `set -e` aborts the entire boot. That trades a rare
ownership repair for a guaranteed crash, and no review finding asked for it.

Each property is now proven discriminating by mutation:
- drop `-H`      -> FAIL symlinked_home_ownership
- top-level only -> FAIL home_contents_ownership (nested/deep.db)
- stop pruning ssh -> FAIL home_contents_ownership (ssh chowned)

Verified: entrypoint self-tests pass on debian:bookworm-slim as a non-root
user; the real-sshd harness still reports 16/16 against the release/1.3.1
baseline; shellcheck clean apart from two pre-existing SC2016 notices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

docker/reborn/entrypoint.sh

@@ -69,6 +69,29 @@ if [ "$(id -u)" = "0" ]; then
   # $IRONCLAW_REBORN_HOME/ssh staying root-owned.
   mkdir -p "$IRONCLAW_REBORN_HOME" /workspace
   chown ironclaw:ironclaw "$IRONCLAW_REBORN_HOME" /workspace
+  # Repair ownership of pre-existing home CONTENTS, not just the directory.
+  # A persistent volume outlives the image, and this image ran as `ironclaw`
+  # until in-worker SSH required a root entrypoint -- so a volume can carry
+  # files a root-run wrote. One unreadable file is fatal rather than
+  # degraded: the provider-registry overlay is fail-closed, so a root-owned
+  # `providers.json` crash-loops the container with
+  # "failed to read provider registry overlay ...: Permission denied", while
+  # a sibling `config.toml` written earlier still loads fine.
+  #
+  # `ssh` is excluded deliberately: start-sshd.sh refuses to start when its
+  # state directory is not root-owned, so recursing into it would trade this
+  # crash loop for a broken SSH listener.
+  # `-H` follows the start path only: $IRONCLAW_REBORN_HOME may itself be a
+  # symlink (find's default physical walk would then match nothing under
+  # -mindepth 1 and silently repair nothing), while symlinks encountered
+  # *during* the walk are still never followed -- combined with `chown -h`,
+  # a symlink planted in the home cannot redirect ownership outside it.
+  # Deliberately NOT filtered with `! -user ironclaw`: that predicate resolves
+  # the name at find time and aborts the whole boot under `set -e` wherever it
+  # does not resolve, trading a rare ownership repair for a guaranteed crash.
+  find -H "$IRONCLAW_REBORN_HOME" -mindepth 1 \
+    -path "$IRONCLAW_REBORN_HOME/ssh" -prune -o \
+    -exec chown -h ironclaw:ironclaw {} +
   # ...but ONLY when the override provably lives inside a directory this
   # entrypoint already manages. The Railway containment check that validates an
   # operator-supplied workspace root runs after the privilege drop (it needs

scripts/ci/test-reborn-docker-entrypoint.sh

@@ -500,6 +500,105 @@ for slash_var in IRONCLAW_REBORN_HOME IRONCLAW_REBORN_WORKSPACE_ROOT; do
   fi
 done
 
+# 9b. The root pass must repair ownership of pre-existing home CONTENTS --
+#     including NESTED ones -- while leaving the `ssh` state directory alone.
+#     A persistent volume outlives the image, and this image ran as `ironclaw`
+#     until in-worker SSH required a root entrypoint, so a volume can carry
+#     files a root run wrote. One unreadable file is fatal rather than
+#     degraded: the provider-registry overlay is fail-closed, so an
+#     unreadable providers.json crash-loops the container.
+#
+#     Two properties are easy to assert weakly and are therefore asserted
+#     explicitly here:
+#       - the NESTED file must appear, not just its top-level parent. A repair
+#         that passed only top-level entries to a non-recursive chown would
+#         still leave `nested/deep.db` broken while a parent-only assertion
+#         stayed green.
+#       - `ssh` must NOT appear: start-sshd.sh refuses to start unless it owns
+#         that directory, so over-repairing trades a boot loop for dead SSH.
+#     The run is also required to SUCCEED; swallowing its exit status would
+#     let a broken root boot path pass as long as the argv looked right.
+own_home="${WORK}/own-repair"
+mkdir -p "$own_home/ssh" "$own_home/nested"
+printf '%s' "$LEGACY_DISABLED" > "${own_home}/config.toml"
+printf '{}' > "${own_home}/providers.json"
+printf 'x' > "${own_home}/nested/deep.db"
+printf 'k' > "${own_home}/ssh/ssh_host_ed25519_key"
+own_chown_record="${WORK}/own-chown-argv"
+
+rm -f "$own_chown_record"
+touch "$own_chown_record"
+if ! (
+  export PATH="${WORK}/root-bin:${WORK}/bin:${PATH}"
+  export IRONCLAW_REBORN_HOME="$own_home"
+  export IRONCLAW_REBORN_DEFAULT_CONFIG=/opt/ironclaw/reborn/config.toml
+  export IRONCLAW_STUB_ARGV_PATH="${own_home}/argv"
+  export IRONCLAW_STUB_WORKSPACE_PATH="${own_home}/workspace-root"
+  export IRONCLAW_STUB_CHOWN_PATH="$own_chown_record"
+  export IRONCLAW_STUB_GOSU_PATH="${own_home}/gosu-argv"
+  export IRONCLAW_STUB_UID=0
+  unset IRONCLAW_REBORN_WORKSPACE_ROOT
+  unset RAILWAY_ENVIRONMENT RAILWAY_PROJECT_ID RAILWAY_SERVICE_ID RAILWAY_VOLUME_MOUNT_PATH
+  sh "$ENTRYPOINT" >/dev/null 2>"${own_home}/entrypoint.err"
+); then
+  echo "FAIL[home_contents_ownership]: entrypoint failed during the root pass" >&2
+  cat "${own_home}/entrypoint.err" >&2
+  failures=$((failures + 1))
+fi
+
+# The chown stub records argv rather than executing, so assert on what the root
+# pass asked for. `nested/deep.db` is the discriminating one.
+for own_expected in config.toml providers.json nested/deep.db; do
+  if ! grep -q "${own_home}/${own_expected}" "$own_chown_record"; then
+    echo "FAIL[home_contents_ownership]: root pass never chowned ${own_expected}; a root-written file would crash-loop the container" >&2
+    cat "$own_chown_record" >&2
+    failures=$((failures + 1))
+  fi
+done
+if grep -q "${own_home}/ssh" "$own_chown_record"; then
+  echo "FAIL[home_contents_ownership]: root pass chowned the ssh state directory; start-sshd.sh requires it to stay root-owned" >&2
+  cat "$own_chown_record" >&2
+  failures=$((failures + 1))
+fi
+
+# 9c. The home may itself be a SYMLINK. `find` defaults to a physical walk, so
+#     a symlinked start path has no descendants under -mindepth 1 and the
+#     repair silently does nothing -- the boot loop would persist with no
+#     diagnostic. `-H` follows the start path only, which is why symlinks
+#     planted *inside* the home still cannot redirect ownership outward.
+link_home_real="${WORK}/own-link-real"
+link_home="${WORK}/own-link"
+mkdir -p "$link_home_real"
+printf '%s' "$LEGACY_DISABLED" > "${link_home_real}/config.toml"
+printf '{}' > "${link_home_real}/providers.json"
+ln -sfn "$link_home_real" "$link_home"
+link_chown_record="${WORK}/own-link-chown-argv"
+
+rm -f "$link_chown_record"
+touch "$link_chown_record"
+if ! (
+  export PATH="${WORK}/root-bin:${WORK}/bin:${PATH}"
+  export IRONCLAW_REBORN_HOME="$link_home"
+  export IRONCLAW_REBORN_DEFAULT_CONFIG=/opt/ironclaw/reborn/config.toml
+  export IRONCLAW_STUB_ARGV_PATH="${link_home}/argv"
+  export IRONCLAW_STUB_WORKSPACE_PATH="${link_home}/workspace-root"
+  export IRONCLAW_STUB_CHOWN_PATH="$link_chown_record"
+  export IRONCLAW_STUB_GOSU_PATH="${link_home}/gosu-argv"
+  export IRONCLAW_STUB_UID=0
+  unset IRONCLAW_REBORN_WORKSPACE_ROOT
+  unset RAILWAY_ENVIRONMENT RAILWAY_PROJECT_ID RAILWAY_SERVICE_ID RAILWAY_VOLUME_MOUNT_PATH
+  sh "$ENTRYPOINT" >/dev/null 2>"${link_home}/entrypoint.err"
+); then
+  echo "FAIL[symlinked_home_ownership]: entrypoint failed with a symlinked home" >&2
+  cat "${link_home}/entrypoint.err" >&2
+  failures=$((failures + 1))
+fi
+if ! grep -q "providers.json" "$link_chown_record"; then
+  echo "FAIL[symlinked_home_ownership]: repair walked nothing through a symlinked home; the boot loop would persist silently" >&2
+  cat "$link_chown_record" >&2
+  failures=$((failures + 1))
+fi
+
 # 10. The dead variable has no reader left anywhere in the tree.
 if grep -rn 'IRONCLAW_REBORN_SLACK_ENABLED' \
   --include='*.rs' --include='*.sh' --include='*.toml' \