mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
* docs: enforce the docs/ publication boundary with a frozen .mintignore and CI gate docs/ mixes the public Mintlify site with internal engineering docs, and omission from docs.json navigation is not a publication boundary: a page left out of navigation is still deployed, reachable by URL, and indexable. docs/design/ and docs/research/ were never added to docs/.mintignore, so both internal docs have been served as hidden pages on the public site. Close the gap and the process hole behind it: * Move docs/design/ and docs/research/ under docs/internal/, the one growing home for internal material — new internal docs now land inside the fence by default instead of requiring a .mintignore edit. * Freeze docs/.mintignore: scripts/ci/docs_publication_boundary.py rejects any new entry (legacy directories stay listed until consolidated into internal/; entries may only be removed). * Gate in CI (Code Style): every .md/.mdx under docs/ must be in docs.json navigation, matched by .mintignore, or carry `hidden: true` frontmatter marking a deliberately unlisted public page; navigation entries must have a source file. The gate has its own has_docs trigger because docs-only PRs skip every Rust lane, and it is checked in the roll-up before the has_code early exit so it blocks docs-only PRs too. Regression coverage: scripts/ci/test_docs_publication_boundary.py (16 cases, run by the CI job before the check; one pins the real docs/ tree as clean). Red/green verified: the checker flagged exactly docs/design/agent-activity-streaming.md and docs/research/pi-agent-deep-dive.md before the fix and passes after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: relocate legacy internal doc directories under docs/internal/ — text-only Move plans/, superpowers/, qa/, adr/, architecture-video/, and reborn-binary.md from docs/ into docs/internal/, and rewrite every repo reference to the old paths (guidance files, script and workflow comments, Rust doc comments, the render-architecture-video VIDEO_DIR, the architecture-video skill, .coderabbit.yaml). Behavior unchanged: all references to these directories were textual except the video script's VIDEO_DIR, the skill paths, and the .coderabbit.yaml ignore, which are updated in step. docs/reborn/ deliberately stays put: its path is load-bearing (ironclaw_capabilities and ironclaw_architecture_tests read contract files from it at test time, and reborn-e2e.yml scope filters match it — pinned by scripts/ci/ws12_workflow_contracts.py). It consolidates into internal/ in a follow-up when those consumers can move with it; docs/.mintignore and FROZEN_MINTIGNORE_PATTERNS shrink to internal/ + reborn/ accordingly. Verified: docs publication boundary check green, its 16 self-tests green, ws12 workflow contracts green (46), touched YAML parses, zero references to the old paths remain outside git history. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover the docs gate's entry point and pin its trigger in ws12 contracts Fixes the three findings from the multi-agent code review of this branch (security/bugs/performance/conventions clean; tests reviewer found 3): * main() was never called by any test — the exit-code contract, the three stderr violation blocks, and main()'s MintignoreSyntaxError handling were uncovered, so a regression returning 0 despite violations would have passed all tests while turning the CI gate into a no-op. Three new tests drive main() directly (clean tree, all violation classes, syntax error). * The has_docs trigger grep and the fail-closed roll-up guard had no pin. ws12_workflow_contracts.py now carries a has_docs CrateScopeFilter (docs/, the gate's own files, and the workflow in scope; crates and README out) plus code_style.yml REQUIRED_MARKERS for the job, both steps, and the roll-up guard — with sabotage tests proving narrowing the grep or removing a marker fails loudly. Red/green verified. * is_ignored()'s slash-glob pattern branch (contains '/' but not trailing) had no fixture; covered with design/*.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin the docs-gate guard's ordering, not just its presence Fixes the three findings from review round 2 (security/bugs/performance clean; tests found 2, conventions found 1): * REQUIRED_MARKERS is presence-only, so relocating the docs-gate roll-up guard to after the has_code early exit — the exact silent-skip bug the guard exists to prevent — passed every contract check. New validate_code_style_docs_guard_order() pins guard-before-early-exit in code_style.yml, with a sabotage test that relocates the guard line and a checked-in-order pass test. Red/green verified. * CrateScopeFilterSabotageTests' docstring still described "the three remaining crate-keyed filters"; updated for the fourth, non-crate-keyed has_docs pin (review-discipline.md: guardrail docs must match the code). * is_ignored()'s nested-directory pattern branch (a trailing-slash entry with an internal slash, e.g. `design/sub/`) never executed under the suite; covered by test_mintignore_nested_directory_pattern_fences. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix relative ADR links missed by the path sweep; align checker hint with the frozen fence Addresses the Copilot review on #7259: * The reference sweep rewrote literal `docs/adr` strings but not relative markdown links: `../../adr/` in target-architecture/{CHECKLIST,PROPOSAL}.md, `../../../adr/` in families/domains.md, and the hooks CLAUDE.md link (which also carried a pre-existing wrong depth from the WS7 family move) all resolved to the old location. A repo-wide relative-link scan found exactly these seven move-caused breaks; the remaining broken links predate this branch (WS7 crate-move fallout in testing-playbook.md, one dead June plan link) or are Mintlify extensionless links that resolve on the site. * The checker's remediation hint said "add its directory to docs/.mintignore", contradicting the frozen-fence rule the same script enforces; it now directs authors to move internal material under docs/internal/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: only executable guard occurrences satisfy the docs-gate order pin CodeRabbit (Major, #7259): validate_code_style_docs_guard_order used a raw str.find, so a commented-out copy of the guard above the has_code early exit — a realistic refactor leftover — satisfied the pin while the executable guard sat below the exit, silently unhooking the gate for docs-only PRs. The validator now strips comment lines before matching and requires EVERY live guard occurrence to precede the first early-exit occurrence; a comment-only occurrence reports the order as unassertable. New decoy sabotage test red/green verified. Also parenthesized the implicit string concatenations Ruff flagged (ISC004). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): align all Remotion packages to 4.0.499 CodeRabbit flagged the moved architecture-video project's manifest: Remotion requires every @remotion/* package at one identical version, but dependabot #6658 bumped only @remotion/cli and @remotion/tailwind-v4 to 4.0.499, leaving remotion, @remotion/transitions, and @remotion/eslint-config-flat at 4.0.447 — a pre-existing break on main that surfaced here because the directory rename presents as a new project. Aligned all five to 4.0.499 and regenerated the lockfile; `npx remotion versions` now reports all packages at the correct version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: assert both signals in the literal-pattern fencing case Copilot (#7259): test_mintignore_literal_file_fences used a pattern outside the frozen allowlist but discarded the `unexpected` result, so it passed while the checker it pins would fail — a misleading regression pin. The case now asserts both independent signals explicitly: the literal entry still fences its file (no publication leak) AND trips the frozen-list rule, with the interplay documented in the test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: mark the architecture video as stale pre-Reborn content Copilot flagged the relocated video scenes for citing crates/ironclaw_engine paths that no longer exist. The scenes are untouched April 2026 content (#2365) presenting as new because of the directory rename; regenerating them against the Reborn architecture is deliberately out of scope for this move-only PR. Until that regeneration happens, a prominent README banner states what the video describes, why it is wrong today, where current docs live (openwiki/), and how to regenerate (architecture-video skill) — so the content cannot mislead contributors in the meantime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: probe docs/docs.json in the has_docs scope pin Copilot (#7259): navigation is half the publication-boundary contract — a nav-only edit can orphan a page into hidden-page territory or reference a missing source file — but no has_docs probe covered docs.json, so a future markdown-only narrowing of the trigger grep (e.g. ^docs/.*\.(md|mdx)$) would silently skip the gate for nav changes while every existing probe stayed green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): map the two sweep-touched dev scripts and satisfy rustfmt Two root causes behind the red CI on #7259, both fallout from the docs path sweep touching files no docs-only change normally touches: * reborn_composition_boundaries.rs reads the (moved) composition pub-use snapshot; the longer docs/internal/plans/ path pushed the line past rustfmt's width. Reformatted. * The Reborn PR test planner fails closed on unmapped repo-root scripts. The sweep touched two local dev tools no workflow invokes — check-type-duplicates.py (docstring path) and render-architecture-video.sh (VIDEO_DIR path) — mapped both with per-file decisions in the planner's established style. Verified by running the planner against this branch's full 173-file changed list (green) plus its 61 self-tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: commit the messaging-framework path rewrites dropped by the previous merge The prior merge commit staged files before running the path sweep, so its rewrites of #6831's new files (docs/superpowers -> docs/internal/superpowers in three Rust doc comments, the plan, and standard-operations.md) were left unstaged and its message wrongly called the sweep a no-op. This commit is those rewrites. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
279 lines
9.7 KiB
Python
279 lines
9.7 KiB
Python
#!/usr/bin/env python3
|
|
"""Self-tests for scripts/ci/docs_publication_boundary.py."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import contextlib
|
|
import io
|
|
import json
|
|
import tempfile
|
|
import unittest
|
|
from pathlib import Path
|
|
from unittest import mock
|
|
|
|
import docs_publication_boundary as boundary
|
|
|
|
|
|
def make_docs_tree(
|
|
root: Path,
|
|
nav_pages: list[object],
|
|
mintignore: str | None,
|
|
files: dict[str, str],
|
|
) -> Path:
|
|
docs = root / "docs"
|
|
docs.mkdir()
|
|
docs_json = {
|
|
"navigation": {
|
|
"languages": [
|
|
{"language": "en", "tabs": [{"tab": " ", "groups": [
|
|
{"group": " ", "pages": nav_pages}
|
|
]}]}
|
|
]
|
|
}
|
|
}
|
|
(docs / "docs.json").write_text(json.dumps(docs_json), encoding="utf-8")
|
|
if mintignore is not None:
|
|
(docs / ".mintignore").write_text(mintignore, encoding="utf-8")
|
|
for rel, content in files.items():
|
|
path = docs / rel
|
|
path.parent.mkdir(parents=True, exist_ok=True)
|
|
path.write_text(content, encoding="utf-8")
|
|
return docs
|
|
|
|
|
|
class DocsPublicationBoundaryTest(unittest.TestCase):
|
|
def setUp(self) -> None:
|
|
self._tmp = tempfile.TemporaryDirectory()
|
|
self.addCleanup(self._tmp.cleanup)
|
|
self.root = Path(self._tmp.name)
|
|
|
|
def test_page_in_nav_is_clean(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["index"], None, {"index.mdx": "# Home"}
|
|
)
|
|
unfenced, missing, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
self.assertEqual(missing, [])
|
|
|
|
def test_new_mintignore_entry_is_rejected(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"internal/\nsecret-notes/\n",
|
|
{"index.mdx": "# Home", "secret-notes/plan.md": "# Internal"},
|
|
)
|
|
unfenced, _, unexpected = boundary.find_violations(docs)
|
|
self.assertEqual(unexpected, ["secret-notes/"])
|
|
# The rogue entry still fences its files — it fails the frozen-list
|
|
# rule, not the leak rule, so the fix message points at internal/.
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_mintignore_subset_of_frozen_list_is_allowed(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"internal/\nreborn/\n*.draft.mdx\n",
|
|
{"index.mdx": "# Home"},
|
|
)
|
|
_, _, unexpected = boundary.find_violations(docs)
|
|
self.assertEqual(unexpected, [])
|
|
|
|
def test_unfenced_page_is_flagged(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"internal/\n",
|
|
{"index.mdx": "# Home", "design/notes.md": "# Internal"},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, ["design/notes.md"])
|
|
|
|
def test_mintignore_directory_pattern_fences(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"internal/\n",
|
|
{"index.mdx": "# Home", "internal/deep/notes.md": "# Internal"},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_mintignore_glob_pattern_fences(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"*.draft.mdx\n",
|
|
{"index.mdx": "# Home", "guide.draft.mdx": "# WIP"},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_mintignore_literal_file_pattern_fences_but_fails_frozen_rule(
|
|
self,
|
|
) -> None:
|
|
# `reborn-binary.md` exercises the literal-filename matching branch,
|
|
# and is deliberately NOT in FROZEN_MINTIGNORE_PATTERNS: the two
|
|
# signals are independent — the entry still fences its file (no
|
|
# publication leak) while the frozen-list rule flags it. Both halves
|
|
# are asserted so neither regression can hide behind the other.
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"reborn-binary.md\n",
|
|
{"index.mdx": "# Home", "reborn-binary.md": "# Internal"},
|
|
)
|
|
unfenced, _, unexpected = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
self.assertEqual(unexpected, ["reborn-binary.md"])
|
|
|
|
def test_hidden_frontmatter_marks_deliberate_page(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
None,
|
|
{
|
|
"index.mdx": "# Home",
|
|
"unlisted.mdx": "---\ntitle: Unlisted\nhidden: true\n---\n# P",
|
|
},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_hidden_after_frontmatter_close_does_not_count(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
None,
|
|
{
|
|
"index.mdx": "# Home",
|
|
"leak.mdx": "---\ntitle: Leak\n---\nhidden: true\n",
|
|
},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, ["leak.mdx"])
|
|
|
|
def test_nav_entry_without_source_file_is_flagged(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["index", "ghost/page"], None, {"index.mdx": "# Home"}
|
|
)
|
|
_, missing, _ = boundary.find_violations(docs)
|
|
self.assertEqual(missing, ["ghost/page"])
|
|
|
|
def test_nav_entry_matches_md_extension_too(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["guide"], None, {"guide.md": "# Guide"}
|
|
)
|
|
unfenced, missing, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
self.assertEqual(missing, [])
|
|
|
|
def test_nested_nav_groups_are_collected(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index", {"group": "Deep", "pages": [{"group": "Deeper", "pages": ["a/b"]}]}],
|
|
None,
|
|
{"index.mdx": "# Home", "a/b.mdx": "# B"},
|
|
)
|
|
unfenced, missing, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
self.assertEqual(missing, [])
|
|
|
|
def test_builtin_ignores_apply_without_mintignore(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
None,
|
|
{
|
|
"index.mdx": "# Home",
|
|
"README.md": "# Readme",
|
|
"snippets/shared.mdx": "shared",
|
|
".claude/skills/x/SKILL.md": "skill",
|
|
},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_negation_pattern_is_rejected_loudly(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"internal/\n!internal/public.md\n",
|
|
{"index.mdx": "# Home"},
|
|
)
|
|
with self.assertRaises(boundary.MintignoreSyntaxError):
|
|
boundary.find_violations(docs)
|
|
|
|
def test_openapi_nav_entries_need_no_source_file(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["index", "GET /users"], None, {"index.mdx": "# Home"}
|
|
)
|
|
_, missing, _ = boundary.find_violations(docs)
|
|
self.assertEqual(missing, [])
|
|
|
|
def test_mintignore_nested_directory_pattern_fences(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"design/sub/\n",
|
|
{"index.mdx": "# Home", "design/sub/notes.md": "# Internal"},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_mintignore_nested_path_glob_pattern_fences(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index"],
|
|
"design/*.md\n",
|
|
{"index.mdx": "# Home", "design/notes.md": "# Internal"},
|
|
)
|
|
unfenced, _, _ = boundary.find_violations(docs)
|
|
self.assertEqual(unfenced, [])
|
|
|
|
def test_main_returns_0_on_clean_tree(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["index"], "internal/\n", {"index.mdx": "# Home"}
|
|
)
|
|
with mock.patch.object(boundary, "DOCS_ROOT", docs):
|
|
with contextlib.redirect_stdout(io.StringIO()) as out:
|
|
rc = boundary.main()
|
|
self.assertEqual(rc, 0)
|
|
self.assertIn("published or fenced", out.getvalue())
|
|
|
|
def test_main_returns_1_and_prints_every_violation_class(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root,
|
|
["index", "ghost/page"],
|
|
"internal/\nrogue/\n",
|
|
{"index.mdx": "# Home", "design/notes.md": "# Internal"},
|
|
)
|
|
with mock.patch.object(boundary, "DOCS_ROOT", docs):
|
|
with contextlib.redirect_stderr(io.StringIO()) as err:
|
|
rc = boundary.main()
|
|
self.assertEqual(rc, 1)
|
|
stderr = err.getvalue()
|
|
self.assertIn("docs/design/notes.md", stderr)
|
|
self.assertIn("ghost/page", stderr)
|
|
self.assertIn("rogue/", stderr)
|
|
self.assertIn("docs/internal/", stderr)
|
|
|
|
def test_main_reports_mintignore_syntax_error(self) -> None:
|
|
docs = make_docs_tree(
|
|
self.root, ["index"], "internal/\n!internal/x.md\n", {"index.mdx": "# Home"}
|
|
)
|
|
with mock.patch.object(boundary, "DOCS_ROOT", docs):
|
|
with contextlib.redirect_stderr(io.StringIO()) as err:
|
|
rc = boundary.main()
|
|
self.assertEqual(rc, 1)
|
|
self.assertIn("negation pattern", err.getvalue())
|
|
|
|
def test_real_repo_docs_pass(self) -> None:
|
|
unfenced, missing, unexpected = boundary.find_violations(boundary.DOCS_ROOT)
|
|
self.assertEqual(unfenced, [])
|
|
self.assertEqual(missing, [])
|
|
self.assertEqual(unexpected, [])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|