mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-03 08:06:01 +08:00
fix(composition,host-runtime): wire ScopedFilesystem for migrated consumer stores
The consumer-crate ScopedFilesystem migrations (commits6ecca195d,5e7688d3b,4ae56769b,81664dd29) changed the public constructors: - FilesystemProcessStore<F> / FilesystemProcessResultStore<F> (lifetime dropped) - ProcessServices::filesystem now takes Arc<ScopedFilesystem<F>> - FilesystemCapabilityLeaseStore<'a, F> now takes &'a ScopedFilesystem<F> Downstream wiring needed catching up: - `ironclaw_reborn_composition` libsql + postgres production builders + the in-process `reborn_app_factory` paths now wrap the raw RootFilesystem in a `ScopedFilesystem` via the new `default_singleton_mount_view()` helper. The view grants read+write+list+delete on the canonical consumer-store aliases (/processes, /secrets, /authorization, /outbound, /engine) mapped to top-level VirtualPath roots — the single-tenant default that preserves current production behaviour while making per-tenant routing a MountView decision instead of a code change. - Type aliases drop the obsolete `'static` lifetime parameter. - `RebornCompositionError::Mount` and `RebornBuildError::Mount` variants surface HostApiError from the MountView construction. - The `reborn_durable_restart_integration` test wraps its LocalFilesystem in a ScopedFilesystem before passing to ProcessServices and FilesystemCapabilityLeaseStore. Per `docs/plans/2026-05-16-scoped-filesystem-tenant-isolation.md`.
This commit is contained in:
62
.anvil/cache/crates/ironclaw_filesystem/13cec0307d888ae1.json
vendored
Normal file
62
.anvil/cache/crates/ironclaw_filesystem/13cec0307d888ae1.json
vendored
Normal file
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"name": "ironclaw_filesystem",
|
||||
"purpose": "Scoped filesystem service that resolves ScopedPath values through a caller's MountView, enforces mount permissions, then performs operations against a trusted root filesystem namespace (VirtualPath).",
|
||||
"public_surface": {
|
||||
"kinds": [
|
||||
"modules",
|
||||
"structs",
|
||||
"enums",
|
||||
"traits",
|
||||
"type_aliases"
|
||||
],
|
||||
"shape": "Exposes backends via RootFilesystem trait; ScopedFilesystem wraps with permission checks; FilesystemCatalog manages mounts; backend impls in local/postgres/libsql/hsm/in_memory; types + record hold domain types and errors.",
|
||||
"stability": "internal — version 0.1.0, publish = false, part of IronClaw workspace"
|
||||
},
|
||||
"error_model": {
|
||||
"primary_error": "thiserror-derived FilesystemError",
|
||||
"convertible_from": [
|
||||
"std::io::Error",
|
||||
"serde_json::Error"
|
||||
],
|
||||
"panics": "unknown — no explicit panic documentation; local backend canonicalization logic may assert or unwrap on unexpected states"
|
||||
},
|
||||
"side_effects": {
|
||||
"filesystem": "both",
|
||||
"network": "writes",
|
||||
"unsafe": "unknown",
|
||||
"global_state": "unknown",
|
||||
"spawns_processes": false
|
||||
},
|
||||
"concurrency_model": {
|
||||
"send_sync_posture": "partial — backends vary (DB-backed vs local vs in-memory)",
|
||||
"async_posture": "tokio",
|
||||
"internal_locks": "unknown — tokio::sync feature present suggests async synchronization primitives"
|
||||
},
|
||||
"dep_posture": {
|
||||
"heavyweights": [
|
||||
"tokio",
|
||||
"deadpool-postgres",
|
||||
"tokio-postgres",
|
||||
"libsql"
|
||||
],
|
||||
"optional_features": [
|
||||
"postgres: enables PostgresRootFilesystem via deadpool-postgres+tokio-postgres",
|
||||
"libsql: enables LibSqlRootFilesystem via libsql with core+replication+remote+tls"
|
||||
],
|
||||
"no_std_compatible": false
|
||||
},
|
||||
"key_files": [
|
||||
"src/lib.rs",
|
||||
"src/backend.rs",
|
||||
"src/catalog.rs",
|
||||
"src/local.rs",
|
||||
"src/scoped.rs",
|
||||
"src/types.rs",
|
||||
"src/root.rs",
|
||||
"src/record.rs"
|
||||
],
|
||||
"confidence": 0.7,
|
||||
"provider": "near_ai",
|
||||
"model": "zai-org/GLM-5.1-FP8",
|
||||
"cached": false
|
||||
}
|
||||
BIN
.anvil/events.db
Normal file
BIN
.anvil/events.db
Normal file
Binary file not shown.
BIN
.anvil/memory.db
Normal file
BIN
.anvil/memory.db
Normal file
Binary file not shown.
214
.claude/rules/architecture.md
Normal file
214
.claude/rules/architecture.md
Normal file
@@ -0,0 +1,214 @@
|
||||
---
|
||||
paths:
|
||||
- "crates/**/*.rs"
|
||||
- "src/**/*.rs"
|
||||
---
|
||||
# Architecture Discipline — Stop the Sprawl Before It Ships
|
||||
|
||||
This rule exists because the engine refactor of 2026-05 found a class
|
||||
of slow-burn architectural decay that no existing rule catches. The
|
||||
individual symptoms look reasonable in isolation (one extra Arc, one
|
||||
extra method arg, one `with_*` builder, one `#[allow(...)]`). The
|
||||
class is recognizable only when you grep for the smoke alarms across
|
||||
the crate: 11 `#[allow(clippy::too_many_arguments)]` annotations in
|
||||
`crates/ironclaw_engine/`, a 7,933-line `runtime/mission.rs`, two
|
||||
parallel action-dispatch pipelines, and the same six Arcs threaded
|
||||
through three layers without ever being given a name.
|
||||
|
||||
The rule is: **listen to the language. When the compiler or clippy
|
||||
complains, the answer is almost never `#[allow]`.**
|
||||
|
||||
## Five smells, with grep-able patterns
|
||||
|
||||
### 1. Argument creep — `#[allow(clippy::too_many_arguments)]` is a smoke alarm
|
||||
|
||||
clippy's default is 7 args. Reaching it means the function has more
|
||||
inputs than a reader can hold in their head. Allowing it once is a
|
||||
trade — allowing it eleven times is a refactor someone declined to
|
||||
do.
|
||||
|
||||
**Required pattern** when introducing the allow:
|
||||
|
||||
```rust
|
||||
// arch-exempt: too_many_args, <one-line reason>, plan #NNNN
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn execute_orchestrator(...)
|
||||
```
|
||||
|
||||
The annotation must name *what aggregation is missing* (a context
|
||||
struct, a service bundle, a config object) and link a tracking
|
||||
issue or plan. "Refactor needed" is not a reason; "needs `EngineServices`
|
||||
bundle, plan #2800" is.
|
||||
|
||||
**Review flag:** any added `#[allow(clippy::too_many_arguments)]`
|
||||
without an `arch-exempt` annotation on the line above it.
|
||||
|
||||
### 2. Optional Arcs that are required in production
|
||||
|
||||
```rust
|
||||
struct Foo {
|
||||
a: Arc<dyn Bar>,
|
||||
b: Option<Arc<dyn Baz>>, // <- smell
|
||||
}
|
||||
|
||||
impl Foo {
|
||||
fn with_baz(mut self, b: Arc<dyn Baz>) -> Self { ... }
|
||||
}
|
||||
```
|
||||
|
||||
If production wires `with_baz` every time and only test code skips
|
||||
it, the type system is lying. The `is_some()` branches that result
|
||||
become dead paths in production and the favourite home of bugs that
|
||||
only one user trips.
|
||||
|
||||
**Rule:** `Option<Arc<…>>` on a runtime struct is allowed only when
|
||||
the dependency is *genuinely* optional (e.g., a feature-flagged
|
||||
component that the binary may legitimately ship without). If the
|
||||
production wiring always sets it, either:
|
||||
|
||||
- Make it required and have tests construct it with a fake.
|
||||
- Or move the conditional behavior into a separate type (split the
|
||||
struct).
|
||||
|
||||
**Review flag:** `Option<Arc<` added on a struct field, paired with a
|
||||
`with_<name>` builder that the production call site always invokes.
|
||||
|
||||
### 3. Re-derived identity / duplicated state
|
||||
|
||||
A field that already lives on a primary entity (`Thread.id`,
|
||||
`Thread.user_id`, `Thread.project_id`) cannot be re-declared on a
|
||||
context struct that the same code path constructs from that entity.
|
||||
|
||||
```rust
|
||||
// Bad: ThreadExecutionContext re-declares thread_id, project_id, user_id
|
||||
// that already exist on Thread, and the constructor copies them.
|
||||
struct ThreadExecutionContext {
|
||||
thread_id: ThreadId,
|
||||
project_id: ProjectId,
|
||||
user_id: String,
|
||||
// ... step-scoped fields
|
||||
}
|
||||
|
||||
// Good: pass the source-of-truth entity by reference; only carry
|
||||
// step-scoped data on the side struct.
|
||||
struct StepFrame<'t> {
|
||||
thread: &'t Thread,
|
||||
step_id: StepId,
|
||||
current_call_id: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
**Why:** identity confusion has shipped four times in `types.md`
|
||||
(PRs #2561, #2473, #2512, #2574). Duplicating identity onto a side
|
||||
struct *adds another copy* of the value the compiler cannot enforce
|
||||
agreement on.
|
||||
|
||||
**Review flag:** a field on type `B` whose name and type match a
|
||||
field on type `A` when `B` is constructed from `A` in the same file.
|
||||
|
||||
### 4. Duplicate dispatch pipelines
|
||||
|
||||
When the same downstream call (`effects.execute_action`,
|
||||
`safety_layer.scan_*`, `dispatcher.dispatch`) is invoked from two
|
||||
places that each implement their own pre-checks (lease, policy,
|
||||
sanitization), they are one pipeline written twice. Every
|
||||
safety/policy change must then land in both — and one always lags.
|
||||
|
||||
This mirrors the rule in `tools.md` ("Everything Goes Through Tools")
|
||||
and `safety-and-sandbox.md` ("Every New Ingress Scans Before Storage
|
||||
or LLM"). The pattern: identify the converging downstream call,
|
||||
extract a single gateway, route both sides through it.
|
||||
|
||||
**Review flag:** a new call site to a registry/executor/dispatcher
|
||||
trait method that re-implements pre-checks (lookup, policy, lease,
|
||||
scan) already implemented at another call site.
|
||||
|
||||
### 5. File size budget
|
||||
|
||||
A `.rs` file > 1,500 lines is a refactor the codebase has been
|
||||
postponing. A file > 3,000 lines is a refactor that is already
|
||||
costing review time. There is no hard cap, but:
|
||||
|
||||
- New files: aim for < 800 lines.
|
||||
- Existing files between 1,500 and 3,000: every PR that touches them
|
||||
should leave them shorter, not longer, unless the PR is explicitly
|
||||
expanding a feature that has nowhere else to live.
|
||||
- Existing files > 3,000: file a tracking issue for decomposition.
|
||||
PRs that *add* > 200 lines need an inline justification.
|
||||
|
||||
This is not a mechanical check — it's a culture norm. The
|
||||
mechanical check is "does this file have a tracking issue."
|
||||
|
||||
## Where each rule is enforced
|
||||
|
||||
The rules above are deliberately split across enforcement layers
|
||||
because each layer catches a different failure mode.
|
||||
|
||||
| Smell | Pre-commit script | CI / clippy | Code review | Agent-facing (this file) |
|
||||
|---|---|---|---|---|
|
||||
| 1. `too_many_arguments` allow | yes — count + annotation grep | clippy default already fires | required | yes |
|
||||
| 2. `Option<Arc<…>>` + `with_*` | yes — paired-pattern grep | — | required | yes |
|
||||
| 3. Re-derived identity | — (heuristic) | — | required | yes |
|
||||
| 4. Duplicate dispatch | partial — known-method grep | — | required | yes |
|
||||
| 5. File size | yes — `wc -l` on staged | — | informational | yes |
|
||||
|
||||
### Why this split
|
||||
|
||||
- **Pre-commit catches the mechanical patterns.** A regex on staged
|
||||
diffs is enough for #1 (annotation grep), #2 (paired patterns),
|
||||
and #5 (line count). These are cheap, deterministic, and run on
|
||||
every commit. Add to `scripts/pre-commit-safety.sh` as Check #10
|
||||
(`ARCH-SPRAWL`) following the existing format.
|
||||
- **CI / clippy catches what compilers can express.** clippy already
|
||||
emits `too_many_arguments`. The rule is "don't silence it without
|
||||
a plan link." No new CI check needed; existing default works once
|
||||
the annotation discipline lands.
|
||||
- **Code review catches semantic patterns.** #3 and #4 require
|
||||
reading the code, not the diff. The annotation `// arch-exempt:
|
||||
<category>, <reason>, plan #NNNN` puts the burden on the proposer
|
||||
to name the aggregation that is missing — reviewers reject
|
||||
exempts without a plan link.
|
||||
- **This rule file is the agent-facing summary.** Loaded into context
|
||||
whenever an agent edits `crates/**/*.rs` or `src/**/*.rs`. The
|
||||
agent's job: *don't be the one who adds the twelfth `#[allow]`.*
|
||||
|
||||
## Annotation format (consistent with other rules)
|
||||
|
||||
```rust
|
||||
// arch-exempt: <category>, <reason>, plan #NNNN
|
||||
```
|
||||
|
||||
Categories:
|
||||
|
||||
- `too_many_args` — function signature grew past clippy default.
|
||||
- `optional_arc` — `Option<Arc<…>>` field on a runtime struct.
|
||||
- `parallel_dispatch` — second call site to a converging downstream.
|
||||
- `large_file` — file size growing past 1,500 lines.
|
||||
|
||||
Each must name a tracking issue or plan that owns the cleanup. An
|
||||
exempt without a plan link is a violation, not an exception.
|
||||
|
||||
## What this rule does NOT cover
|
||||
|
||||
- **Trait shape.** Trait method signatures are part of the public
|
||||
contract; their argument count is governed by API design, not
|
||||
this rule.
|
||||
- **Test code.** Tests are allowed to construct things with the
|
||||
full Arc bag explicitly when that makes the test clearer.
|
||||
`#[cfg(test)]` blocks are skipped by the pre-commit check.
|
||||
- **Generated code.** WIT bindings, `serde` derive output, and
|
||||
similar machine-emitted code are exempt (they are not what a
|
||||
reader maintains).
|
||||
- **One-off scripts under `scripts/`.** Architectural sprawl in a
|
||||
shell script or migration helper is a different conversation.
|
||||
|
||||
## References
|
||||
|
||||
- The diagnosis that motivated this rule:
|
||||
`docs/plans/2026-05-02-engine-architecture-simplification.md`.
|
||||
- Adjacent rules with the same shape (extract a single gateway,
|
||||
route everything through it): `tools.md`, `safety-and-sandbox.md`,
|
||||
`gateway-events.md`.
|
||||
- Annotation discipline reference: `gateway-events.md` —
|
||||
`// projection-exempt: <category>, <detail>` is the canonical
|
||||
shape this rule borrows.
|
||||
@@ -11,7 +11,7 @@ use ironclaw_events::{
|
||||
DurableAuditSink, DurableEventSink, EventStreamKey, ReadScope, RuntimeEventKind,
|
||||
};
|
||||
use ironclaw_extensions::{ExtensionManifest, ExtensionPackage, ExtensionRegistry};
|
||||
use ironclaw_filesystem::LocalFilesystem;
|
||||
use ironclaw_filesystem::{LocalFilesystem, ScopedFilesystem};
|
||||
use ironclaw_host_api::*;
|
||||
use ironclaw_host_runtime::{
|
||||
CapabilitySurfaceVersion, HostRuntime, HostRuntimeServices, RuntimeCapabilityOutcome,
|
||||
@@ -319,15 +319,15 @@ async fn jsonl_event_and_audit_replay_survive_reopen_without_raw_sentinels() {
|
||||
}
|
||||
|
||||
type DurableProcessServices = ProcessServices<
|
||||
FilesystemProcessStore<'static, LocalFilesystem>,
|
||||
FilesystemProcessResultStore<'static, LocalFilesystem>,
|
||||
FilesystemProcessStore<LocalFilesystem>,
|
||||
FilesystemProcessResultStore<LocalFilesystem>,
|
||||
>;
|
||||
|
||||
type DurableHostRuntimeServices = HostRuntimeServices<
|
||||
LocalFilesystem,
|
||||
InMemoryResourceGovernor,
|
||||
FilesystemProcessStore<'static, LocalFilesystem>,
|
||||
FilesystemProcessResultStore<'static, LocalFilesystem>,
|
||||
FilesystemProcessStore<LocalFilesystem>,
|
||||
FilesystemProcessResultStore<LocalFilesystem>,
|
||||
>;
|
||||
|
||||
struct DurableServices {
|
||||
@@ -342,10 +342,10 @@ async fn durable_services(engine_root: &Path, event_root: &Path) -> DurableServi
|
||||
let event_stores = jsonl_event_stores(event_root).await;
|
||||
let run_state_fs = leaked_engine_filesystem(engine_root);
|
||||
let approval_fs = leaked_engine_filesystem(engine_root);
|
||||
let lease_fs = leaked_engine_filesystem(engine_root);
|
||||
let lease_scoped_fs = leaked_scoped_engine_filesystem(engine_root);
|
||||
let run_state = Arc::new(FilesystemRunStateStore::new(run_state_fs));
|
||||
let approval_requests = Arc::new(FilesystemApprovalRequestStore::new(approval_fs));
|
||||
let capability_leases = Arc::new(FilesystemCapabilityLeaseStore::new(lease_fs));
|
||||
let capability_leases = Arc::new(FilesystemCapabilityLeaseStore::new(lease_scoped_fs));
|
||||
let services = base_services(
|
||||
engine_root,
|
||||
event_stores.clone(),
|
||||
@@ -403,7 +403,41 @@ async fn jsonl_event_stores(event_root: &Path) -> RebornEventStores {
|
||||
}
|
||||
|
||||
fn filesystem_process_services(engine_root: &Path) -> DurableProcessServices {
|
||||
ProcessServices::filesystem(Arc::new(mounted_engine_filesystem(engine_root)))
|
||||
let scoped = Arc::new(ScopedFilesystem::new(
|
||||
Arc::new(mounted_engine_filesystem(engine_root)),
|
||||
durable_mount_view(),
|
||||
));
|
||||
ProcessServices::filesystem(scoped)
|
||||
}
|
||||
|
||||
/// Mount view granting the migrated consumer crates full per-user-owner
|
||||
/// permissions on their canonical aliases. Test-only: production
|
||||
/// composition (`ironclaw_reborn_composition::default_singleton_mount_view`)
|
||||
/// has the equivalent shape but is `pub(crate)`.
|
||||
fn durable_mount_view() -> MountView {
|
||||
MountView::new(vec![
|
||||
MountGrant::new(
|
||||
MountAlias::new("/processes").unwrap(),
|
||||
VirtualPath::new("/processes").unwrap(),
|
||||
MountPermissions::read_write_list_delete(),
|
||||
),
|
||||
MountGrant::new(
|
||||
MountAlias::new("/authorization").unwrap(),
|
||||
VirtualPath::new("/authorization").unwrap(),
|
||||
MountPermissions::read_write_list_delete(),
|
||||
),
|
||||
])
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn leaked_scoped_engine_filesystem(
|
||||
engine_root: &Path,
|
||||
) -> &'static ScopedFilesystem<LocalFilesystem> {
|
||||
let scoped = ScopedFilesystem::new(
|
||||
Arc::new(mounted_engine_filesystem(engine_root)),
|
||||
durable_mount_view(),
|
||||
);
|
||||
Box::leak(Box::new(scoped))
|
||||
}
|
||||
|
||||
fn mounted_engine_filesystem(engine_root: &Path) -> LocalFilesystem {
|
||||
|
||||
@@ -32,6 +32,8 @@ pub enum RebornBuildError {
|
||||
CapabilityLease(#[from] ironclaw_authorization::CapabilityLeaseError),
|
||||
#[error("reborn turn state build failed")]
|
||||
Turn(#[from] ironclaw_turns::TurnError),
|
||||
#[error("reborn mount view construction failed")]
|
||||
Mount(#[from] ironclaw_host_api::HostApiError),
|
||||
}
|
||||
|
||||
impl From<ironclaw_host_runtime::ProductionWiringReport> for RebornBuildError {
|
||||
|
||||
@@ -262,12 +262,13 @@ async fn build_libsql_production(
|
||||
auth_token,
|
||||
};
|
||||
|
||||
let scoped_filesystem = crate::wrap_scoped(Arc::clone(&filesystem))?;
|
||||
let services = HostRuntimeServices::new(
|
||||
Arc::new(ExtensionRegistry::new()),
|
||||
Arc::clone(&filesystem),
|
||||
Arc::new(InMemoryResourceGovernor::new()),
|
||||
Arc::new(GrantAuthorizer::new()),
|
||||
ProcessServices::filesystem(Arc::clone(&filesystem)),
|
||||
ProcessServices::filesystem(scoped_filesystem),
|
||||
CapabilitySurfaceVersion::new("reborn-app-v1")?,
|
||||
)
|
||||
.with_trust_policy(production_wiring.trust_policy)
|
||||
@@ -324,12 +325,13 @@ async fn build_postgres_production(
|
||||
|
||||
let event_store = ironclaw_reborn_event_store::RebornEventStoreConfig::Postgres { url };
|
||||
|
||||
let scoped_filesystem = crate::wrap_scoped(Arc::clone(&filesystem))?;
|
||||
let services = HostRuntimeServices::new(
|
||||
Arc::new(ExtensionRegistry::new()),
|
||||
Arc::clone(&filesystem),
|
||||
Arc::new(InMemoryResourceGovernor::new()),
|
||||
Arc::new(GrantAuthorizer::new()),
|
||||
ProcessServices::filesystem(Arc::clone(&filesystem)),
|
||||
ProcessServices::filesystem(scoped_filesystem),
|
||||
CapabilitySurfaceVersion::new("reborn-app-v1")?,
|
||||
)
|
||||
.with_trust_policy(production_wiring.trust_policy)
|
||||
|
||||
@@ -32,7 +32,11 @@ use ironclaw_filesystem::LibSqlRootFilesystem;
|
||||
#[cfg(feature = "postgres")]
|
||||
use ironclaw_filesystem::PostgresRootFilesystem;
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
use ironclaw_host_api::{ResourceScope, SecretHandle};
|
||||
use ironclaw_filesystem::{RootFilesystem, ScopedFilesystem};
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
use ironclaw_host_api::{
|
||||
MountAlias, MountGrant, MountPermissions, MountView, ResourceScope, SecretHandle, VirtualPath,
|
||||
};
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
use ironclaw_host_runtime::{CapabilitySurfaceVersion, HostRuntimeServices};
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
@@ -71,18 +75,63 @@ use thiserror::Error;
|
||||
pub type LibSqlProductionHostRuntimeServices = HostRuntimeServices<
|
||||
LibSqlRootFilesystem,
|
||||
PersistentResourceGovernor<LibSqlResourceGovernorStore>,
|
||||
FilesystemProcessStore<'static, LibSqlRootFilesystem>,
|
||||
FilesystemProcessResultStore<'static, LibSqlRootFilesystem>,
|
||||
FilesystemProcessStore<LibSqlRootFilesystem>,
|
||||
FilesystemProcessResultStore<LibSqlRootFilesystem>,
|
||||
>;
|
||||
|
||||
#[cfg(feature = "postgres")]
|
||||
pub type PostgresProductionHostRuntimeServices = HostRuntimeServices<
|
||||
PostgresRootFilesystem,
|
||||
PersistentResourceGovernor<PostgresResourceGovernorStore>,
|
||||
FilesystemProcessStore<'static, PostgresRootFilesystem>,
|
||||
FilesystemProcessResultStore<'static, PostgresRootFilesystem>,
|
||||
FilesystemProcessStore<PostgresRootFilesystem>,
|
||||
FilesystemProcessResultStore<PostgresRootFilesystem>,
|
||||
>;
|
||||
|
||||
/// Build the default single-tenant [`MountView`] for production composition.
|
||||
///
|
||||
/// Wires the canonical consumer-store aliases (`/processes`, `/secrets`,
|
||||
/// `/authorization`, `/outbound`, `/engine`) to top-level
|
||||
/// [`VirtualPath`] roots and grants full per-user-owner permissions.
|
||||
///
|
||||
/// This is the **single-tenant** default: every alias maps to the
|
||||
/// root-level prefix with no `tenants/<tenant_id>/users/<user_id>/...`
|
||||
/// rewriting. Multi-tenant deployments build a per-invocation MountView
|
||||
/// that points each alias to a tenant/user-scoped subtree of the same
|
||||
/// underlying [`RootFilesystem`] — see
|
||||
/// `docs/plans/2026-05-16-scoped-filesystem-tenant-isolation.md`.
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
pub(crate) fn default_singleton_mount_view() -> Result<MountView, ironclaw_host_api::HostApiError> {
|
||||
let aliases: &[(&str, &str)] = &[
|
||||
("/processes", "/processes"),
|
||||
("/secrets", "/secrets"),
|
||||
("/authorization", "/authorization"),
|
||||
("/outbound", "/outbound"),
|
||||
("/engine", "/engine"),
|
||||
];
|
||||
let grants = aliases
|
||||
.iter()
|
||||
.map(|(alias, target)| {
|
||||
Ok(MountGrant::new(
|
||||
MountAlias::new(*alias)?,
|
||||
VirtualPath::new(*target)?,
|
||||
MountPermissions::read_write_list_delete(),
|
||||
))
|
||||
})
|
||||
.collect::<Result<Vec<_>, ironclaw_host_api::HostApiError>>()?;
|
||||
MountView::new(grants)
|
||||
}
|
||||
|
||||
#[cfg(any(feature = "libsql", feature = "postgres"))]
|
||||
pub(crate) fn wrap_scoped<F>(
|
||||
root: Arc<F>,
|
||||
) -> Result<Arc<ScopedFilesystem<F>>, ironclaw_host_api::HostApiError>
|
||||
where
|
||||
F: RootFilesystem,
|
||||
{
|
||||
let view = default_singleton_mount_view()?;
|
||||
Ok(Arc::new(ScopedFilesystem::new(root, view)))
|
||||
}
|
||||
|
||||
/// libSQL substrate handles needed to build production host-runtime services.
|
||||
#[cfg(feature = "libsql")]
|
||||
pub struct LibSqlProductionSubstrateConfig<TPolicy, TWake>
|
||||
@@ -117,6 +166,8 @@ where
|
||||
pub enum RebornCompositionError {
|
||||
#[error("reborn production composition requires explicit secret master key")]
|
||||
MissingSecretMasterKey,
|
||||
#[error("reborn mount view construction failed: {0}")]
|
||||
Mount(#[from] ironclaw_host_api::HostApiError),
|
||||
#[error("reborn filesystem substrate failed: {0}")]
|
||||
Filesystem(#[from] ironclaw_filesystem::FilesystemError),
|
||||
#[error("reborn resource governor substrate failed: {0}")]
|
||||
@@ -156,7 +207,8 @@ where
|
||||
let filesystem = Arc::new(LibSqlRootFilesystem::new(Arc::clone(&config.database)));
|
||||
filesystem.run_migrations().await?;
|
||||
|
||||
let process_services = ProcessServices::filesystem(Arc::clone(&filesystem));
|
||||
let scoped_filesystem = wrap_scoped(Arc::clone(&filesystem))?;
|
||||
let process_services = ProcessServices::filesystem(Arc::clone(&scoped_filesystem));
|
||||
|
||||
let resource_store = LibSqlResourceGovernorStore::new(Arc::clone(&config.database));
|
||||
resource_store.run_migrations().await?;
|
||||
@@ -219,7 +271,8 @@ where
|
||||
let filesystem = Arc::new(PostgresRootFilesystem::new(config.pool.clone()));
|
||||
filesystem.run_migrations().await?;
|
||||
|
||||
let process_services = ProcessServices::filesystem(Arc::clone(&filesystem));
|
||||
let scoped_filesystem = wrap_scoped(Arc::clone(&filesystem))?;
|
||||
let process_services = ProcessServices::filesystem(Arc::clone(&scoped_filesystem));
|
||||
|
||||
let resource_store = PostgresResourceGovernorStore::new(config.pool.clone());
|
||||
resource_store.run_migrations().await?;
|
||||
|
||||
192
docs/plans/2026-05-02-engine-architecture-simplification.md
Normal file
192
docs/plans/2026-05-02-engine-architecture-simplification.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# IronClaw Engine: Argument & Layer Simplification
|
||||
|
||||
**Date:** 2026-05-02
|
||||
**Status:** Proposed
|
||||
**Scope:** `crates/ironclaw_engine/`
|
||||
**Goal:** Reduce dependency-arg threading and collapse duplicate dispatch
|
||||
paths in the engine without changing externally visible behavior.
|
||||
|
||||
---
|
||||
|
||||
## Diagnosis
|
||||
|
||||
The five primitives (Thread, Step, Capability, MemoryDoc, Project) are
|
||||
clean. The runtime plumbing around them is not. Five symptoms point at the
|
||||
same root cause: there is no aggregation type for engine-wide services,
|
||||
so every layer re-threads the same Arcs.
|
||||
|
||||
### Symptom 1 — Service Arcs replicated at every layer
|
||||
|
||||
Six dependencies — `LlmBackend`, `EffectExecutor`, `Store`,
|
||||
`CapabilityRegistry`, `LeaseManager`, `PolicyEngine` — appear at:
|
||||
|
||||
- `runtime/manager.rs:34-47` (stored as fields on `ThreadManager`)
|
||||
- `executor/loop_engine.rs:105-124` (re-passed into `ExecutionLoop`)
|
||||
- `executor/orchestrator.rs:436-449` (re-passed as 12 args to
|
||||
`execute_orchestrator`, gated by `#[allow(clippy::too_many_arguments)]`)
|
||||
- `runtime/mission.rs` (subset, plus optional `EffectExecutor`)
|
||||
|
||||
Three layers, identical payload, no shared name.
|
||||
|
||||
### Symptom 2 — `with_*` builders hide required dependencies
|
||||
|
||||
`ExecutionLoop::new` takes 5 Arcs; five further deps come in via
|
||||
`with_capabilities`, `with_store`, `with_retrieval`, `with_event_tx`,
|
||||
`with_platform_info` (`loop_engine.rs:152-186`). Production wires all
|
||||
five. Tests wire some. The type system says "optional" so the runtime
|
||||
carries `is_some()` branches forever, even though the production
|
||||
invariant is "always present." Dead paths are bug homes.
|
||||
|
||||
### Symptom 3 — `Thread` and `ThreadExecutionContext` duplicate state
|
||||
|
||||
`ThreadExecutionContext` (`traits/effect.rs:21-45`) carries `thread_id`,
|
||||
`thread_type`, `project_id`, `user_id` — all already on `Thread`. It
|
||||
also carries `available_actions_snapshot` and
|
||||
`available_action_inventory_snapshot` that the orchestrator stuffs back
|
||||
mid-step to redeliver work the loop already did. Two sources of truth
|
||||
for "what is executing right now."
|
||||
|
||||
### Symptom 4 — Two parallel action-dispatch pipelines
|
||||
|
||||
- Tier 0 (`executor/structured.rs:55-67`): per call → fetch lease →
|
||||
policy check → `effects.execute_action`.
|
||||
- Tier 1 (`executor/orchestrator.rs` host-fn match around line 534+,
|
||||
plus `executor/scripting.rs`): same shape, written separately.
|
||||
|
||||
Every safety/policy/lease change has to land in both. Recurring bug
|
||||
shape: PRs #2470, #2491, #2676.
|
||||
|
||||
### Symptom 5 — Manager-of-managers stack
|
||||
|
||||
`ConversationManager → ThreadManager → ExecutionLoop →
|
||||
execute_orchestrator → host-fn handlers`. Five frames before a tool
|
||||
runs. `runtime/mission.rs` is 7,933 lines; `executor/orchestrator.rs`
|
||||
is 5,212. Length is the layering tax made visible.
|
||||
|
||||
---
|
||||
|
||||
## Target Design
|
||||
|
||||
One principle: **a value travels through the engine by reference once,
|
||||
not by argument N times.**
|
||||
|
||||
### A. `EngineServices` bundle
|
||||
|
||||
```rust
|
||||
pub struct EngineServices {
|
||||
pub llm: Arc<dyn LlmBackend>,
|
||||
pub effects: Arc<dyn EffectExecutor>,
|
||||
pub store: Arc<dyn Store>,
|
||||
pub capabilities: Arc<CapabilityRegistry>,
|
||||
pub leases: Arc<LeaseManager>,
|
||||
pub policy: Arc<PolicyEngine>,
|
||||
pub retrieval: RetrievalEngine,
|
||||
pub platform: PlatformInfo,
|
||||
pub event_tx: broadcast::Sender<ThreadEvent>,
|
||||
}
|
||||
```
|
||||
|
||||
- Constructed once in `app.rs`, held as `Arc<EngineServices>`.
|
||||
- `ThreadManager`, `ExecutionLoop`, `MissionManager`, and
|
||||
`execute_orchestrator` all take `Arc<EngineServices>`.
|
||||
- The `with_*` optional builders go away: production and test wire
|
||||
the same struct (tests pass an `EngineServices` built around
|
||||
`InMemoryStore` + a fake LLM).
|
||||
- After: `ExecutionLoop::new(thread, services, signal_rx)`;
|
||||
`execute_orchestrator(code, thread, services, signal_rx,
|
||||
persisted_state)`.
|
||||
|
||||
When a manager genuinely needs only a subset, give it a narrower trait
|
||||
view (`trait HasStore`, etc.) over `EngineServices` rather than going
|
||||
back to per-Arc params.
|
||||
|
||||
### B. `ActionGateway` — single dispatch path
|
||||
|
||||
```rust
|
||||
pub struct ActionGateway { /* leases, policy, effects, capabilities */ }
|
||||
|
||||
impl ActionGateway {
|
||||
pub async fn execute(&self, ctx: &StepFrame, call: ActionCall) -> ActionResult;
|
||||
pub async fn execute_batch(&self, ctx: &StepFrame, calls: Vec<ActionCall>) -> Vec<ActionResult>;
|
||||
}
|
||||
```
|
||||
|
||||
Both Tier 0 (`structured.rs`) and the Python host functions
|
||||
(`orchestrator.rs::__execute_action__` / `__execute_actions_parallel__`)
|
||||
delegate to `ActionGateway`. Lease consumption, policy evaluation,
|
||||
capability lookup, action snapshot caching live in one place. Future
|
||||
safety rules (`tool-evidence.md` empty-fast gate, the side-effect
|
||||
intent gate) get one home.
|
||||
|
||||
This also eliminates the snapshot-passing in
|
||||
`ThreadExecutionContext` — the gateway *is* the snapshot.
|
||||
|
||||
### C. `Thread` is the source of truth; derive `StepFrame` from it
|
||||
|
||||
Delete `thread_id`/`project_id`/`user_id`/`thread_type` from
|
||||
`ThreadExecutionContext`. Replace its in-step usage with `&Thread`.
|
||||
The remaining step-scoped fields (`step_id`, `current_call_id`,
|
||||
`source_channel`, `user_timezone`) form a small `StepFrame` that lives
|
||||
only inside the gateway call. No more re-deriving identity per layer.
|
||||
|
||||
### D. Decompose `runtime/mission.rs`
|
||||
|
||||
7,933 lines mixing scheduling/cron, gate evaluation, learning-mission
|
||||
seeding, fire-rate limiting, budget gates, notifications. Split along
|
||||
those seams — each gets a sibling file. Do this *after* (A) lands so
|
||||
the constructor noise has shrunk first.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing — least risky path
|
||||
|
||||
1. **Land `EngineServices`** as a pure refactor. Same fields, new
|
||||
home. Touches every constructor; no behavior change. Every later
|
||||
refactor is cheaper because deps stop multiplying. **Acceptance:**
|
||||
`cargo test -p ironclaw_engine` green; constructor argument count
|
||||
drops at every site by ≥4.
|
||||
2. **Make `ExecutionLoop` deps required** from `EngineServices`.
|
||||
Delete `with_*` builders and the `Option<…>` checks they enable.
|
||||
**Acceptance:** zero `Option<Arc<…>>` fields on `ExecutionLoop`;
|
||||
no runtime behavior diff in trace recordings of equivalent threads.
|
||||
3. **Extract `ActionGateway`.** Migrate Tier 0 first (smaller surface).
|
||||
Then port the four Python host fns to delegate. Keep behavior
|
||||
bit-identical; the existing executor tests catch drift.
|
||||
**Acceptance:** `structured.rs` no longer references `leases` or
|
||||
`policy` directly; orchestrator host fns match.
|
||||
4. **Slim `ThreadExecutionContext`** to step-scoped data; introduce
|
||||
`StepFrame`. **Acceptance:** all duplicate identity fields gone;
|
||||
snapshot fields removed; orchestrator no longer re-stuffs them.
|
||||
5. **Decompose `runtime/mission.rs`** along scheduling /
|
||||
gate-evaluation / learning-seed / notifications seams.
|
||||
**Acceptance:** no single file > 1500 lines in `runtime/`.
|
||||
|
||||
---
|
||||
|
||||
## Tradeoffs
|
||||
|
||||
`EngineServices` makes "what does this layer actually need" less
|
||||
visible at the constructor — fine-grained DI traded for ergonomics.
|
||||
In a crate with one process, one set of services, and no plugin
|
||||
points at the manager layer, this trade is correct. The escape
|
||||
hatch (subset traits) preserves the option to narrow later when a
|
||||
manager genuinely diverges.
|
||||
|
||||
The `ActionGateway` extraction adds an indirection — but the
|
||||
indirection already exists, twice, in two places. Naming it once
|
||||
is a net reduction.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Public engine API surface (`lib.rs` re-exports stay).
|
||||
- Trait shapes of `LlmBackend` / `Store` / `EffectExecutor`.
|
||||
- The Python orchestrator code itself.
|
||||
- Persistence schema.
|
||||
|
||||
## Related
|
||||
|
||||
- `crates/ironclaw_engine/CLAUDE.md` — primitives and module map.
|
||||
- `docs/plans/2026-03-20-engine-v2-architecture.md` — original v2 plan.
|
||||
- `.claude/rules/architecture.md` — companion rule (filed alongside
|
||||
this plan) capturing the discipline that would have prevented the
|
||||
sprawl in the first place.
|
||||
Reference in New Issue
Block a user