chore(e2e): scope Tier 2 networks and gate pentagi on a healthy mock LLM

The base compose file pins fixed network names, so any network left un-overridden
in this e2e override joins the stack to the developer's live observability and
langfuse networks. Give all three networks e2e-scoped names. Add a healthcheck to
the mock-LLM service and make pentagi depend on service_healthy, so a crash-looping
mock can never read as a started dependency and silently mask itself.

Update frontend/docs/e2e.md to match reality: the stand tier runs @stand only
(flow-run drives a real paid agent and stays local), every mock run rebuilds its
own bundle by design, and the trend/affected-routes notes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sergey Kozyrenko
2026-07-21 03:59:07 +07:00
parent 25b0229282
commit 179c814bb3
2 changed files with 28 additions and 12 deletions

View File

@@ -11,9 +11,16 @@
# developer's .env and the stack inherits live provider keys and DOCKER_HOST.
# (frontend/e2e/tools/run-local-tier.sh wraps all of this.)
# All three base networks get e2e names — the base file gives them fixed
# `name:`s, so leaving any un-overridden would join this stack to the
# developer's live observability/langfuse networks.
networks:
pentagi-network:
name: pentagi-e2e-network
observability-network:
name: pentagi-e2e-observability-network
langfuse-network:
name: pentagi-e2e-langfuse-network
services:
mock-llm:
@@ -25,6 +32,13 @@ services:
- ./frontend/e2e/mock-llm:/app:ro
networks:
- pentagi-network
# Keeps a crash-looping mock (restart would silently mask it) from ever
# reading as a started dependency.
healthcheck:
test: ["CMD-SHELL", "wget -qO /dev/null http://127.0.0.1:8080/v1/models"]
interval: 3s
timeout: 5s
retries: 20
pentagi:
container_name: pentagi-e2e
@@ -32,7 +46,7 @@ services:
pgvector:
condition: service_healthy
mock-llm:
condition: service_started
condition: service_healthy
environment:
- LLM_SERVER_URL=http://mock-llm:8080/v1
# The langchaingo client refuses to construct without a token even though

View File

@@ -28,14 +28,18 @@ CI=1 pnpm e2e # byte-identical reproduction of a CI run
./e2e/tools/run-local-tier.sh # Tier 2: branch image + isolated docker
# stack + mock LLM; runs specs/real/**
E2E_TIER=stand E2E_BASE_URL=https://… pnpm e2e # against a live stand
E2E_TIER=stand E2E_BASE_URL=https://… pnpm e2e # live stand: @stand smoke only
# (flow-run drives a real paid agent run
# and is scoped to the local tier)
pnpm e2e:visual # visual snapshots (pinned container)
pnpm e2e:visual:update # regenerate baselines after a UI change
```
Tips for the mock tier: `vite preview` is reused between runs if you keep it
running (`reuseExistingServer`), so only the first run pays the build.
Every mock-tier run rebuilds the bundle and starts its own `vite preview`
deliberately: reusing a server already listening on the port would silently
test a previous commit's dist. A stray listener on 8100 fails the run loudly;
kill it and rerun.
## Tiers
@@ -105,7 +109,7 @@ Key conventions:
## Stand tier (Tier 3)
Runs the LLM-independent `@stand` smoke against a real deployment. Label a PR
`e2e:stand` (or dispatch the workflow); the job runs in a protected
`e2e:stand` (or dispatch the workflow with `tier: stand`); the job runs in a protected
`e2e-stand` Environment whose required reviewers approve before any secret is
exposed, so even a mislabeled run blocks on a human. Fork PRs never reach it.
@@ -119,10 +123,11 @@ prefix `NODE_TLS_REJECT_UNAUTHORIZED=0`; a real stand has a valid cert).
## Trends and selective runs
- **Trend:** `node e2e/tools/trend.mjs` turns a run's `results.json` into one
JSONL record (p50/p95 spec duration, slowest three, pass/flaky/fail counts).
CI appends it to the retained `e2e-trend` artifact so duration creep and flake
are visible, not just green/red.
- **Trend:** `node e2e/tools/trend.mjs` turns a run's `results.json` (written
under `CI=1`) into one JSONL record (p50/p95 spec duration, slowest three,
pass/flaky/fail counts). Each CI run uploads its own `e2e-trend` artifact;
aggregate across runs offline (`gh run download`) to see duration creep and
flake, not just green/red.
- **Affected routes:** `pnpm exec tsx e2e/tools/affected.ts <base>` prints the
manifest routes a diff touches (backed by each route's owning `sources` in
`e2e/routes.ts`). Empty output = no frontend route changed. This is the
@@ -153,6 +158,3 @@ bespoke bot:
This section is the recipe, not yet wired — the deterministic tiers above are
the foundation it plugs into.
The full design (tiers, CI topology, mocking contract, roadmap) lives in the
team's E2E testing plan; scenario coverage maps 1:1 to the scenario catalog IDs.