2 Commits

Author	SHA1	Message	Date
Miranda Limonczenko	6d3a4bcc48	feat(www) Add scaffolding for WWW E2E tests and CI check (#48861) ... Closes DOCS-1278 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature. Adds E2E test scaffolding and a CI check for the marketing site. ## What is the current behavior? Closes [FE-4047](https://linear.app/supabase/issue/FE-4047). The marketing site has no E2E coverage. Docs has a suite in `e2e/docs`, but its runner, git helpers and axe reporting are private to that package, so a second site cannot reuse them. ## What is the new behavior? * **A www suite scoped to changed content.** Changed `.mdx` files in `_blog`, `_events`, `_customers` and `_alternatives` map to the URLs they render. Pages with `disable_page_build: true` are skipped because they 404 by design. Capped at 20 pages. Enforces `heading-order` and `page-has-heading-one`, matching docs. * **`e2e/shared` The docs site is also static with similar needs. This folder shares the docs logic with www. * **A CI check that is safe to mark required.** Path scoping lives in a `Detect changed paths` step rather than a `paths:` trigger, so the check reports on every pull request instead of being skipped. `waitForVercelDocsPreview.js` becomes `waitForVercelPreview.js`, shared by both workflows. ## How the check behaves The job always reports a check run, so it is safe to mark required. Path scoping happens in a step rather than a `paths:` trigger, which would leave non-www pull requests waiting on a check that never reports. | Case | Behavior | | --- | --- | | Fork pull request adds new pages | Passes without testing. The Vercel wait is gated on `head.repo.full_name == github.repository`, so forks resolve no preview URL. The job emits a `::warning` and a job summary containing a ready-to-run `gh workflow run www-e2e.yml` command with the resolved page paths, so a maintainer can run it against the preview. | | Vercel preview times out or fails | Passes without testing. The wait step is `continue-on-error: true`, so a 900s timeout or a failed deployment leaves the URL unset and the suite skips. Vercel's own `Vercel – zone-www-dot-com` check already reports the failure. | | Draft pull request | Job does not run at all, gated at the job level on `pull_request.draft == false`. `ready_for_review` is in the trigger's `types`, so marking it ready runs the check. | | Another app changed, www untouched | Job runs and every step skips. The `www` filter matches only the four content directories, `e2e/www`, `e2e/shared`, the lockfile, and this workflow. | | Only the harness changed | Passes without testing. Scope resolves to zero pages, and the Vercel wait is additionally gated on `www_app`, so it does not wait for a preview Vercel skipped. | | No preview resolves, any reason | Skips rather than falling back to production. Production does not serve pages the pull request adds, so testing it would fail a valid change. | ### Not covered Changes to `apps/www` components and routes do not trigger this check — only the four content directories do. A follow-up can check global components such as the navigation and the footer. ## Manual testing 1. Start the site: `pnpm dev:www` 2. Run `pnpm e2e:www` with no www content changed. It should resolve zero pages and skip Playwright, not fail. 3. Touch a post, then run `pnpm e2e:www` again: `echo "" >> apps/www/_blog/2024-01-01-some-post.mdx`. The resolved `/blog/...` path should be listed before Playwright starts. 4. Run against production with no local server: `PLAYWRIGHT_BASE_URL=https://supabase.com WWW_E2E_PAGE_PATHS=/blog/postgres-language-server pnpm e2e:www` 5. Point step 4 at a page with a known heading problem. The failure should name the rule, the CSS selector and the markup. 6. Confirm docs still passes on the shared runner: `pnpm dev:docs`, then `pnpm e2e:docs` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added WWW end-to-end testing for affected content pages, including accessibility checks. * Added standard and full-site test commands, configurable preview testing, and failure reports. * Added shared utilities for page discovery, accessibility scanning, and test execution. * **Documentation** * Documented WWW test setup, coverage, debugging, CI behavior, and running checks against production or preview environments. * **Improvements** * Updated documentation test workflows to better identify affected changes and handle preview environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-11 22:06:51 +00:00
Miranda Limonczenko	52cb1c2600	feat(docs) Dynamically E2E test all docs-owned content (#48320) ... Closes DOCS-1203 ## Problem The docs E2E workflow only ever tested one hardcoded page: the Next.js quickstart. All other docs content had no E2E coverage. ## Solution This PR expands the initial scaffolding to generalize the Next.js quickstart tests, page runs and checks local links, to all pages affecting Docs content: - Add `resolveDocsScope` (`e2e/docs/utils/resolve-docs-scope.ts`) to map changed guide and troubleshooting `.mdx` files to their `/docs/...` page paths, and to expand changed `_partials` to every page that includes them (including transitively, through partials nested inside other partials). Federated guide sections (`graphql`, `database/extensions/wrappers`, `ai/python`, `deployment/terraform`, `deployment/ci`) and reference docs stay out of scope, and resolution is capped at 20 pages to keep runtime bounded. - Replace the single `quickstarts.spec.ts` test with a generic `docs-pages.spec.ts` that loads whatever pages are resolved, asserting each renders with an `<h1>` and that its docs-owned links resolve. - Add `run-e2e-docs.ts` so `pnpm e2e:docs` resolves scope locally (from commits since `origin/master`, plus staged/unstaged changes) and skips Playwright entirely when nothing in scope changed. - Update `.github/workflows/docs-e2e.yml` to widen the trigger paths to all guides/troubleshooting/partials, resolve scope in a dedicated step, skip the rest of the job when scope is empty, and accept a `page_paths` input for manual `workflow_dispatch` runs. - Rewrite `e2e/docs/README.md` to document the new scoping behavior, the override envs (`DOCS_E2E_PAGE_PATHS`, `DOCS_E2E_BASE_REF`), and how CI uses the suite. - `pnpm e2e:docs:all` is also added to run tests on every page locally. Good for scoping issues but should not be included in CI. ## Manual testing Walk through the following steps to verify this works: - [x] `pnpm e2e:docs` from repo root resolves the expected pages for a local guide edit and can run against local dev **Note:** Challenges with testing on local in part because of the long lag for first page load. Recommendation to use a hosted URL is added to docs. - [x] Editing a shared `_partials` file resolves to every page that includes it (including through nested partials) - [x] `pnpm e2e:docs` exits cleanly with no Playwright run when no in-scope files changed - [x] `git diff --name-only ... | pnpm -C e2e/docs resolve-docs-scope` prints the expected page list for a sample diff - [x] Workflow run on a PR that only touches `e2e/docs`/workflow files skips the Playwright steps - [x] Manual `workflow_dispatch` run with `page_paths` set tests only those pages - [x] Run `pnpm e2e:docs:all` to run the suite on all docs content, which takes awhile ## Next steps After this PR merges, we have the scaffolding to add more fun tests like a11y 😁 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added scoped Docs E2E runs that target eligible doc pages based on changes, plus manual page-targeted runs and an “all eligible pages” mode. * Introduced `DOCS_E2E_PAGE_PATHS` (and updated base ref/base URL behavior) to control which pages are tested. * **Bug Fixes** * Automatically skips Playwright setup when no relevant pages are in scope; Playwright reporting now uploads only on failure. * **Documentation** * Updated the Docs E2E README with new run/CI behavior, troubleshooting notes, and commands to inspect the resolved page list. * **Tests** * Added a Docs-owned pages E2E suite; removed the Next.js quickstart E2E spec. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-28 00:04:58 +00:00