Files
supabase/e2e/docs/features/docs-pages.spec.ts
Miranda Limonczenko 52cb1c2600 feat(docs) Dynamically E2E test all docs-owned content (#48320)
Closes DOCS-1203

## Problem

The docs E2E workflow only ever tested one hardcoded page: the Next.js
quickstart. All other docs content had no E2E coverage.

## Solution

This PR expands the initial scaffolding to generalize the Next.js
quickstart tests, page runs and checks local links, to all pages
affecting Docs content:


- Add `resolveDocsScope` (`e2e/docs/utils/resolve-docs-scope.ts`) to map
changed guide and troubleshooting `.mdx` files to their `/docs/...` page
paths, and to expand changed `_partials` to every page that includes
them (including transitively, through partials nested inside other
partials). Federated guide sections (`graphql`,
`database/extensions/wrappers`, `ai/python`, `deployment/terraform`,
`deployment/ci`) and reference docs stay out of scope, and resolution is
capped at 20 pages to keep runtime bounded.
- Replace the single `quickstarts.spec.ts` test with a generic
`docs-pages.spec.ts` that loads whatever pages are resolved, asserting
each renders with an `<h1>` and that its docs-owned links resolve.
- Add `run-e2e-docs.ts` so `pnpm e2e:docs` resolves scope locally (from
commits since `origin/master`, plus staged/unstaged changes) and skips
Playwright entirely when nothing in scope changed.
- Update `.github/workflows/docs-e2e.yml` to widen the trigger paths to
all guides/troubleshooting/partials, resolve scope in a dedicated step,
skip the rest of the job when scope is empty, and accept a `page_paths`
input for manual `workflow_dispatch` runs.
- Rewrite `e2e/docs/README.md` to document the new scoping behavior, the
override envs (`DOCS_E2E_PAGE_PATHS`, `DOCS_E2E_BASE_REF`), and how CI
uses the suite.
- `pnpm e2e:docs:all` is also added to run tests on every page locally.
Good for scoping issues but should not be included in CI.

## Manual testing

Walk through the following steps to verify this works:

- [x] `pnpm e2e:docs` from repo root resolves the expected pages for a
local guide edit and can run against local dev
**Note:** Challenges with testing on local in part because of the long
lag for first page load. Recommendation to use a hosted URL is added to
docs.
- [x] Editing a shared `_partials` file resolves to every page that
includes it (including through nested partials)
- [x] `pnpm e2e:docs` exits cleanly with no Playwright run when no
in-scope files changed
- [x] `git diff --name-only ... | pnpm -C e2e/docs resolve-docs-scope`
prints the expected page list for a sample diff
- [x] Workflow run on a PR that only touches `e2e/docs`/workflow files
skips the Playwright steps
- [x] Manual `workflow_dispatch` run with `page_paths` set tests only
those pages
- [x] Run `pnpm e2e:docs:all` to run the suite on all docs content,
which takes awhile

## Next steps

After this PR merges, we have the scaffolding to add more fun tests like
a11y 😁

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added scoped Docs E2E runs that target eligible doc pages based on
changes, plus manual page-targeted runs and an “all eligible pages”
mode.
* Introduced `DOCS_E2E_PAGE_PATHS` (and updated base ref/base URL
behavior) to control which pages are tested.
* **Bug Fixes**
* Automatically skips Playwright setup when no relevant pages are in
scope; Playwright reporting now uploads only on failure.
* **Documentation**
* Updated the Docs E2E README with new run/CI behavior, troubleshooting
notes, and commands to inspect the resolved page list.
* **Tests**
* Added a Docs-owned pages E2E suite; removed the Next.js quickstart E2E
spec.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 00:04:58 +00:00

68 lines
2.4 KiB
TypeScript

import { expect, test } from '@playwright/test'
import {
articleSelectorForPagePath,
browserLikeUserAgent,
collectDocsOwnedLinks,
parseDocsE2EPagePaths,
} from '../utils/docs-links.js'
const pagePaths = parseDocsE2EPagePaths(process.env.DOCS_E2E_PAGE_PATHS)
test.describe('Docs owned pages', () => {
// playwright.config.ts sets fullyParallel: false, and Playwright shards
// work by file rather than by test in that mode — without this, every test
// in this single spec file runs on one worker no matter what --workers is
// passed. Opt this describe block into parallel scheduling explicitly.
test.describe.configure({ mode: 'parallel' })
test('resolved page list must not be empty', () => {
expect(
pagePaths.length,
'No pages to test. `pnpm e2e:docs` resolves pages from git changes by default, ' +
'or set DOCS_E2E_PAGE_PATHS explicitly.'
).toBeGreaterThan(0)
})
for (const pagePath of pagePaths) {
test(`${pagePath} loads and docs-owned article links resolve`, async ({ page }, testInfo) => {
const baseURL = testInfo.project.use.baseURL
expect(baseURL, 'A Playwright base URL should be configured').toBeTruthy()
const articleSelector = articleSelectorForPagePath(pagePath)
const response = await page.goto(pagePath)
expect(response, `Expected a response for ${pagePath}`).not.toBeNull()
expect(
response!.ok(),
`Page should return a successful status, got ${response!.status()}`
).toBeTruthy()
const article = page.locator(articleSelector)
await expect(article, 'Page article should be present').toBeVisible()
await expect(
article.getByRole('heading', { level: 1 }),
'Page article should include an h1'
).toBeVisible()
const links = await collectDocsOwnedLinks(page, baseURL!, articleSelector)
const userAgent = await browserLikeUserAgent(page)
for (const url of links) {
try {
const linkResponse = await page.request.get(url, { headers: { 'user-agent': userAgent } })
expect
.soft(linkResponse.ok(), `${url} should resolve (status ${linkResponse.status()})`)
.toBeTruthy()
} catch (error) {
expect
.soft(
null,
`${url} should be reachable (${error instanceof Error ? error.message : error})`
)
.toBeTruthy()
}
}
})
}
})