mirror of
https://github.com/supabase/supabase.git
synced 2026-09-07 02:20:52 +08:00
Closes DOCS-1203 ## Problem The docs E2E workflow only ever tested one hardcoded page: the Next.js quickstart. All other docs content had no E2E coverage. ## Solution This PR expands the initial scaffolding to generalize the Next.js quickstart tests, page runs and checks local links, to all pages affecting Docs content: - Add `resolveDocsScope` (`e2e/docs/utils/resolve-docs-scope.ts`) to map changed guide and troubleshooting `.mdx` files to their `/docs/...` page paths, and to expand changed `_partials` to every page that includes them (including transitively, through partials nested inside other partials). Federated guide sections (`graphql`, `database/extensions/wrappers`, `ai/python`, `deployment/terraform`, `deployment/ci`) and reference docs stay out of scope, and resolution is capped at 20 pages to keep runtime bounded. - Replace the single `quickstarts.spec.ts` test with a generic `docs-pages.spec.ts` that loads whatever pages are resolved, asserting each renders with an `<h1>` and that its docs-owned links resolve. - Add `run-e2e-docs.ts` so `pnpm e2e:docs` resolves scope locally (from commits since `origin/master`, plus staged/unstaged changes) and skips Playwright entirely when nothing in scope changed. - Update `.github/workflows/docs-e2e.yml` to widen the trigger paths to all guides/troubleshooting/partials, resolve scope in a dedicated step, skip the rest of the job when scope is empty, and accept a `page_paths` input for manual `workflow_dispatch` runs. - Rewrite `e2e/docs/README.md` to document the new scoping behavior, the override envs (`DOCS_E2E_PAGE_PATHS`, `DOCS_E2E_BASE_REF`), and how CI uses the suite. - `pnpm e2e:docs:all` is also added to run tests on every page locally. Good for scoping issues but should not be included in CI. ## Manual testing Walk through the following steps to verify this works: - [x] `pnpm e2e:docs` from repo root resolves the expected pages for a local guide edit and can run against local dev **Note:** Challenges with testing on local in part because of the long lag for first page load. Recommendation to use a hosted URL is added to docs. - [x] Editing a shared `_partials` file resolves to every page that includes it (including through nested partials) - [x] `pnpm e2e:docs` exits cleanly with no Playwright run when no in-scope files changed - [x] `git diff --name-only ... | pnpm -C e2e/docs resolve-docs-scope` prints the expected page list for a sample diff - [x] Workflow run on a PR that only touches `e2e/docs`/workflow files skips the Playwright steps - [x] Manual `workflow_dispatch` run with `page_paths` set tests only those pages - [x] Run `pnpm e2e:docs:all` to run the suite on all docs content, which takes awhile ## Next steps After this PR merges, we have the scaffolding to add more fun tests like a11y 😁 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added scoped Docs E2E runs that target eligible doc pages based on changes, plus manual page-targeted runs and an “all eligible pages” mode. * Introduced `DOCS_E2E_PAGE_PATHS` (and updated base ref/base URL behavior) to control which pages are tested. * **Bug Fixes** * Automatically skips Playwright setup when no relevant pages are in scope; Playwright reporting now uploads only on failure. * **Documentation** * Updated the Docs E2E README with new run/CI behavior, troubleshooting notes, and commands to inspect the resolved page list. * **Tests** * Added a Docs-owned pages E2E suite; removed the Next.js quickstart E2E spec. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
68 lines
2.4 KiB
TypeScript
68 lines
2.4 KiB
TypeScript
import { expect, test } from '@playwright/test'
|
|
|
|
import {
|
|
articleSelectorForPagePath,
|
|
browserLikeUserAgent,
|
|
collectDocsOwnedLinks,
|
|
parseDocsE2EPagePaths,
|
|
} from '../utils/docs-links.js'
|
|
|
|
const pagePaths = parseDocsE2EPagePaths(process.env.DOCS_E2E_PAGE_PATHS)
|
|
|
|
test.describe('Docs owned pages', () => {
|
|
// playwright.config.ts sets fullyParallel: false, and Playwright shards
|
|
// work by file rather than by test in that mode — without this, every test
|
|
// in this single spec file runs on one worker no matter what --workers is
|
|
// passed. Opt this describe block into parallel scheduling explicitly.
|
|
test.describe.configure({ mode: 'parallel' })
|
|
|
|
test('resolved page list must not be empty', () => {
|
|
expect(
|
|
pagePaths.length,
|
|
'No pages to test. `pnpm e2e:docs` resolves pages from git changes by default, ' +
|
|
'or set DOCS_E2E_PAGE_PATHS explicitly.'
|
|
).toBeGreaterThan(0)
|
|
})
|
|
|
|
for (const pagePath of pagePaths) {
|
|
test(`${pagePath} loads and docs-owned article links resolve`, async ({ page }, testInfo) => {
|
|
const baseURL = testInfo.project.use.baseURL
|
|
expect(baseURL, 'A Playwright base URL should be configured').toBeTruthy()
|
|
|
|
const articleSelector = articleSelectorForPagePath(pagePath)
|
|
const response = await page.goto(pagePath)
|
|
expect(response, `Expected a response for ${pagePath}`).not.toBeNull()
|
|
expect(
|
|
response!.ok(),
|
|
`Page should return a successful status, got ${response!.status()}`
|
|
).toBeTruthy()
|
|
|
|
const article = page.locator(articleSelector)
|
|
await expect(article, 'Page article should be present').toBeVisible()
|
|
await expect(
|
|
article.getByRole('heading', { level: 1 }),
|
|
'Page article should include an h1'
|
|
).toBeVisible()
|
|
|
|
const links = await collectDocsOwnedLinks(page, baseURL!, articleSelector)
|
|
const userAgent = await browserLikeUserAgent(page)
|
|
|
|
for (const url of links) {
|
|
try {
|
|
const linkResponse = await page.request.get(url, { headers: { 'user-agent': userAgent } })
|
|
expect
|
|
.soft(linkResponse.ok(), `${url} should resolve (status ${linkResponse.status()})`)
|
|
.toBeTruthy()
|
|
} catch (error) {
|
|
expect
|
|
.soft(
|
|
null,
|
|
`${url} should be reachable (${error instanceof Error ? error.message : error})`
|
|
)
|
|
.toBeTruthy()
|
|
}
|
|
}
|
|
})
|
|
}
|
|
})
|