mirror of
https://github.com/supabase/supabase.git
synced 2026-09-06 09:59:03 +08:00
Closes DOCS-1278 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature. Adds E2E test scaffolding and a CI check for the marketing site. ## What is the current behavior? Closes [FE-4047](https://linear.app/supabase/issue/FE-4047). The marketing site has no E2E coverage. Docs has a suite in `e2e/docs`, but its runner, git helpers and axe reporting are private to that package, so a second site cannot reuse them. ## What is the new behavior? * **A www suite scoped to changed content.** Changed `.mdx` files in `_blog`, `_events`, `_customers` and `_alternatives` map to the URLs they render. Pages with `disable_page_build: true` are skipped because they 404 by design. Capped at 20 pages. Enforces `heading-order` and `page-has-heading-one`, matching docs. * **`e2e/shared` The docs site is also static with similar needs. This folder shares the docs logic with www. * **A CI check that is safe to mark required.** Path scoping lives in a `Detect changed paths` step rather than a `paths:` trigger, so the check reports on every pull request instead of being skipped. `waitForVercelDocsPreview.js` becomes `waitForVercelPreview.js`, shared by both workflows. ## How the check behaves The job always reports a check run, so it is safe to mark required. Path scoping happens in a step rather than a `paths:` trigger, which would leave non-www pull requests waiting on a check that never reports. | Case | Behavior | | --- | --- | | Fork pull request adds new pages | Passes without testing. The Vercel wait is gated on `head.repo.full_name == github.repository`, so forks resolve no preview URL. The job emits a `::warning` and a job summary containing a ready-to-run `gh workflow run www-e2e.yml` command with the resolved page paths, so a maintainer can run it against the preview. | | Vercel preview times out or fails | Passes without testing. The wait step is `continue-on-error: true`, so a 900s timeout or a failed deployment leaves the URL unset and the suite skips. Vercel's own `Vercel – zone-www-dot-com` check already reports the failure. | | Draft pull request | Job does not run at all, gated at the job level on `pull_request.draft == false`. `ready_for_review` is in the trigger's `types`, so marking it ready runs the check. | | Another app changed, www untouched | Job runs and every step skips. The `www` filter matches only the four content directories, `e2e/www`, `e2e/shared`, the lockfile, and this workflow. | | Only the harness changed | Passes without testing. Scope resolves to zero pages, and the Vercel wait is additionally gated on `www_app`, so it does not wait for a preview Vercel skipped. | | No preview resolves, any reason | Skips rather than falling back to production. Production does not serve pages the pull request adds, so testing it would fail a valid change. | ### Not covered Changes to `apps/www` components and routes do not trigger this check — only the four content directories do. A follow-up can check global components such as the navigation and the footer. ## Manual testing 1. Start the site: `pnpm dev:www` 2. Run `pnpm e2e:www` with no www content changed. It should resolve zero pages and skip Playwright, not fail. 3. Touch a post, then run `pnpm e2e:www` again: `echo "" >> apps/www/_blog/2024-01-01-some-post.mdx`. The resolved `/blog/...` path should be listed before Playwright starts. 4. Run against production with no local server: `PLAYWRIGHT_BASE_URL=https://supabase.com WWW_E2E_PAGE_PATHS=/blog/postgres-language-server pnpm e2e:www` 5. Point step 4 at a page with a known heading problem. The failure should name the rule, the CSS selector and the markup. 6. Confirm docs still passes on the shared runner: `pnpm dev:docs`, then `pnpm e2e:docs` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added WWW end-to-end testing for affected content pages, including accessibility checks. * Added standard and full-site test commands, configurable preview testing, and failure reports. * Added shared utilities for page discovery, accessibility scanning, and test execution. * **Documentation** * Documented WWW test setup, coverage, debugging, CI behavior, and running checks against production or preview environments. * **Improvements** * Updated documentation test workflows to better identify affected changes and handle preview environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
109 lines
3.3 KiB
TypeScript
109 lines
3.3 KiB
TypeScript
import { spawn } from 'node:child_process'
|
|
|
|
import { collectChangedFiles, repoRootFrom } from './git.ts'
|
|
|
|
export type ScopeResult = { pages: string[]; skip: boolean }
|
|
|
|
export type SuiteConfig = {
|
|
root: string
|
|
label: string
|
|
devCommand: string
|
|
pagePathsEnv: string
|
|
baseRefEnv: string
|
|
defaultBaseUrl: string
|
|
resolveScope: (options: { changedFiles: string[]; repoRoot: string }) => Promise<ScopeResult>
|
|
resolveAllPages?: (repoRoot: string) => Promise<string[]>
|
|
}
|
|
|
|
const PREFLIGHT_TIMEOUT_MS = 3_000
|
|
|
|
async function isBaseUrlReachable(baseUrl: string): Promise<boolean> {
|
|
const controller = new AbortController()
|
|
const timeout = setTimeout(() => controller.abort(), PREFLIGHT_TIMEOUT_MS)
|
|
try {
|
|
await fetch(baseUrl, { signal: controller.signal })
|
|
return true
|
|
} catch {
|
|
return false
|
|
} finally {
|
|
clearTimeout(timeout)
|
|
}
|
|
}
|
|
|
|
async function resolvePagePaths(config: SuiteConfig): Promise<string[] | null> {
|
|
const explicit = process.env[config.pagePathsEnv]?.trim()
|
|
if (explicit) {
|
|
return explicit
|
|
.split(/[\n,]/)
|
|
.map((path) => path.trim())
|
|
.filter(Boolean)
|
|
}
|
|
|
|
const baseRef = process.env[config.baseRefEnv]?.trim() || 'origin/master'
|
|
const repoRoot = repoRootFrom(config.root)
|
|
const changedFiles = collectChangedFiles(repoRoot, baseRef)
|
|
const result = await config.resolveScope({ changedFiles, repoRoot })
|
|
|
|
if (result.skip) {
|
|
console.error(
|
|
`No in-scope ${config.label} pages changed vs ${baseRef} (including working tree). Skipping Playwright.`
|
|
)
|
|
return null
|
|
}
|
|
|
|
console.error(
|
|
`Resolved ${result.pages.length} ${config.label} page(s) from changes vs ${baseRef}:`
|
|
)
|
|
for (const page of result.pages) {
|
|
console.error(` ${page}`)
|
|
}
|
|
return result.pages
|
|
}
|
|
|
|
export async function runSuite(config: SuiteConfig): Promise<void> {
|
|
const rawArgs = process.argv.slice(2)
|
|
const runAll = rawArgs.includes('--all')
|
|
const playwrightArgs = rawArgs.filter((arg) => arg !== '--all' && arg !== '--')
|
|
|
|
let pages: string[] | null
|
|
if (runAll && config.resolveAllPages) {
|
|
pages = await config.resolveAllPages(repoRootFrom(config.root))
|
|
console.error(`Resolved all ${pages.length} in-scope ${config.label} page(s).`)
|
|
} else {
|
|
pages = await resolvePagePaths(config)
|
|
}
|
|
if (pages === null) process.exit(0)
|
|
|
|
const hasMaxFailuresArg = playwrightArgs.some(
|
|
(arg) => arg === '-x' || arg.startsWith('--max-failures')
|
|
)
|
|
const finalArgs =
|
|
runAll && !hasMaxFailuresArg ? [...playwrightArgs, '--max-failures=0'] : playwrightArgs
|
|
|
|
const baseUrl = process.env.PLAYWRIGHT_BASE_URL?.trim() || config.defaultBaseUrl
|
|
if (!(await isBaseUrlReachable(baseUrl))) {
|
|
console.error(`No ${config.label} server responding at ${baseUrl}.`)
|
|
if (!process.env.PLAYWRIGHT_BASE_URL) {
|
|
console.error(`Start it with \`${config.devCommand}\`, or point at a deployed site.`)
|
|
} else {
|
|
console.error('Check that the URL is correct and reachable.')
|
|
}
|
|
process.exit(1)
|
|
}
|
|
|
|
const child = spawn('pnpm', ['exec', 'playwright', 'test', ...finalArgs], {
|
|
cwd: config.root,
|
|
env: { ...process.env, [config.pagePathsEnv]: pages.join(',') },
|
|
stdio: 'inherit',
|
|
shell: process.platform === 'win32',
|
|
})
|
|
|
|
child.on('exit', (code, signal) => {
|
|
if (signal) {
|
|
process.kill(process.pid, signal)
|
|
return
|
|
}
|
|
process.exit(code ?? 1)
|
|
})
|
|
}
|