Files
supabase/scripts/waitForVercelPreview.js
Miranda Limonczenko 6d3a4bcc48 feat(www) Add scaffolding for WWW E2E tests and CI check (#48861)
Closes DOCS-1278

## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature. Adds E2E test scaffolding and a CI check for the marketing
site.

## What is the current behavior?

Closes [FE-4047](https://linear.app/supabase/issue/FE-4047).

The marketing site has no E2E coverage. Docs has a suite in `e2e/docs`,
but its
runner, git helpers and axe reporting are private to that package, so a
second
site cannot reuse them.

## What is the new behavior?

* **A www suite scoped to changed content.** Changed `.mdx` files in
`_blog`,
`_events`, `_customers` and `_alternatives` map to the URLs they render.
Pages
with `disable_page_build: true` are skipped because they 404 by design.
Capped
at 20 pages. Enforces `heading-order` and `page-has-heading-one`,
matching docs.
* **`e2e/shared` The docs site is also static with similar needs. This
folder shares the docs logic with www.
* **A CI check that is safe to mark required.** Path scoping lives in a
`Detect changed paths` step rather than a `paths:` trigger, so the check
  reports on every pull request instead of being skipped.
`waitForVercelDocsPreview.js` becomes `waitForVercelPreview.js`, shared
by both
  workflows.

## How the check behaves

The job always reports a check run, so it is safe to mark required. Path
scoping
happens in a step rather than a `paths:` trigger, which would leave
non-www pull
requests waiting on a check that never reports.

| Case | Behavior |
| --- | --- |
| Fork pull request adds new pages | Passes without testing. The Vercel
wait is gated on `head.repo.full_name == github.repository`, so forks
resolve no preview URL. The job emits a `::warning` and a job summary
containing a ready-to-run `gh workflow run www-e2e.yml` command with the
resolved page paths, so a maintainer can run it against the preview. |
| Vercel preview times out or fails | Passes without testing. The wait
step is `continue-on-error: true`, so a 900s timeout or a failed
deployment leaves the URL unset and the suite skips. Vercel's own
`Vercel – zone-www-dot-com` check already reports the failure. |
| Draft pull request | Job does not run at all, gated at the job level
on `pull_request.draft == false`. `ready_for_review` is in the trigger's
`types`, so marking it ready runs the check. |
| Another app changed, www untouched | Job runs and every step skips.
The `www` filter matches only the four content directories, `e2e/www`,
`e2e/shared`, the lockfile, and this workflow. |
| Only the harness changed | Passes without testing. Scope resolves to
zero pages, and the Vercel wait is additionally gated on `www_app`, so
it does not wait for a preview Vercel skipped. |
| No preview resolves, any reason | Skips rather than falling back to
production. Production does not serve pages the pull request adds, so
testing it would fail a valid change. |

### Not covered

Changes to `apps/www` components and routes do not trigger this check —
only the
four content directories do. A follow-up can check global components
such as the navigation and the footer.

## Manual testing

1. Start the site: `pnpm dev:www`
2. Run `pnpm e2e:www` with no www content changed. It should resolve
zero pages
   and skip Playwright, not fail.
3. Touch a post, then run `pnpm e2e:www` again:
`echo "" >> apps/www/_blog/2024-01-01-some-post.mdx`. The resolved
`/blog/...`
   path should be listed before Playwright starts.
4. Run against production with no local server:
`PLAYWRIGHT_BASE_URL=https://supabase.com
WWW_E2E_PAGE_PATHS=/blog/postgres-language-server pnpm e2e:www`
5. Point step 4 at a page with a known heading problem. The failure
should name
   the rule, the CSS selector and the markup.
6. Confirm docs still passes on the shared runner: `pnpm dev:docs`, then
   `pnpm e2e:docs`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added WWW end-to-end testing for affected content pages, including
accessibility checks.
* Added standard and full-site test commands, configurable preview
testing, and failure reports.
* Added shared utilities for page discovery, accessibility scanning, and
test execution.

* **Documentation**
* Documented WWW test setup, coverage, debugging, CI behavior, and
running checks against production or preview environments.

* **Improvements**
* Updated documentation test workflows to better identify affected
changes and handle preview environments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5 <[email protected]>
2026-08-11 22:06:51 +00:00

113 lines
3.9 KiB
JavaScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// Vercel's GitHub App has stopped writing GitHub Deployment objects since
// 2026-02-17 (broken app auth), so polling the Deployments API (as
// vercel/wait-for-deployment-action does) times out even though the preview
// builds fine. Poll the project's Vercel commit status instead, then resolve
// the actual preview URL via Vercel's own deployments API.
//
// Set VERCEL_STATUS_CONTEXT to the project's commit status name, e.g.
// "Vercel docs" or "Vercel zone-www-dot-com".
const { appendFileSync } = require('fs')
const TIMEOUT_MS = 900_000
const POLL_INTERVAL_MS = 15_000
async function fetchLatestStatus(repository, sha, githubToken, statusContext) {
const url = `https://api.github.com/repos/${repository}/commits/${sha}/statuses`
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${githubToken}`,
Accept: 'application/vnd.github+json',
},
})
if (!response.ok) {
throw new Error(`Failed to fetch commit statuses: ${response.status} ${response.statusText}`)
}
const statuses = await response.json()
return statuses
.filter((status) => status.context === statusContext)
.sort((a, b) => new Date(b.created_at).getTime() - new Date(a.created_at).getTime())[0]
}
async function resolveDeploymentUrl(targetUrl, vercelToken, teamId) {
const rawId = targetUrl.split('/').filter(Boolean).pop()
if (!rawId) {
throw new Error(`Could not parse a deployment ID from target_url: ${targetUrl}`)
}
const deploymentId = rawId.startsWith('dpl_') ? rawId : `dpl_${rawId}`
const url = teamId
? `https://api.vercel.com/v13/deployments/${deploymentId}?teamId=${teamId}`
: `https://api.vercel.com/v13/deployments/${deploymentId}`
const response = await fetch(url, {
headers: { Authorization: `Bearer ${vercelToken}` },
})
if (!response.ok) {
throw new Error(
`Failed to resolve Vercel deployment ${deploymentId}: ${response.status} ${response.statusText}`
)
}
const deployment = await response.json()
return `https://${deployment.url}`
}
function writeOutput(name, value) {
const outputFile = process.env.GITHUB_OUTPUT
if (!outputFile) {
throw new Error('GITHUB_OUTPUT environment variable is required')
}
appendFileSync(outputFile, `${name}=${value}\n`)
}
async function main() {
const repository = process.env.GITHUB_REPOSITORY
const sha = process.env.HEAD_SHA
const githubToken = process.env.GITHUB_TOKEN
const vercelToken = process.env.VERCEL_TOKEN
const teamId = process.env.VERCEL_TEAM_ID
const statusContext = process.env.VERCEL_STATUS_CONTEXT
if (!repository) throw new Error('GITHUB_REPOSITORY environment variable is required')
if (!sha) throw new Error('HEAD_SHA environment variable is required')
if (!githubToken) throw new Error('GITHUB_TOKEN environment variable is required')
if (!vercelToken) throw new Error('VERCEL_TOKEN environment variable is required')
if (!statusContext) throw new Error('VERCEL_STATUS_CONTEXT environment variable is required')
const start = Date.now()
for (;;) {
const latest = await fetchLatestStatus(repository, sha, githubToken, statusContext)
if (latest?.state === 'success') {
if (!latest.target_url) {
throw new Error(
`"${statusContext}" commit status succeeded but had no target_url to resolve a deployment from`
)
}
const deploymentUrl = await resolveDeploymentUrl(latest.target_url, vercelToken, teamId)
writeOutput('deployment-url', deploymentUrl)
return
}
if (latest?.state === 'failure' || latest?.state === 'error') {
throw new Error(`"${statusContext}" deployment failed (commit status: ${latest.state})`)
}
if (Date.now() - start > TIMEOUT_MS) {
throw new Error(`Timed out after 900s waiting for the "${statusContext}" preview deployment`)
}
await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS))
}
}
main().catch((error) => {
console.error('Fatal error:', error)
process.exit(1)
})