mirror of
https://github.com/supabase/supabase.git
synced 2026-09-09 03:19:36 +08:00
Closes DOCS-1278 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature. Adds E2E test scaffolding and a CI check for the marketing site. ## What is the current behavior? Closes [FE-4047](https://linear.app/supabase/issue/FE-4047). The marketing site has no E2E coverage. Docs has a suite in `e2e/docs`, but its runner, git helpers and axe reporting are private to that package, so a second site cannot reuse them. ## What is the new behavior? * **A www suite scoped to changed content.** Changed `.mdx` files in `_blog`, `_events`, `_customers` and `_alternatives` map to the URLs they render. Pages with `disable_page_build: true` are skipped because they 404 by design. Capped at 20 pages. Enforces `heading-order` and `page-has-heading-one`, matching docs. * **`e2e/shared` The docs site is also static with similar needs. This folder shares the docs logic with www. * **A CI check that is safe to mark required.** Path scoping lives in a `Detect changed paths` step rather than a `paths:` trigger, so the check reports on every pull request instead of being skipped. `waitForVercelDocsPreview.js` becomes `waitForVercelPreview.js`, shared by both workflows. ## How the check behaves The job always reports a check run, so it is safe to mark required. Path scoping happens in a step rather than a `paths:` trigger, which would leave non-www pull requests waiting on a check that never reports. | Case | Behavior | | --- | --- | | Fork pull request adds new pages | Passes without testing. The Vercel wait is gated on `head.repo.full_name == github.repository`, so forks resolve no preview URL. The job emits a `::warning` and a job summary containing a ready-to-run `gh workflow run www-e2e.yml` command with the resolved page paths, so a maintainer can run it against the preview. | | Vercel preview times out or fails | Passes without testing. The wait step is `continue-on-error: true`, so a 900s timeout or a failed deployment leaves the URL unset and the suite skips. Vercel's own `Vercel – zone-www-dot-com` check already reports the failure. | | Draft pull request | Job does not run at all, gated at the job level on `pull_request.draft == false`. `ready_for_review` is in the trigger's `types`, so marking it ready runs the check. | | Another app changed, www untouched | Job runs and every step skips. The `www` filter matches only the four content directories, `e2e/www`, `e2e/shared`, the lockfile, and this workflow. | | Only the harness changed | Passes without testing. Scope resolves to zero pages, and the Vercel wait is additionally gated on `www_app`, so it does not wait for a preview Vercel skipped. | | No preview resolves, any reason | Skips rather than falling back to production. Production does not serve pages the pull request adds, so testing it would fail a valid change. | ### Not covered Changes to `apps/www` components and routes do not trigger this check — only the four content directories do. A follow-up can check global components such as the navigation and the footer. ## Manual testing 1. Start the site: `pnpm dev:www` 2. Run `pnpm e2e:www` with no www content changed. It should resolve zero pages and skip Playwright, not fail. 3. Touch a post, then run `pnpm e2e:www` again: `echo "" >> apps/www/_blog/2024-01-01-some-post.mdx`. The resolved `/blog/...` path should be listed before Playwright starts. 4. Run against production with no local server: `PLAYWRIGHT_BASE_URL=https://supabase.com WWW_E2E_PAGE_PATHS=/blog/postgres-language-server pnpm e2e:www` 5. Point step 4 at a page with a known heading problem. The failure should name the rule, the CSS selector and the markup. 6. Confirm docs still passes on the shared runner: `pnpm dev:docs`, then `pnpm e2e:docs` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added WWW end-to-end testing for affected content pages, including accessibility checks. * Added standard and full-site test commands, configurable preview testing, and failure reports. * Added shared utilities for page discovery, accessibility scanning, and test execution. * **Documentation** * Documented WWW test setup, coverage, debugging, CI behavior, and running checks against production or preview environments. * **Improvements** * Updated documentation test workflows to better identify affected changes and handle preview environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
209 lines
7.9 KiB
YAML
209 lines
7.9 KiB
YAML
name: WWW E2E Tests
|
||
|
||
# Path scoping lives in "Detect changed paths" rather than a `paths` trigger, so
|
||
# this reports a check run on every pull request and stays safe to mark required.
|
||
on:
|
||
pull_request:
|
||
types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]
|
||
branches: ['master']
|
||
workflow_dispatch:
|
||
inputs:
|
||
base_url:
|
||
description: 'Base URL to test against'
|
||
required: false
|
||
default: 'https://supabase.com'
|
||
type: string
|
||
page_paths:
|
||
description: 'Comma-separated site-relative paths to test (required for manual runs)'
|
||
required: false
|
||
default: ''
|
||
type: string
|
||
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
permissions:
|
||
contents: read
|
||
statuses: read
|
||
pull-requests: read
|
||
|
||
env:
|
||
CI: true
|
||
|
||
jobs:
|
||
e2e:
|
||
name: WWW E2E
|
||
if: github.event_name == 'workflow_dispatch' || github.event.pull_request.draft == false
|
||
timeout-minutes: 30
|
||
runs-on: blacksmith-4vcpu-ubuntu-2404
|
||
|
||
steps:
|
||
- name: Detect changed paths
|
||
id: changes
|
||
if: github.event_name == 'pull_request'
|
||
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
|
||
with:
|
||
filters: |
|
||
www:
|
||
- 'apps/www/_blog/**/*.mdx'
|
||
- 'apps/www/_events/**/*.mdx'
|
||
- 'apps/www/_customers/**/*.mdx'
|
||
- 'apps/www/_alternatives/**/*.mdx'
|
||
- 'e2e/www/**'
|
||
- 'e2e/shared/**'
|
||
- 'pnpm-lock.yaml'
|
||
- '.github/workflows/www-e2e.yml'
|
||
www_app:
|
||
- 'apps/www/**'
|
||
|
||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||
if: github.event_name == 'workflow_dispatch' || steps.changes.outputs.www == 'true'
|
||
with:
|
||
persist-credentials: false
|
||
# String '0' — numeric 0 is falsy in Actions expressions.
|
||
fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }}
|
||
sparse-checkout: |
|
||
e2e/www
|
||
e2e/shared
|
||
scripts
|
||
patches
|
||
apps/www/_blog
|
||
apps/www/_events
|
||
apps/www/_customers
|
||
apps/www/_alternatives
|
||
|
||
- name: Use Node.js
|
||
if: github.event_name == 'workflow_dispatch' || steps.changes.outputs.www == 'true'
|
||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||
with:
|
||
node-version-file: '.nvmrc'
|
||
|
||
- name: Resolve www E2E scope
|
||
id: scope
|
||
if: github.event_name == 'workflow_dispatch' || steps.changes.outputs.www == 'true'
|
||
env:
|
||
EVENT_NAME: ${{ github.event_name }}
|
||
BASE_REF: ${{ github.base_ref }}
|
||
PAGE_PATHS_INPUT: ${{ inputs.page_paths }}
|
||
run: |
|
||
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
|
||
if [ -z "$PAGE_PATHS_INPUT" ]; then
|
||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||
echo "paths=" >> "$GITHUB_OUTPUT"
|
||
echo "Manual run requires the page_paths input."
|
||
exit 0
|
||
fi
|
||
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||
printf 'paths=%s\n' "$PAGE_PATHS_INPUT" >> "$GITHUB_OUTPUT"
|
||
exit 0
|
||
fi
|
||
|
||
git diff --name-only --diff-filter=ACMR "origin/$BASE_REF"...HEAD \
|
||
| node --experimental-strip-types e2e/www/scripts/resolve-www-scope.ts
|
||
|
||
- name: Skip Playwright (no in-scope pages)
|
||
if: steps.scope.outputs.skip == 'true'
|
||
run: echo "No in-scope www pages changed; skipping Playwright suite."
|
||
|
||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||
if: steps.scope.outputs.skip == 'false'
|
||
name: Install pnpm
|
||
with:
|
||
run_install: false
|
||
|
||
- name: Enable pnpm store cache
|
||
if: steps.scope.outputs.skip == 'false'
|
||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||
with:
|
||
node-version-file: '.nvmrc'
|
||
cache: 'pnpm'
|
||
|
||
# Vercel skips the preview when only the harness changed, so wait for one
|
||
# only when apps/www changed. See scripts/waitForVercelPreview.js.
|
||
- name: Wait for Vercel www preview
|
||
if: steps.scope.outputs.skip == 'false' && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && steps.changes.outputs.www_app == 'true'
|
||
id: deployment
|
||
continue-on-error: true
|
||
run: node scripts/waitForVercelPreview.js
|
||
env:
|
||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||
VERCEL_STATUS_CONTEXT: 'Vercel – zone-www-dot-com'
|
||
VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
|
||
VERCEL_TEAM_ID: ${{ secrets.VERCEL_TEAM_ID }}
|
||
|
||
- name: Resolve base URL
|
||
if: steps.scope.outputs.skip == 'false'
|
||
id: base-url
|
||
env:
|
||
EVENT_NAME: ${{ github.event_name }}
|
||
BASE_URL_INPUT: ${{ inputs.base_url }}
|
||
DEPLOYMENT_URL: ${{ steps.deployment.outputs.deployment-url }}
|
||
PAGE_PATHS: ${{ steps.scope.outputs.paths }}
|
||
run: |
|
||
set -euo pipefail
|
||
|
||
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
|
||
printf 'url=%s\n' "$BASE_URL_INPUT" >> "$GITHUB_OUTPUT"
|
||
if [ "$BASE_URL_INPUT" = "https://supabase.com" ]; then
|
||
echo "use_bypass=false" >> "$GITHUB_OUTPUT"
|
||
else
|
||
echo "use_bypass=true" >> "$GITHUB_OUTPUT"
|
||
fi
|
||
echo "should_test=true" >> "$GITHUB_OUTPUT"
|
||
exit 0
|
||
fi
|
||
|
||
if [ -n "$DEPLOYMENT_URL" ]; then
|
||
printf 'url=%s\n' "$DEPLOYMENT_URL" >> "$GITHUB_OUTPUT"
|
||
echo "use_bypass=true" >> "$GITHUB_OUTPUT"
|
||
echo "should_test=true" >> "$GITHUB_OUTPUT"
|
||
exit 0
|
||
fi
|
||
|
||
# Production lacks pages this pull request adds, so it is not a fallback.
|
||
echo "url=" >> "$GITHUB_OUTPUT"
|
||
echo "use_bypass=false" >> "$GITHUB_OUTPUT"
|
||
echo "should_test=false" >> "$GITHUB_OUTPUT"
|
||
echo "::warning::No Vercel www preview URL for this pull request. Skipping Playwright rather than testing production, which does not have pages this pull request adds."
|
||
{
|
||
echo "### WWW E2E skipped: no preview to test against"
|
||
echo
|
||
echo "Fork pull requests reach this path because they run without repository"
|
||
echo "secrets. A maintainer can run the suite against the preview manually:"
|
||
echo
|
||
echo '```'
|
||
echo "gh workflow run www-e2e.yml \\"
|
||
echo " -f base_url=<preview-url> \\"
|
||
echo " -f page_paths=$PAGE_PATHS"
|
||
echo '```'
|
||
} >> "$GITHUB_STEP_SUMMARY"
|
||
|
||
- name: Install dependencies
|
||
if: steps.base-url.outputs.should_test == 'true'
|
||
run: pnpm install --frozen-lockfile --filter=e2e-www...
|
||
|
||
- name: Install Playwright Chromium
|
||
if: steps.base-url.outputs.should_test == 'true'
|
||
run: pnpm -C e2e/www exec playwright install chromium --with-deps --only-shell
|
||
|
||
- name: Run www E2E
|
||
if: steps.base-url.outputs.should_test == 'true'
|
||
working-directory: e2e/www
|
||
run: pnpm run e2e:www
|
||
env:
|
||
PLAYWRIGHT_BASE_URL: ${{ steps.base-url.outputs.url }}
|
||
WWW_E2E_PAGE_PATHS: ${{ steps.scope.outputs.paths }}
|
||
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ steps.base-url.outputs.use_bypass == 'true' && secrets.VERCEL_AUTOMATION_BYPASS_WWW || '' }}
|
||
|
||
- name: Upload Playwright report
|
||
if: failure() && steps.scope.outputs.skip == 'false'
|
||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||
with:
|
||
name: www-playwright-report
|
||
path: |
|
||
e2e/www/playwright-report/
|
||
e2e/www/test-results/
|
||
retention-days: 7
|