mirror of
https://github.com/supabase/supabase.git
synced 2026-09-06 09:59:03 +08:00
create-pull-request/patch
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Anthony Lio
|
085d218bff |
fix(docs): button a11y enhancements (#49952)
## What kind of change does this PR introduce? bug fix for accessibility, fixes [docs-1265](https://linear.app/supabase/issue/DOCS-1365/add-aria-label-to-icon-only-buttons-button-name) ## What is the current behavior? some controles in the docs app have no accessible name, so screen readers announce them as an unlabelled "button" ## What is the new behavior? - adds each control a name the way its neighbours already do - adds menu toggle a state-dependent `aria-label` and `aria-expanded` - adds `button-name` to `enforced_rules` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Accessibility** - Added descriptive labels to configuration selectors and mobile menu controls. - Mobile menu controls now announce whether the menu is open or closed. - Expanded accessibility validation to check button names across documentation pages. - **UI Improvements** - Updated the raw throughput table control with a clearer button appearance and expandable chevron indicator. - Replaced a schema-migration tooltip with a direct in-page link. - **Documentation** - Removed the Info Tooltip component guidance from the contributing documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Saxon Fletcher
|
e0280cb650 |
docs: restructure observability navigation and overview (#49505)
<!-- CURSOR_AGENT_PR_BODY_BEGIN --> ## Stack Draft stack extracted from `docs/monitoring`. Merge bottom-up. The troubleshooting *catalog* rewrite (`content/troubleshooting` and the Diagnosing UI) stays out of scope. 1. #49503 move inspect and advisors 2. #49501 split Studio logs from ClickHouse queries 3. #49500 treat reports as signal dashboards 4. #49502 add Observe the data hub 5. #49506 add agent setup components 6. #49504 add hire-an-agent templates 7. **#49505** restructure observability nav, overview, Detecting, and flatten Observe the data ← **this PR** ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Docs update. Top layer in the observability stack. ## What is the current behavior? The section is still titled Monitoring and Debugging, with a Debugging / Monitoring split that does not match the new pages. The debugging guide is still the master layer-isolation + symptom table. Observe the data is split into “what data” vs “where to observe it,” which duplicates the source pages. ## What is the new behavior? - Section title is Observability - Overview groups Observe the data, Detect and resolve, Hire an agent, and Export - **Observe the data is flattened by source.** Logs, Metrics API, Database, Advisors, and Reports each list where to read that source. There is no separate MCP/API/CLI/Studio nav group. - **Observe vs Detecting:** Observe is the catalog (what exists, how to access it). Detecting is how to *use* those sources to pick up a Health / Security / Performance / Usage signal. Named errors skip to Diagnosing. - Studio Logs sits under Logs. Reports sits beside the other sources. - Troubleshooting stays in the global menu and also appears as Diagnosing under Detect and resolve ## Additional context This is the last PR in the stack. Together the seven PRs reconstruct the `docs/monitoring` observability IA and guide content, without shipping the troubleshooting catalog overhaul. <!-- CURSOR_AGENT_PR_BODY_END --> <div><a href="https://cursor.com/agents/bc-a3cb5ece-925b-4046-b58a-5d69e9a9d794?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-a3cb5ece-925b-4046-b58a-5d69e9a9d794&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Saxon Fletcher <SaxonF@users.noreply.github.com> Co-authored-by: Nik Richers <nik@validmind.ai> |
||
|
Saxon Fletcher
|
0bbd64743c |
docs: move inspect and advisors into observability (#49503)
<!-- CURSOR_AGENT_PR_BODY_BEGIN --> ## Stack Draft stack extracted from `docs/monitoring`. Merge bottom-up. Troubleshooting / debugging-guide rewrite is out of scope. 1. **#49503** move inspect and advisors ← **this PR** 2. #49501 split Studio logs from ClickHouse queries 3. #49500 treat reports as signal dashboards 4. #49502 add Observe the data hub 5. #49506 add agent setup components 6. #49504 add hire-an-agent templates 7. #49505 restructure observability nav and overview ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Docs update. First layer in the observability stack. ## What is the current behavior? Inspect and advisors live under Database (`/guides/database/inspect`, `/guides/database/database-advisors`). Observability readers have to leave the monitoring section to find them. ## What is the new behavior? - Moves inspect into `/guides/monitoring-and-debugging/inspect` - Adds `/guides/monitoring-and-debugging/advisors` (replaces the Database Advisors page) - Adds redirects and updates Studio/docs links so old URLs keep working - Adds both pages to the existing Monitoring nav so they are discoverable before the later IA PR ## Additional context Inspect and advisors pages render as standard MDX. Redirects cover `/docs/guides/database/inspect`, `/docs/guides/database/database-advisors`, and `/docs/guides/database/database-linter`. Debugging-guide content is unchanged except the inspect URL. ## Self-review - No leftover `/guides/database/inspect` or `/guides/database/database-advisors` links in docs guides or Studio linter/AI surfaces (historical blog posts left as-is) - Smoke test path updated to `/docs/guides/monitoring-and-debugging/advisors` <!-- CURSOR_AGENT_PR_BODY_END --> <div><a href="https://cursor.com/agents/bc-a3cb5ece-925b-4046-b58a-5d69e9a9d794?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-a3cb5ece-925b-4046-b58a-5d69e9a9d794&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a centralized Advisors guide for security and performance checks. - Updated database inspection guidance with live Postgres statistics, cache hit-rate context, and query-analysis resources. - **Documentation** - Reorganized Advisors and database inspection content under Monitoring and Debugging. - Updated navigation, cross-references, in-product help links, and CLI documentation links. - Added permanent redirects from previous documentation URLs to preserve access. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Saxon Fletcher <SaxonF@users.noreply.github.com> Co-authored-by: Nik Richers <nik@validmind.ai> |
||
|
Miranda Limonczenko
|
f10f00ae69 |
fix(e2e): install e2e-shared when CI filters to a single suite (#48960)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Bug fix. Unblocks the WWW E2E check on `master`. ## What is the current behavior? The WWW E2E job fails before running any test: ``` Error: Cannot find package '@axe-core/playwright' imported from /home/runner/_work/supabase/supabase/e2e/shared/axe.ts Error: No tests found ``` Both E2E workflows install with a filter: ``` pnpm install --frozen-lockfile --filter=e2e-www... ``` The `...` suffix pulls in a package's declared dependencies. Neither `e2e-www` nor `e2e-docs` declared `e2e-shared`; both reach it through relative imports such as `../../shared/axe.ts`, which pnpm's dependency graph cannot see. So the filter selected one project, `e2e/shared/node_modules` was never created, and Node resolving `@axe-core/playwright` from `e2e/shared/axe.ts` walked up to a root that does not carry it under pnpm's isolated layout. `e2e-docs` is broken the same way. It had not run against the shared module yet, so it has not gone red. ## What is the new behavior? `e2e-shared` is declared as a workspace dependency of both suites, so the filter installs it. | | Filter scope | Importing `e2e/shared/axe.ts` | | --- | --- | --- | | Before | 1 of 28 projects | `Cannot find package '@axe-core/playwright'` | | After | 2 of 28 projects | Imports cleanly | The lockfile gains two `link:../shared` entries and no new downloads. ## Manual Testing 1. Check out this branch and delete the shared package's modules: `rm -rf e2e/shared/node_modules` 2. Run the command CI runs: `pnpm install --frozen-lockfile --filter=e2e-www...` 3. Confirm the output reports `Scope: 2 of 28 workspace projects` and that `e2e/shared/node_modules` exists again. 4. Repeat steps 1 - 3 with `--filter=e2e-docs...`. ## Additional context Fixing only the workflow lines, by adding a second `--filter=e2e-shared`, would work as well. Declaring the dependency was chosen instead because the dependency is real and every consumer of the filter gets it, not just the two workflow files. The imports stay relative. Declaring the workspace dependency is enough to get the package installed, so no import paths change in this PR. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated end-to-end test packages to use shared testing utilities at runtime. * Improved consistency between documentation and website test suites. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
6d3a4bcc48 |
feat(www) Add scaffolding for WWW E2E tests and CI check (#48861)
Closes DOCS-1278 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature. Adds E2E test scaffolding and a CI check for the marketing site. ## What is the current behavior? Closes [FE-4047](https://linear.app/supabase/issue/FE-4047). The marketing site has no E2E coverage. Docs has a suite in `e2e/docs`, but its runner, git helpers and axe reporting are private to that package, so a second site cannot reuse them. ## What is the new behavior? * **A www suite scoped to changed content.** Changed `.mdx` files in `_blog`, `_events`, `_customers` and `_alternatives` map to the URLs they render. Pages with `disable_page_build: true` are skipped because they 404 by design. Capped at 20 pages. Enforces `heading-order` and `page-has-heading-one`, matching docs. * **`e2e/shared` The docs site is also static with similar needs. This folder shares the docs logic with www. * **A CI check that is safe to mark required.** Path scoping lives in a `Detect changed paths` step rather than a `paths:` trigger, so the check reports on every pull request instead of being skipped. `waitForVercelDocsPreview.js` becomes `waitForVercelPreview.js`, shared by both workflows. ## How the check behaves The job always reports a check run, so it is safe to mark required. Path scoping happens in a step rather than a `paths:` trigger, which would leave non-www pull requests waiting on a check that never reports. | Case | Behavior | | --- | --- | | Fork pull request adds new pages | Passes without testing. The Vercel wait is gated on `head.repo.full_name == github.repository`, so forks resolve no preview URL. The job emits a `::warning` and a job summary containing a ready-to-run `gh workflow run www-e2e.yml` command with the resolved page paths, so a maintainer can run it against the preview. | | Vercel preview times out or fails | Passes without testing. The wait step is `continue-on-error: true`, so a 900s timeout or a failed deployment leaves the URL unset and the suite skips. Vercel's own `Vercel – zone-www-dot-com` check already reports the failure. | | Draft pull request | Job does not run at all, gated at the job level on `pull_request.draft == false`. `ready_for_review` is in the trigger's `types`, so marking it ready runs the check. | | Another app changed, www untouched | Job runs and every step skips. The `www` filter matches only the four content directories, `e2e/www`, `e2e/shared`, the lockfile, and this workflow. | | Only the harness changed | Passes without testing. Scope resolves to zero pages, and the Vercel wait is additionally gated on `www_app`, so it does not wait for a preview Vercel skipped. | | No preview resolves, any reason | Skips rather than falling back to production. Production does not serve pages the pull request adds, so testing it would fail a valid change. | ### Not covered Changes to `apps/www` components and routes do not trigger this check — only the four content directories do. A follow-up can check global components such as the navigation and the footer. ## Manual testing 1. Start the site: `pnpm dev:www` 2. Run `pnpm e2e:www` with no www content changed. It should resolve zero pages and skip Playwright, not fail. 3. Touch a post, then run `pnpm e2e:www` again: `echo "" >> apps/www/_blog/2024-01-01-some-post.mdx`. The resolved `/blog/...` path should be listed before Playwright starts. 4. Run against production with no local server: `PLAYWRIGHT_BASE_URL=https://supabase.com WWW_E2E_PAGE_PATHS=/blog/postgres-language-server pnpm e2e:www` 5. Point step 4 at a page with a known heading problem. The failure should name the rule, the CSS selector and the markup. 6. Confirm docs still passes on the shared runner: `pnpm dev:docs`, then `pnpm e2e:docs` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added WWW end-to-end testing for affected content pages, including accessibility checks. * Added standard and full-site test commands, configurable preview testing, and failure reports. * Added shared utilities for page discovery, accessibility scanning, and test execution. * **Documentation** * Documented WWW test setup, coverage, debugging, CI behavior, and running checks against production or preview environments. * **Improvements** * Updated documentation test workflows to better identify affected changes and handle preview environments. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
777c02c205 |
test(docs): scan changed pages for WCAG 2.1 A/AA in warn mode (#48727)
Closes DOCS-1233 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Test coverage. The docs accessibility check now covers the full WCAG 2.1 A/AA rule set instead of two rules. **Note:** This PR tests _only_ the main article of changed pages (meaning, the content itself). A follow-up Linear issue is to address scanning the pieces outside of that: header, navigation, and interactive elements. ## What is the current behavior? The `@a11y` test in `e2e/docs` runs two axe rules against each in-scope page, `heading-order` and `page-has-heading-one`. Both already pass everywhere, so the check only guards a result we have. Nothing else in WCAG A/AA is checked. ## What is the new behavior? The same test runs the full WCAG 2.1 A/AA rule set. - **Existing debt does not block PRs.** Only the two heading rules fail. Everything else reports. - **The check stays fast.** It scans the article only and skips nine rules that cannot fire there. Scan time drops from 2405ms to 981ms. - **Findings belong to us.** Legacy mode excludes cross-origin frames. YouTube embeds were counting against us, 11 of 15 violations on one page. - **A pass carries meaning.** A 404 reports as a load failure, not an a11y bug. A page scanned before it hydrates warns instead of quietly reporting clean. ## How the findings appear The test is named `has no blocking accessibility violations`, so a failure listed by CI is always something to fix. It is not named for the full rule set, because a green check would then claim more than the check verifies. | | Rules | Where you see it | | --- | --- | --- | | Blocking | `heading-order`, `page-has-heading-one` | Test failure, so the runner reports it on the PR | | Reported | Everything else in WCAG A/AA | `::warning` annotation on the run | An annotation looks like this, on a run that still passes: ``` ::warning title=Accessibility::/docs/guides/database/functions has 1 non-blocking accessibility finding(s): frame-title (4) ``` The full axe result for each page is attached to the report as `axe-results.json`. ## Matching the Studio ratchet This follows the ESLint ratchet in `apps/studio`. That pattern warns on pre-existing debt rather than blocking on it, surfaces findings as annotations rather than PR comments, and promotes a rule to an error once its violations reach zero. The mechanism here is `ENFORCED_RULES` in `utils/axe-helpers.ts`. The two heading rules are on it because the heading-hierarchy work drove them to zero site-wide. The intent is to migrate rules into that list one at a time. Pick a rule, fix its violations, then move it into `ENFORCED_RULES` so it cannot come back. An exhaustive scan of the site groups the current backlog by root cause to sequence that work, and two fixes cover 99.1% of it. Studio keeps per-file baseline counts, which this does not. A whole-rule list is coarser, and it works here because docs violations reach zero across the site rather than per file. ## Manual testing Install the browser once, then run each step from the repo root. Every command scans production, so you do not need a local docs server. ```bash pnpm -C e2e/docs exec playwright install chromium ``` 1. Confirm a reported finding does not fail the check. ```bash DOCS_E2E_PAGE_PATHS=/docs/guides/database/functions PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y ``` Expect `1 passed`, and the `::warning` annotation above in the output. 2. Confirm the scan finds that violation. Same page, now failing on every rule. ```bash A11Y_ENFORCE_ALL=1 DOCS_E2E_PAGE_PATHS=/docs/guides/database/functions PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y ``` Expect `1 failed`, reporting `frame-title (serious, 4 node(s))`. Steps 1 and 2 together are the point of this PR. 3. Confirm the skipped rules stay skipped. ```bash A11Y_ENFORCE_ALL=1 DOCS_E2E_PAGE_PATHS=/docs/guides/getting-started/quickstarts/nextjs PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y ``` Expect `button-name (critical, 2 node(s))` and `label (critical, 2 node(s))`, and no `color-contrast`. 4. Confirm a page that does not load reports a load failure. ```bash DOCS_E2E_PAGE_PATHS=/docs/guides/does-not-exist-xyz PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y ``` Expect `Expected a successful response for /docs/guides/does-not-exist-xyz, got 404`, and no axe assertion. 5. Confirm the link checker still passes alongside the a11y test. ```bash DOCS_E2E_PAGE_PATHS=/docs/guides/auth/passwords PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs ``` Expect `3 passed`. ## Known gaps - `/docs/reference/*` is not scanned. Those routes render client-side into tens of thousands of elements, where axe exceeds its timeout and results depend on whether the scan caught the page mid-render. - Shared chrome is outside the article scope, so nav, sidebar, footer, menus, and drawers are not covered. - axe catches roughly 30-40% of WCAG issues. Keyboard navigation, focus management, and screen reader behavior still need manual testing. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
d45e0cd3d5 |
fix(docs ci): stop Docs E2E blocking pull requests it shouldn't (#48726)
Supersedes #48725, which GitHub closed when its head branch was renamed. Same commits, same diff. Fixes [DOCS-1270](https://linear.app/supabase/issue/DOCS-1270/fail-the-e2e-pipeline-if-the-docs-preview-never-loads). `Docs E2E` is a required check on `master`, so anything that turns it red blocks a merge. It had three ways of going red that had nothing to do with whether the author's docs were correct. ## Problem **1. Every troubleshooting page could fail, with nothing actionable.** Troubleshooting entries were selected by `article.prose`. That class is not unique — `apps/docs/app/not-found.tsx` renders `<article className="prose …">` too — and nothing guaranteed it matched the entry's article at all. When it missed, the link test failed with `Page article should be present` and the a11y test failed inside axe with `No elements found for include in page Context` plus a stack trace. Neither tells the author what to do. This is what DOCS-1270 actually was. The ticket describes tests running "against a preview build that was never created", but the [failing run](https://github.com/supabase/supabase/actions/runs/30949924515/job/92132543658) for #48719 shows the preview resolved fine and `response.ok()` passed — it broke at the article assertion. **Blocked:** anyone adding or editing a troubleshooting entry. **2. Fork pull requests failed for being forks.** Fork runs get no `VERCEL_TOKEN`, so no preview URL resolves, and the base-URL step fell back to `https://supabase.com`. The page paths under test can include pages the pull request *adds*, which do not exist on production, so they 404. **Blocked:** every external contributor adding a docs page, unconditionally, with no action available to them. **3. A Vercel problem failed the docs check.** `waitForVercelDocsPreview.js` throws when Vercel reports a failed deployment, omits a `target_url`, or does not post a status within 900s. The step had no `continue-on-error`, so any of those turned `Docs E2E` red. **Blocked:** any author whose pull request coincided with a Vercel incident. This is live right now — two Vercel checks on this very pull request are failing with "unable to fetch required git information", a git-integration auth error that happens before any build runs. ## Solution **1. Select on a stable, purpose-named attribute.** Add `id="sb-docs-troubleshooting-main-article"` on the troubleshooting article, mirroring `#sb-docs-guide-main-article` on guides, and select on that instead of the class. Per review feedback, a plain id doesn't say it's a test hook, so both articles also get `data-testid` with the same value — matching the convention `apps/studio` already uses with Playwright's `getByTestId` — and the e2e selectors target that attribute instead. Guides keep their `id` — `GuidesMdx.client.tsx` and `GuidesSidebar.tsx` both query it directly for the table of contents and the "copy article" fallback — and gain `data-testid` alongside it. **2 and 3. Resolve a preview or skip — never substitute production, never fail on Vercel.** The production fallback is gone. `continue-on-error: true` on the preview wait means a Vercel failure resolves no URL instead of failing the job, which lands in the same path as a fork: `should_test=false`, so Playwright is skipped and the check passes. Both cases emit a `::warning::` and a job summary with the exact `gh workflow run` command to test the preview by hand, and manual runs against a non-production base URL now send the protection bypass so that command actually works. Skipping does not let a broken preview through: `Vercel – docs` is itself a required check on `master`, so a genuine preview failure still blocks the merge — via the check that describes the real problem. ## Manual test **1. The selector matches the markup, and it needs this pull request's preview.** `data-testid` isn't deployed anywhere yet — not on production, not on any other branch — so this is the one claim in this PR that production cannot confirm. Verified directly against this branch's own Vercel preview: ```bash curl -s https://docs-git-docs-e2e-stop-false-blocks-supabase.vercel.app/docs/guides/database/overview \ | grep -o 'data-testid="[^"]*"' curl -s https://docs-git-docs-e2e-stop-false-blocks-supabase.vercel.app/docs/guides/troubleshooting/42501--permission-denied-for-table-httprequestqueue-KnozmQ \ | grep -o 'data-testid="[^"]*"' ``` Expect `data-testid="sb-docs-guide-main-article"` and `data-testid="sb-docs-troubleshooting-main-article"` respectively. Then run the suite against that same preview — expect all page/link/a11y checks to pass: ```bash PLAYWRIGHT_BASE_URL=https://docs-git-docs-e2e-stop-false-blocks-supabase.vercel.app \ DOCS_E2E_PAGE_PATHS=/docs/guides/database/overview,/docs/guides/troubleshooting/42501--permission-denied-for-table-httprequestqueue-KnozmQ \ pnpm -C e2e/docs exec playwright test --reporter=list ``` Running the same command with `PLAYWRIGHT_BASE_URL=https://supabase.com` fails both pages right now — expected until this merges, not a regression. Once merged, exercise it through the real pipeline: ```bash gh workflow run docs-e2e.yml --ref docs-e2e/stop-false-blocks \ -f base_url=<preview-url> \ -f page_paths=/docs/guides/troubleshooting/42501--permission-denied-for-table-httprequestqueue-KnozmQ ``` **2. No preview means skip, not a run against production.** Exercise the base-URL step's three paths from the repository root: ```bash export GITHUB_OUTPUT=$(mktemp) GITHUB_STEP_SUMMARY=$(mktemp) PAGE_PATHS=/docs/guides/a script=$(python3 -c "import yaml;print([s for s in yaml.safe_load(open('.github/workflows/docs-e2e.yml'))['jobs']['e2e']['steps'] if s.get('name')=='Resolve base URL'][0]['run'])") for c in "workflow_dispatch|https://supabase.com|" "pull_request||https://docs-abc.vercel.app" "pull_request||"; do IFS='|' read -r ev url dep <<< "$c" : > "$GITHUB_OUTPUT" EVENT_NAME="$ev" BASE_URL_INPUT="${url:-https://supabase.com}" DEPLOYMENT_URL="$dep" bash -c "$script" >/dev/null 2>&1 echo "$ev deployment=[${dep:-none}] -> $(tr '\n' ' ' < "$GITHUB_OUTPUT")" done tail -4 "$GITHUB_STEP_SUMMARY" ``` Expected: ``` workflow_dispatch deployment=[none] -> url=https://supabase.com use_bypass=false should_test=true pull_request deployment=[https://docs-abc.vercel.app] -> url=https://docs-abc.vercel.app use_bypass=true should_test=true pull_request deployment=[none] -> url= use_bypass=false should_test=false ``` followed by a runnable `gh workflow run docs-e2e.yml` command in the job summary. The third line covers both the fork case and the Vercel-failure case: no base URL, no test, no block. **3. A Vercel failure no longer fails the job.** `continue-on-error: true` on the wait step is what routes a throw into that third line: ```bash python3 -c " import yaml s=[x for x in yaml.safe_load(open('.github/workflows/docs-e2e.yml'))['jobs']['e2e']['steps'] if x.get('name')=='Wait for Vercel docs preview'][0] print('continue-on-error:', s.get('continue-on-error')) for n in ('Install dependencies','Install Playwright Chromium','Run docs E2E'): print(n, '->', [x for x in yaml.safe_load(open('.github/workflows/docs-e2e.yml'))['jobs']['e2e']['steps'] if x.get('name')==n][0]['if']) " ``` Expect `continue-on-error: True` and all three run steps gated on `steps.base-url.outputs.should_test == 'true'`. **Note on this pull request's own check.** The scope resolver only maps `apps/docs/content/**` to pages, and this pull request changes none, so `Docs E2E` resolves zero pages and skips — which is correct, and why the dispatch above is the real test. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved documentation preview checks so unavailable or delayed previews no longer cause unnecessary workflow failures. * Added clearer handling for manual documentation checks and missing preview deployments. * **Tests** * Improved end-to-end documentation testing reliability across preview and production environments. * Added stable targeting for the troubleshooting article to reduce test failures caused by page structure changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
8dda0c3910 |
Add heading-hierarchy a11y check to docs E2E tests (#48422)
Closes DOCS-1232 ## Problem We do not have any tests to verify that we are following a proper heading hierarchy. For a documentation site that deals in mostly static content, this test is important. Single h1 + logical heading hierarchy (h1→h2→h3, no skips) matters because screen reader users navigate by jumping between headings — broken structure breaks that navigation. Relevant: WCAG 1.3.1 Info and Relationships (Level A) — https://www.w3.org/WAI/WCAG21/Understanding/info-and-relationships.html ## Solution Add Playwright axe-core, which we plan to expand later, to test only the h1 and header-hierarchy rule. This is added to our current suite that dynamically checks only pages that are edited. ## Manual testing 1. Find a docs guide and intentionally break the header hierarchy. 2. Run `pnpm e2e:docs:a11y` and see your errors. 3. Resolve the issue and run again to see errors resolved. Ensure there is at least a line changed to see the page tested. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated accessibility checks for documentation pages. * Verified heading order and the presence of a level-one heading on each page. * Added a dedicated command to run documentation accessibility tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
52cb1c2600 |
feat(docs) Dynamically E2E test all docs-owned content (#48320)
Closes DOCS-1203 ## Problem The docs E2E workflow only ever tested one hardcoded page: the Next.js quickstart. All other docs content had no E2E coverage. ## Solution This PR expands the initial scaffolding to generalize the Next.js quickstart tests, page runs and checks local links, to all pages affecting Docs content: - Add `resolveDocsScope` (`e2e/docs/utils/resolve-docs-scope.ts`) to map changed guide and troubleshooting `.mdx` files to their `/docs/...` page paths, and to expand changed `_partials` to every page that includes them (including transitively, through partials nested inside other partials). Federated guide sections (`graphql`, `database/extensions/wrappers`, `ai/python`, `deployment/terraform`, `deployment/ci`) and reference docs stay out of scope, and resolution is capped at 20 pages to keep runtime bounded. - Replace the single `quickstarts.spec.ts` test with a generic `docs-pages.spec.ts` that loads whatever pages are resolved, asserting each renders with an `<h1>` and that its docs-owned links resolve. - Add `run-e2e-docs.ts` so `pnpm e2e:docs` resolves scope locally (from commits since `origin/master`, plus staged/unstaged changes) and skips Playwright entirely when nothing in scope changed. - Update `.github/workflows/docs-e2e.yml` to widen the trigger paths to all guides/troubleshooting/partials, resolve scope in a dedicated step, skip the rest of the job when scope is empty, and accept a `page_paths` input for manual `workflow_dispatch` runs. - Rewrite `e2e/docs/README.md` to document the new scoping behavior, the override envs (`DOCS_E2E_PAGE_PATHS`, `DOCS_E2E_BASE_REF`), and how CI uses the suite. - `pnpm e2e:docs:all` is also added to run tests on every page locally. Good for scoping issues but should not be included in CI. ## Manual testing Walk through the following steps to verify this works: - [x] `pnpm e2e:docs` from repo root resolves the expected pages for a local guide edit and can run against local dev **Note:** Challenges with testing on local in part because of the long lag for first page load. Recommendation to use a hosted URL is added to docs. - [x] Editing a shared `_partials` file resolves to every page that includes it (including through nested partials) - [x] `pnpm e2e:docs` exits cleanly with no Playwright run when no in-scope files changed - [x] `git diff --name-only ... | pnpm -C e2e/docs resolve-docs-scope` prints the expected page list for a sample diff - [x] Workflow run on a PR that only touches `e2e/docs`/workflow files skips the Playwright steps - [x] Manual `workflow_dispatch` run with `page_paths` set tests only those pages - [x] Run `pnpm e2e:docs:all` to run the suite on all docs content, which takes awhile ## Next steps After this PR merges, we have the scaffolding to add more fun tests like a11y 😁 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added scoped Docs E2E runs that target eligible doc pages based on changes, plus manual page-targeted runs and an “all eligible pages” mode. * Introduced `DOCS_E2E_PAGE_PATHS` (and updated base ref/base URL behavior) to control which pages are tested. * **Bug Fixes** * Automatically skips Playwright setup when no relevant pages are in scope; Playwright reporting now uploads only on failure. * **Documentation** * Updated the Docs E2E README with new run/CI behavior, troubleshooting notes, and commands to inspect the resolved page list. * **Tests** * Added a Docs-owned pages E2E suite; removed the Next.js quickstart E2E spec. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Miranda Limonczenko
|
b5cae478bc |
fix(docs) Add smoke test for local development without credentials (#48218)
Closes DOCS-1210 Closes DOCS-1209 #48226 needs to merge first for CI failure ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Test coverage and several small bug fixes discovered during implementation. ## What is the current behavior? Nothing verified that `pnpm run dev:docs` keeps working without private credentials. We value this command working, especially for community contributors. However, this issue can go undetected by employees at Supabase since many of us have credentials in place. We do not want this to go a week before finding and fixing like in the previous instance. ## What is the new behavior? - **New Playwright test suite**: `e2e/docs/local-smoke/no-credentials.spec.ts` boots the docs dev server with zero GitHub App/Supabase secrets and checks 5 routes covering each known failure point. - **CI**: a new `local-dev-smoke` job in `docs-tests.yml` runs this suite with no credentials configured. ## Additional bugs resolved Setting up this test exposed other issues that are fixed in this PR: - **Troubleshooting.utils.ts crash** — Unguarded Supabase call pattern, crashing every troubleshooting article. Added the same guard as previous fixes. - **Missing manifest.json** — middleware.ts statically imports public/markdown/manifest.json, which is gitignored and only generated by a build step that's skipped in local dev. On a fresh checkout it doesn't exist, so middleware fails to compile and takes down every page. Fixed by committing a placeholder [] (real builds still regenerate the full file). - **Phantom @code-hike/mdx import** — apps/docs/app/layout.tsx imported @code-hike/mdx/styles.css, but only apps/www actually declares that dependency. Worked by accident whenever both apps were installed together; broke in CI's docs-only install. Turned out to be dead code (nothing in docs actually uses code-hike), so fixed by deleting the unused imports rather than adding the dependency. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit * **New Features** * Added a credential-free “local smoke” end-to-end test suite for key documentation routes. * **Bug Fixes** * Improved troubleshooting behavior when required external service credentials are missing. * Updated federated “wrappers” documentation pages to gracefully show a fallback message when external content can’t be fetched. * **Tests** * Added a dedicated local-smoke Playwright runner and enhanced CI path-based triggering and reporting (failure-focused artifacts). * **Chores** * Refined docs workflow path filters and adjusted docs markdown manifest/ignore rules for generated content. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Miranda Limonczenko
|
9199aad57e |
feat(docs) Add scaffolding and CI/CD step for Docs Playwright (#48120)
Closes DOCS-1197 ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## Problem We do not have any E2E testing established. ## Solution This PR creates an ultra-lean starting place for Docs Playwright: - A CI/CD step that skips on draft and relies on Preview for testing - One simple broken link check for one page The goal: - Playwright is implemented where we want it, with an architecture we want, with set-up steps we can build from The anti-goal of this PR: - We have meaningful tests running ## CI/CD steps <img width="1191" height="72" alt="Screenshot 2026-07-21 at 10 17 06 AM" src="https://github.com/user-attachments/assets/eeb2454c-d864-4574-a050-ce39bb3f083f" /> 1. Checkout a thin slice of the repo (`apps/docs`, `packages`, `patches`). 2. Wait for the Vercel **docs** preview for that commit SHA. 3. Use that preview URL as `PLAYWRIGHT_BASE_URL`. 4. Install Node deps and Chromium. 5. Run `pnpm run e2e:docs` (`--grep @quickstart`). 6. If anything fails, upload the HTML report + traces. Manual runs skip the Vercel wait and default to `https://supabase.com` (or whatever URL you enter), then run the full suite (`pnpm run e2e`). ## What the test checks Because this PR is scaffolding, it is doing something very basic: 1. Opens `/docs/guides/getting-started/quickstarts/nextjs` only if a connected file was edited in CI/CD step 2. Asserts the page loaded and the H1 is visible. 3. Collects docs-owned `/docs/**` links from `#sb-docs-guide-main-article`. 4. HTTP-checks each link (no full navigation) and soft-fails so every broken link is reported. Config keeps it cheap: Chromium only, 1 worker, 2 CI retries, failure screenshots/traces. ## Docs vs Studio/Dashboard The setup of Docs Playwright differs from Studio. | | Docs E2E | Studio E2E | |---|---|---| | Location |`e2e/docs/` | `e2e/studio/` | | What it tests | One published docs page + its links | Many Studio UI flows (tables, auth, storage, …) | | Where the app runs | Already-deployed **Vercel preview** | Built and started **on the runner** | | Backend needed | None | Local Supabase via Docker | | Path filtering | Native `on.pull_request.paths` (skip whole workflow) | `dorny/paths-filter` after checkout (workflow starts, heavy steps gated) | | Parallelism | 1 worker, no shards | Matrix of frameworks × 2 shards | | Retries | 2 in CI | 5 in CI | | Reports | HTML report on failure | Blob reports per shard → merge → PR comment | | Draft handling | Explicit draft skip | No draft skip today | | Manual broader run | Yes (`workflow_dispatch`) | No | The big conceptual difference: **Studio owns the environment** (build Studio, start Supabase, hit `localhost`). **Docs borrows Vercel’s preview** and only asks “does this page and its docs links work on the deployed site?” ## Docs architecture justification The docs architecture is deliberately lightweight because docs are **static, published content served by Vercel**, not an interactive app with a backend. That single fact justifies every difference: - **Borrow the Vercel preview instead of building on the runner.** The preview is already the exact artifact users will see, and Vercel builds it for free on every PR. Rebuilding docs on the runner would duplicate that work and risk testing something different from what ships. Studio, by contrast, needs a running app plus a local Supabase, so it *has* to own its environment. - **No backend.** Docs pages don't need a database or auth to render, so there's nothing to spin up. This is what keeps the job cheap enough to run per-PR. - **Native `paths` filtering.** Since the job is cheap and self-contained, an all-or-nothing skip at the workflow level is sufficient—no need for `dorny/paths-filter` to gate expensive setup steps mid-run like Studio does. - **Low parallelism and modest retries.** One page and its links is a tiny surface, so 1 worker is plenty and there's no sharding to coordinate. Retries exist only to absorb transient network flakiness against a live URL, hence 2 rather than Studio's 5 (which also cushions a heavier, stateful environment). - **Non-blocking + draft skip + manual dispatch.** As initial scaffolding checking link health on a deployed site, it should inform rather than gate merges, avoid burning minutes on drafts, and still be runnable on demand against production. In short: **Studio owns its environment because it must; docs borrows Vercel's preview because it can.** The scope is intentionally minimal today. ## Testing 1. Break a docs-owned link in the Next.js quickstart. 1. Follow README instructions to set up and run e2e docs test. 1. Confirm the suite fails. 1. Restore the broken link and re-run. 1. Confirm the suite **passes** (`1 passed`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary - **New Features** - Added a GitHub Actions workflow to run Playwright docs end-to-end tests on PRs and via manual dispatch (with optional base URL), including docs-preview waiting and concurrency cancellation. - **Documentation** - Added `e2e/docs` README with setup, how to run the suite (including UI/debug and single-spec), and how base URL selection works. - **Tests** - Added a quickstarts E2E spec that validates the page and soft-checks docs-owned links resolve. - **Chores** - Added shared Playwright configuration/package scripts and an `e2e/docs` `.gitignore` for test outputs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
