lywsvip/supabase
1
0
Fork 0
mirror of https://github.com/supabase/supabase.git synced 2026-09-07 02:20:52 +08:00
Code Issues Packages Projects Releases Wiki Activity
37,661 Commits 1,672 Branches 60 Tags
3c305148186c311db7ec03f2e17e91ff94de1200
Commit Graph

3 Commits

Author SHA1 Message Date
Miranda Limonczenko
52cb1c2600 feat(docs) Dynamically E2E test all docs-owned content (#48320) 
Closes DOCS-1203

## Problem

The docs E2E workflow only ever tested one hardcoded page: the Next.js
quickstart. All other docs content had no E2E coverage.

## Solution

This PR expands the initial scaffolding to generalize the Next.js
quickstart tests, page runs and checks local links, to all pages
affecting Docs content:


- Add `resolveDocsScope` (`e2e/docs/utils/resolve-docs-scope.ts`) to map
changed guide and troubleshooting `.mdx` files to their `/docs/...` page
paths, and to expand changed `_partials` to every page that includes
them (including transitively, through partials nested inside other
partials). Federated guide sections (`graphql`,
`database/extensions/wrappers`, `ai/python`, `deployment/terraform`,
`deployment/ci`) and reference docs stay out of scope, and resolution is
capped at 20 pages to keep runtime bounded.
- Replace the single `quickstarts.spec.ts` test with a generic
`docs-pages.spec.ts` that loads whatever pages are resolved, asserting
each renders with an `<h1>` and that its docs-owned links resolve.
- Add `run-e2e-docs.ts` so `pnpm e2e:docs` resolves scope locally (from
commits since `origin/master`, plus staged/unstaged changes) and skips
Playwright entirely when nothing in scope changed.
- Update `.github/workflows/docs-e2e.yml` to widen the trigger paths to
all guides/troubleshooting/partials, resolve scope in a dedicated step,
skip the rest of the job when scope is empty, and accept a `page_paths`
input for manual `workflow_dispatch` runs.
- Rewrite `e2e/docs/README.md` to document the new scoping behavior, the
override envs (`DOCS_E2E_PAGE_PATHS`, `DOCS_E2E_BASE_REF`), and how CI
uses the suite.
- `pnpm e2e:docs:all` is also added to run tests on every page locally.
Good for scoping issues but should not be included in CI.

## Manual testing

Walk through the following steps to verify this works:

- [x] `pnpm e2e:docs` from repo root resolves the expected pages for a
local guide edit and can run against local dev
**Note:** Challenges with testing on local in part because of the long
lag for first page load. Recommendation to use a hosted URL is added to
docs.
- [x] Editing a shared `_partials` file resolves to every page that
includes it (including through nested partials)
- [x] `pnpm e2e:docs` exits cleanly with no Playwright run when no
in-scope files changed
- [x] `git diff --name-only ... | pnpm -C e2e/docs resolve-docs-scope`
prints the expected page list for a sample diff
- [x] Workflow run on a PR that only touches `e2e/docs`/workflow files
skips the Playwright steps
- [x] Manual `workflow_dispatch` run with `page_paths` set tests only
those pages
- [x] Run `pnpm e2e:docs:all` to run the suite on all docs content,
which takes awhile

## Next steps

After this PR merges, we have the scaffolding to add more fun tests like
a11y 😁

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added scoped Docs E2E runs that target eligible doc pages based on
changes, plus manual page-targeted runs and an “all eligible pages”
mode.
* Introduced `DOCS_E2E_PAGE_PATHS` (and updated base ref/base URL
behavior) to control which pages are tested.
* **Bug Fixes**
* Automatically skips Playwright setup when no relevant pages are in
scope; Playwright reporting now uploads only on failure.
* **Documentation**
* Updated the Docs E2E README with new run/CI behavior, troubleshooting
notes, and commands to inspect the resolved page list.
* **Tests**
* Added a Docs-owned pages E2E suite; removed the Next.js quickstart E2E
spec.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-07-28 00:04:58 +00:00
Miranda Limonczenko
b5cae478bc fix(docs) Add smoke test for local development without credentials (#48218) 
Closes DOCS-1210
Closes DOCS-1209

#48226 needs to merge first for CI failure

## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Test coverage and several small bug fixes discovered during
implementation.

## What is the current behavior?

Nothing verified that `pnpm run dev:docs` keeps working without private
credentials.
We value this command working, especially for community contributors.

However, this issue can go undetected by employees at Supabase since
many of us have credentials in place. We do not want this to go a week
before finding and fixing like in the previous instance.

## What is the new behavior?

- **New Playwright test suite**:
`e2e/docs/local-smoke/no-credentials.spec.ts` boots the docs dev server
with zero GitHub App/Supabase secrets and checks 5 routes covering each
known failure point.
- **CI**: a new `local-dev-smoke` job in `docs-tests.yml` runs this
suite with no credentials configured.


## Additional bugs resolved

Setting up this test exposed other issues that are fixed in this PR:

- **Troubleshooting.utils.ts crash** — Unguarded Supabase call pattern,
crashing every troubleshooting article. Added the same guard as previous
fixes.
- **Missing manifest.json** — middleware.ts statically imports
public/markdown/manifest.json, which is gitignored and only generated by
a build step that's skipped in local dev. On a fresh checkout it doesn't
exist, so middleware fails to compile and takes down every page. Fixed
by committing a placeholder [] (real builds still regenerate the full
file).
- **Phantom @code-hike/mdx import** — apps/docs/app/layout.tsx imported
@code-hike/mdx/styles.css, but only apps/www actually declares that
dependency. Worked by accident whenever both apps were installed
together; broke in CI's docs-only install. Turned out to be dead code
(nothing in docs actually uses code-hike), so fixed by deleting the
unused imports rather than adding the dependency.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* Added a credential-free “local smoke” end-to-end test suite for key
documentation routes.

* **Bug Fixes**
* Improved troubleshooting behavior when required external service
credentials are missing.
* Updated federated “wrappers” documentation pages to gracefully show a
fallback message when external content can’t be fetched.

* **Tests**
* Added a dedicated local-smoke Playwright runner and enhanced CI
path-based triggering and reporting (failure-focused artifacts).

* **Chores**
* Refined docs workflow path filters and adjusted docs markdown
manifest/ignore rules for generated content.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-24 10:29:12 -07:00
Miranda Limonczenko
9199aad57e feat(docs) Add scaffolding and CI/CD step for Docs Playwright (#48120) 
Closes DOCS-1197



## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## Problem

We do not have any E2E testing established. 

## Solution

This PR creates an ultra-lean starting place for Docs Playwright:

- A CI/CD step that skips on draft and relies on Preview for testing
- One simple broken link check for one page

The goal: 
- Playwright is implemented where we want it, with an architecture we
want, with set-up steps we can build from

The anti-goal of this PR:
- We have meaningful tests running

## CI/CD steps

<img width="1191" height="72" alt="Screenshot 2026-07-21 at 10 17 06 AM"
src="https://github.com/user-attachments/assets/eeb2454c-d864-4574-a050-ce39bb3f083f"
/>

1. Checkout a thin slice of the repo (`apps/docs`, `packages`,
`patches`).
2. Wait for the Vercel **docs** preview for that commit SHA.
3. Use that preview URL as `PLAYWRIGHT_BASE_URL`.
4. Install Node deps and Chromium.
5. Run `pnpm run e2e:docs` (`--grep @quickstart`).
6. If anything fails, upload the HTML report + traces.

Manual runs skip the Vercel wait and default to `https://supabase.com`
(or whatever URL you enter), then run the full suite (`pnpm run e2e`).

## What the test checks

Because this PR is scaffolding, it is doing something very basic:

1. Opens `/docs/guides/getting-started/quickstarts/nextjs` only if a
connected file was edited in CI/CD step
2. Asserts the page loaded and the H1 is visible.
3. Collects docs-owned `/docs/**` links from
`#sb-docs-guide-main-article`.
4. HTTP-checks each link (no full navigation) and soft-fails so every
broken link is reported.

Config keeps it cheap: Chromium only, 1 worker, 2 CI retries, failure
screenshots/traces.


## Docs vs Studio/Dashboard

The setup of Docs Playwright differs from Studio.

| | Docs E2E | Studio E2E |
|---|---|---|
| Location |`e2e/docs/` | `e2e/studio/` |
| What it tests | One published docs page + its links | Many Studio UI
flows (tables, auth, storage, …) |
| Where the app runs | Already-deployed **Vercel preview** | Built and
started **on the runner** |
| Backend needed | None | Local Supabase via Docker |
| Path filtering | Native `on.pull_request.paths` (skip whole workflow)
| `dorny/paths-filter` after checkout (workflow starts, heavy steps
gated) |
| Parallelism | 1 worker, no shards | Matrix of frameworks × 2 shards |
| Retries | 2 in CI | 5 in CI |
| Reports | HTML report on failure | Blob reports per shard → merge → PR
comment |
| Draft handling | Explicit draft skip | No draft skip today |
| Manual broader run | Yes (`workflow_dispatch`) | No |

The big conceptual difference: **Studio owns the environment** (build
Studio, start Supabase, hit `localhost`). **Docs borrows Vercel’s
preview** and only asks “does this page and its docs links work on the
deployed site?”

## Docs architecture justification

The docs architecture is deliberately lightweight because docs are
**static, published content served by Vercel**, not an interactive app
with a backend. That single fact justifies every difference:

- **Borrow the Vercel preview instead of building on the runner.** The
preview is already the exact artifact users will see, and Vercel builds
it for free on every PR. Rebuilding docs on the runner would duplicate
that work and risk testing something different from what ships. Studio,
by contrast, needs a running app plus a local Supabase, so it *has* to
own its environment.

- **No backend.** Docs pages don't need a database or auth to render, so
there's nothing to spin up. This is what keeps the job cheap enough to
run per-PR.

- **Native `paths` filtering.** Since the job is cheap and
self-contained, an all-or-nothing skip at the workflow level is
sufficient—no need for `dorny/paths-filter` to gate expensive setup
steps mid-run like Studio does.

- **Low parallelism and modest retries.** One page and its links is a
tiny surface, so 1 worker is plenty and there's no sharding to
coordinate. Retries exist only to absorb transient network flakiness
against a live URL, hence 2 rather than Studio's 5 (which also cushions
a heavier, stateful environment).

- **Non-blocking + draft skip + manual dispatch.** As initial
scaffolding checking link health on a deployed site, it should inform
rather than gate merges, avoid burning minutes on drafts, and still be
runnable on demand against production.

In short: **Studio owns its environment because it must; docs borrows
Vercel's preview because it can.** The scope is intentionally minimal
today.

## Testing

1. Break a docs-owned link in the Next.js quickstart.
1. Follow README instructions to set up and run e2e docs test.
1. Confirm the suite fails.
1. Restore the broken link and re-run.
1. Confirm the suite **passes** (`1 passed`).



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary

- **New Features**
- Added a GitHub Actions workflow to run Playwright docs end-to-end
tests on PRs and via manual dispatch (with optional base URL), including
docs-preview waiting and concurrency cancellation.
- **Documentation**
- Added `e2e/docs` README with setup, how to run the suite (including
UI/debug and single-spec), and how base URL selection works.
- **Tests**
- Added a quickstarts E2E spec that validates the page and soft-checks
docs-owned links resolve.
- **Chores**
- Added shared Playwright configuration/package scripts and an
`e2e/docs` `.gitignore` for test outputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-21 14:59:10 -07:00