diff --git a/.claude/skills/e2e-studio-tests/SKILL.md b/.claude/skills/e2e-studio-tests/SKILL.md new file mode 100644 index 00000000000..6f9797f720d --- /dev/null +++ b/.claude/skills/e2e-studio-tests/SKILL.md @@ -0,0 +1,233 @@ +--- +name: e2e-studio-tests +description: Run e2e tests in the Studio app. Use when asked to run e2e tests, run studio tests, playwright tests, or test the feature. +--- + +# E2E Studio Tests + +Run Playwright end-to-end tests for the Studio application. + +## Running Tests + +Tests must be run from the `e2e/studio` directory: + +```bash +cd e2e/studio && pnpm run e2e +``` + +### Run specific file + +```bash +cd e2e/studio && pnpm run e2e -- features/cron-jobs.spec.ts +``` + +### Run with grep filter + +```bash +cd e2e/studio && pnpm run e2e -- --grep "test name pattern" +``` + +### UI mode for debugging + +```bash +cd e2e/studio && pnpm run e2e -- --ui +``` + +## Environment Setup + +- Tests auto-start Supabase local containers via web server config +- Self-hosted mode (`IS_PLATFORM=false`) runs tests in parallel (3 workers) +- No manual setup needed for self-hosted tests + +## Test File Structure + +- Tests are in `e2e/studio/features/*.spec.ts` +- Use custom test utility: `import { test } from '../utils/test.js'` +- Test fixtures provide `page`, `ref`, and other helpers + +## Common Patterns + +Wait for elements with generous timeouts: + +```typescript +await expect(locator).toBeVisible({ timeout: 30000 }) +``` + +Add messages to expects for debugging: + +```typescript +await expect(locator).toBeVisible({ timeout: 30000 }, 'Element should be visible after page load') +``` + +Use serial mode for tests sharing database state: + +```typescript +test.describe.configure({ mode: 'serial' }) +``` + +## Writing Robust Selectors + +### Selector priority (best to worst) + +1. **`getByRole` with accessible name** - Most robust, tests accessibility + ```typescript + page.getByRole('button', { name: 'Save' }) + page.getByRole('button', { name: 'Configure API privileges' }) + ``` + +2. **`getByTestId`** - Stable, explicit test hooks + ```typescript + page.getByTestId('table-editor-side-panel') + ``` + +3. **`getByText` with exact match** - Good for unique text + ```typescript + page.getByText('Data API Access', { exact: true }) + ``` + +4. **`locator` with CSS** - Use sparingly, more fragile + ```typescript + page.locator('[data-state="open"]') + ``` + +### Patterns to avoid + +- **XPath selectors** - Fragile to DOM changes + ```typescript + // BAD + locator('xpath=ancestor::div[contains(@class, "space-y")]') + ``` + +- **Parent traversal with `locator('..')`** - Breaks when structure changes + ```typescript + // BAD + element.locator('..').getByRole('button') + ``` + +- **Broad `filter({ hasText })` on generic elements** - May match multiple elements + ```typescript + // BAD - popover may have more than one combobox + // Could consider scoping down the container or filtering the combobox more specifically + popover.getByRole('combobox') + ``` + +### Add accessible labels to components + +When a component lacks a good accessible name, add one in the source code: + +```tsx +// In the React component + +``` + +Then use it in tests: +```typescript +page.getByRole('button', { name: 'Configure API privileges' }) +``` + +### Narrowing search scope + +Scope selectors to specific containers to avoid matching wrong elements: + +```typescript +// Good - scoped to side panel +const sidePanel = page.getByTestId('table-editor-side-panel') +const toggle = sidePanel.getByRole('switch') + +// Good - find unique element, then scope from there +const popover = page.locator('[data-radix-popper-content-wrapper]') +const roleSection = popover.getByText('Anonymous (anon)', { exact: true }) +``` + +## Avoiding `waitForTimeout` + +Never use `waitForTimeout` - always wait for something specific: + +```typescript +// BAD +await page.waitForTimeout(1000) + +// GOOD - wait for UI element +await expect(page.getByText('Success')).toBeVisible() + +// GOOD - wait for API response +const apiPromise = waitForApiResponse(page, 'pg-meta', ref, 'query?key=table-create') +await saveButton.click() +await apiPromise + +// GOOD - wait for toast indicating operation complete +await expect(page.getByText('Table created successfully')).toBeVisible({ timeout: 15000 }) +``` + +## Avoiding `force: true` on clicks + +Instead of forcing clicks on hidden elements, make them visible first: + +```typescript +// BAD +await menuButton.click({ force: true }) + +// GOOD - hover to reveal, then click +await tableRow.hover() +await expect(menuButton).toBeVisible() +await menuButton.click() +``` + +## Debugging + +### View trace + +```bash +cd e2e/studio && pnpm exec playwright show-trace +``` + +### View HTML report + +```bash +cd e2e/studio && pnpm exec playwright show-report +``` + +### Error context + +Error context files are saved in the `test-results/` directory. + +### Playwright MCP tools + +Use Playwright MCP tools to inspect UI when debugging locally. + +## CI vs Local Development + +The key difference is **cold start vs warm state**: + +### CI (cold start) + +Tests run from a blank database slate. Each test run resets the database and starts fresh containers. Extensions like pg_cron are NOT enabled by default. + +### Local dev with `pnpm dev:studio-local` + +When debugging with a running dev server, the database may already have state from previous runs (extensions enabled, test data present). + +## Handling Cold Start Bugs + +Tests that work locally but fail in CI often have assumptions about existing state. + +### Common issues + +1. Extension not enabled (must enable in test setup) +2. Race conditions when parallel tests try to modify shared state (use `test.describe.configure({ mode: 'serial' })`) +3. Locators matching wrong elements because the page structure differs when state isn't set up + +### Reproducing CI behavior locally + +The test framework automatically resets the database when running `pnpm run e2e`. This matches CI behavior. + +If using `pnpm dev:studio-local` for Playwright MCP debugging, remember the state differs from CI. + +## Debugging Workflow for CI Failures + +1. First, run the test locally with `pnpm run e2e -- features/.spec.ts` (cold start) +2. Check error context in `test-results/` directory +3. If you need to inspect UI state, start `pnpm dev:studio-local` and use Playwright MCP tools +4. Remember: what you see in the dev server may have state that doesn't exist in CI diff --git a/.gitignore b/.gitignore index 5d4494d1cbd..4ba0158ba06 100644 --- a/.gitignore +++ b/.gitignore @@ -120,6 +120,8 @@ next-env.d.ts .claude/* !.claude/settings.json !.claude/scripts/ +!.claude/skills/ +.claude/skills/me-* CLAUDE.md #include template .env file for docker-compose