## Problem
Since the upgrade to NextJS 16.1.6, many e2e tests run fails with
timeout, probably when building or starting the studio
## Solution
- [x] Add dedicated steps for supabase setup and NextJS build to ease
debugging
- [x] Fix flakyness
## Problem
e2e tests are still flaky and not as fast as they could be
## Solution
- [x] Reset supabase instance: this makes the database visualiser tests
faster as there are less tables to screenshot
- [x] Improve the single file setup utilities so that they never block
local tests by cleaning them up before starting
- [x] Disable animations while running the tests (less time waiting for
animations to complete
- [x] Add utility functions that help reproducing flaky tests locally
* attempt to make table editor tests less flaky
* updated race conditions for table editor
* updated playwright config with optimized launch options
* reduce workers
* updated timeouts for tests
* updated config to have a global timeout of 20 seconds
* updated to be visible
Table editor tests were being run serially because they would otherwise
stomp all over each other in setup. But this made them really slow. This
PR changes them to run in parallel, and makes sure the setup step runs
only once by using a file lock.
Also cleans up miscellaneous other sources of flakiness in tests.
Co-authored-by: Ali Waseem <waseema393@gmail.com>
* a new hope
* run tests in ci against cli mode
* summary
* try vercel action to run e2e against studio self hosted preview
* believe
* debug
* gh pages artifact
* test
* rm pages step
* fix automation bypass missing
* continue on error
* only install necessary deps for CI
* fix bypass
* remove
* fail job if test fails
* disable customer query if is_platform false
* vercel check
* fix var name, make comment update instead
* check bypass on runtime
* add env var
* fix tests going to project ref instead of default
* fix
* better dates in comment
* Update E2E test workflow to include flaky test detection and improve summary output
* fix
* fix dumb mistake
* chore: add e2e test for sql editor snippets
* trigger build
* chore: fix failing test and add rls
---------
Co-authored-by: Terry Sutton <saltcod@gmail.com>
* add new e2e folder
* add local supabase and confitional storage
* fix e2e selfhosted
* update actions
* add correct e2e folder
* fix e2e actions
* fix action project ids
* fix permissions
* fix script
* fix playwright install
* playwright root
* pnpm i
* fix api rul
* add env docs
* update run script
* only install deps for e2e
* use same dep
* only install deps for tests
* upd lockfile
* use official vercel integration
* use vercel cli
* remove old folder
* fix script
* rm filter
* rename e2e studio package
* fix install browsers
* add polling for vercel build
* use vercel-preview-url package
* undo actions
* rename ci env to ci
* chore:add rls check and make playwright test less flakey (#35348)
* update ci action
* fix paths
* fix browser install
* run ci against staging
* try caching builds
* fix envs
* fix env check
* fix sign in
* fix sign in url
* fix envs and url
* fix caching
* fix race condition in sign in page
* fix race condition in sign in page
* add check to see if being redirected
* fix caching, check IS_PLATFORM var
* log is_platform
* try vercel build
* fix vercel project id
* fix path
* add temp vercel.json file
* fix paths
* undo project id stuff
* rm cwd
* fix path
* fix paths again
* fix path
* fix base url
* try different fix
* fix config base url
* fix base studio url issues
* retain video on fails
* Update e2e/studio/README.md
Co-authored-by: Copple <10214025+kiwicopple@users.noreply.github.com>
* Update e2e/studio/README.md
Co-authored-by: Copple <10214025+kiwicopple@users.noreply.github.com>
* fix env file naming
* undo caching
* rm old tests folder
* fix readme scripts
* rm vercel deploy for now, just run build locally
* fix url
* fix build script
* fix is_platform
* fix stuck studio start
* fix env vars
* retain network and logs on fail for better debugging
* add apiurl env
* back to vercel
* disable catpcha
* fix test
* update environment configuration to remove default URLs for CI and streamline API base URL handling
* fix typeerr
* fix urls in home.spec
* fix urls in logs.spec
* fix urls in sqleditor spec
* fix table editor spec
* add tourl util
* use staging api in ci
* re add base url env var
* fix url in projects page
* fix url in sql editor spec
* fix sign in not waiting for cookies omfg
* fix env var name
* fix sql-editor test
* simplify table removal
* add opt out telemetry step
* fix logs tests
* fix table editor spec
* remove flaky steps from table editor tests
* use vercel deployment events instead of build
* add studio check
* fix condition
* debug event
* rm if
* trigger deploy
* undo ac
* make opt out button step optional, some envs dont hav eit
* use testid for sql run button
* use id instaed of timestamp in logs tests
* empty
* rm retries
* up glbal timeout
* chore: fix failing sql-editor playwright test (#35767)
* chore: fix failing sql-editor playwright test
* chore: minor fixes
* Chore/update playwright config (#35826)
chore: update playwright config
* rm supabase project from e2e tests
* refactor and simplify environments
* fix sql editor test
* fix ci env vars
* fix
* fix on windows
* update readme
* add playwright install script to readme
* rm turbopack trace flag
* npm to pnpm for scripts
* delete ivan lines
---------
Co-authored-by: Michael Ong <minghao_3728@hotmail.com>
Co-authored-by: Copple <10214025+kiwicopple@users.noreply.github.com>
