Files
supabase/apps/reference/docs/guides/cli/environments.mdx

418 lines
13 KiB
Plaintext

---
id: managing-environments
title: Managing Environments
description: How to deploy Supabase schema changes with a CI / CD pipeline.
---
import Tabs from '@theme/Tabs'
import TabItem from '@theme/TabItem'
## Overview
The Supabase CLI provides the tools you need to manage multiple environments.
This guide shows you how to set up your local Supabase development environment that integrates with GitHub Actions to automatically
test and release schema changes to staging and production Supabase projects.
To get started:
- [Install the Supabase CLI](/docs/guides/cli)
- Create a [Supabase project](https://app.supabase.com) or use an existing one
- Initialize a local Git repository
## Set up a local environment
The first step is to set up your local repository with the Supabase CLI:
```bash
supabase init
```
You should see a new `supabase` directory. Then you need to link your local repository with your Supabase project:
```bash
supabase login
supabase link --project-ref $PROJECT_ID
```
You can get your `$PROJECT_ID` from your project's dashboard URL:
```
https://app.supabase.com/project/<project-id>
```
If you're using an existing Supabase project, you might have made schema changes through the Dashboard.
Run the following command to pull these changes before making local schema changes from the CLI:
```sql
supabase db remote commit
```
This command creates a new migration in `supabase/migrations/<timestamp>_remote_commit.sql` which reflects the schema changes you have made previously.
Now commit your local changes to Git and run the local development setup:
```bash
git add .
git commit -m "init supabase"
supabase start
```
You are now ready to develop schema changes locally and create your first migration.
## Create a new migration
There are two ways to make schema changes:
1. Manual migration: Write DDL statements manually into a migration file
1. Auto schema diff: Make changes through Studio UI and auto generate a schema diff
### Manual migration
<!-- ![Manual migration](/img/guides/cli/diff-manual.png) -->
Create a new migration script by running:
```bash
supabase migration new new_employee
```
You should see a new file created: `supabase/migrations/<timestamp>_new_employee.sql`. You can then write SQL statements in this script using a text editor:
```sql
create table public.employees (
id integer primary key generated always as identity,
name text
);
```
Apply the new migration to your local database:
```bash
supabase db reset
```
This command recreates your local database from scratch and applies all migration scripts under `supabase/migrations` directory. Now your local database is up to date.
:::tip
The new migration command also supports stdin as input.
This allows you to pipe in an existing script from another file or stdout:
`supabase migration new new_employee < create_employees_table.sql`
:::
### Auto schema diff
Unlike manual migrations, auto schema diff creates a new migration script from changes **already** applied to your local database.
<!-- ![Auto schema diff](/img/guides/cli/diff-auto.png) -->
Create an `employees` table under the `public` schema using Studio UI, accessible at [localhost:54323](http://localhost:54323/) by default.
Next, generate a schema diff by running the following command:
```bash
supabase db diff -f new_employee
```
You should see that a new file `supabase/migrations/<timestamp>_new_employee.sql` is created. Open the file and verify that the generated DDL statements are the same as below.
```sql
-- This script was generated by the Schema Diff utility in pgAdmin 4
-- For the circular dependencies, the order in which Schema Diff writes the objects is not very sophisticated
-- and may require manual changes to the script to ensure changes are applied in the correct order.
-- Please report an issue for any failure with the reproduction steps.
CREATE TABLE IF NOT EXISTS public.employees
(
id integer NOT NULL GENERATED ALWAYS AS IDENTITY ( INCREMENT 1 START 1 MINVALUE 1 MAXVALUE 2147483647 CACHE 1 ),
name text COLLATE pg_catalog."default",
CONSTRAINT employees_pkey PRIMARY KEY (id)
)
TABLESPACE pg_default;
ALTER TABLE IF EXISTS public.employees
OWNER to postgres;
GRANT ALL ON TABLE public.employees TO anon;
GRANT ALL ON TABLE public.employees TO authenticated;
GRANT ALL ON TABLE public.employees TO postgres;
GRANT ALL ON TABLE public.employees TO service_role;
```
You may notice that the auto-generated migration script is more verbose than the manually written one.
This is because the default schema diff tool does not account for default privileges added by the initial schema.
Commit the new migration script to git and you are ready to deploy.
:::tip
Alternatively, you may pass in the `--use-migra` experimental flag to generate a more concise migration using [migra](https://github.com/djrobstep/migra).
Without the `-f` file flag, the output is written to stdout by default.
`supabase db diff --use-migra`
:::
## Deploy a migration
In a production environment, we recommend using a CI/CD pipeline to deploy new migrations with GitHub Actions rather than deploying from your local machine.
![Deploy migration](/img/guides/cli/cicd-github.png)
This example uses two Supabase projects, one for production and one for staging.
Prepare your environments by:
- Creating separate Supabase projects for staging and production
- Pushing your git repository to GitHub and enabling GitHub Actions
:::caution
You need a _new_ project for staging. A project which has already been modified to reflect the production project's schema can't be used because the CLI would reapply these changes.
:::
### Configure GitHub Actions
The Supabase CLI requires a few environment variables to run in non-interactive mode.
- `SUPABASE_ACCESS_TOKEN` is your personal access token
- `SUPABASE_DB_PASSWORD` is your project specific database password
We recommend adding these as [encrypted secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets) to your GitHub Actions runners.
Create the following files inside the `.github/workflows` directory:
<Tabs
defaultValue="CI"
values={[
{label: 'ci.yaml', value: 'CI'},
{label: 'staging.yaml', value: 'STAGING'},
{label: 'production.yaml', value: 'PRODUCTION'},
]}>
<TabItem value="CI">
```yaml title=".github/workflows/ci.yml"
name: CI
on:
pull_request:
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: supabase/setup-cli@v1
with:
version: 1.0.0
- name: Start Supabase local development setup
run: supabase start
- name: Verify generated types are up-to-date
run: |
supabase gen types typescript --local > types.ts
if [ "$(git diff --ignore-space-at-eol types.ts | wc -l)" -gt "0" ]; then
echo "Detected uncommitted changes after build. See status below:"
git diff
exit 1
fi
```
</TabItem>
<TabItem value="STAGING">
```yaml title=".github/workflows/staging.yml"
name: Deploy Migrations to Staging
on:
push:
branches:
- develop
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-22.04
env:
SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
SUPABASE_DB_PASSWORD: ${{ secrets.STAGING_DB_PASSWORD }}
STAGING_PROJECT_ID: abcdefghijklmnopqrst
steps:
- uses: actions/checkout@v3
- uses: supabase/setup-cli@v1
with:
version: 1.0.0
- run: |
supabase link --project-ref $STAGING_PROJECT_ID
supabase db push
```
</TabItem>
<TabItem value="PRODUCTION">
```yaml title=".github/workflows/production.yml"
name: Deploy Migrations to Production
on:
push:
branches:
- main
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-22.04
env:
SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
SUPABASE_DB_PASSWORD: ${{ secrets.PRODUCTION_DB_PASSWORD }}
PRODUCTION_PROJECT_ID: abcdefghijklmnopqrst
steps:
- uses: actions/checkout@v3
- uses: supabase/setup-cli@v1
with:
version: 1.0.0
- run: |
supabase link --project-ref $PRODUCTION_PROJECT_ID
supabase db push
```
</TabItem>
</Tabs>
The full example code is available in the [demo repository](https://github.com/supabase/supabase-action-example).
Commit these files to git and push to your `main` branch on GitHub. Update these environment variables to match your Supabase projects:
- `SUPABASE_ACCESS_TOKEN`
- `PRODUCTION_PROJECT_ID`
- `PRODUCTION_DB_PASSWORD`
- `STAGING_PROJECT_ID`
- `STAGING_DB_PASSWORD`
When configured correctly, your repository will have CI and Release workflows that trigger on new commits pushed to `main` and `develop` branches.
![Correctly configured repo](/img/guides/cli/ci-main.png)
### Open a PR with new migration
Follow the [migration steps](#create-a-new-migration) to create a `supabase/migrations/<timestamp>_new_employee.sql` file.
Checkout a new branch `feat/employee` from `develop` , commit the migration file, and push to GitHub.
```bash
git checkout -b feat/employee
git add supabase/migrations/<timestamp>_new_employee.sql
git commit -m "Add employee table"
git push --set-upstream origin feat/employee
```
Open a PR from `feat/employee` to the `develop` branch to see that the CI workflow has been triggered.
Once the test error is resolved, merge this PR and watch the deployment in action.
### Release to production
After verifying your staging project has successfully migrated, create another PR from `develop` to `main` and merge it to deploy the migration to the production project.
<!-- ![Merge new PR](/img/guides/cli/ci-release.png) -->
The `release` job applies all new migration scripts merged in `supabase/migrations` directory to a linked Supabase project. You can control which project the job links to via `PROJECT_ID` environment variable.
## Troubleshooting
### Sync production project to staging
When setting up a new staging project, you might need to sync the initial schema with migrations previously applied to the production project.
One way is to leverage the Release workflow:
- Create a new branch `develop` and choose `main` as the branch source
- Push the `develop` branch to GitHub
The GitHub Actions runner will deploy your existing migrations to the staging project.
Alternatively, you can also apply migrations through your local CLI to a linked remote database.
```sql
supabase db push
```
Once pushed, check that the migration version is up to date for both local and remote databases.
```sql
supabase migration list
```
### Permission denied on db remote commit
If you have been using Supabase hosted projects for a long time, you might encounter the following permission error when executing `db remote commit`.
```bash
Error: Error running pg_dump on remote database: pg_dump: error: query failed: ERROR: permission denied for table _type
pg_dump: error: query was: LOCK TABLE "graphql"."_type" IN ACCESS SHARE MODE
```
To resolve this error, you need to grant `postgres` role permissions to `graphql` schema. You can do that by running the following query from Supabase dashboard's SQL Editor.
```sql
grant all on all tables in schema graphql to postgres, anon, authenticated, service_role;
grant all on all functions in schema graphql to postgres, anon, authenticated, service_role;
grant all on all sequences in schema graphql to postgres, anon, authenticated, service_role;
```
### Permission denied on db push
If you created a table through Supabase dashboard, and your new migration script contains `ALTER TABLE` statements, you might run into permission error when applying them on staging or production databases.
```bash
ERROR: must be owner of table employees (SQLSTATE 42501); while executing migration <timestamp>
```
This is because tables created through Supabase dashboard are owned by `supabase_admin` role while the migration scripts executed through CLI are under `postgres` role.
One way to solve this is to reassign the owner of those tables to `postgres` role. For example, if your table is named `users` in the public schema, you can run the following command to reassign owner.
```sql
ALTER TABLE users OWNER TO postgres;
```
Apart from tables, you also need to reassign owner of other entities using their respective commands, including [types](https://www.postgresql.org/docs/current/sql-alterschema.html), [functions](https://www.postgresql.org/docs/current/sql-alterroutine.html), and [schemas](https://www.postgresql.org/docs/current/sql-altertype.html).
### Rebasing new migrations
Sometimes your teammate may merge a new migration file to git main branch, and now you need to rebase your local schema changes on top.
<!-- ![Rebase on main](/img/guides/cli/rebase.png) -->
We can handle this scenario gracefully by renaming your old migration file with a new timestamp.
```bash
git pull
supabase migration new dev_A
# Assume the new file is: supabase/migrations/<t+2>_dev_A.sql
mv <time>_dev_A.sql <t+2>_dev_A.sql
supabase db reset
```
In case [`reset`](/reference/cli/usage#supabase-db-reset) fails, you can manually resolve conflicts by editing `<t+2>_dev_A.sql` file.
Once validated locally, commit your changes to Git and push to GitHub.