mirror of
https://github.com/supabase/supabase.git
synced 2026-09-09 11:30:17 +08:00
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## Problem The Docs E2E link checker found broken links throughout docs, starting with `phone-login.mdx` pointing to `/docs/guides/cli/config` (404). Old links like `/docs/guides/cli/config` still work on the live site because `supabase.com` has redirects set up for them, but these links break on the docs preview site, which is what the E2E check tests against. These issues look clean on the live site, and I didn't catch them in my first pass because I was testing production instead of the preview. The E2E check only tests the ~20 pages a given PR happens to touch, so fixing the pages it flagged kept exposing more of the same problem one page at a time as each fix pulled in a new file. To stop chasing this incrementally, I cross-referenced every `/docs/guides/*` and `/docs/reference/*` redirect source in `apps/www/lib/redirects.js` against actual usage across all of `apps/docs`, and verified each candidate against the live preview. ## Solution Rather than updating the Docs E2E link checker, this PR resolves the links. **Why:** we own these docs, so keeping the links clean without redirects is keeping the house maintained. See [Broken Window Theory](https://blog.codinghorror.com/the-broken-window-theory/). Updated every link still using an old path to point straight at the current page instead of relying on a redirect. This covers old links like: - `/docs/guides/cli/config` → `/docs/guides/local-development/cli/config` - `/docs/guides/cli/getting-started` → `/docs/guides/local-development/cli/getting-started` - `/docs/guides/cli/local-development` → `/docs/guides/local-development/database-migrations` - `/docs/guides/cli/managing-environments` → `/docs/guides/deployment/managing-environments` - `/docs/guides/cli/seeding-your-database` → `/docs/guides/local-development/seeding-your-database` - bare `/docs/guides/cli` → `/docs/guides/local-development` - `/docs/guides/platform/compute-add-ons` → `/docs/guides/platform/compute-and-disk` - `/docs/guides/platform/shared-responsibility-model` → `/docs/guides/deployment/shared-responsibility-model` - `/docs/guides/database` → `/docs/guides/database/overview` - `/docs/reference/javascript`, `/docs/reference/dart`, `/docs/reference/kotlin`, `/docs/reference/python`, `/docs/reference/csharp` → their `/introduction` pages (the redirect's own destination, `/start`, turned out to be dead even on production — a separate bug in `redirects.js` I didn't touch here) - and about 35 more of the same pattern, listed in the commit messages Also fixed a handful of dead heading anchors found along the way (links that resolve to the right page but point at a `#section` that got renamed or moved), including the original `#bigquery` anchor and a few in `connecting-to-postgres.mdx` where content moved to its own dedicated page. Left alone on purpose: - `content/guides/cli.mdx` — this page has no route in the docs app at all (no `app/guides/cli/` directory), so it 404s even in production before the `www` redirect ever fires. Fixing its internal link wouldn't change that; it needs an actual routing/content decision, not a link fix. - A few candidates that already resolve fine as-is (`pg_partman`, bare `/docs/reference/api`, bare `/docs/reference/cli`) — confirmed via curl, left untouched. ## Manual testing 1. Confirmed every new link target actually exists by checking the destination file/page and matching heading anchors. 2. Cross-referenced every `/docs/guides/*` and `/docs/reference/*` redirect source in `apps/www/lib/redirects.js` against real usage in `apps/docs`, and curl-verified each old path (404) and new path (200) against the live PR preview before fixing it. 3. Ran the Docs E2E link checker locally against changed pages. 4. Spot-checked the original broken link from CI (`/docs/guides/cli/config`) to confirm it now points to a working page.
130 lines
7.3 KiB
Plaintext
130 lines
7.3 KiB
Plaintext
---
|
||
id: 'database'
|
||
title: 'Import data into Supabase'
|
||
---
|
||
|
||
You can import data into Supabase in multiple ways. The best method depends on your data size and app requirements.
|
||
|
||
If you're working with small datasets in development, you can experiment with CSV import in the Supabase dashboard. If you're working with a large dataset in production, you should plan your data import to minimize app latency and ensure data integrity.
|
||
|
||
## How to import data into Supabase
|
||
|
||
You have multiple options for importing your data into Supabase:
|
||
|
||
1. [CSV import via the Supabase dashboard](#option-1-csv-import-via-supabase-dashboard)
|
||
2. [Bulk import using `pgloader`](#option-2-bulk-import-using-pgloader)
|
||
3. [Using the Postgres `COPY` command](#option-3-using-postgres-copy-command)
|
||
4. [Using the Supabase API](#option-4-using-the-supabase-api)
|
||
|
||
<Admonition type="note">
|
||
|
||
If you're importing a large dataset or importing data into production, plan ahead and [prepare your database](#preparing-to-import-data).
|
||
|
||
</Admonition>
|
||
|
||
### Option 1: CSV import via Supabase dashboard
|
||
|
||
Supabase dashboard provides a user-friendly way to import data. However, for very large datasets, this method may not be the most efficient choice, given the size limit is 100MB. It's generally better suited for smaller datasets and quick data imports. Consider using alternative methods like pgloader for large-scale data imports.
|
||
|
||
1. Navigate to the relevant table in the [Table Editor.](/dashboard/project/_/editor)
|
||
2. Click on _+ New table_ (for new, empty projects) or _Insert_ (for existing tables), then choose _Import Data from CSV_ and follow the on-screen instructions to upload your CSV file.
|
||
|
||
### Option 2: Bulk import using pgloader
|
||
|
||
[pgloader](https://pgloader.io/) is a powerful tool for efficiently importing data into a Postgres database that supports a wide range of source database engines, including MySQL and MS SQL.
|
||
|
||
You can use it in conjunction with Supabase by following these steps:
|
||
|
||
1. Install pgloader on your local machine or a server. For more info, you can refer to the [official pgloader installation page](https://pgloader.readthedocs.io/en/latest/install.html).
|
||
|
||
```bash
|
||
$ apt-get install pgloader
|
||
```
|
||
|
||
2. Create a configuration file that specifies the source data and the target Supabase database (e.g., config.load).
|
||
Here's an example configuration file:
|
||
|
||
```sql
|
||
LOAD DATABASE
|
||
FROM sourcedb://USER:PASSWORD@HOST/SOURCE_DB
|
||
INTO postgres://postgres.xxxx:password@xxxx.pooler.supabase.com:6543/postgres
|
||
ALTER SCHEMA 'public' OWNER TO 'postgres';
|
||
set wal_buffers = '64MB', max_wal_senders = 0, statement_timeout = 0, work_mem to '2GB';
|
||
```
|
||
|
||
Customize the source and Supabase database URL and options to fit your specific use case:
|
||
- `wal_buffers`: This parameter is set to '64MB' to allocate 64 megabytes of memory for write-ahead logging buffers. A larger value can help improve write performance by caching more data in memory before writing it to disk. This can be useful during data import operations to speed up the writing of transaction logs.
|
||
- `max_wal_senders`: It is set to 0, to disable replication connections. This is done during the data import process to prevent replication-related conflicts and issues.
|
||
- `statement_timeout`: The value is set to 0, which means it's disabled, allowing SQL statements to run without a time limit.
|
||
- `work_mem`: It is set to '2GB', allocating 2 GB of memory for query operations. This enhances the performance of complex queries by allowing larger in-memory datasets.
|
||
|
||
3. Run pgloader with the configuration file.
|
||
```jsx
|
||
pgloader config.load
|
||
```
|
||
|
||
For databases using the Postgres engine, we recommend using the [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html) and [psql](https://www.postgresql.org/docs/current/app-psql.html) command line tools.
|
||
|
||
### Option 3: Using Postgres copy command
|
||
|
||
Read more about [Bulk data loading.](/docs/guides/database/tables#bulk-data-loading)
|
||
|
||
### Option 4: Using the Supabase API
|
||
|
||
The Supabase API allows you to programmatically import data into your tables. You can use various client libraries to interact with the API and perform data import operations. This approach is useful when you need to automate data imports, and it gives you fine-grained control over the process. Refer to our [API guide](/docs/guides/api) for more details.
|
||
|
||
<Admonition type="note">
|
||
|
||
When importing data via the Supabase API, it's advisable to refrain from bulk imports. This helps ensure a smooth data transfer process and prevents any potential disruptions.
|
||
|
||
Read more about [Rate Limiting, Resource Allocation, & Abuse Prevention.](/docs/guides/deployment/going-into-prod#rate-limiting-resource-allocation--abuse-prevention)
|
||
|
||
</Admonition>
|
||
|
||
## Preparing to import data
|
||
|
||
Large data imports can affect your database performance. Failed imports can also cause data corruption. Importing data is a safe and common operation, but you should plan ahead if you're importing a lot of data, or if you're working in a production environment.
|
||
|
||
### 1. Back up your data
|
||
|
||
Backups help you restore your data if something goes wrong. Databases on Pro, Team and Enterprise Plans are automatically backed up on schedule, but you can also take your own backup. See [Database Backups](/docs/guides/platform/backups) for more information.
|
||
|
||
### 2. Increase statement timeouts
|
||
|
||
By default, Supabase enforces query statement timeouts to ensure fair resource allocation and prevent long-running queries from affecting the overall system. When importing large datasets, you may encounter timeouts. To address this:
|
||
|
||
- **Increase the Statement Timeout**: You can adjust the statement timeout for your session or connection to accommodate longer-running queries. Be cautious when doing this, as excessively long queries can negatively impact system performance. Read more about [Statement Timeouts](/docs/guides/database/postgres/configuration).
|
||
|
||
### 3. Estimate your required disk size
|
||
|
||
Large datasets consume disk space. Ensure your Supabase project has sufficient disk capacity to accommodate the imported data. If you know how big your database is going to be, you can manually increase the size in your [projects database settings](/dashboard/project/_/database/settings).
|
||
|
||
Read more about [disk management](/docs/guides/platform/database-size#disk-management).
|
||
|
||
### 4. Disable triggers
|
||
|
||
When importing large datasets, it's often beneficial to disable triggers temporarily. Triggers can significantly slow down the import process, especially if they involve complex logic or referential integrity checks. After the import, you can re-enable the triggers.
|
||
|
||
To disable triggers, use the following SQL commands:
|
||
|
||
```sql
|
||
-- Disable triggers on a specific table
|
||
ALTER TABLE table_name DISABLE TRIGGER ALL;
|
||
|
||
-- To re-enable triggers
|
||
ALTER TABLE table_name ENABLE TRIGGER ALL;
|
||
```
|
||
|
||
### 5. Rebuild indices after data import is complete
|
||
|
||
Indexing is crucial for query performance, but building indices while importing a large dataset can be time-consuming. Consider building or rebuilding indices after the data import is complete. This approach can significantly speed up the import process and reduce the overall time required.
|
||
|
||
To build an index after the data import:
|
||
|
||
```sql
|
||
-- Create an index on a table
|
||
create index index_name on table_name (column_name);
|
||
```
|
||
|
||
Read more about [Managing Indexes in Postgres](/docs/guides/database/postgres/indexes).
|