mirror of
https://github.com/supabase/supabase.git
synced 2026-09-06 09:59:03 +08:00
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Bug fix (performance), plus a regression-guard test suite and docs. ## What is the current behavior? Studio's introspection queries in `@supabase/pg-meta` do `O(catalog)` work for per-table requests. On databases with very large catalogs (hundreds of thousands of relations/constraints — real deployments reach this) they take tens of seconds per dashboard interaction, trip `statement_timeout`, and create heavy CPU/memory pressure when several tabs open concurrently. Two instances of the same bug class: **1. Table Editor query (`getTableEditorSql`)** — fetches metadata for ONE table by OID, but five catalog scans are unscoped and only filtered at the top-level join: - `primary_keys` CTE — scans all of `pg_index` (`where i.indisprimary`) - `index_cols` CTE — scans all unique indexes - `relationships` CTE — scans every FK in `pg_constraint` (and is scanned twice by the two subplans) - `uniques` subquery (inside `columns`) — scans all single-column unique constraints - `check_constraints` subquery (inside `columns`) — scans all single-column check constraints The planner cannot push the outer join qual into grouped / `distinct on` subqueries, so each is computed over the full catalog and thrown away. `tables-paginated.ts` was previously rewritten to avoid exactly this pattern; the single-table query never got the same treatment. **2. Entity definitions (`getTableDefinitionSql` / `getEntityDefinitionsSql`)** — the vendored `pg_get_tabledef` plpgsql function scans the entire `information_schema.columns` view once **per column** (plus `information_schema.tables` once per call) just to decide whether a name needs double-quoting — a pure string property of a name it already holds — and its per-index partial-index lookup casts `relnamespace::regnamespace::text` across every `pg_class` row. On a 12K-table catalog this makes a single entity's DDL cost ~3.7s and a default 100-entity definitions page ~6 minutes. ## What is the new behavior? **Fix 1 — scope the Table Editor CTEs to the requested OID** (`id` is validated non-null and interpolated via `literal()`, same as the existing `base_table_info` filter): - `primary_keys` / `index_cols`: `and i.indrelid = <id>` - `relationships`: `and (c.conrelid = <id> or c.confrelid = <id>)` - `uniques` / `check_constraints`: `and conrelid = <id>` Semantics are unchanged: the top-level select already filtered every CTE to the target table, so rows for other tables were computed and discarded. The `pg_index`/`pg_constraint` lookups become index scans returning a handful of rows. One residual scan is structural: PostgreSQL has no index on `pg_constraint.confrelid`, so the incoming-FK half of `relationships` is a single filtered seq scan of `pg_constraint` — still one cheap pass instead of materializing every FK row twice. **Fix 2 — remove the O(catalog) scans inside `pg_get_tabledef`**: the information_schema uppercase checks are replaced with direct regex tests on the name in hand (preserving the original's `quote_ident` behavior for schemas that need quoting), and the partial-index lookup is scoped by the already-resolved table OID. Original statements are kept as comments, matching the vendored file's convention. **Regression guard** — so this bug class stays out: - `test/db/stress-catalog.ts` builds a synthetic catalog (default 2,000 tables with PKs, unique + check constraints, FK chains and an FK hub; `PG_META_STRESS_TABLES` scales it to incident size). - `test/db/plan-guard.ts` provides `EXPLAIN (ANALYZE, FORMAT JSON)`-based budget assertions: a query's plan may only seq-scan a scaling catalog if its budget entry carries a written structural justification (e.g. no index on `pg_constraint.confrelid`; no index on `pg_class.relnamespace` for per-schema listings), plus a per-query time bound (the only guard available for opaque plpgsql internals like `pg_get_tabledef`). - `test/sql/studio/catalog-plan-guard.test.ts` applies budgets to the hot-path studio queries: table editor, constraints, FK listing, entity types, tables-paginated, columns, indexes, table/entity definitions, views. Reverting either fix makes the suite fail immediately with the offending scans listed. - `test/sql/studio/table-editor.test.ts` (new — none existed) asserts the Table Editor query's semantics: primary keys, unique indexes, both FK directions, `is_unique`, check definitions, column comments. - A new package `README.md` documents the plan-guard budget entry as a requirement for any new introspection query. ### Validation (synthetic 12,000-table catalog, PostgreSQL 17.6) - **Output equivalence, fix 1:** for 12 relation types (regular, composite PK, partitioned parent + partition, view, materialized view, constraint-free table, FK hub/chain/tail, and a fixture with enums/domains/generated/identity columns and duplicate check constraints), the `entity` jsonb from the old and new query is byte-identical. - **Output equivalence, fix 2:** byte-identical DDL across 13 fixture combinations (serial/identity/generated/array columns, case-sensitive and keyword names, mixed-case schemas, partitions, unlogged + reloptions, partial/expression indexes, external PK/FK/comments/trigger variants). - **Performance, fix 1:** Table Editor query `EXPLAIN ANALYZE` ~1,630ms → ~30ms (~50×); the gap grows with catalog size since the old query is O(catalog) per call. - **Performance, fix 2:** single entity definition 3,672ms → 63ms; a 100-entity definitions page ~6min → 0.87s. The plan-guard bound for `getEntityDefinitionsSql` tightens accordingly from 15s/25 entities to 3s/100 entities (330ms measured at default test scale). Verified locally: `catalog-plan-guard` (12 tests), `table-editor`, `tables-paginated` (16 tests) pass; `typecheck` clean. ### Rollout Per review, the new behavior ships **dark** behind the `pgMetaScopedIntrospection` ConfigCat flag (default off = legacy SQL, kept as full duplicated templates in pg-meta and verified byte-identical to the pre-PR queries). Studio reads the flag in the query hooks and threads it through (flag state is part of the React Query keys). The rollout is staged in the ConfigCat dashboard via user-email targeting (like every other ConfigCat flag): target the reporting user's email first, then a percentage rollout, then 100%. Server-side AI callers of `getEntityDefinitionsSql` stay on the legacy path. Once fully rolled out, delete the legacy templates + flag in a cleanup PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Closes: PGMETA-122 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved table editor SQL to correctly scope primary keys, indexes, uniques, checks, and relationships to the selected table. - Optimized table definition SQL to reduce unnecessary catalog scanning for uppercase-name detection and partial-index detection. - **Tests** - Added SQL generator tests for table editor metadata (keys, indexes, relationships, comments, and constraints). - Added catalog query plan guard coverage with a stress catalog and EXPLAIN-based scoping/performance budgets. - **Documentation** - Expanded documentation on catalog query plan safeguards and how to keep new introspection queries properly scoped. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
133 lines
4.1 KiB
TypeScript
133 lines
4.1 KiB
TypeScript
import { afterAll, expect, test } from 'vitest'
|
|
|
|
import { getTableEditorSql } from '../../../src'
|
|
import { cleanupRoot, createTestDatabase } from '../../db/utils'
|
|
|
|
afterAll(async () => {
|
|
await cleanupRoot()
|
|
})
|
|
|
|
type Entity = {
|
|
entity_type: string
|
|
id: number
|
|
schema: string
|
|
name: string
|
|
primary_keys: Array<{ table_id: number; schema: string; table_name: string; name: string }>
|
|
unique_indexes: Array<{ table_id: number; schema: string; table_name: string; columns: string[] }>
|
|
relationships: Array<{
|
|
id: number
|
|
constraint_name: string
|
|
source_schema: string
|
|
source_table_name: string
|
|
source_column_name: string
|
|
target_table_schema: string
|
|
target_table_name: string
|
|
target_column_name: string
|
|
}>
|
|
columns: Array<{
|
|
name: string
|
|
is_unique: boolean
|
|
check: string | null
|
|
comment: string | null
|
|
}>
|
|
}
|
|
|
|
const withTestDatabase = (
|
|
name: string,
|
|
fn: (db: Awaited<ReturnType<typeof createTestDatabase>>) => Promise<void>
|
|
) => {
|
|
test(name, async () => {
|
|
const db = await createTestDatabase()
|
|
try {
|
|
await fn(db)
|
|
} finally {
|
|
await db.cleanup()
|
|
}
|
|
})
|
|
}
|
|
|
|
// The `scoped` flag gates the PR #47894 scoping predicates (default OFF for
|
|
// progressive rollout). Both code paths must return semantically identical
|
|
// entities; running the same assertions for scoped:true and scoped:false is the
|
|
// CI equivalence guard. Once the rollout completes, drop scoped:false here.
|
|
for (const scoped of [true, false]) {
|
|
withTestDatabase(
|
|
`scopes primary keys, unique indexes, relationships and columns to the target table (scoped=${scoped})`,
|
|
async ({ executeQuery }) => {
|
|
await executeQuery(`
|
|
create schema if not exists editor_scope;
|
|
|
|
create table editor_scope.authors (
|
|
id serial primary key
|
|
);
|
|
|
|
create table editor_scope.books (
|
|
id serial primary key,
|
|
isbn text not null unique,
|
|
price numeric not null check (price > 0),
|
|
author_id int not null references editor_scope.authors (id)
|
|
);
|
|
comment on column editor_scope.books.isbn is 'International Standard Book Number';
|
|
|
|
create table editor_scope.reviews (
|
|
id serial primary key,
|
|
book_id int not null references editor_scope.books (id)
|
|
);
|
|
`)
|
|
|
|
const [{ id: booksId }] = await executeQuery<{ id: number }[]>(
|
|
`select 'editor_scope.books'::regclass::oid::int8 as id;`
|
|
)
|
|
|
|
const sql = getTableEditorSql({ id: booksId, scoped })
|
|
const [{ entity }] = await executeQuery<{ entity: Entity }[]>(sql)
|
|
|
|
// Basic identity.
|
|
expect(entity.id).toBe(booksId)
|
|
expect(entity.schema).toBe('editor_scope')
|
|
expect(entity.name).toBe('books')
|
|
|
|
// Primary key scoped to `books`.
|
|
expect(entity.primary_keys).toEqual([
|
|
{ schema: 'editor_scope', table_name: 'books', table_id: booksId, name: 'id' },
|
|
])
|
|
|
|
// Unique index scoped to `books`.
|
|
expect(entity.unique_indexes).toHaveLength(1)
|
|
expect(entity.unique_indexes[0]).toMatchObject({
|
|
schema: 'editor_scope',
|
|
table_name: 'books',
|
|
table_id: booksId,
|
|
columns: ['isbn'],
|
|
})
|
|
|
|
// Relationships include both the outgoing FK (books -> authors) and the
|
|
// incoming FK (reviews -> books).
|
|
expect(entity.relationships).toContainEqual(
|
|
expect.objectContaining({
|
|
source_table_name: 'books',
|
|
source_column_name: 'author_id',
|
|
target_table_name: 'authors',
|
|
target_column_name: 'id',
|
|
})
|
|
)
|
|
expect(entity.relationships).toContainEqual(
|
|
expect.objectContaining({
|
|
source_table_name: 'reviews',
|
|
source_column_name: 'book_id',
|
|
target_table_name: 'books',
|
|
target_column_name: 'id',
|
|
})
|
|
)
|
|
|
|
// Columns: unique flag, check constraint definition, and comment.
|
|
const isbnCol = entity.columns.find((c) => c.name === 'isbn')!
|
|
expect(isbnCol.is_unique).toBe(true)
|
|
expect(isbnCol.comment).toBe('International Standard Book Number')
|
|
|
|
const priceCol = entity.columns.find((c) => c.name === 'price')!
|
|
expect(priceCol.check).toContain('price > 0')
|
|
}
|
|
)
|
|
}
|