Files
supabase/apps/studio/data/privileges/table-api-access-query.ts
Andrew Valleteau 6c6a721cb7 fix(pg-meta): scope remaining O(catalog) introspection queries behind pgMetaScopedIntrospection (#48148)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Bug fix (performance), follow-up to #47894, plus regression-guard tests.

## What is the current behavior?

#47894 scoped the Table Editor and entity-definition introspection
queries, but four more `@supabase/pg-meta` query families still do
O(catalog) work per request. On a production project with a very large
catalog (hundreds of schemas, ~465K `pg_constraint` rows) they run 5 to
55 seconds each, trip the 58s `statement_timeout`, and spill sorts to
temp files. During a recent "DB CPU > 85%" incident on such a project,
24 of 27 active backends were running these queries concurrently.

1. **`tables.retrieve()` (single-table lookup by name+schema or id)**:
the `tables`/`columns` CTEs scan the whole catalog (`pg_class`,
`pg_constraint`, `pg_index`, all of `pg_attribute`, per-table sizes) and
the one-table predicate is applied only on the outer select. Same bug
class #47894 fixed for the OID-based table editor query; this sibling
path never got the treatment. It accounted for 94 of the 96
statement-timeout cancellations in the incident.
2. **Types listing**: the `t_enums` and `t_attributes` subqueries
aggregate the entire `pg_enum` and every composite relation before the
wrapper's schema filter applies.
3. **Table privileges**: `aclexplode` + double `pg_roles` join + GROUP
BY over every relation in the database; schema/OID filters applied only
after aggregation, in both `list()` and `retrieve()`.
4. **Row counts**: `getTableRowsCountSql` treats `reltuples = -1`
(never-analyzed table) as "small table, run exact count(*)". A freshly
bulk-loaded multi-million-row table times out on every Table Editor
pagination render.

Two Studio-side amplifiers turned one slow query into a sustained load
storm:

- `useTableQuery` (behind `tables.retrieve()`) mounts once per visible
foreign-key grid cell via `ForeignKeyFormatter`, so a single Table
Editor view fires ~20 concurrent copies against the FK target table. A
timed-out query caches nothing, and TanStack retries errored no-data
queries on every observer mount by default, so scrolling kept re-issuing
the 58s scan.
- `useTableApiAccessQuery` fetched table privileges for the entire
database and filtered down to one schema client-side.

## What is the new behavior?

**pg-meta (all behind the existing `pgMetaScopedIntrospection` flag,
same rollout mechanism as #47894; `scoped: false` keeps serving the
current SQL):**

- `tables.retrieve()`: the identifier is resolved to a scalar
`targetOid` init-plan and pushed into the base scan, primary-key,
relationships (both FK directions kept: `conrelid` or `confrelid`) and
columns CTEs. A materialized `target` CTE was deliberately avoided: it
acts as an optimization barrier and forces the very seq scans being
removed.
- Types: filter `pg_type`/`pg_namespace` first, then compute
enums/attributes per surviving row via correlated index-scan subqueries
(`pg_enum(enumtypid, enumsortorder)`, `pg_attribute(attrelid, attnum)`).
- Table privileges: schema/OID predicates injected into the base WHERE
before `aclexplode`/GROUP BY for `list()` and `retrieve()`.
- Row counts: `reltuples = -1` is treated as "unknown" and gated on
physical size via `pg_relation_size` (a cheap stat call; `relpages` is
equally stale pre-vacuum). At or below `THRESHOLD_ESTIMATE_BYTES`
(~10MB, derived from `THRESHOLD_COUNT` at a conservative ~200 bytes/row)
the exact count runs as before: fast by construction, and it avoids
bogus estimates since Postgres floors never-vacuumed heaps at 10 pages,
so an empty table would otherwise report ~2K estimated rows. Above the
gate the count routes through the EXPLAIN-based
`pg_temp.count_estimate`, or returns `-1`/`is_estimate = true` in
read-only contexts where the temp function cannot be created. The scoped
branch embeds the estimated select via `literal()` instead of legacy's
apostrophe-only escaping, so it stays correct under
`standard_conforming_strings = off`. `enforceExactCount` unchanged.

**Studio:**

- The flag decision is contained in the data layer instead of
prop-drilled: a small imperative accessor
(`apps/studio/data/scoped-introspection.ts`) is hydrated from `useFlag`
via a one-line `useSyncScopedIntrospection()` call in `DefaultLayout`,
and the query functions read it internally when building the pg-meta
SQL. `DefaultLayout` is shared by both the Next and TanStack router
trees; hydrating from `_app.tsx` alone would leave TanStack-served pages
permanently unscoped since `routes/__root.tsx` mounts its own flag
provider. Cold loads cannot race the flag: the query functions await a
readiness promise that resolves only after the sync hook has hydrated
the accessor with a loaded flag store (immediately on self-hosted where
flags are disabled; a 5s safety net armed lazily on the first `ready()`
call - not at module import, which would let the timer expire before a
project page ever mounts - bounds genuine ConfigCat outages). No
component threading, no query-key changes (remaining tradeoff,
documented in the module: a mid-session flag flip can serve stale-keyed
caches until refetch, fine for a session-stable rollout flag). #47894's
existing threading is left as-is and gets deleted together with the flag
in the cleanup PR. Also fixes the previously-missing `scoped`
pass-through in `getTableRowsCount`.
- Flag-independent hardening: `useTableQuery` now sets `retryOnMount:
false`, `refetchOnWindowFocus: false` and `staleTime: 5min`. Errored
(timed-out) queries no longer refire on every grid cell remount, while
stale successful metadata still revalidates on mount after `staleTime`.
- `useTableApiAccessQuery` now passes `includedSchemas: [schemaName]`;
the client-side filter stays as a safety net.
- The rows-count query is `enabled`-gated on the permission check
settling, so a transiently-false `canSQLAdminWrite` can no longer cache
a read-only `-1` count for a writable user (read replicas short-circuit
synchronously as before).

**Regression guards (extending the #47894 infrastructure):**

- Execution-based scoped-vs-legacy equivalence tests for all four
queries: both variants run against the test database and are compared
with raw `toEqual` - no normalization, ids included (types across 6
option combos, privileges incl. multi-grantee + PUBLIC,
`tables.retrieve` for both identifier branches, row counts for every
case where the two paths must agree). Two documented exceptions where
only the LEGACY side is sorted, because a de-normalized diagnostic run
proved legacy emits genuinely plan-dependent order there (an
adversarial-FK fixture shows it is neither oid, name, nor creation
order): the `types.list` outer row order (scoped adds `order by t.oid`;
legacy has no ORDER BY) and the `tables.retrieve` relationships array
(scoped orders by `constraint_name` + column-name tie-breakers - a
composite two-column FK expands to 4 entries sharing one
constraint_name). Everything else (privileges via `aclexplode` over the
same relacl, columns by `ordinal_position`, primary keys by `indkey`
order, enums by `enumsortorder`) is byte-identical between the two paths
with no test-side help. The one intentional value divergence,
never-analyzed tables above the size gate where legacy's exact count is
the timeout bug itself, is asserted explicitly as a divergence.
- Plan-guard budgets for every scoped query against the stress catalog
(extended with 200 enums + 200 composite types). Residual seq scans are
justified in-budget: `pg_constraint` max 2 (no index on `confrelid`),
`pg_attrdef` max 1, `pg_authid` max 2 (scales with role count, not
schema count).
- Legacy templates carry a FROZEN do-not-edit marker (they must keep
matching production behavior until the flag cleanup deletes them); the
ordinary test suite runs against the legacy default, so behavioral drift
there fails regular tests.

### Validation

- pg-meta: typecheck clean; the affected suites (types,
table-privileges, tables, rows-count, catalog-plan-guard) pass in full.
- Cross-version: the scoped-vs-legacy equivalence and rows-count
behavioral suites were validated on PostgreSQL 14, 15, and 17 (identical
results on all three). Two version-marginal planner choices surfaced on
17 (`pg_type` / `pg_class` seq scan vs full-index bitmap for per-schema
listings, both structurally unavoidable without an index leading on the
namespace column) and are carried as justified plan-guard budget
entries. A full 468-test suite run sequentially: 452 passed, 16 failures
verified environmental (13 timeouts in an untouched file that passes
27/27 in isolation on the marathon-run cluster, 3 cluster-global role
collisions from container reuse).
- Studio: `pnpm --filter studio typecheck` clean; 39/39 tests across the
touched data hooks; eslint clean on touched files.

### Rollout

Same staged ConfigCat rollout as #47894 via `pgMetaScopedIntrospection`
(user-email targeting first, then percentage, then 100%). The
`useTableQuery` hardening and the API-access schema scoping ship
unflagged (behavior-safe). Gate before percentage rollout: functionally
verify the FK popover/selector UX under the new
`staleTime`/`retryOnMount` settings (a just-edited FK target must not
look stale anywhere Studio does not already refetch on save). Once fully
rolled out, the legacy templates and flag get deleted together with
#47894's in one cleanup PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 08:07:08 +02:00

246 lines
6.6 KiB
TypeScript

import { useMemo } from 'react'
import {
useTablePrivilegesQuery,
type TablePrivilegesData,
type TablePrivilegesError,
} from './table-privileges-query'
import type { ConnectionVars } from '@/data/common.types'
import { useIsSchemaExposed } from '@/hooks/misc/useIsSchemaExposed'
import {
API_ACCESS_ROLES,
API_PRIVILEGE_TYPES,
isApiAccessRole,
isApiPrivilegeType,
type ApiPrivilegesByRole,
} from '@/lib/data-api-types'
import type { Prettify } from '@/lib/type-helpers'
import type { UseCustomQueryOptions } from '@/types'
// The contents of this array are never used, so any will allow
// it to be used anywhere an array of any type is required.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const STABLE_EMPTY_ARRAY: any[] = []
const STABLE_EMPTY_OBJECT = {}
const getApiPrivilegesByRole = (
privileges: TablePrivilegesData[number]['privileges']
): ApiPrivilegesByRole => {
const privilegesByRole: ApiPrivilegesByRole = {
anon: [],
authenticated: [],
service_role: [],
}
privileges.forEach((privilege) => {
const { grantee, privilege_type } = privilege
if (isApiAccessRole(grantee) && isApiPrivilegeType(privilege_type)) {
privilegesByRole[grantee].push(privilege_type)
}
})
return privilegesByRole
}
const mapPrivilegesByTableName = (
privileges: TablePrivilegesData | undefined,
schemaName: string,
tableNames: Set<string>
): Record<string, ApiPrivilegesByRole> => {
if (!privileges) return {}
const result: Record<string, ApiPrivilegesByRole> = {}
privileges.forEach((entry) => {
if (entry.schema !== schemaName) return
if (!tableNames.has(entry.name)) return
result[entry.name] = getApiPrivilegesByRole(entry.privileges)
})
return result
}
export type UseTableApiAccessQueryParams = Prettify<
ConnectionVars & {
schemaName: string
tableNames: string[]
}
>
export type DataApiAccessType = 'none' | 'exposed-schema-no-grants' | 'access'
/**
* Mirrors the "granted | custom | revoked" classification used by the Data API
* settings page (see getTableGrantsCTEs in packages/pg-meta privileges.ts).
* - `granted`: all 3 API roles (anon/authenticated/service_role) have all 4
* CRUD privileges — the standard Data API exposure.
* - `custom`: at least one grant exists but it's not the full standard set.
* - `revoked`: no API role has any privilege (mapped to `exposed-schema-no-grants`).
*/
export type TableGrantStatus = 'granted' | 'custom'
export type TableApiAccessData =
| {
apiAccessType: 'access'
grantStatus: TableGrantStatus
privileges: ApiPrivilegesByRole
}
| {
apiAccessType: 'none' | 'exposed-schema-no-grants'
}
/**
* Matches the "granted" branch of getTableGrantsCTEs in packages/pg-meta's
* privileges.ts — all 3 API roles must have all 4 CRUD privileges.
*/
export const isFullyGranted = (privileges: ApiPrivilegesByRole): boolean =>
API_ACCESS_ROLES.every((role) =>
API_PRIVILEGE_TYPES.every((priv) => privileges[role].includes(priv))
)
export type TableApiAccessMap = Prettify<Record<string, TableApiAccessData>>
export type UseTableApiAccessQueryReturn =
| {
data: TableApiAccessMap
status: 'success'
isSuccess: true
isPending: false
isError: false
}
| {
data: undefined
status: 'pending'
isSuccess: false
isPending: true
isError: false
}
| {
data: undefined
status: 'error'
isSuccess: false
isPending: false
isError: true
}
export const useTableApiAccessQuery = (
{
projectRef,
connectionString,
schemaName,
tableNames = STABLE_EMPTY_ARRAY,
}: UseTableApiAccessQueryParams,
{
enabled = true,
...options
}: { enabled?: boolean } & Omit<
UseCustomQueryOptions<TablePrivilegesData, TablePrivilegesError>,
'enabled'
> = {}
): UseTableApiAccessQueryReturn => {
const uniqueTableNames = useMemo(() => {
return new Set(
tableNames.filter((tableName) => typeof tableName === 'string' && tableName.length > 0)
)
}, [tableNames])
const hasTables = uniqueTableNames.size > 0
const schemaExposureStatus = useIsSchemaExposed({ projectRef, schemaName }, { enabled })
const isSchemaExposed = schemaExposureStatus.isSuccess && schemaExposureStatus.data === true
const enablePrivilegesQuery = enabled && hasTables
const privilegeStatus = useTablePrivilegesQuery(
{ projectRef, connectionString, includedSchemas: [schemaName] },
{ enabled: enablePrivilegesQuery, ...options }
)
const result: UseTableApiAccessQueryReturn = useMemo(() => {
const isPending =
!enabled ||
schemaExposureStatus.status === 'pending' ||
(enablePrivilegesQuery && privilegeStatus.isPending)
if (isPending) {
return {
data: undefined,
status: 'pending',
isSuccess: false,
isPending: true,
isError: false,
}
}
const isError =
schemaExposureStatus.status === 'error' || (enablePrivilegesQuery && privilegeStatus.isError)
if (isError) {
return {
data: undefined,
status: 'error',
isSuccess: false,
isPending: false,
isError: true,
}
}
if (!hasTables) {
return {
data: STABLE_EMPTY_OBJECT,
status: 'success',
isSuccess: true,
isPending: false,
isError: false,
}
}
const resultData: TableApiAccessMap = {}
const tablePrivilegesByName = isSchemaExposed
? mapPrivilegesByTableName(privilegeStatus.data, schemaName, uniqueTableNames)
: {}
uniqueTableNames.forEach((tableName) => {
if (!isSchemaExposed) {
resultData[tableName] = { apiAccessType: 'none' }
return
}
const tablePrivileges = tablePrivilegesByName[tableName] ?? {
anon: [],
authenticated: [],
service_role: [],
}
const hasApiPrivileges =
tablePrivileges.anon.length > 0 ||
tablePrivileges.authenticated.length > 0 ||
tablePrivileges.service_role.length > 0
resultData[tableName] = hasApiPrivileges
? {
apiAccessType: 'access',
grantStatus: isFullyGranted(tablePrivileges) ? 'granted' : 'custom',
privileges: tablePrivileges,
}
: { apiAccessType: 'exposed-schema-no-grants' }
})
return {
data: resultData,
status: 'success',
isSuccess: true,
isPending: false,
isError: false,
}
}, [
enabled,
enablePrivilegesQuery,
hasTables,
schemaExposureStatus.status,
isSchemaExposed,
privilegeStatus.isPending,
privilegeStatus.isError,
privilegeStatus.data,
schemaName,
uniqueTableNames,
])
return result
}