In this PR:
- The `llms.ts` script is removed from the pipeline. It was just
iterating over the sections of the SDK JSON files to generate _dummy_
text files for LLMS to consume. But with a lot of inconsistencies and
mistakes.
- There's a new script that builds markdown files from the SDK
definition files.
- The Management API now has a definition file too, it was missing
before.
- On the www project now there are redirects for the reference text
files to the new markdown files in the docs project.
- NPM scripts have received better names so that prebuild ones are
smaller and all are easier to differentiate.
# Second try of making a new better process for SDK automation
Instead of building a new pipeline. We will take the lessons learned
form round 1, plus the good design and improvement on DX quality for
drop-in file as a single step required from SDK team and produce almost
identical set of files as used right now to render using the current
pipeline.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* New reference-content pipeline producing per-library reference
artifacts and integrating into prebuilds, search ingestion, and
rendering (type-aware examples).
* **Documentation**
* Added comprehensive JavaScript SDK v2 reference content and partials
(Auth MFA, passkeys, admin, TypeScript support, filters, modifiers,
Installing, Initializing, Buckets, etc.).
* **Tests & CI**
* Added regression snapshot test and updated workflows to refresh
reference snapshots and ensure spec downloads.
* **Chores**
* Updated ignore rules, build scripts, Makefile targets, and package
lifecycle hooks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Katerina Skroumpelou <mandarini@users.noreply.github.com>
Co-authored-by: Katerina Skroumpelou <sk.katherine@gmail.com>
This PR
- reverts changes made in
https://github.com/supabase/supabase/pull/42568 and
https://github.com/supabase/supabase/pull/42570 because they had some
unneeded changes (`@types/node` bumped to v25+, for example)
- bumps only the vulnerable dependency (which is in `cms` app and it's
not currently used).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Adjusted CMS dependencies to earlier versions
* Optimized documentation API code organization and type handling
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.
YES
## What kind of change does this PR introduce?
Update lock file changes
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Reorganized internal imports and improved type compatibility handling
for the AI documentation API to ensure robust operation across different
OpenAI client versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Content API requests were all accepted via POST (as detailed in the
GraphQL spec) and not amenable to caching.
Now, requests are also accepted via GET (as optionally allowed for query
requests only), and cached.
Also sent the request method to Logflare so we can track the ratio of
GET/POST requests.
* Move typescript version definition to pnpm-workspace.
* Bump typescript to 5.9.
* Minor fixes to satisfy the compiler.
---------
Co-authored-by: Raúl Barroso <code@raulb.dev>
* fix(www): types for awaited params
* fix(docs): no extraneous exports from route page
* fix(studio): api handler types, no non-handlers allowed in pages
* feat: alternate search index for nimbus
Create an alternate search index for Nimbus that filters out
feature-flagged pages (equivalent to setting all feature flags to
false).
Notes:
- Creates two new DB tables, `page_nimbus` and `page_section_nimbus`,
which are filtered versions of `page` and `page_section`
- Makes `nimbus` versions of all the DB search functions
- Refactored the embedding upload script. Changes to make it faster (got
annoyed by how slow it was when testing...), incorporate retries, and
produce better summary logs.
- Upload script, when run with the environment variable
ENABLED_FEATURES_OVERRIDE_DISABLE_ALL, produces and uploads the
alternate search index
- Changed all the search calls in frontend/API to check for
`isFeatureEnabled('search:fullIndex')` to determine whether to search
the full or alternate index
* ci: produce nimbus search indexes on merge
* fix: turn full search index on
* refactor: reading markdown docs files
Refactor how Markdown docs files are read:
- Reuses the same logic across search index generation & page generation
- Improves the indexed content for search:
- Stops removing MDX components, which often contain useful
information like Admonitions
- Denormalizes Partials and CodeSamples for more complete content
This is a prerequisite step for implementing the "Copy docs as Markdown"
functionality.
Only touches regular guides for now, not federated ones.
* fix: tailwind build error (#37728)
We changed to default to ESM imports a while ago, which means local
builds are now breaking because the Tailwind uses a require. Changed to
CJS for Tailwind config file. (I have no idea how this has been working
on Vercel all this time.)
* style: prettier
Implement hybrid search for the /docs/api/graphql searchDocs endpoint. Prepend a more descriptive title and introduction to database advisor docs so they rank more highly when directly searched for.
Add the ability to look up error code explanation in API Gateway logs.
Also a bunch of GraphQL-related utilities and generated types for
calling the Content API.
* docs: add cursor rule for embedding generation process
Add documentation for cursor IDE about how docs embeddings are generated,
including the workflow for creating and uploading semantic search content.
* feat: improve API reference metadata upload with descriptive content
- Add preembeddings script to run codegen before embedding generation
- Enhance OpenApiReferenceSource to generate more descriptive content including
parameters, responses, path information, and better structured documentation
* feat: add Management API references to searchDocs GraphQL query
- Add ManagementApiReference GraphQL type and model for API endpoint search results
- Integrate Management API references into global search results
- Update test snapshots and add comprehensive test coverage for Management API search
* style: format
Sentry errors captured on routes run on Edge are not getting properly
sent. This is apparently a known issue where Vercel closes the process
before Sentry can flush, so the exceptions need to be manually flushed:
https://github.com/getsentry/sentry-javascript/issues/9626
- Add support for filtering errors by specific error codes
- Implement `errors` query that accepts array of error code IDs
- Add tests for error code filtering functionality
- Update GraphQL schema with new error filtering capabilities
* feat(graphql): add paginated errors collection query
- Add new GraphQL query field 'errors' with cursor-based pagination
- Add UUID id column to content.error table for cursor pagination
- Implement error collection resolver with forward/backward pagination
- Add comprehensive test suite for pagination functionality
- Update database types and schema to support new error collection
- Add utility functions for handling collection queries and errors
- Add seed data for testing pagination scenarios
This change allows clients to efficiently paginate through error codes using cursor-based pagination, supporting both forward and backward traversal. The implementation follows the Relay connection specification and includes proper error handling and type safety.
* docs(graphql): add comprehensive GraphQL architecture documentation
Add detailed documentation for the docs GraphQL endpoint architecture, including:
- Modular query pattern and folder structure
- Step-by-step guide for creating new top-level queries
- Best practices for error handling, field optimization, and testing
- Code examples for schemas, models, resolvers, and tests
* feat(graphql): add service filtering to errors collection query
Enable filtering error codes by Supabase service in the GraphQL errors collection:
- Add optional service argument to errors query resolver
- Update error model to support service-based filtering in database queries
- Maintain pagination compatibility with service filtering
- Add comprehensive tests for service filtering with and without pagination
* feat(graphql): add service filtering and fix cursor encoding for errors collection
- Add service parameter to errors GraphQL query for filtering by Supabase service
- Implement base64 encoding/decoding for pagination cursors in error resolver
- Fix test cursor encoding to match resolver implementation
- Update GraphQL schema snapshot to reflect new service filter field
* docs(graphql): fix codegen instruction
Add a script for syncing error codes from the repo to the database. This
is part of the newly created rootSync script, where all sync scripts
should be moved eventually.
strictNullChecks was off for docs, which lets errors slip through and
leads to incorrect required/optional typing on Zod-inferred types. This
PR enables strictNullChecks and fixes all the existing violations.
* feat(content api): add error endpoint
Add an endpoint to return the details of a Supabase error, given the
error code and service.
Schema additions:
```graphql
type RootQueryType {
"...previous root queries"
"""Get the details of an error code returned from a Supabase service"""
error(code: String!, service: Service!): Error
}
"""An error returned by a Supabase service"""
type Error {
"""
The unique code identifying the error. The code is stable, and can be used for string matching during error handling.
"""
code: String!
"""The Supabase service that returns this error."""
service: Service!
"""The HTTP status code returned with this error."""
httpStatusCode: Int
"""
A human-readable message describing the error. The message is not stable, and should not be used for string matching during error handling. Use the code instead.
"""
message: String
}
enum Service {
AUTH
REALTIME
STORAGE
}
```
* test(content api): add tests for top-level query `error`
Alter the docs test script so it spins up Supabase locally. Will allow
for end-to-end tests against the local DB.
Also small alteration to one of the test files to silence an annoying
(but intended) error.
# Before
Crawlers pages that used MDX components with defined attributes errored.
Example: /docs/reference/kotlin/installing
# After
The pages do not error.
* Bump all versions of postcss to 8.5.3.
* Run next/codemod on the docs app.
* Move two experimental flags into stable. Add next-mdx-remote as a transpiled package.
* Add extra folders to the clean command in docs.
* Fix type errors in docs test.
* Run prettier on the new files.
* remove turbopack, fix fetch revalidation, fix metadata awaits
Couple of minor fixes:
- Turbopack doesn't work in dev because of known MDX loader limitations
(cannot load functions in MDX plugin config)
- Fetches not cached by default anymore in Next 15 so need to manually
cache the ones we need
- Missing a few awaits for metadata generation with page params
* Bump the graphiql version because headlessui/react is not building with Next 15.
---------
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
* feat(content api): add client library api reference search results
Allow searchDocs results to also return function references from the
client library APIs
* fix(content api): refine language enum handling
Add a top-level field to search docs globally. Right now this only
returns Markdown guides (not references, GitHub discussions, or partner
pages.
The full GraphQL schema at this point:
```
schema {
query: RootQueryType
}
type RootQueryType {
"""Get the GraphQL schema for this endpoint"""
schema: String!
"""Search the Supabase docs for content matching a query string"""
searchDocs(query: String!, limit: Int): SearchResultCollection
}
"""A collection of search results containing content from Supabase docs"""
type SearchResultCollection {
"""A list of edges containing nodes in this collection"""
edges: [SearchResultEdge!]!
"""The nodes in this collection, directly accessible"""
nodes: [SearchResult!]!
"""The total count of items available in this collection"""
totalCount: Int!
}
"""An edge in a collection of SearchResults"""
type SearchResultEdge {
"""The SearchResult at the end of the edge"""
node: SearchResult!
}
"""Document that matches a search query"""
interface SearchResult {
"""The title of the matching result"""
title: String
"""The URL of the matching result"""
href: String
"""The full content of the matching result"""
content: String
}
```
Towards DOCS-214
* feat (content api): add top-level schema query
Adds a top-level schema query that returns the stringified GraphQL
schema for this endpoint. Useful for LLMs to access the stringified
schema for prompting and tool calls.
Query shape:
```
query Search {
schema
}
```
Query response:
```
{
"data": {
"schema": "schema {\n query: RootQueryType\n}\n\ntype RootQueryType
{\n \"\"\"Get the GraphQL schema for this endpoint\"\"\"\n schema:
String\n}"
}
}
```
Towards DOCS-214
* test (content api): add schema snapshot
Add schema snapshot test to prevent unintentional changes to public API
for this endpoint
* fixup! feat (content api): add top-level schema query
* fix (content api): local dev introspection
The validation rules that are used for security in production are
preventing GraphiQL from introspecting the schema in dev. Since this DX
feature is incredibly useful, I'm turning off the complexity and depth
validations for dev GraphiQL only. Local curls, etc., will still have
production-standard validation so it can be tested locally.
GraphQL errors were occasionally returned as normal HTTP status code
errors, switched to using only GraphQL-spec compliant errors for the
/docs/api/graphql endpoint.
Continue building out the docs/api/graphql endpoint by adding more
validation:
-A query depth limiter
- A query complexity limiter
Also added dummy (empty) root resolver and schema, plus a dev-only
GraphiQL page for better DX.
Endpoint should still return a 404 for now as there is no resolver.
The first PR in a series to build out the Content API. Adds a dummy
GraphQL route with some tests: for now all this does is validate the
request body and return a 404.
Before:
Revalidation was turned off entirely for wrappers pages, and done daily for GraphQL pages.
After:
Revalidation is via tag. There is an /api/revalidate endpoint that is used by GitHub Actions on the federated repos to trigger the revalidation. This decreases the number of unnecessary page generations by Vercel, and gives the maintainers of federated repos more fine-grained control over when to reploy their docs (see the docs deployment playbook for more info).
Before:
There was a bug with middleware rewrites for reference pages without
crawler versions (basically all references except the client SDKs).
There was a branch that was not handled (the requesting user agent is a
a bot, but the reference is not a client SDK), so it failed through to
the base case of NextResponse.next(), which 404s as that page does not
exist for a path like /reference/api/v1-deploy-a-function
After:
Fixed the branch. If all of the following are true:
- Is a reference page
- Requesting user agent is a bot
- Reference is not a client SDK
Then it should follow the same path as a non-bot request, because there
is no bot-specific version. That means it should be rewritten to teh
slugless page: /reference/api/v1-deploya-function => /reference/api
Add a route for manually revalidating cache contents by tag.
The route is protected by authentication to prevent abuse. Automated
actions in CI should be set up with a basic API key, which has a rate
limit of 6 hours between changes. Overriding is possible with an
override key, which should be used as an escape hatch.
Usage:
- API key provided in header `Authorization: Bearer <KEY>`
- Body has shape `{ tags: string[] }`
The end of the Great Migration is here!!!
Moves the last pages over, deleting pages like the FAQ that we don't use
and that contain duplicated information anyway.
Dev secret auth page URL had to change as App Router doesn't like the
leading underscores in the path.
Also fixes the not-found recommendations to use the proper Next.js
not-found page so it will return a 404 as it should. (I was under the
erroneous impression that I couldn't get the pathname in not-found.tsx,
that is not true, so this works better.)
