Pages that are disabled in navigation/site rendering are still shown in
llms.txt. This change filters those pages out of the llms.txt build as
well.
Co-authored-by: Chris Chinchilla <chris.ward@supabase.io>
* refactor: reading markdown docs files
Refactor how Markdown docs files are read:
- Reuses the same logic across search index generation & page generation
- Improves the indexed content for search:
- Stops removing MDX components, which often contain useful
information like Admonitions
- Denormalizes Partials and CodeSamples for more complete content
This is a prerequisite step for implementing the "Copy docs as Markdown"
functionality.
Only touches regular guides for now, not federated ones.
* fix: tailwind build error (#37728)
We changed to default to ESM imports a while ago, which means local
builds are now breaking because the Tailwind uses a require. Changed to
CJS for Tailwind config file. (I have no idea how this has been working
on Vercel all this time.)
* style: prettier
strictNullChecks was off for docs, which lets errors slip through and
leads to incorrect required/optional typing on Zod-inferred types. This
PR enables strictNullChecks and fixes all the existing violations.
Scripts currently use CJS, which is causing a bit of a mess when trying
to use shared utilities from the app. Converting everything to ESM so
there are fewer conflicts when adding new scripts going forward.
* feat: llms.txt
* feat: split llms.txt into multiple files
We have too many docs, so the concatenated text file uses an unreasonable amount of tokens. Chunk it up a little so it's more usable.
Begin the process of moving our MDX files into their own content directory.
Fixed a few minor bugs re: ToC and tabs not rerendering consistently on page navigation. (The ToC thing wasn't a problem before the refactor, the tabs thing is a problem on prod.)
Moved MDX files can't import their own components, so everything they require needs to be back in the component prop for mdx-remote's serializer. Cleaned this up a bit and lazy-loaded heavy/rare stuff. Also, the component prop doesn't take arbitrary objects (only actual components), so imported data has to be wrapped in a component.
Improve the way docs are indexed and FTS results are ranked, so results are more relevant.
Also improve debouncing and searching UX so it feels a bit faster and you can scan the results more easily.
