mirror of
https://github.com/supabase/supabase.git
synced 2026-09-09 11:30:17 +08:00
21265b2e59
Ref DOCS-1274 Follow-up to #49017, now merged. This is the go-to-green piece: everything aimed at the three failing eval checks, and nothing else. Technical corrections follow in the PR stacked on this one. ## Problem `build-docs-002-rls-guide` points an agent at this guide with a vibe-coder prompt that never says RLS, policy, role, or test. Grants, policies, access probes, indexes, and security-definer placement all pass. Three checks fail, and have failed on every recorded run: | Check | What it measures | Why it failed | | --- | --- | --- | | `pgTAP test file(s) written under supabase/tests/` | Any `.sql` file exists | The agent never wrote one. | | `supabase test db runs at least 8 assertions and all pass` | Suite runs, ≥8 assertions, none failing | Nothing to run. The only example was `plan(4)`, under the floor even if copied perfectly. | | `tests assert allow and deny per operation … for anon and authenticated` | LLM judge on coverage | Never reached the judge: "no test files to review". | The guide already had a `Test your policies` section, so this isn't a strength problem. Agents don't read the page. They fetch it through an LLM extraction guided by their own query, and that query asked for enabling RLS, policy syntax, `auth.uid()`, indexes, and security definer functions. It never mentioned tests. A section about testing never enters the extract, so more testing prose cannot reach the agent. There was also a plain documentation bug underneath it: `Secure a table with RLS` said a table isn't secured until the suite passes, but the procedure beneath it ran 1–3 and ended on `grant`. A reader following the numbered steps finished without ever being told to write a test. ## Solution Put the tests where the procedure and the examples already are. - **`Secure a table with RLS`** opens with the four steps that finish a table, ending on `supabase test db`. Until the suite passes, you don't know whether the policies do what you intended. - **`Enable RLS and set the grants` gains step 4** — `supabase test new <table>_rls.test`, then `supabase test db`. The procedure ends on a passing suite instead of a grant. - **The public-read example** gains its policy and `announcements_rls.test.sql`, so a test file rides along in the enable-RLS extract. - **The four policy examples** are followed immediately by `profiles_rls.test.sql`, so one rides along in the `create policy` extract too. - **`Run the test suite` shrinks** to creating and running the files. It no longer carries content that has to survive extraction. - Each file leads with its own path as a comment, so it survives if the fence metadata is dropped. ### How that maps to the three checks | Check | Addressed by | | --- | --- | | Test files written | A complete test file now sits inside both extracts an agent's own query pulls, and step 4 of the procedure names the command that creates one. | | ≥8 assertions, all passing | `announcements_rls.test.sql` is `plan(10)`, `profiles_rls.test.sql` is `plan(14)`. Either alone clears the floor; together, 24. | | Coverage judge | `profiles` asserts allow **and** deny for all four operations. Allowed writes use `returning` + `results_eq`, proving state changed rather than that nothing raised. `using`-filtered denials use `is_empty`, asserting the row is unchanged rather than that an error was raised — the case the rubric explicitly fails suites for getting wrong. Both files switch role with `set local role` and identity with `set local request.jwt.claim.sub`, and cover `anon` as well as `authenticated`. | ## Manual testing 1. Open the [Row Level Security guide](https://docs-git-docs-rls-tests-in-procedure-supabase.vercel.app/docs/guides/database/postgres/row-level-security) on the preview. `Secure a table with RLS` opens with a four-step definition of done ending on `supabase test db`. 2. Read `Enable RLS and set the grants`. The procedure runs 1–4 and ends on writing and running the test, not on the grant. 3. Scroll to `DELETE policies`. The four policies are followed immediately by `profiles_rls.test.sql`, not a pointer to a later section. 4. Open the [markdown version](https://docs-git-docs-rls-tests-in-procedure-supabase.vercel.app/docs/guides/database/postgres/row-level-security.md), which is what agents fetch. Both test files are present, each leading with its path. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Documentation - Updated database security guidance for enabling row-level security and configuring grants. - Added per-table pgTAP testing requirements and revised `supabase test db` examples. - Expanded examples for permitted and denied access across public and authenticated roles. - Added dedicated guidance for profile testing and security-definer member/non-member cases. - Documented recursive-policy `42P17` failures and the security-definer workaround. - Clarified indexing, denial diagnosis, returned-row verification, and table-hardening links. - Streamlined the general policy-testing guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>