Commit Graph

105 Commits

Author SHA1 Message Date
Saxon Fletcher
9b17ce8f2c chore(studio): default assistant to GPT-5.6 Luna (#49749)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature / chore: hide assistant model selection in the UI and default
chats to GPT-5.6 Luna.

## What is the current behavior?

The assistant composer exposes a model picker. Paid orgs default to
`gpt-5.3-codex`; everyone else defaults to `gpt-5.4-nano`.

## What is the new behavior?

- The model picker is hidden in the assistant composer and Explorer
home.
- Chats default to `gpt-5.6-luna` with `reasoningEffort: medium`.
- Model selection plumbing is kept (registry, entitlements, `setModel`,
generate-v4 request body) so a requested model can still be honored when
provided.
- Other completion endpoints still use `gpt-5.4-nano`.

## Additional context

Model selector UI can be re-enabled by passing `selectedModel` /
`onSelectModel` to `AssistantChatForm`.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for the GPT-5.6 Luna model with medium reasoning
capability.
  * Made GPT-5.6 Luna the default assistant model.
* **Improvements**
* Simplified assistant chat by removing model selection from the primary
chat experience.
  * Updated model fallback behavior to use the standard assistant model.
* Chat forms can now optionally display model selection when configured.
* **Tests**
* Updated model coverage and assistant chat tests for the new defaults
and behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-02 10:24:37 +10:00
claude[bot]
8790e657e9 feat(ai): include org slug in Assistant Braintrust span metadata (#49692)
<!-- ccr-slack-attribution -->
_Requested by **Matt Rossman** · [Slack
thread](https://supabase.slack.com/archives/D0A79RYJKRB/p1787926891744399)_


# Problem

Assistant spans in Braintrust record only the numeric `orgId`, whereas
support tickets show org slug.

This incurs an extra manual step to resolve the ID through admin studio
before the trace can be found.

# Fix

Adds `orgSlug` to spans, sourced from the same verified org lookup that
produces `orgId`.

Renamed the request body's `orgSlug` to `rawOrgSlug` to distinguish the
verified slug from getAIDetails, following the existing
rawRequestedModel / requestedModel pattern.

## How to review

See sample trace
[94863b6d-aaa9-449a-a9c9-981ad40e614a](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=223112cd-33f4-45c4-a273-8d3781689448&s=223112cd-33f4-45c4-a273-8d3781689448)
produced from sending a chat from the
[Preview](https://studio-staging-git-mattrossman-ai-1149-include-698a5f-supabase.vercel.app/dashboard/org)
on this PR.

Note it now includes the org slug in span metadata:

<img width="873" height="548" alt="CleanShot 2026-08-28 at 10 59 49@2x"
src="https://github.com/user-attachments/assets/bcf47a94-6782-434a-9006-c7b9c95f1c37"
/>

If desired you can test yourself too by chatting with Assistant in the
preview and looking up the corresponding Chat ID from Braintrust
[logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs).

Closes AI-1149

---
_Generated by [Claude
Code](https://claude.ai/code/session_01N2ziJech9dV19pJ9MisYdX)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-28 11:48:38 -04:00
Charis
ebd616fa90 fix(studio): auto-retry notebook updates on stale/invalid conflicts (#49323)
## Summary

- **Removed dead client-side refresh UI** in
`NotebookProposalRenderer.tsx` and its test — the diff preview is always
computed from live data, so the check was redundant with the tool's
server-side re-validation
- **Added typed `NotebookToolError`** in `notebook-tools.ts` with
structured metadata (`{ exposeToAssistant: boolean }`) validated by a
zod schema with a literal discriminant tag (`tag:
'notebook_tool_error'`) — tracks the two retryable failures: staleness
conflict and invalid operations (unknown cell id)
- **Encoded errors in `generate-v4.ts` onError** — the one place in the
pipeline that holds the live `Error` before it becomes a string in the
persisted message
- **Extracted and fixed message history filter** into new
`generate-assistant-response.utils.ts` — any tool-error whose
`errorText` decodes against the `NotebookToolError` schema is let
through (with `errorText` rewritten to plain prose so the model sees the
message, not JSON), while other errors stay filtered as before

Net effect: the assistant detects the specific, actionable rejection
reason and retries on its own with no dead button or human intervention
needed.

## Test plan

- Existing unit tests in `NotebookProposalRenderer.test.tsx` pass (dead
button test removed)
- New unit tests in `notebook-tools.test.ts` cover encode/decode
round-trips and error discrimination
- New unit tests in `generate-assistant-response.utils.test.ts` cover
message history filtering with all error states
- `pnpm typecheck` is clean
- `pnpm --filter studio run lint:ratchet` passes (no new ESLint
warnings)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Notebook update errors now provide clearer, structured explanations to
the AI assistant.
* Assistant responses preserve relevant notebook error details while
filtering invalid or temporary tool states.
* **Bug Fixes**
* Improved handling of stale notebook revisions and invalid notebook
update operations.
* Notebook proposal rendering proceeds without an unnecessary refresh
step.
* **Tests**
* Expanded coverage for notebook errors, message filtering,
serialization, and error handling.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-20 16:08:19 -04:00
Saxon Fletcher
c80f8ad78d chore(studio): upgrade AI SDK to v7 (#49167)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Chore / dependency upgrade.

## What is the current behavior?

Studio is on AI SDK 6 (`ai` ^6.0.174, `@ai-sdk/react` ^3). Tool
approvals still use the v6 `needsApproval` flag on individual tools.

## What is the new behavior?

Upgrades Studio to AI SDK 7 (`ai` 7.0.59) and the matching `@ai-sdk/*`
packages. Aligns call sites with v7 names (`instructions`,
`isStepCount`, `onEnd`, `ToolExecutionOptions`).

This is the bottom of stack #49171. Later layers add a shared Confirm
card and AssistantQueryCell.

## Additional context

- Stack: #49167#49168#49169#49170
- `needsApproval` on tools is left as-is in this PR so the upgrade can
land independently. A follow-up can move those gates to `streamText({
toolApproval })` and `experimental_toolApprovalSecret`.
- Independent of the notebook preview stack
([#49112](https://github.com/supabase/supabase/pull/49112),
[#49159](https://github.com/supabase/supabase/pull/49159)), which should
merge first before we wrap notebook proposals in Confirm.

## Test plan

- [ ] `pnpm --filter studio test` for `lib/ai/tools/*` and assistant
generate path
- [ ] Assistant chat still streams and tool-approval SQL / Edge Function
still pause for confirm
- [ ] Evals still run with mock tools (`needsApproval: false` overrides)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Improvements**
* Updated AI-powered chat, onboarding, SQL, code completion, and recipe
generation workflows for more reliable responses.
* Streaming responses now better preserve reasoning and source
information where available.
* Improved tool privacy notices while preserving dynamically generated
tool descriptions.
* Refined AI response handling, including step limits and structured
policy results.
* **Bug Fixes**
* Improved compatibility across AI-powered tool interactions and
execution scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-18 17:04:36 +08:00
Charis
ddb3e2c442 feat(studio): create_notebook AI tool (#48938)
## Summary
- Adds a `create_notebook` AI assistant tool (`needsApproval: true`)
that lets the assistant create a new notebook after explicit user
approval.
- Cell SQL is promoted from untrusted to safe via
`acceptUntrustedSql`/`acceptUntrustedLogsSql` inside `execute`, using
the approval gate as the confirming user gesture (same pattern as
`execute_sql`).
- Input is validated against the existing agent-writable notebook
schema, which rejects any agent-supplied cell `id` at the schema level.
- Threads an optional auth-headers param through
`upsertContent`/`createNotebook`/`updateNotebook` so the tool can pass
its own bearer token server-side.
- Registers the tool in the tool-filter (`SCHEMA` category, alongside
`list_notebooks`/`get_notebook`) and adds a `## Notebooks` prompt
section guiding the assistant on when to use `create_notebook` vs.
one-off `execute_sql`.

Resolves FE-4082

## Test plan
- [x] `notebook-tools.test.ts` covers: tool registration,
`needsApproval`, cell-id rejection, valid input, PUT body shape, and the
returned id — all passing
- [x] Typecheck clean
- [x] Lint clean (no new warnings)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added AI-assisted notebook creation for saving multi-step
investigations.
* Added support for database and log SQL cells in newly created
notebooks.
* Notebook creation requires approval before saving and returns the
notebook’s name and identifier.
* Added support for custom request headers during notebook and content
operations.
* Added guidance for choosing between one-time SQL execution and
reusable notebooks when Explorer is enabled.

* **Improvements**
* Improved validation and normalization of notebook content before
saving.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-11 11:54:49 -04:00
Charis
7798e42435 feat(studio): notebook read tools (#48908)
## Summary
- Adds `list_notebooks` (cursor-paginated) and `get_notebook` AI tools
in `lib/ai/tools/notebook-tools.ts`, modeled directly on
`report-tools.ts`: server-side `getContent`/`getNotebook` with the
`authorization` header forwarded, zod-validated input.
- `get_notebook` resolves every cell and exposes `unchecked_sql` as a
plain `sql` field for the agent to read — display only, per the
`safe-sql-execution` skill; nothing here executes SQL.
- Registers both tools in `lib/ai/tools/index.ts` (same platform branch
as reports) and in `lib/ai/tool-filter.ts`'s `toolSetValidationSchema` +
`TOOL_CATEGORY_MAP` (`SCHEMA` tier).
- Adds an optional `headers` param to `content-infinite-query.ts`'s
`getContent`, mirroring the sibling `content-query.ts`, so the
cursor-paginated fetch can carry the `Authorization` header from a
server context.
- New tools are behind the Explorer feature flag.

Stacked on #48907 (1.4 — notebook query and mutation hooks), per the
Notebooks implementation plan (stack 2.1).

Resolves FE-4081
Resolves FE-4080

## Test plan
- [x] `pnpm exec tsc --noEmit` — no new errors
- [x] `pnpm exec vitest run lib/ai/tools/notebook-tools.test.ts
lib/ai/tools/index.test.ts lib/ai/tools/report-tools.test.ts
data/content/notebooks` — 36/36 passing
- [x] `pnpm --filter studio run lint` — no new warnings
- [x] `pnpm exec prettier --check` on changed files — clean

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added AI tools to list project notebooks with pagination.
* Added AI support for retrieving notebook markdown and resolved SQL
cell content.
  * Notebook tools now respect project and authorization context.
* Notebook features are available only when Explorer access is enabled.
  * Content requests can forward custom request headers.

* **Tests**
* Added coverage for notebook tools, Explorer access, feature flags,
authorization, pagination, and error handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-11 08:40:51 -04:00
Ali Waseem
b04d14856a Resolve AI opt-in and tracing settings server-side (#48855)
The AI endpoints resolved organization and project settings
independently and applied them together without confirming they belonged
to the same pairing. Consolidates both into a single `getAIDetails` that
reconciles them and falls back to the most restrictive posture when
unconfirmed, and applies the HIPAA sensitivity gate to the opt-in level,
which previously only existed on the client.

Fixes FE-4110

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Consolidated AI access details across organization and project
settings.
- AI functionality now validates project ownership and disables access
for mismatched or HIPAA-sensitive projects.
- AI responses include plan, region, opt-in status, sensitivity,
authorization, and advanced model access information.

- **Bug Fixes**
- Improved fail-closed behavior when project or organization data is
missing or inconsistent.
- Updated AI generation, feedback, rate, and policy flows to
consistently apply consolidated access settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-10 10:34:31 -06:00
Charis
21511042a3 feat(studio): assistant logs context and reports guard (#48514)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature — final PR (9/9) of the SQL editor logs-source stack.

**Base branch:** `charislam/sql-editor-inline-ai-clickhouse-dialect` (PR
8). Nothing here is user-visible: entry points stay behind
`sqlEditorLogsSource` + `otelLegacyLogs`, and flag rollout happens after
the whole stack merges.

## What is the current behavior?

- The Assistant has no idea a SQL editor snippet targets the logs
backend. Ask it about a logs snippet and it answers in Postgres, because
the attached query is fenced as ` ```sql ` and nothing tells the model
otherwise.
- Because the `sql` fence is what `MessageMarkdown` treats as runnable
Postgres, an attached ClickHouse query is rendered with a
Run-against-Postgres affordance and branded with `untrustedSql`.
- "Debug with Assistant" on a failed logs query produces a dialect-less
prompt, so both the in-app assistant and the copyable version get
debugged as Postgres.
- A report referencing a `log_sql` snippet runs its ClickHouse SQL
against the user's Postgres database and surfaces the resulting error.

## What is the new behavior?

**Assistant panel.** The "Current Query" chip records which backend the
attached query targets. That reaches the model two ways: each attachment
is fenced with its own dialect (` ```clickhouse ` vs ` ```sql `), and a
`containsLogsSnippets` flag rides on the user message as AI SDK
`metadata`. The server reads the flag off the conversation and prepends
the ClickHouse dialect rules plus the logs schema reference as a
non-cached context message.

Two design points worth calling out in review:

- The flag lives on the **message**, not the request body, so Retry and
the tool-approval continuation reproduce the context a message was
originally asked in — neither of those passes a per-call body.
- It's derived from **what's actually attached**, so detaching the chip
drops the claim rather than leaving the two able to disagree.

The `clickhouse` fence also keeps a logs query out of
`MessageMarkdown`'s `sql` branch, so it's no longer offered as runnable
Postgres or branded with `untrustedSql` — a boundary this stack's
distinct brands exist to prevent crossing.

**Debug flow.** `buildDebugChatArgs` attaches its query with a source
for the same reason, and names the dialect in the prompt text so the
copyable version stands on its own outside the app.

**Reports.** A report only stores a snippet id, so whether it queries
the logs backend is only knowable once the content loads. `ReportBlock`
guards on the fetched type and renders a `LogsSnippetReportBlock`
placeholder instead of executing. Double-guarded: no `sql` for a logs
snippet (so it's out of the query key and `queryFn` short-circuits even
on an explicit `refetch`) and `enabled` excludes it.

**Incidental cleanups.** `buildAssistantContextMessages` extracted out
of `generate-assistant-response`; a schema-access sentinel that was
duplicated as a string literal across two files (and compared against)
replaced with one exported constant; `SqlSnippet` deduplicated to a
single declaration; `resolveSnippetSource` / `isLogsSource` shared
instead of re-implemented per surface.

**Tests.** 4 new/extended suites. Notable cases pinned: a message with
no metadata must validate (`safeValidateUIMessages` applies
`metadataSchema` to *every* message, so a required schema would 400
every existing conversation); only *user* messages count, so a model
reply can't talk the server into a different dialect; a mixed-attachment
message is flagged without overclaiming a single source; and
`ReportBlock` registers no pg-meta mock for the logs cases, so an
unhandled request failing the test *is* the assertion that logs SQL
never reaches Postgres.

Verified: `pnpm typecheck`, `lint:ratchet` (no regression), Prettier,
and the full Studio suite (459 files / 4969 tests).

## Additional context

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Added support for recognizing log snippets in reports, with clear
guidance to open them in the SQL editor or remove them.
- AI Assistant now understands log snippets and provides
ClickHouse-specific context, formatting, and troubleshooting guidance.
- Snippets retain their source information when shared with the AI
Assistant.

- **Bug Fixes**
- Prevented unsupported log snippets from being executed as regular
database queries.
  - Improved source detection when opening snippets directly from links.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-04 09:02:40 -04:00
Joshen Lim
dc23320e43 Add sentry capture exception to apiWrapper (#47804)
## Context

As per PR title - also adjusts the imports for files consuming
`apiWrapper` to remove the default export for `apiWrapper`

Have tested locally by throwing an error in one of the API routes -
verified that the event shows up on Sentry

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* API errors are now captured in Sentry before returning server error
responses, improving production visibility while keeping endpoint
behavior the same.
* **Tests**
* Added coverage to confirm rejected handler executions are reported to
Sentry and return the expected HTTP 500 JSON payload.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-10 16:28:42 +08:00
Carel de Waal
0acc0eb8b3 feat: Support Form - Sync AI assistant conversation to Front (#46778)
# Sync AI assistant conversation to Front

## What & why

When a user submits a support ticket, an AI assistant chat opens so they
get help
immediately while waiting for a human agent. This PR mirrors every turn
of that chat into
the Front conversation the support form already created, so the support
team sees the full
context and Front automations (routing, emails, CSAT) can act on it.

Studio holds no Front credentials — it calls the platform endpoints (see
the platform PR)
to do the syncing. The assistant card is gated behind the
`supportAssistantFollowUp`
ConfigCat flag.

## How it works

1. **Submit** — `SupportFormV3` generates a stable `threadRef` (via the
`uuid` package —
`crypto.randomUUID()` is `undefined` in insecure contexts like
non-localhost HTTP and
would throw, silently aborting the submit) and sends it on
`/platform/feedback/send`.
The response returns the Front `conversationId`. Both are stored on
`SubmittedSupportRequest`.
2. **Open chat** — `SupportAssistantSuccessCardContent` opens a chat
seeded with
`supportMetadata` (`threadRef`, `frontConversationId`, subject,
category, severity, …).
   The first message is a `<support>…</support>` XML block.
3. **First user message** — the chat is tagged `isSupportChat = true`;
the `onFinish` hook
   fires `syncSupportChatToFront`.
4. **Subsequent turns** — each `onFinish` slices the unsynced delta,
strips the XML
metadata block from the seed message, and posts to the platform messages
endpoint.
5. **Escalation / resolve** — the `escalate_to_human` /
`resolve_support_conversation` tools
(and manual **Escalate**/**Resolve** buttons in the assistant input)
flip lifecycle status
via `setSupportLifecycleStatus` → `syncSupportLifecycleToFront`, which
calls the
escalation/resolve endpoints. Front rules act on `ai_support_status`.
The assistant only
   resolves after the user explicitly confirms the issue is fixed.

## Key design decisions

- **`threadRef` as the shared key** — one UUID travels as `threadRef` on
submit and as
`chatId` on every sync, so all messages thread into a single Front
conversation.
- **`conversationId` from the form response** — passed to all
sync/lifecycle calls so the
  platform skips lazy derivation and PATCHes custom fields directly.
- **Delta-only sync** — `lastSyncedMessageCount` tracks what's been
sent; the boundary is
snapshotted before the async call to avoid skipping messages that arrive
mid-flight.
- **Server-side de-dup** — stable `external_id` (`chatId:msg.id`) means
retries don't
  duplicate in Front.
- **Fire-and-forget** — sync failures log to Sentry, never break the
chat; `isSyncing`
resets on rehydration so the next `onFinish` retries the same delta.
Message and lifecycle
syncs use separate guards (`isSyncing` / `isLifecycleSyncing`) so an
in-flight message
  sync can't drop an escalate/resolve.
- **Lifecycle queued until the conversation exists** — if a lifecycle
transition is requested
before the initial message sync has returned a `frontConversationId`,
it's stored as
`pendingLifecycleStatus` and flushed once the id is assigned, rather
than dropped.
- **Tools return immediately** — the lifecycle tools return a stub to
the AI SDK; the real
Front call happens in `onFinish`, keeping async I/O out of the tool
execute path.
- **XML seed stripped before sync** — only the user's actual `<message>`
is sent to Front
  (or dropped entirely if the form already created the conversation).

## Changes

| Area | File(s) |
| --- | --- |
| Support form state | `SupportForm.state.ts` — `threadRef` /
`frontConversationId` on `SubmittedSupportRequest` |
| Support form submit | `support-ticket-send.ts` — sends `threadRef`,
reads `conversationId` |
| Support form UI | `SupportFormV3.tsx` — generates `threadRef`, stores
`conversationId` |
| AI assistant state | `ai-assistant-state.tsx` — `SupportChatMetadata`,
`setSupportLifecycleStatus`, `onFinish` wiring, tool handling |
| Message sync | `state/ai-chat-front-sync.ts` — delta tracking, message
filtering, initial vs. incremental |
| API data layer | `data/feedback/ai-chat-front-sync.ts` — typed
platform-client wrappers for the three conversation endpoints |
| Support tools | `lib/ai/tools/support-tools.ts` — `escalate_to_human`,
`resolve_support_conversation` |
| Tool integration | `lib/ai/tool-filter.ts`, `tools/index.ts`,
`generate-assistant-response.ts` |
| Success card | `SupportAssistantSuccessCardContent.tsx` — tags chat on
first engagement |
| Assistant panel UI | `AIAssistant.tsx` — Escalate/Resolve buttons,
disabled input on closed chats, support placeholders |



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
  - Support chats now include “Escalate to human” and “Resolve” actions.
- Support submissions can be associated with a stable Front thread via a
generated `threadRef`, preserving linkage across follow-ups.
- AI assistant responses and input hints adapt when support mode is
active.

- **Bug Fixes**
- Improved support chat state management and lifecycle handling to keep
conversation metadata and message history synchronized more reliably
with Front.

- **Chores**
- Added/updated coverage to reflect the new support-chat state and
syncing behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
2026-07-08 12:41:53 +02:00
Pedro Rodrigues
c4c213ce3d feat(studio): switch dashboard assistant to remote MCP server (#47479)
## I have read the
[CONTRIBUTING.md](<https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md>)
file.

YES

## What kind of change does this PR introduce?

Feature / refactor.

## What is the current behavior?

The dashboard assistant runs `@supabase/mcp-server-supabase` in-process
over an in-memory transport (`lib/ai/supabase-mcp.ts`).

## What is the new behavior?

The assistant connects to the **remote MCP server** over HTTP
(`@ai-sdk/mcp`), forwarding the dashboard session token as a bearer. URL
comes from `NEXT_PUBLIC_MCP_URL` with a local-dev fallback;
platform-only, and Nimbus works via the same env var.

* **Tool model unchanged:** UI-controlled `execute_sql` (with
`needsApproval`) and `deploy_edge_function` still come from Studio; the
allowlist (`TOOL_CATEGORY_MAP`) remains the gate keeping the remote's
write tools away from the assistant (`read_only` is defense-in-depth).
* **Attribution:** sends `x-source-name: supabase-studio` (+
`x-source-version`) → logged as `source_name`/`client_name`.
* **Connection lifecycle:** the HTTP client is closed via the request's
`AbortSignal` (tools execute later during streaming); `signal` is
required on `getTools`/`getMcpTools`.
* **Resilience:** a remote-MCP failure degrades to the remaining tools
instead of failing the assistant.
* **Drift protection:** relied-upon tools are typed against `keyof
typeof supabaseMcpToolSchemas`, so a package bump that renames/removes
one fails `pnpm typecheck`; a runtime check also warns if the deployed
server returns fewer tools.
* Adds unit tests for the above.

## Additional context

* Verified end-to-end against a local remote MCP server with a dashboard
token: `initialize` 200, tools listed, a tool executed, client closed
cleanly.
* The remote MCP (mgmt-api) already accepts dashboard session tokens
(GoTrue-JWT auth path) — no backend change needed. `NEXT_PUBLIC_MCP_URL`
must point at each env's `/mcp`.
* `@supabase/mcp-server-supabase` is kept — still used by the
self-hosted `/api/mcp` routes.

Closes
[AI-137](https://linear.app/supabase/issue/AI-137/switch-dashboard-assistant-to-remote-mcp)

## Rollout

* **Rollout:** merges with `USE_REMOTE_MCP` off (in-process); flip it to
`true` per environment (staging → prod → Nimbus) once each one's
prerequisites land.
* **Rollback:** unset `USE_REMOTE_MCP` and redeploy to fall back to the
in-process client — no revert needed.

## Summary by CodeRabbit

* **Bug Fixes**
* Improved AI request handling so tool loading and generation clean up
properly when a request is cancelled or the browser connection closes.
* Added safer fallback behavior when remote tool loading fails, so AI
features can continue with available tools instead of stopping entirely.
* Updated remote tool access to use the current project reference and
preserve the correct access headers.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* AI tools now connect more reliably to remote services and stop cleanly
when requests end or are canceled.
* Tool loading is more resilient, continuing with available tools if
remote access is unavailable.

* **Bug Fixes**
* Improved cleanup to prevent lingering connections during SQL
generation and policy workflows.
  * Added safer handling for remote tool changes and invalid responses.

* **Tests**
* Expanded automated coverage for remote tool setup, cancellation, and
fallback behavior.


<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:38:21 +01:00
Joshen Lim
1baaded0bb Consolidate execute-sql-query into execute-sql-mutation (#46944)
## Context

Just some clean up as I was going through stuff
- `useExecuteSqlQuery` is deprecated and not used at all
- As such `execute-sql-query` is technically irrelevant, the more
relevant file is `execute-sql-mutation`
- Hence opting to consolidate `execute-sql-query` into
`execute-sql-mutation`
- Also removing `ExecuteSqlError` since its just re-exporting the
`ResponseError` type

There's a lot of file changes but its essentially just updating the
importing statements across the files
2026-06-16 00:07:16 +08:00
Matt Rossman
d143571586 feat(assistant): trace-level scorers + server-side tool execution with needsApproval (#45654)
## Motivation

When Assistant runs a potentially destructive tool like `execute_sql`,
it stops the LLM request and prompts for client-side approval and
execution of the tool. After approval, a second request kicks off under
a separate trace. This has made scoring and
[Topics](https://www.braintrust.dev/blog/topics) classification
challenging, as the generated `output` is split across stateless
requests. The [span-level
scoring](https://www.braintrust.dev/docs/evaluate/custom-code#score-spans)
approach we've used thusfar (after the LLM call, we massage the result
into an `output` payload that's stuck onto the root span) has been
cumbersome and led to invalid scores / topics where only part of the
assistant response is considered. It's also inefficient, as we're
duplicating potentially large info (like the `search_docs` output) that
already exists within the trace.

An alternative to scoring spans is to [score
traces](https://www.braintrust.dev/docs/evaluate/custom-code#score-traces).
Braintrust [best
practices](https://www.braintrust.dev/docs/evaluate/score-online#best-practices)
advise:

> Use span scope for evaluating individual operations or outputs. Use
trace scope for evaluating multi-turn conversations, overall workflow
completion, or when your scorer needs access to the full execution
context.

We've also received [direct
guidance](https://supabase.slack.com/archives/C05QYJBLX89/p1777925770927149?thread_ts=1777905716.911979&cid=C05QYJBLX89)
from their team to use this approach.

## Changes

Migrates eval scorers from custom `AssistantEvalOutput` shape to
trace-level scoring via `trace.getThread()` / `trace.getSpans()`, with
thread parsing that scores the full latest Assistant turn and passes
prior conversation separately where relevant.

Moves `execute_sql` and `deploy_edge_function` from client-side
execution after approval to AI SDK `needsApproval` + server-side
`execute()`. SQL results returned to the model are gated by AI opt-in
level, so row data is only included with `schema_and_log_and_data`;
otherwise the tool returns the no-data-permissions sentinel.

Adds `metadata.isFinalStep` to disambiguate multiple LLM requests within
an "assistant" turn due to tool call requests/responses. For online
evals, this means we should configure automations to only score traces
with `metadata.isFinalStep = true` to ensure we're judging the complete
generated response.

Other minor kaizen changes:
- Renamed `promptProviderOptions` to `systemProviderOptions` to clarify
that this is associated with the "system" message and disambiguate from
the root `providerOptions`
- Adds `evals/trace-utils.ts` to handle Zod validation of the `unknown`
span shapes from Braintrust, to more easily access typed inputs/output
on tool spans.
- Bumps AI SDK floor version `^6.0.116` → `^6.0.174`
- Tweaked the "Conciseness" scorer to not unfairly dock points for the
new `[called tool_name]` labels in serialized assistant response

## Verification

In the studio staging build, I asked Assistant to create a todos table
with 3 sample todos. I manually approved the `execute_sql` call and saw
Assistant generate text before & after the call.

In Braintrust I verified two traces were produced (see [filtered
logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?v=Staging&tvt=trace&search={%22filter%22:[{%22text%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22label%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22originType%22:%22btql%22},{%22text%22:%22%2560Chat%2520ID%2560%2520%253D%2520%25221cb2ac45-e5e7-458c-9da4-3bf6863b8842%2522%22,%22label%22:%22Chat%2520ID%2520equals%25201cb2ac45-e5e7-458c-9da4-3bf6863b8842%22,%22originType%22:%22form%22}]})),
the first with `metadata.isFinalStep = false` and the second with
`metadata.isFinalStep = true`.

In the Braintrust staging scorers, I ran the preview Completeness scorer
on the second trace and verified it sees the complete Assistant response
including markers for tool calls ([link to
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)/trace?object_type=project_logs&object_id=b5214b62-ad1e-4929-9d5b-40b1daebe948&r=0ed0a4f8-8aff-4a34-bb1d-1df1d88a5070&s=ff9015f8-6bf7-4ab3-83a9-ca4e69e27e82))

<img width="1193" height="960" alt="CleanShot 2026-05-07 at 11 27 10@2x"
src="https://github.com/user-attachments/assets/509d4858-c3a1-4068-986d-3aa4d5617d1a"
/>

I also tested the `deploy_edge_function` workflow and verified it still
prompts for permission and warns on deployment of existing functions.

**References**
- https://www.braintrust.dev/docs/evaluate/custom-code#score-traces
-
https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval

Supercedes https://github.com/supabase/supabase/pull/45556 and
https://github.com/supabase/supabase/pull/45339

Closes AI-473

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Tool actions (SQL execution, edge-function deploy) now require
explicit user Approve/Deny before proceeding.

* **Improvements**
* Assistant pauses for approval responses before sending follow-ups,
giving clearer control over risky actions.
  * Deploy/replace flows show confirmation and clearer replace warnings.
* Evaluation/scoring updated to use richer trace data for more accurate
assistant performance signals.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-12 15:24:21 -04:00
Charis
0433eeb5f5 feat(studio): mark sql provenance for safety (#45336)
Mark provenance of SQL via the branded types SafeSqlFragment and
UntrustedSqlFragment. Only SafeSqlFragment should be executed;
UntrustedSqlFragments require some kind of implicit user approval (show
on screen + user has to click something) before they are promoted to
SafeSqlFragment.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Editor and RLS tester show loading states for inferred/generated SQL
and include a dedicated user SQL editor for safer edits.

* **Refactor**
* Platform-wide SQL handling tightened: snippets and AI-generated SQL
are treated as untrusted/display-only until promoted, improving safety
and consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-04 13:08:06 -04:00
Joshen Lim
5f867e5f6c Feature Preview: RLS Tester (#45121)
## Context

Resolves FE-3077
Related discussion: https://github.com/orgs/supabase/discussions/45233

Verifying the correctness of your RLS policies set up has always been a
gap, as highlighted by a number of GitHub discussions like
[here](https://github.com/orgs/supabase/discussions/12269) and
[here](https://github.com/orgs/supabase/discussions/14401). As such,
we're piloting a dedicated UI for RLS testing (using role impersonation
as the base), in which you'll be able to
- Run a SQL query as a user (not logged in / logged in - this is the
role impersonation part)
- See which RLS policies are being evaluated as part of the query
- And hopefully be able to debug which policies are not set up correctly

Changes are currently set as a feature preview - and we'll iterate as we
get feedback from everyone 🙂 🙏

<img width="613" height="957" alt="image"
src="https://github.com/user-attachments/assets/83c37f8a-28fc-43b3-b0ff-e28571d8710c"
/>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* RLS Tester: run queries as anon or authenticated users, view inferred
SQL, per-table policy summaries, and data previews of accessible rows.
* UI preview: new RLS Tester preview card and modal with opt-in toggle;
RLS Tester sheet with role/user selector and query editor.
  * SQLEditor: “Explain” tab is always visible.

* **Chores**
* Added supporting API endpoints, background checks for table RLS
status, and a local-storage flag to persist the preview opt-in.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-28 15:02:49 +08:00
Matt Rossman
36ae9beb0c chore(ai): remove DPA signer killswitch for assistant tracing (#45134)
Removes the temporary killswitch added when Braintrust was onboarded as
a subprocessor, to satisfy the 30-day DPA notice obligation. The window
has elapsed and legal has cleared removal.

Drops the `orgIsDpaSigned` check from `isTracingAllowed`, removes the
extra `/platform/organizations/{slug}/documents/dpa-signed` network hop
from `getOrgAIDetails`, and cleans up all call sites and tests.

Closes AI-596

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Simplified AI tracing eligibility logic by removing DPA signing status
checks. Tracing authorization decisions now depend solely on region,
HIPAA addon status, and project sensitivity settings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-27 13:53:24 -04:00
Joshen Lim
7f5865872a Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context

Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix
all related issues
2026-04-27 17:42:34 +08:00
Charis
205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Matt Rossman
8aeacc6152 feat(assistant): disable Braintrust tracing for EU regions and DPA signers (#44504)
**Changes**

- Extracted tracing conditional to an `isTracingAllowed` helper with
unit tests (the function is simple but sensitive hence the extra testing
precaution)
- Disables Braintrust tracing for projects in EU database regions
(region prefix `eu-`) to address GDPR data residency concerns
- Disables Braintrust tracing for orgs whose owners have signed the
previous DPA, as a stopgap during the 30-day notice period for the
updated DPA that adds Braintrust as a subprocessor
- Refactored `org-ai-details.ts` → `ai-details.ts`, splitting
`getOrgAIDetails` into separate org and project helpers to cleanly scope
the EU-region check at the project level

DPA check uses the newly added `/documents/dpa-signed` endpoint from
https://github.com/supabase/platform/pull/31060. This PR includes
regenerated `api.d.ts` and `platform.d.ts` from running `pnpm codegen`
in `packages/api-types` to get type safety on this new endpoint.

Note tracing is still yet to be activated in production, this is a
preparatory step.

**To verify**

Send a chat message and check for the `x-braintrust-span-id` response
header on `POST /api/ai/sql/generate-v4` — it should be absent for
DPA-signed orgs or EU-region projects, and present otherwise.

<img width="3594" height="1992" alt="CleanShot 2026-04-03 at 14 28
58@2x"
src="https://github.com/user-attachments/assets/4c91d7ad-2604-4531-a78e-dedf41632fa5"
/>

If you have access to the Braintrust dashboard, you can also verify
whether logs are produced or not in the Assistant project there.


Closes AI-570
Closes AI-569

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Tracks organization DPA signing and detects EU-region projects
* Assistant tracing now follows a combined compliance policy (HIPAA
addon, DPA, project sensitivity, region)
  * Added helpers to fetch org and project AI details

* **Documentation**
* Expanded API docs with additional examples and clarified parameter
descriptions
  * Added response schemas for subscription preview and document status

* **Tests**
* Added/updated tests covering DPA/region behavior and tracing policy
enforcement
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-07 13:09:34 -04:00
Matt Rossman
adf8b0c67c feat(assistant): per-endpoint reasoningEffort + model config cleanup (#43981)
We're exploring support for newer models like
[gpt-5.4-nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/)
in Assistant. This model doesn't support the `'minimal'` reasoning
effort level we use for gpt-5-mini which leads to vague errors.

<img width="595" height="263" alt="CleanShot 2026-03-18 at 17 13 05@2x"
src="https://github.com/user-attachments/assets/cf7c2370-322d-4a8a-be55-23e680db0aa0"
/>


Also, we've [previously
discussed](https://supabase.slack.com/archives/C0161K73J1J/p1771544464850199?thread_ts=1771493920.775699&cid=C0161K73J1J)
that reasoning adds unnecessary latency to otherwise simple AI
completion endpoints like `title-v2`. We want more control of reasoning
level independent of model/endpoint.

This PR aims to solve both problems by:
- making reasoning effort configurable on a per-request basis
- adding compile-time guardrails to prevent selecting an incompatible
reasoning level for models
- adding a `DEFAULT_COMPLETION_MODEL` with minimal reasoning that we can
update with newer models that support disabling reasoning (independent
of Assistant chat model reasoning)

Other improvements to our model config logic:
- Fixes bug in `onboarding/design.ts` and `assistant.eval.ts` where
`providerOptions` was being dropped
- `getModel()` now returns a bundled `modelParams` object (spread into
AI SDK calls) so `providerOptions` can't be accidentally omitted (this
[has happened
before](https://supabase.slack.com/archives/C0161K73J1J/p1771518443534309?thread_ts=1771493920.775699&cid=C0161K73J1J))
- Introduces an `ASSISTANT_MODELS` registry as a single source of truth
for assistant model config, eliminating hardcoded model IDs across the
codebase
- Aligns free/pro model conditional logic with `assistant.advance_model`
entitlement naming conventions instead of the `isLimited` pattern
- Adds `console.error` logging of Assistant stream errors so we can
interpret reasoning effort compatibility errors in the future (instead
of just opaque "Sorry, I'm having trouble responding right now" card)
- Removes unnecessary type casts and generally making the model config
logic stricter
- Removes pre-existing dead code: `anthropic` provider variant in
`GetModelParams` / `PROVIDERS` registry that was never implemented in
`getModel()`

Now if you try to select an unsupported reasoning level you get a type
error:

<img width="1306" height="320" alt="CleanShot 2026-03-20 at 14 37 24@2x"
src="https://github.com/user-attachments/assets/a6ac234b-5ea5-4d81-8e01-ac4be34a0800"
/>

And if for some reason an invalid reasoning level slips through, you now
get a server-side error surfacing the issue:

<img width="1268" height="204" alt="CleanShot 2026-03-20 at 14 58 14@2x"
src="https://github.com/user-attachments/assets/aadc1b7a-9495-475f-9741-39979bd27cd7"
/>

I've tested gpt-5 and gpt-5-mini are still working on the staging
preview and verified the models were selected properly in Braintrust
logs. Both models are available on my Pro test account, and my Free test
account shows the Pro upgrade CTA.


Closes AI-446
Closes AI-551
2026-03-25 11:29:23 -04:00
Ali Waseem
aa12ae790a fix: flatten AI generation schema for filters (#44092)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

OpenAI claims to support recursive schemas with $defs/$ref, but in
practice it's unreliable. When Zod's z.lazy() is converted to JSON
Schema, it produces recursive $ref entries that OpenAI's structured
output frequently rejects with errors like "Recursive reference
detected" or "Invalid schema for response_format".

Simplify the AI generation schema since we only support AND and don't
need the recursion because we don't support nesting of groups.
2026-03-24 07:38:40 -06:00
Matt Rossman
d29fbf6eb7 feat(assistant): upgrade AI SDK v5 → v6 (#43931)
Upgrades `ai` from v5 to v6 and all related packages.

**Package bumps:**
- `ai`: `5.0.52` → `^6.0.116`
- `@ai-sdk/openai`: `2.0.32` → `^3.0.41`
- `@ai-sdk/react`: `2.0.52` → `^3.0.118`
- `@ai-sdk/provider`: `^2.0.0` → `^3.0.8`
- `@ai-sdk/provider-utils`: `^3.0.0` → `^4.0.19`
- `@ai-sdk/amazon-bedrock`: `^3.0.0` → `^4.0.81`
- `@ai-sdk/mcp`: N/A → `^1.0.25`
- `openai`: bumped to `^4.104.0`
- `braintrust`: `3.0.x` → `^3.4.0`

**Breaking change migrations:**
- `generateObject` removed in v6 — migrated 5 API routes to
`generateText` with `Output.object({ schema })`, returning
`result.output`
- `convertToModelMessages` is now async — added `await`
- MCP import path changed: `experimental_createMCPClient` from `ai` →
`createMCPClient` from `@ai-sdk/mcp`
- `openai()` defaults to Responses API — added `store: false` to
provider options for ZDR org compatibility

**Streaming fix:**
Added `Content-Encoding: none` header to `pipeUIMessageStreamToResponse`
calls. Without it, proxy middleware buffers the entire SSE response
before flushing, causing the full reply to appear at once.

**Zero Data Retention fix:**
In recent AI SDK versions, `openai()` default to Responses API instead
of the legacy chat completions API. This produces a 404 from OpenAI with
message `"Items are not persisted for Zero Data Retention organizations.
Remove this item from your input and try again."` The Responses API is
OpenAI's [recommended
endpoint](https://developers.openai.com/api/docs/guides/migrate-to-responses).

This PR adds `store: false` as mentioned in
https://github.com/vercel/ai/issues/10060 to avoid incompatible
persistence attempts.

**References:**
- https://ai-sdk.dev/docs/migration-guides/migration-guide-6-0
-
https://ai-sdk.dev/docs/troubleshooting/streaming-not-working-when-proxied
- https://github.com/vercel/ai/issues/10060

Closes AI-514
Related AI-509
2026-03-19 09:07:33 -04:00
Ali Waseem
6cd3857d70 fix: added support for is and is not null (#43404)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

- For the new filters experience, we are missing null changes for dates
and also no support for isNull
- Updated E2E tests to account for new null cases
2026-03-05 07:07:54 -07:00
Matt Rossman
3fcb6cbe2c fix(studio): pass providerOptions to all AI SDK calls (#43031)
`reasoningEffort: 'minimal'` was
[configured](https://github.com/supabase/supabase/blob/d5cc70560d/apps/studio/lib/ai/model.utils.ts#L55-L59)
in the provider registry but `getModel()` returns it as a separate value
that callers must destructure and forward — and 7 of 8 endpoints weren't
doing so. This meant `gpt-5-mini` (a reasoning model) was running at
default reasoning effort for every call.

This PR destructures `providerOptions` from `getModel()` and passes it
to `generateObject`/`generateText` in all affected endpoints.

## Benchmark (local, median of 5 runs)

| Endpoint | Before (s) | After (s) | Speedup |
|----------|-----------|----------|---------|
| title-v2 | 7.0 | 1.9 | 3.7x |
| cron-v2 | 2.3 | 0.9 | 2.6x |
| filter-v1 | 5.8 | 2.2 | 2.6x |
| feedback/classify | 3.5 | 0.9 | 3.9x |
| feedback/rate | 2.9 | 0.9 | 3.2x |

`code/complete` and `policy` also received the fix but aren't
benchmarked here as they require a live DB connection and use multi-step
tool calls (separate latency concern tracked in AI-419).

To test the SQL naming, visit the SQL Editor in sidebar, add some SQL
like:

```sql
create table todos (                                                                                                 
  id serial primary key,                                                                                             
  task text not null,                                                                                                
  completed boolean default false
);
```

Right click on the snippet, "Rename" and "Rename with Supabase AI"

Closes AI-443
2026-02-20 11:36:06 -05:00
Matt Rossman
20d1ac0f83 feat(assistant): send user feedback to Braintrust traces (#43021)
- Expose Braintrust span ID to client via `x-braintrust-span-id`
response header, captured in chat transport
- On feedback submission, call `logFeedback()` with `scores["User
Rating"]` (1/0), `comment`, and on the root span assign
`metadata.feedbackCategory`
- Silently skipped when tracing is disabled (HIPAA, missing env vars)
- Log `requestedModel` in trace metadata so we can see what model the
user selected vs what was actually used after throttling

Example traces:

- [Thumbs
up](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=0bb71680-784c-45c1-a234-cba0242562d6&s=0bb71680-784c-45c1-a234-cba0242562d6)
- [Thumbs down + negative
feedback](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=d5a78084-6c9a-4230-8615-1e864bb9bac7&s=d5a78084-6c9a-4230-8615-1e864bb9bac7)

<img width="645" height="173" alt="CleanShot 2026-02-19 at 13 30 25@2x"
src="https://github.com/user-attachments/assets/6c463e83-27c6-4afb-a8d0-a329ed61270a"
/>

Closes AI-442
2026-02-19 15:00:07 -05:00
Matt Rossman
e8ab92408f feat(assistant): enable Braintrust tracing for non-sensitive chats (#42963)
Enables Braintrust tracing for AI Assistant chats to support debugging
and future online evals.

**Code Changes**

- Wraps `generateAssistantResponse` in a Braintrust `traced()` span,
logging the user's latest message as input along with metadata
(`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.)
- Threads JWT claims from `apiWrapper` → handler to log `userId` in
Braintrust without an extra API call (+ expanded `apiWrapper` tests)
- Threads `orgId` and `planId` from `getOrgAIDetails` to log in
Braintrust

**Infrastructure Changes**

- Created a "Vercel" service account in Braintrust
- Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the
studio-staging project in Vercel using a service token for the above
service account
- Added an "Overview" view to the Logs tab in the Braintrust Assistant
project to surface the new metadata

**Precautions**

- HIPAA sensitive projects are excluded from logging (see
https://github.com/supabase/supabase/pull/42787 for the detection logic)
- Production is temporarily excluded from logging until we're confident
in the setup

**Testing steps**

- Chat with the AI Assistant in the [studio-staging preview
build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023)
below
- Visit the [Logs tab in the Braintrust Assistant
project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs)
and inspect the trace

<img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x"
src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f"
/>

<img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45
04@2x"
src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa"
/>

**References**
- https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel
- https://www.braintrust.dev/docs/instrument/custom-tracing

Closes AI-438
2026-02-19 11:43:47 -05:00
Matt Rossman
2fc062a725 feat(assistant): detect HIPAA customers in assistant logic (#42787)
Detects HIPAA customers server-side in the assistant code path. Threads
`isHipaaEnabled` boolean through `getOrgAIDetails` → `generate-v4` →
`generateAssistantResponse`. The motivation is to support online evals
down the road, where we'll want to exclude HIPAA projects from Assistant
tracing.

This PR follows existing patterns for checking if HIPAA is enabled for a
project (org has HIPAA addon + project is sensitive). Example
[[1]](a5dd0a9671/apps/studio/components/interfaces/Settings/Addons/Addons.tsx (L75)),
[[2]](6858d4e18d/apps/studio/hooks/misc/useOrgOptedIntoAi.ts (L69)).

```ts
const hasHipaaAddon = subscriptionHasHipaaAddon(subscription) && settings?.is_sensitive
```

(I call it `isHipaaEnabled` in this PR to avoid it being misunderstood
as just the org-level addon, rather it's a combo of that addon being
present AND high compliance being enabled on the project).

### Verification steps

<details><summary>Click to view the steps I followed to sanity check it
works with the local stack</summary>

Tested locally with `mise fullstack`:

1. Found my org's subscription ID:

   ```sh
docker exec platform-db-1 psql -U postgres -c "SELECT id, customer_id,
status FROM orb.subscriptions;"
   ```

2. Added HIPAA addon to it:

   ```sh
   docker exec platform-db-1 psql -U postgres -c "
     UPDATE orb.subscriptions
SET price_intervals = price_intervals || '[{\"price\": {\"unit_config\":
{\"unit_amount\": \"350.00\"}, \"external_price_id\":
\"addon_security_hipaa\", \"item\": {\"name\": \"HIPAA\"}}}]'::jsonb
     WHERE id = '<subscription_id>';"
   ```

2. Toggled on High Compliance (Project Settings → General)

3. Added a temporary log after `getOrgAIDetails` in `generate-v4.ts`:

   ```ts
   console.log('[HIPAA]', { isHipaaEnabled })
   ```

4. Sent a message in the AI Assistant → `isHipaaEnabled: true`

5. Toggled off High Compliance → resent → `isHipaaEnabled: false`

6. Removed addon from subscription, left project toggle on →
`isHipaaEnabled: false`

   ```sql
   -- Find addon index:
   SELECT ordinality - 1 as idx FROM orb.subscriptions,
jsonb_array_elements(price_intervals) WITH ORDINALITY AS elem(val,
ordinality)
     WHERE id = '<subscription_id>'
     AND val->'price'->>'external_price_id' = 'addon_security_hipaa';

   -- Remove by index:
UPDATE orb.subscriptions SET price_intervals = price_intervals - <idx>
     WHERE id = '<subscription_id>';
   ```

All three cases confirm `isHipaaEnabled` requires both the org addon and
the project-level toggle.


</details> 

Closes AI-434

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added HIPAA mode detection and exposed it in AI workflows.
* API request functions now accept optional custom authorization headers
for downstream calls.

* **Tests**
* Added tests covering HIPAA scenarios and verifying authorization
header propagation in related flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-17 15:59:26 -05:00
Danny White
05dc676d36 feat(studio): incident-aware AI assistant (#41603)
* handling and mock data support

* admonition

* only show in empty state

* rabbit

* mock without local overrides

* remove admonition

* make incident banner more prominent

* remove mock data

* move to tool

* prettier

* fix(studio): get_active_incidents tool

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
2026-01-21 13:52:29 +11:00
Saxon Fletcher
d2f4e808e5 Filter pattern refactor (#41545)
* filter refactor

* update tests

* prettier

* fix ts

* remove filter preview

* Filter bar in table editor behind flag (#41546)

filter bar in table editor behind flag

* fixes

* ts fix

* filter v1 refactor

* rename helpers

* unmount timeout

* fixes

* fix

* more
2026-01-06 08:04:28 +10:00
Matt Rossman
072883bcec feat: assistant evals (#41311)
* chore: bump `supabase` CLI

* chore: stricter message types in `generate-v4.ts`

* feat: tutorial eval

https://www.braintrust.dev/docs/evaluation

* feat: project ID for eval

* refactor: `generateAssistantResponse` out of `handlePost`

* refactor: generateAssistantResponse to lib/ai

* feat: factuality eval with assistant response

* chore: upgrade braintrust to v1.0.1

* chore: silence tsconfig warning

* feat: assertion scorer

* fix: aggregate tools across all steps

* refactor: strict tool names, remove need for `as const`

* refactor: generic tool name type in assertions

* feat: transfer mocks from `feature/braintrust`

* feat: LLM criteria assertion

* feat: braintrust evals workflow

* fix: BRAINTRUST_PROJECT_ID

* feat: `sql_similar` assertion

* fix: `OPENAI_API_KEY` in workflow env

* feat: split AssertionScorer into separate scorers

* feat: remove tutorial eval

* feat: 20 minute CI timeout

* feat: category in test case metadata

* feat: score with gpt-5

* refactor: dataset to own file, colocate scorers

* feat: "gpt-5.2-2025-12-11" for llm as a judge

* feat: SQL syntax scorer with `libpg-query`

* feat: `evals:setup` and `evals:run` scripts

* feat: `evals:setup` in CI

* feat: human readable scorer names

* chore: rename to "SQL Validity"

* feat: add 2 "sql_generation" test cases

* feat: update requiredTools in test cases

* chore: ignore Cursor MCP config

* feat: "Conciseness" score

* feat: "Completeness" scorer

* fix: generate-v4 test mocks

* feat: serialize "steps" for scorer inputs

* updated node mem options for typecheck

* updated runner

* remove ram update as actions handle this

* feat: read `BRAINTRUST_PROJECT_ID` from secrets

* feat: score helpfulness, remove old scorers

* feat: separate `evals:run` and `evals:upload` scripts

* feat: passthrough entire classifier result

* feat: use live `search_docs` impl, store docs result in metadata

* feat: reduce classifier options

* feat: filter workflow by `run-evals` PR label or `master` branch

* chore: cleanup stubbed mock tools

* fix: checkout actual branch with `ref:`

* fix: capture search_docs results from all content parts

* feat: simplify sql syntax score calculation

* feat: use AI SDK's UI message validator

* docs: justification for relative `extends`

* fix: cleanup leftover validatedMessages

* doc: note mock token isn't secret for snyk

* fix: mock ui message to pass validation

* feat: revert ignoring Cursor MCP config

Using `.git/info/exclude` instead until we have an opinion on this

* feat: add "tsconfig" as shared-data devDependency, revert relative path in tsconfig

* refactor: tool call parsing into function

* Update apps/studio/evals/assistant.eval.ts

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* refactor: organize mock schemas and tool factories

---------

Co-authored-by: Ali Waseem <waseema393@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-12-22 23:45:48 -05:00
Saxon Fletcher
9f3af6e502 Generate policies experiment (#40881)
* policy generation

* add ai

* refactor

* table create performance

* policy list

* style

* simplify

* refactor

* flag

* tracking

* track

* ts

* fixes

* connection string

* re-add rls and realtime toggle

* restore old logic

* base path

* badge

* false rls

* copy

* add permissions button

* Refactor based on comments

* Fix TS

* More nudge

* Update tests

* Fix test

* Fixx

* Fix

* Address feedback

* Address issues

* Improve experiment telemetry for generate policies A/B test (#41172)

* Address code rabbit catch

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Sean Oliver <882952+seanoliver@users.noreply.github.com>
2025-12-12 12:44:11 +08:00
Saxon Fletcher
7e6796125c AI model selection (#39472)
* expose model and model selection

* tooltip

* simplify

* Update ai-assistant-state.tsx

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update ModelSelector.tsx

---------

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>
2025-10-17 11:17:43 +10:00
Greg Richardson
d94f1f9b3a fix(ai): increase request body size limit (#39330)
* fix(ai): increase request body size limit

* fix: remove incorrect config type
2025-10-07 11:31:54 -06:00
Saxon Fletcher
54ad0b7495 Chore/ai realtime (#39145)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* ai realtime

* ai realtime

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* add limitations prompt

* limitations prompt

* link prompt

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

* empty states

* prompt otpimise

* refine prompt

* prompt optimizer

* remove realtime for now

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* updates

* feature flag

* use flag and additional check

* remove sort

* messages copy

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-06 10:31:40 +10:00
Saxon Fletcher
1973615a42 Prompt GPT-5 Optimizations (#39031)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* add limitations prompt

* limitations prompt

* link prompt

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

* prompt optimizer

* remove realtime for now

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-29 14:07:53 +10:00
Saxon Fletcher
626eb30e77 Assistant action orientated approach (#38806)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-29 03:57:36 +00:00
Greg Richardson
9820707d71 feat: local mcp server (#38797)
* feat: local mcp server

* feat(local-mcp): implement migrations

* fix: remove unsupported mcp args

* feat(local-mcp): tests

* fix(local-mcp): packages to adhere to minimumReleaseAge

* fix(mcp): import path for createSupabaseApiPlatform

* fix(local-mcp): move tests out of pages/api dir

* refactor: self-hosted execute sql logic

* fix: deps
2025-09-23 12:00:08 -06:00
Saxon Fletcher
d276aca6cb Chore/ai flow model (#38805)
* update onboarding

* update model and fix part issue

* lock

* remove unused filter

* fix tests

* fix again

* update package

* fix merge issue
2025-09-23 12:46:20 +10:00
Saxon Fletcher
bf004b8a1d Choose provider and model (#38267)
* choose provider and model

* fix test

* Fix lint

* Nit clean up

* add provider options and remove anthropic

* remove package

* use chat model and update prompt

* prompt rendering

* default to pro

* fix test

* low

* remove log

* remove redundant part.text

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-09 10:05:39 +00:00
Saxon Fletcher
cddc472705 Test GPT-OSS (#37710)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* testing new gptoss models

* sonnet as pro

* back to 3.7

* revert test

* add policy list tool

* fix

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* use limited model for completions

* remove duplicate

* Update bedrock.ts

Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>

* add default check

* only add prefix if more then one region

* temp use api

* refactor to use openai api

* test

* remove unused import

* Prettier lint

* Fix test

* revert back to bedrock

* Tiny style fix for spacing of code blocks from edit message button

* try generate text normally

* prompt adjust

* add list style to prompt

* prompt adjust

* prompt adjust emphasise display query

* update prompts

* prompt adjust

* remove package

* prompt adjust

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-27 11:34:44 +07:00
Saxon Fletcher
adfc138450 Remove error display temporarily (#37804) 2025-08-09 14:56:05 +10:00
Taishi
81e1acc2d8 Add ability to stop AI Assistant responses mid-stream (#37628)
* get `stop()` from useChat and use it

* Implement abort signal in `generate-v4` API

* Enhance `onStop` functionality in AIAssistant to store partial message when aborted

* Update dependencies in AIAssistant to include `updateMessage` in effect hook (lint warning fix)

* change onStop prop to be required in AssistantChatForm

* add comment for saving partial AI responses

* remove redundant text size class

* Make onStop prop optional in AssistantChatForm and conditionally render stop button

* Add loading spinner to stop button in AssistantChatForm

* simplify opacity handling for submit button

* Add tooltips

* Improve UI error handler for assistant panel

* Small tweak

* Filter out any tool calls that are streaming in generate-v4, happens when aborting the assistant while a tool is being called

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-08 17:47:46 +07:00
Saxon Fletcher
d60aceb562 Prompt and tool refactoring (#37500)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* add policy list tool

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* slight prompt change

* Minor clean up

* More clean up

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-08 15:25:57 +07:00
Saxon Fletcher
d52e2c252b Update AI SDK 5 (#37656)
* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* zod

* v4 again

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
2025-08-08 14:15:39 +07:00
Joshen Lim
e70242f822 Chore/clean up assistant mcp feature flags (#36772)
* Clean up usage of newOrgAiOptIn and useBedrockAssistant feature flags

* Remove all OpenAI endpoints

* Fix for self-hosted

* Default isLimited to false

* Update PG meta tests

* Fix unit tests for model

* Revert pg meta tests

* Fix test

---------

Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
2025-07-22 16:05:16 +08:00
Saxon Fletcher
8375880f9d remove dynamic chat name from prompt for caching (#37281) 2025-07-18 00:43:25 +00:00
Greg Richardson
2afd0c92ab fix(ai): temporarily remove log tools (#37274)
* fix(ai): temporarily remove log tools

* fix: temp remove log tools from tool filter logic

* fix: temp remove log tools from tests

* fix: remove references to get_logs in system prompt
2025-07-17 21:27:54 +00:00
Saxon Fletcher
09c1c95d7e Upgrades pro users and above (#37183)
* feat: bedrock prompt caching

* fix: prompt cache point not sending to bedrock

* upgrade paid orgs to higher model

---------

Co-authored-by: Greg Richardson <greg.nmr@gmail.com>
2025-07-16 14:58:33 +10:00
Greg Richardson
283826cbd9 feat: bedrock prompt caching (#37140)
* feat: bedrock prompt caching

* fix: prompt cache point not sending to bedrock
2025-07-15 20:08:31 -05:00
Saxon Fletcher
fd670667ab Context reduction (#37032)
* context saving

* fix model name

* add results comment

* fix errors
2025-07-11 16:03:37 +10:00