Files
supabase/apps/studio/lib/ai/assistant-message-metadata.test.ts
Charis 21511042a3 feat(studio): assistant logs context and reports guard (#48514)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature — final PR (9/9) of the SQL editor logs-source stack.

**Base branch:** `charislam/sql-editor-inline-ai-clickhouse-dialect` (PR
8). Nothing here is user-visible: entry points stay behind
`sqlEditorLogsSource` + `otelLegacyLogs`, and flag rollout happens after
the whole stack merges.

## What is the current behavior?

- The Assistant has no idea a SQL editor snippet targets the logs
backend. Ask it about a logs snippet and it answers in Postgres, because
the attached query is fenced as ` ```sql ` and nothing tells the model
otherwise.
- Because the `sql` fence is what `MessageMarkdown` treats as runnable
Postgres, an attached ClickHouse query is rendered with a
Run-against-Postgres affordance and branded with `untrustedSql`.
- "Debug with Assistant" on a failed logs query produces a dialect-less
prompt, so both the in-app assistant and the copyable version get
debugged as Postgres.
- A report referencing a `log_sql` snippet runs its ClickHouse SQL
against the user's Postgres database and surfaces the resulting error.

## What is the new behavior?

**Assistant panel.** The "Current Query" chip records which backend the
attached query targets. That reaches the model two ways: each attachment
is fenced with its own dialect (` ```clickhouse ` vs ` ```sql `), and a
`containsLogsSnippets` flag rides on the user message as AI SDK
`metadata`. The server reads the flag off the conversation and prepends
the ClickHouse dialect rules plus the logs schema reference as a
non-cached context message.

Two design points worth calling out in review:

- The flag lives on the **message**, not the request body, so Retry and
the tool-approval continuation reproduce the context a message was
originally asked in — neither of those passes a per-call body.
- It's derived from **what's actually attached**, so detaching the chip
drops the claim rather than leaving the two able to disagree.

The `clickhouse` fence also keeps a logs query out of
`MessageMarkdown`'s `sql` branch, so it's no longer offered as runnable
Postgres or branded with `untrustedSql` — a boundary this stack's
distinct brands exist to prevent crossing.

**Debug flow.** `buildDebugChatArgs` attaches its query with a source
for the same reason, and names the dialect in the prompt text so the
copyable version stands on its own outside the app.

**Reports.** A report only stores a snippet id, so whether it queries
the logs backend is only knowable once the content loads. `ReportBlock`
guards on the fetched type and renders a `LogsSnippetReportBlock`
placeholder instead of executing. Double-guarded: no `sql` for a logs
snippet (so it's out of the query key and `queryFn` short-circuits even
on an explicit `refetch`) and `enabled` excludes it.

**Incidental cleanups.** `buildAssistantContextMessages` extracted out
of `generate-assistant-response`; a schema-access sentinel that was
duplicated as a string literal across two files (and compared against)
replaced with one exported constant; `SqlSnippet` deduplicated to a
single declaration; `resolveSnippetSource` / `isLogsSource` shared
instead of re-implemented per surface.

**Tests.** 4 new/extended suites. Notable cases pinned: a message with
no metadata must validate (`safeValidateUIMessages` applies
`metadataSchema` to *every* message, so a required schema would 400
every existing conversation); only *user* messages count, so a model
reply can't talk the server into a different dialect; a mixed-attachment
message is flagged without overclaiming a single source; and
`ReportBlock` registers no pg-meta mock for the logs cases, so an
unhandled request failing the test *is* the assertion that logs SQL
never reaches Postgres.

Verified: `pnpm typecheck`, `lint:ratchet` (no regression), Prettier,
and the full Studio suite (459 files / 4969 tests).

## Additional context

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Added support for recognizing log snippets in reports, with clear
guidance to open them in the SQL editor or remove them.
- AI Assistant now understands log snippets and provides
ClickHouse-specific context, formatting, and troubleshooting guidance.
- Snippets retain their source information when shared with the AI
Assistant.

- **Bug Fixes**
- Prevented unsupported log snippets from being executed as regular
database queries.
  - Improved source detection when opening snippets directly from links.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-04 09:02:40 -04:00

91 lines
3.3 KiB
TypeScript

import type { UIMessage } from 'ai'
import { describe, expect, it } from 'vitest'
import {
assistantMessageMetadataSchema,
messagesIncludeLogsSnippets,
} from '@/lib/ai/assistant-message-metadata'
function userMessage(id: string, text: string, metadata?: unknown): UIMessage {
return { id, role: 'user', parts: [{ type: 'text', text }], metadata } as UIMessage
}
function assistantMessage(id: string, text: string): UIMessage {
return { id, role: 'assistant', parts: [{ type: 'text', text }] } as UIMessage
}
describe('assistantMessageMetadataSchema', () => {
// safeValidateUIMessages applies this schema to EVERY message's metadata, so a
// message with none (i.e. every message written before this field existed) has to
// pass — otherwise an existing conversation 400s on its next turn.
it('accepts a message with no metadata', () => {
expect(assistantMessageMetadataSchema.safeParse(undefined).success).toBe(true)
})
it('accepts metadata flagging attached logs queries', () => {
const result = assistantMessageMetadataSchema.safeParse({ containsLogsSnippets: true })
expect(result.success).toBe(true)
expect(result.data?.containsLogsSnippets).toBe(true)
})
it('rejects a non-boolean flag', () => {
expect(assistantMessageMetadataSchema.safeParse({ containsLogsSnippets: 'yes' }).success).toBe(
false
)
})
})
describe('messagesIncludeLogsSnippets', () => {
it('detects a message that attached a logs query', () => {
expect(
messagesIncludeLogsSnippets([
userMessage('1', 'show me 500s', { containsLogsSnippets: true }),
])
).toBe(true)
})
it('stays true once any earlier message attached a logs query', () => {
expect(
messagesIncludeLogsSnippets([
userMessage('1', 'show me 500s', { containsLogsSnippets: true }),
assistantMessage('2', 'select ...'),
userMessage('3', 'now count my users', { containsLogsSnippets: false }),
])
).toBe(true)
})
it('is false for a conversation that only attached database queries', () => {
expect(
messagesIncludeLogsSnippets([
userMessage('1', 'count my users', { containsLogsSnippets: false }),
assistantMessage('2', 'select ...'),
])
).toBe(false)
})
it('is false when no message carries metadata', () => {
expect(messagesIncludeLogsSnippets([userMessage('1', 'hello')])).toBe(false)
expect(messagesIncludeLogsSnippets([userMessage('1', 'hello', {})])).toBe(false)
expect(messagesIncludeLogsSnippets([])).toBe(false)
})
// Only the user states which query they attached; an assistant message must not be
// able to talk the server into a different dialect.
it('ignores metadata on assistant messages', () => {
const assistantWithMetadata = {
id: '1',
role: 'assistant',
parts: [{ type: 'text', text: 'hi' }],
metadata: { containsLogsSnippets: true },
} as UIMessage
expect(messagesIncludeLogsSnippets([assistantWithMetadata])).toBe(false)
})
it('is false rather than throwing on malformed persisted metadata', () => {
expect(
messagesIncludeLogsSnippets([userMessage('1', 'hi', { containsLogsSnippets: 'yes' })])
).toBe(false)
expect(messagesIncludeLogsSnippets([userMessage('1', 'hi', 'not an object')])).toBe(false)
})
})