Files
supabase/apps/studio/lib/api/apiWrapper.ts
Matt Rossman e8ab92408f feat(assistant): enable Braintrust tracing for non-sensitive chats (#42963)
Enables Braintrust tracing for AI Assistant chats to support debugging
and future online evals.

**Code Changes**

- Wraps `generateAssistantResponse` in a Braintrust `traced()` span,
logging the user's latest message as input along with metadata
(`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.)
- Threads JWT claims from `apiWrapper` → handler to log `userId` in
Braintrust without an extra API call (+ expanded `apiWrapper` tests)
- Threads `orgId` and `planId` from `getOrgAIDetails` to log in
Braintrust

**Infrastructure Changes**

- Created a "Vercel" service account in Braintrust
- Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the
studio-staging project in Vercel using a service token for the above
service account
- Added an "Overview" view to the Logs tab in the Braintrust Assistant
project to surface the new metadata

**Precautions**

- HIPAA sensitive projects are excluded from logging (see
https://github.com/supabase/supabase/pull/42787 for the detection logic)
- Production is temporarily excluded from logging until we're confident
in the setup

**Testing steps**

- Chat with the AI Assistant in the [studio-staging preview
build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023)
below
- Visit the [Logs tab in the Braintrust Assistant
project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs)
and inspect the trace

<img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x"
src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f"
/>

<img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45
04@2x"
src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa"
/>

**References**
- https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel
- https://www.braintrust.dev/docs/instrument/custom-tracing

Closes AI-438
2026-02-19 11:43:47 -05:00

58 lines
1.5 KiB
TypeScript

import type { JwtPayload } from '@supabase/supabase-js'
import type { NextApiRequest, NextApiResponse } from 'next'
import { ResponseError, ResponseFailure } from 'types'
import { IS_PLATFORM } from '../constants'
import { apiAuthenticate } from './apiAuthenticate'
export function isResponseOk<T>(response: T | ResponseFailure | undefined): response is T {
if (response === undefined || response === null) {
return false
}
if (response instanceof ResponseError) {
return false
}
if (typeof response === 'object' && 'error' in response && Boolean(response.error)) {
return false
}
return true
}
// Purpose of this apiWrapper is to function like a global catchall for ANY errors
// It's a safety net as the API service should never drop, nor fail
export default async function apiWrapper(
req: NextApiRequest,
res: NextApiResponse,
handler: (
req: NextApiRequest,
res: NextApiResponse,
claims?: JwtPayload
) => Promise<Response | void>,
options?: { withAuth: boolean }
): Promise<Response | void> {
try {
const { withAuth } = options || {}
let claims: JwtPayload | undefined
if (IS_PLATFORM && withAuth) {
const response = await apiAuthenticate(req, res)
if (!isResponseOk(response)) {
return res.status(401).json({
error: {
message: `Unauthorized: ${response.error.message}`,
},
})
}
claims = response
}
return handler(req, res, claims)
} catch (error) {
return res.status(500).json({ error })
}
}