Files
supabase/apps/studio/lib/telemetry/funnel-errors.ts
Pamela Chia 5db8a0e960 feat(studio): instrument sign-in attempts and failures (#49853)
The /sign-in page emitted only a pageview on entry and the success-side
`sign_in` event on exit: failed or abandoned attempts were invisible, so
"never interacted" and "tried and failed silently" could not be told
apart in the sign-in funnel. I added an unsampled `sign_in_submitted`
event at every initiation point and classified failure capture via
`dashboard_error_created` with a new `signin` origin.

**Changed:**
- **Submit attempts observable**: `sign_in_submitted` (method: `email`,
provider id, `sso`, or partner) fires from the DOM submit handler on the
password and SSO forms (so submits that fail client-side validation
still count), and from the OAuth, custom-provider, and partner
initiation handlers.
- **Failures classified**: each sign-in error path feeds the existing
funnel-error pipe with origin `signin` and a controlled reason slug
(`invalid_credentials`, `email_not_confirmed`, `captcha_failed`,
`sso_provider_not_found`, ...). GoTrue auth errors now classify via
their numeric `status`, guarded so transport failures (`status: 0`) stay
`network_error`.
- **Attempt events survive the OAuth redirect**: the telemetry event
POST sends with `keepalive` (scoped to `sign_in_submitted`, since
keepalive requests share a per-page in-flight body quota), so a
dispatched request is no longer aborted by the provider navigation; send
rejections are caught centrally instead of surfacing as unhandled
rejections. The fetch still dispatches after an async token lookup, so
preview testing verifies the GitHub-path event actually lands on the
wire.
- **Captcha rejection is no longer silent**: a rejected hCaptcha
challenge resolves the stuck loading toast with an error message, emits
`captcha_challenge_failed` (distinct from `captcha_failed`, which stays
reserved for the auth server rejecting a submitted token), reports to
error monitoring, and resets the captcha widget (previously: unhandled
promise rejection and a spinner that never resolved).
- **Partner method validated**: the partner sign-in page resolves the
URL-hash value against the provider registry and forwards the canonical
provider id into `method` on both `sign_in_submitted` and `sign_in`;
anything unregistered records as `unregistered_partner`, so a crafted
link can't poison the breakdown on either event.

**Note:** failure events stay on the shared 10%
`dashboard_error_created` sampling rate (a per-origin carve-out would
break cross-source volume comparability); the unsampled attempt event
carries the tried-vs-never-interacted signal at full volume.

## To test

Tested on Vercel preview (studio-staging, wire-level network capture +
staging ingestion check):
- [x] On `/sign-in`, submit a bogus email + password: expect a `POST
*/platform/telemetry/event` request with `action: sign_in_submitted`,
`method: email` in the network tab, plus an error toast. Observed: 201,
auth returned 400 as expected.
- [x] Submit with an empty password: expect `sign_in_submitted` to still
fire (validation failures count as attempts). Observed: event fired with
201 and no auth call followed.
- [x] Click "Continue with GitHub": expect `sign_in_submitted` with
`method: github` on the wire before the provider redirect. Observed: the
POST completed (201) before the browser landed on github.com, so the
keepalive path holds.
- [x] Negative case: fresh page load with no interaction fires no
`sign_in_submitted`.
- [x] Ingestion: all fired events (methods `email`, `github`, plus
organic `sso` submits from a real login on the same preview) arrived in
the staging project with the expected properties.
- [x] Re-ran the email and GitHub paths on the scoped-keepalive build
(`129bf8d`): both `sign_in_submitted` POSTs returned 201 (the GitHub one
completed despite the provider redirect), and both events ingested into
the staging project with the expected `method`/`category` properties.

## Linear
- GROWTH-1165 (no `fixes` keyword on purpose: the evidence checks run on
prod data post-deploy, and the issue closes manually after they pass)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Improved sign-in protection with more reliable invisible CAPTCHA
handling.
* Added sign-in submission tracking across password, SSO, partner,
custom OAuth, and external-provider flows.
* Added detailed classification for authentication, validation, CAPTCHA,
provider, and network errors.

* **Bug Fixes**
  * Sign-in now stops safely and resets CAPTCHA when verification fails.
* Improved error reporting for failed sign-in attempts, including
redirects and OAuth flows.
* Ensured sign-in telemetry is delivered reliably during OAuth
redirects.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-02 19:02:37 +08:00

179 lines
6.2 KiB
TypeScript

import type { FieldErrors } from 'react-hook-form'
export type FunnelOrigin = 'signup' | 'signin' | 'project_creation' | 'org_creation'
export type ErrorCategory = 'validation' | 'api' | 'network' | 'payment' | 'unknown'
export interface FunnelErrorClassification {
errorCategory: ErrorCategory
errorReason: FunnelErrorReason
errorCode?: number
}
const RATE_LIMIT_STATUS = 429
const API_REASON_PATTERNS = {
signup: [
[/already registered|already been registered|already exists/i, 'email_already_registered'],
[/rate limit|too many requests|after \d+ second/i, 'rate_limited'],
[/captcha/i, 'captcha_failed'],
[/password/i, 'password_rejected'],
[/valid email|invalid email|email address/i, 'email_invalid'],
],
signin: [
[/invalid login credentials/i, 'invalid_credentials'],
[/email not confirmed/i, 'email_not_confirmed'],
[/rate limit|too many requests|after \d+ second/i, 'rate_limited'],
[/captcha/i, 'captcha_failed'],
[/sso provider/i, 'sso_provider_not_found'],
[/redirect|requested path is invalid/i, 'redirect_not_allowed'],
[/provider is not enabled|unsupported provider/i, 'provider_not_enabled'],
[/valid email|invalid email|email address/i, 'email_invalid'],
],
project_creation: [
[/already exists/i, 'project_name_taken'],
[/free plan|free tier/i, 'free_tier_limit'],
[/limit|maximum number|can only have/i, 'project_limit_reached'],
[/payment|invoice|overdue|past due|billing/i, 'billing_issue'],
[/region/i, 'region_unavailable'],
[/db_pass|password/i, 'db_password_rejected'],
],
org_creation: [
[/already exists|name.*taken/i, 'org_name_taken'],
[/payment|card|invoice|billing/i, 'billing_issue'],
[/limit/i, 'org_limit_reached'],
],
} as const satisfies Record<FunnelOrigin, ReadonlyArray<readonly [RegExp, string]>>
const VALIDATION_FIELD_REASONS = {
signup: {
email: 'email_invalid',
password: 'password_invalid',
},
signin: {
email: 'email_invalid',
password: 'password_invalid',
},
project_creation: {
organization: 'organization_missing',
projectName: 'project_name_invalid',
dbPass: 'db_password_weak',
dbPassStrength: 'db_password_weak',
dbRegion: 'region_missing',
cloudProvider: 'cloud_provider_invalid',
postgresVersion: 'postgres_version_missing',
highAvailability: 'incompatible_options',
useOrioleDb: 'incompatible_options',
},
org_creation: {
name: 'org_name_missing',
kind: 'org_kind_invalid',
size: 'org_size_invalid',
},
} as const satisfies Record<FunnelOrigin, Readonly<Record<string, string>>>
const STRIPE_DECLINE_REASONS = {
insufficient_funds: 'card_insufficient_funds',
card_declined: 'card_declined',
expired_card: 'card_expired',
incorrect_cvc: 'card_incorrect_cvc',
incorrect_number: 'card_incorrect_number',
processing_error: 'card_processing_error',
} as const satisfies Record<string, string>
const GENERIC_REASONS = [
'captcha_challenge_failed',
'rate_limited',
'server_error',
'connection_timeout',
'network_error',
'payment_failed',
'payment_error',
'oriole_unavailable',
'unauthorized',
'forbidden',
'not_found',
'other',
] as const
type ValuesOf<T> = T extends Readonly<Record<string, infer V extends string>> ? V : never
export type FunnelErrorReason =
| (typeof API_REASON_PATTERNS)[FunnelOrigin][number][1]
| ValuesOf<(typeof VALIDATION_FIELD_REASONS)[FunnelOrigin]>
| ValuesOf<typeof STRIPE_DECLINE_REASONS>
| (typeof GENERIC_REASONS)[number]
const STATUS_REASONS: Readonly<Partial<Record<number, FunnelErrorReason>>> = {
401: 'unauthorized',
403: 'forbidden',
404: 'not_found',
}
export function classifyApiError(origin: FunnelOrigin, error: unknown): FunnelErrorClassification {
const err = error as { code?: unknown; status?: unknown; errorType?: unknown; message?: unknown }
// GoTrue AuthErrors carry a numeric `status` and a string `code` slug; auth-js uses
// status 0 for transport failures (AuthRetryableFetchError), which must classify as
// network_error, so the fallback only accepts positive statuses.
const code =
typeof err?.code === 'number'
? err.code
: typeof err?.status === 'number' && err.status > 0
? err.status
: undefined
const message = typeof err?.message === 'string' ? err.message : ''
if (err?.errorType === 'connection-timeout') {
return { errorCategory: 'network', errorReason: 'connection_timeout' }
}
if (code === undefined) {
return { errorCategory: 'network', errorReason: 'network_error' }
}
if (code === RATE_LIMIT_STATUS) {
return { errorCategory: 'api', errorReason: 'rate_limited', errorCode: code }
}
if (code >= 500) {
return { errorCategory: 'api', errorReason: 'server_error', errorCode: code }
}
for (const [pattern, reason] of API_REASON_PATTERNS[origin]) {
if (pattern.test(message)) {
return { errorCategory: 'api', errorReason: reason, errorCode: code }
}
}
const statusReason = STATUS_REASONS[code]
if (statusReason) {
return { errorCategory: 'api', errorReason: statusReason, errorCode: code }
}
return { errorCategory: 'api', errorReason: 'other', errorCode: code }
}
export function classifyValidationError(
origin: FunnelOrigin,
errors: FieldErrors
): FunnelErrorClassification {
const fieldErrors = errors as Record<string, unknown>
const reasons = VALIDATION_FIELD_REASONS[origin] as Readonly<Record<string, FunnelErrorReason>>
for (const field of Object.keys(reasons)) {
if (fieldErrors[field]) {
return { errorCategory: 'validation', errorReason: reasons[field] }
}
}
return { errorCategory: 'validation', errorReason: 'other' }
}
export function classifyStripeError(error: unknown): FunnelErrorClassification {
const err = error as { code?: unknown; decline_code?: unknown }
const key =
typeof err?.decline_code === 'string'
? err.decline_code
: typeof err?.code === 'string'
? err.code
: undefined
const reason = key
? (STRIPE_DECLINE_REASONS as Readonly<Record<string, FunnelErrorReason>>)[key]
: undefined
if (reason) {
return { errorCategory: 'payment', errorReason: reason }
}
return { errorCategory: 'payment', errorReason: 'payment_failed' }
}