mirror of
https://github.com/nearai/ironclaw.git
synced 2026-06-09 03:24:59 +08:00
Gemini review pointed out that sqlite3 .dump output is often incompatible with PostgreSQL (PRAGMA statements, type differences, quoting). - Add warning callout explaining the incompatibility - Recommend pgloader as the primary migration tool - Keep manual export as fallback with editing caveat
314 lines
8.3 KiB
Plaintext
314 lines
8.3 KiB
Plaintext
---
|
|
title: Database Backends
|
|
description: PostgreSQL vs libSQL — choosing your database
|
|
---
|
|
|
|
IronClaw supports two database backends: **PostgreSQL** and **libSQL** (embedded SQLite). **PostgreSQL is the default** — if you don't set `DATABASE_BACKEND`, IronClaw will use PostgreSQL and require a `DATABASE_URL`. Choose libSQL for zero-config personal use.
|
|
|
|
<Warning>
|
|
If you start IronClaw without configuring a database, you'll see: `DATABASE_URL is required`. Run `ironclaw onboard` or set the environment variables manually.
|
|
</Warning>
|
|
|
|
## Quick Comparison
|
|
|
|
| Feature | PostgreSQL (default) | libSQL |
|
|
|---------|----------------------|--------|
|
|
| **Setup** | Requires PostgreSQL server | Zero-dependency, auto-created |
|
|
| **Best For** | Production, multi-user | Personal use, single-user |
|
|
| **Search** | Hybrid (FTS + vector) | FTS only (vector via Turso) |
|
|
| **Scaling** | Horizontal (read replicas) | Single node |
|
|
| **Backup** | pg_dump, replication | File copy, Turso sync |
|
|
| **Size** | 100MB+ installed | ~5MB binary |
|
|
|
|
## PostgreSQL
|
|
|
|
Recommended for production deployments, multi-user scenarios, and high-throughput use cases.
|
|
|
|
### Requirements
|
|
|
|
- PostgreSQL 15 or later
|
|
- pgvector extension for embeddings
|
|
|
|
### Installation
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Docker Compose (recommended)" icon="container">
|
|
The fastest way to get started. A `docker-compose.yml` is included in the IronClaw repository:
|
|
|
|
```bash
|
|
cd ironclaw
|
|
docker compose up -d postgres
|
|
```
|
|
|
|
This starts `pgvector/pgvector:pg16` on `127.0.0.1:5432` with database, user, and password all set to `ironclaw`.
|
|
|
|
<Warning>
|
|
These credentials are for local development only. Change `POSTGRES_PASSWORD` for any non-local deployment.
|
|
</Warning>
|
|
</Accordion>
|
|
|
|
<Accordion title="Ubuntu/Debian" icon="terminal">
|
|
```bash
|
|
sudo apt install postgresql-15 postgresql-15-pgvector
|
|
sudo systemctl enable --now postgresql
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="macOS (Homebrew)" icon="terminal">
|
|
```bash
|
|
brew install postgresql
|
|
brew install pgvector
|
|
brew services start postgresql
|
|
```
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
### Configuration
|
|
|
|
```bash
|
|
# Create database
|
|
sudo -u postgres psql -c "CREATE DATABASE ironclaw;"
|
|
sudo -u postgres psql -c "CREATE USER ironclaw WITH PASSWORD 'your-password';"
|
|
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE ironclaw TO ironclaw;"
|
|
|
|
# Enable pgvector
|
|
sudo -u postgres psql -d ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
|
|
```
|
|
|
|
### IronClaw Configuration
|
|
|
|
```bash
|
|
# Select backend (optional — postgres is the default)
|
|
# Accepted values: postgres, postgresql, pg | libsql, turso, sqlite
|
|
export DATABASE_BACKEND=postgres
|
|
|
|
# Required for PostgreSQL
|
|
export DATABASE_URL="postgres://ironclaw:***@localhost/ironclaw"
|
|
export DATABASE_POOL_SIZE=30 # default; increase for high-throughput workloads
|
|
```
|
|
|
|
<Warning>
|
|
`DATABASE_URL` is **required** when using PostgreSQL. If not set, IronClaw will fail to start with the error: `DATABASE_URL is required — Run 'ironclaw onboard' or set DATABASE_URL environment variable`.
|
|
</Warning>
|
|
|
|
Or in the wizard:
|
|
1. Select "PostgreSQL"
|
|
2. Enter connection string
|
|
3. Test connection
|
|
|
|
### SSL Modes
|
|
|
|
| Mode | Behavior | Use Case |
|
|
|------|----------|----------|
|
|
| `disable` | Never use TLS | Local development |
|
|
| `prefer` | Try TLS, fallback to plaintext | **Default** — works everywhere |
|
|
| `require` | Require TLS | Production with TLS |
|
|
|
|
```bash
|
|
export DATABASE_SSLMODE=require
|
|
```
|
|
|
|
## libSQL
|
|
|
|
Recommended for personal use, development, and single-user deployments. Zero setup required.
|
|
|
|
### How It Works
|
|
|
|
libSQL is an embedded SQLite-compatible database:
|
|
- Database is a single file (`~/.ironclaw/ironclaw.db`)
|
|
- No separate server process
|
|
- Auto-created on first connection
|
|
- Full SQLite feature set
|
|
|
|
### IronClaw Configuration
|
|
|
|
```bash
|
|
export DATABASE_BACKEND=libsql
|
|
export LIBSQL_PATH="/home/user/.ironclaw/ironclaw.db"
|
|
```
|
|
|
|
Or just use the wizard defaults:
|
|
1. Select "libSQL"
|
|
2. Accept default path
|
|
3. Done!
|
|
|
|
### Turso Cloud Sync
|
|
|
|
libSQL supports syncing to Turso for cloud backup:
|
|
|
|
```bash
|
|
export DATABASE_BACKEND=libsql
|
|
export LIBSQL_PATH="/home/user/.ironclaw/ironclaw.db"
|
|
export LIBSQL_URL="libsql://your-db.turso.io"
|
|
export LIBSQL_AUTH_TOKEN="***"
|
|
```
|
|
|
|
<Warning>
|
|
`LIBSQL_AUTH_TOKEN` is **required** when `LIBSQL_URL` is set. IronClaw will fail to start without it.
|
|
</Warning>
|
|
|
|
This keeps a local copy with automatic cloud sync.
|
|
|
|
## Feature Comparison
|
|
|
|
### Hybrid Search
|
|
|
|
**PostgreSQL:** Full hybrid search (FTS + vector via RRF)
|
|
```
|
|
Keyword matches + semantic similarity
|
|
Reciprocal Rank Fusion ranking
|
|
```
|
|
|
|
**libSQL:** FTS only (text search)
|
|
```
|
|
Keyword matching via FTS5
|
|
Vector search via Turso cloud only
|
|
```
|
|
|
|
### Embeddings
|
|
|
|
Both backends support embeddings, but with different implementations:
|
|
|
|
| Backend | Embeddings | Notes |
|
|
|---------|------------|-------|
|
|
| PostgreSQL | Yes | pgvector for vector storage |
|
|
| libSQL local | FTS only | No local vector storage |
|
|
| libSQL + Turso | Yes | Via Turso vector indexes |
|
|
|
|
<Warning>
|
|
**Encryption at rest:** The local SQLite database stores conversation and workspace data in plaintext. Only secrets (API tokens) are encrypted with AES-256-GCM. If you handle sensitive data, use full-disk encryption (FileVault, LUKS, BitLocker) or choose PostgreSQL with TDE.
|
|
</Warning>
|
|
|
|
## Migration
|
|
|
|
### From libSQL to PostgreSQL
|
|
|
|
<Warning>
|
|
SQLite and PostgreSQL have different SQL dialects. A raw `sqlite3 .dump` output may fail on PostgreSQL due to PRAGMA statements, type differences, and quoting. Use a conversion tool like [pgloader](https://pgloader.io/) or [pgloader Docker](https://hub.docker.com/r/dimitri/pgloader) for reliable migration.
|
|
</Warning>
|
|
|
|
1. **Using pgloader (recommended):**
|
|
```bash
|
|
pgloader ~/.ironclaw/ironclaw.db postgresql://ironclaw:***@localhost/ironclaw
|
|
```
|
|
|
|
2. **Manual export (may require editing):**
|
|
```bash
|
|
sqlite3 ~/.ironclaw/ironclaw.db ".dump" > ironclaw.sql
|
|
# Edit ironclaw.sql to remove PRAGMAs and fix type incompatibilities
|
|
psql -d ironclaw -f ironclaw.sql
|
|
```
|
|
|
|
3. **Update IronClaw config:**
|
|
```bash
|
|
export DATABASE_BACKEND=postgres
|
|
export DATABASE_URL="postgres://user:***@localhost/ironclaw"
|
|
```
|
|
|
|
4. **Restart IronClaw**
|
|
|
|
### From PostgreSQL to libSQL
|
|
|
|
1. **Export:**
|
|
```bash
|
|
pg_dump -h localhost -U ironclaw ironclaw > ironclaw.sql
|
|
```
|
|
|
|
2. **Convert and import to SQLite** (requires conversion tools)
|
|
|
|
3. **Update IronClaw config**
|
|
|
|
## When to Choose Which
|
|
|
|
### Choose libSQL if:
|
|
|
|
- Running IronClaw on a personal laptop/desktop
|
|
- Single-user deployment
|
|
- Want zero database administration
|
|
- Don't need horizontal scaling
|
|
- FTS-only search is sufficient
|
|
|
|
### Choose PostgreSQL if:
|
|
|
|
- Production multi-user deployment
|
|
- Need hybrid (FTS + vector) search locally
|
|
- High-throughput scenario
|
|
- Existing PostgreSQL infrastructure
|
|
- Require advanced backup/recovery
|
|
- Team or shared deployment
|
|
|
|
## Backup
|
|
|
|
### PostgreSQL
|
|
|
|
```bash
|
|
# Backup
|
|
pg_dump -h localhost -U ironclaw ironclaw > backup.sql
|
|
|
|
# Restore
|
|
psql -d ironclaw -f backup.sql
|
|
```
|
|
|
|
### libSQL
|
|
|
|
```bash
|
|
# Backup (simple file copy)
|
|
cp ~/.ironclaw/ironclaw.db ~/.ironclaw/ironclaw.db.backup
|
|
|
|
# Restore
|
|
cp ~/.ironclaw/ironclaw.db.backup ~/.ironclaw/ironclaw.db
|
|
|
|
# With Turso: automatic cloud backup
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="pgvector not found" icon="x-circle">
|
|
```bash
|
|
# Install pgvector
|
|
sudo apt install postgresql-15-pgvector
|
|
|
|
# Or compile manually
|
|
git clone https://github.com/pgvector/pgvector.git
|
|
cd pgvector
|
|
make
|
|
sudo make install
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="Database file is locked" icon="lock">
|
|
```bash
|
|
# Find and kill process
|
|
lsof ~/.ironclaw/ironclaw.db
|
|
kill -9 <pid>
|
|
|
|
# Or wait for it to release
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="Connection refused (PostgreSQL)" icon="network">
|
|
```bash
|
|
# Check PostgreSQL is running
|
|
sudo systemctl status postgresql
|
|
|
|
# Check listen addresses
|
|
sudo -u postgres psql -c "SHOW listen_addresses;"
|
|
|
|
# Should be '*' or 'localhost'
|
|
```
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Next Steps
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Configuration" icon="settings" href="/capabilities/configuration">
|
|
Full environment variable reference
|
|
</Card>
|
|
|
|
<Card title="VPS Install" icon="server" href="/infrastructure/droplet">
|
|
Production deployment guide with PostgreSQL
|
|
</Card>
|
|
</CardGroup>
|