mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-02 23:56:24 +08:00
172 lines
4.0 KiB
Plaintext
172 lines
4.0 KiB
Plaintext
---
|
|
title: OpenAI-Compatible
|
|
sidebarTitle: OpenAI-Compatible
|
|
description: OpenRouter, Together AI, Fireworks, vLLM, LiteLLM, LM Studio
|
|
---
|
|
|
|
IronClaw supports any OpenAI-compatible API endpoint. This includes OpenRouter, Together AI, Fireworks AI, self-hosted inference, and more.
|
|
|
|
## Overview
|
|
|
|
These providers use the same OpenAI API format:
|
|
|
|
```
|
|
POST /v1/chat/completions
|
|
Authorization: Bearer {key}
|
|
Content-Type: application/json
|
|
```
|
|
|
|
## Configuration
|
|
|
|
```bash
|
|
# ~/.ironclaw/.env
|
|
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=https://api.openrouter.ai/api/v1
|
|
LLM_API_KEY=sk-or-...
|
|
```
|
|
|
|
## Supported Providers
|
|
|
|
### OpenRouter
|
|
|
|
[OpenRouter](https://openrouter.ai) — 300+ models, single API key.
|
|
|
|
```bash
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=https://openrouter.ai/api/v1
|
|
export LLM_API_KEY=sk-or-...
|
|
export LLM_MODEL=anthropic/claude-sonnet-4
|
|
|
|
# For attribution (optional)
|
|
export LLM_EXTRA_HEADERS="HTTP-Referer:https://your-site.com,X-Title:Your App"
|
|
```
|
|
|
|
**Popular models:**
|
|
|
|
| Model | ID |
|
|
|-------|-----|
|
|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
|
|
| GPT-4o | `openai/gpt-4o` |
|
|
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
|
|
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
|
|
|
|
Browse all: https://openrouter.ai/models
|
|
|
|
### Together AI
|
|
|
|
[Together AI](https://www.together.ai) — Fast inference for open-source models.
|
|
|
|
```bash
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=https://api.together.xyz/v1
|
|
export LLM_API_KEY=...
|
|
export LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
|
|
```
|
|
|
|
### Fireworks AI
|
|
|
|
[Fireworks AI](https://fireworks.ai) — Fast inference with compound AI.
|
|
|
|
```bash
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=https://api.fireworks.ai/inference/v1
|
|
export LLM_API_KEY=fw-...
|
|
export LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
|
|
```
|
|
|
|
### vLLM (Self-Hosted)
|
|
|
|
[vLLM](https://github.com/vllm-project/vllm) — High-throughput inference.
|
|
|
|
```bash
|
|
# Start vLLM server
|
|
python -m vllm.entrypoints.openai.api_server \
|
|
--model meta-llama/Meta-Llama-3-8B-Instruct
|
|
|
|
# IronClaw config
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=http://localhost:8000/v1
|
|
export LLM_API_KEY=token-abc123 # any value if auth not configured
|
|
export LLM_MODEL=meta-llama/Meta-Llama-3-8B-Instruct
|
|
```
|
|
|
|
### LiteLLM Proxy
|
|
|
|
[LiteLLM](https://github.com/BerriAI/litellm) — Universal proxy for any provider.
|
|
|
|
```bash
|
|
# LiteLLM config (config.yaml)
|
|
model_list:
|
|
- model_name: gpt-4o
|
|
litellm_params:
|
|
model: openai/gpt-4o
|
|
api_key: sk-...
|
|
|
|
# IronClaw config
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=http://localhost:4000/v1
|
|
export LLM_API_KEY=sk-...
|
|
export LLM_MODEL=gpt-4o
|
|
```
|
|
|
|
### LM Studio
|
|
|
|
[LM Studio](https://lmstudio.ai) — Local GUI with OpenAI-compatible server.
|
|
|
|
1. Download and install LM Studio
|
|
2. Load a model
|
|
3. Start local server
|
|
4. Configure IronClaw:
|
|
|
|
```bash
|
|
export LLM_BACKEND=openai_compatible
|
|
export LLM_BASE_URL=http://localhost:1234/v1
|
|
export LLM_MODEL=llama-3.2-3b-instruct
|
|
# No API key needed
|
|
```
|
|
|
|
## Extra Headers
|
|
|
|
Add custom headers with `LLM_EXTRA_HEADERS`:
|
|
|
|
```bash
|
|
export LLM_EXTRA_HEADERS="Key1:Value1,Key2:Value2"
|
|
```
|
|
|
|
Useful for OpenRouter attribution.
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Invalid base URL" icon="link">
|
|
- Must end with `/v1` for most providers
|
|
- Include protocol (`https://`)
|
|
- No trailing slash after `/v1`
|
|
</Accordion>
|
|
|
|
<Accordion title="Model not found" icon="search">
|
|
- Each provider uses different model IDs
|
|
- Check provider's model list
|
|
- Use exact ID from provider docs
|
|
</Accordion>
|
|
|
|
<Accordion title="Authentication failed" icon="key">
|
|
- Verify API key format
|
|
- Check for expired keys
|
|
- Some providers don't need keys (LM Studio)
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Next Steps
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Ollama" icon="download" href="/providers/ollama">
|
|
Free local inference alternative
|
|
</Card>
|
|
|
|
<Card title="Configuration" icon="settings" href="/setup/configuration">
|
|
Full environment variable reference
|
|
</Card>
|
|
</CardGroup>
|