lywsvip/ironclaw
1
0
Fork 0
mirror of https://github.com/nearai/ironclaw.git synced 2026-09-03 08:06:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
faf2ed446534a4bb403b375da05061ed636427fb
ironclaw/providers.json
Illia Polosukhin cfeae9e678 refactor(llm): hide provider-specific auth, model fetch, and embeddings config behind facades (#3416) 
* refactor(llm): hide provider-specific auth, model fetch, and embeddings config behind facades

External callers were reaching into provider-specific modules of
`ironclaw_llm` (`gemini_oauth::CredentialManager`,
`github_copilot_auth::*`, `OpenAiCodexSessionManager`,
`codex_auth::*`, `BedrockConfig`, etc.). Closes those leaks behind a
small set of verb-based public surfaces while keeping per-provider
behaviour inside the LLM crate.

Changes:

1. Extract `oauth_helpers.rs` into a new `ironclaw_oauth` crate. The
   loopback OAuth callback listener (port 9876, landing pages,
   `OAUTH_CALLBACK_HOST` rules) is shared by every IronClaw OAuth flow
   (NEAR AI session login, WASM tool auth, MCP) and never depended on
   `ironclaw_llm`. `src/auth/oauth.rs` now `pub use ironclaw_oauth::*`
   directly. `ironclaw_llm` no longer depends on `ironclaw_oauth` —
   the helper had zero internal callers.

2. Add `ironclaw_llm::auth` facade (`start_login`, `validate_token`,
   `default_headers`, `load_persisted_credentials`,
   `default_credentials_path`) with backend-agnostic types
   (`AuthPrompt`, `LoginRequest`, `AuthOutcome`, `PersistedCredentials`,
   `OpenAiCodexLoginOptions`, `AuthBackend`, `CredentialSource`).
   Privatize `gemini_oauth`, `github_copilot_auth`, `openai_codex_session`,
   `codex_auth` (`pub(crate) mod`). Migrate the wizard, the
   `ironclaw login --openai-codex` CLI subcommand, and the LLM config
   loader to the facade. Wizard introduces a single `WizardAuthPrompt`
   that handles device-code prompts + browser launch for all backends.

3. Add `ironclaw_llm::models::fetch_models_for(provider_id, &opts)`
   facade. Privatize `fetch_anthropic_models`, `fetch_openai_models`,
   `fetch_ollama_models`, `fetch_openai_compatible_models`,
   `is_openai_chat_model`, `openai_model_priority`, `sort_openai_models`.
   Wizard's per-backend match collapses to one call. Move classifier
   unit tests into `crates/ironclaw_llm/src/models.rs`; rewrite the
   two wizard fallback tests through the public API.

4. Decouple embeddings from `ironclaw_llm::BedrockConfig`. New
   `crate::workspace::BedrockEmbeddingSetup { region, profile }` carries
   only what `BedrockEmbeddings` actually needs. `EmbeddingsConfig::create_provider`
   and `BedrockEmbeddings::new` take the new type; callers translate from
   `LlmConfig.bedrock` at the boundary (`src/app.rs`, `src/cli/mod.rs`).

5. Add `ironclaw_llm::testing::nearai_test_config(model)` helper for
   tests that need a minimal `LlmConfig` shape (no retries, no caching,
   NEAR AI backend). Replaces two duplicated 30-line struct literals
   in the gateway settings hot-reload tests.

Boundary cleanup is behaviour-preserving: 4,932 main-binary unit tests,
729 ironclaw_llm unit tests, 4 ironclaw_oauth tests, 3 architecture
boundary tests all pass; `cargo clippy --all --benches --tests
--examples --all-features` is clean.

Three `pub` methods on `gemini_oauth::CredentialManager` /
`GeminiOauthProvider` (`get_valid_access_token`, `last_response_meta`,
`count_tokens`) and the `GeminiResponseMeta` struct are now reachable
only crate-internally and have no callers; marked `#[allow(dead_code)]`
with a comment rather than deleted to keep this PR purely a boundary
move (delete in a follow-up if no caller emerges).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(llm): promote dedicated backends into the registry; absorb config validation, defaults, and per-provider overrides into ironclaw_llm

Continues the LLM boundary cleanup from 0addf3ac2. After that commit
provider-specific auth, model fetch, and embeddings config lived behind
facades inside `ironclaw_llm`, but four backend-specific knowledge
sources still leaked out:

  1. Validation rules and default values for the dedicated-config
     backends (Bedrock cross-region prefixes, OpenAI Codex endpoints
     and client_id, Gemini OAuth credentials path defaults) lived
     inline in `src/config/llm.rs::resolve`.
  2. The dispatcher in `create_llm_provider` matched on backend strings
     ("nearai", "bedrock", ...) instead of a typed protocol value. The
     same booleans (`is_nearai`, `is_bedrock`, `is_gemini_oauth`,
     `is_openai_codex`) recurred across `src/config/llm.rs`,
     `src/app.rs`, `src/cli/models.rs`, and the wizard.
  3. The setup wizard had per-backend specialization in
     `step_inference_provider` and `run_provider_setup` (manual menu
     pushes for nearai/bedrock/codex/gemini_oauth, four dedicated
     `setup_*` entry points dispatched on string compares).
  4. `Settings` carried named `bedrock_region`, `bedrock_cross_region`,
     `bedrock_profile` columns even though no other dedicated backend
     had named columns and adding a new one would mean schema churn.

Layers A-D address each in turn:

* Layer A — `BedrockConfig::build`, `OpenAiCodexConfig::build`, and
  `GeminiOauthConfig::build` own validation + defaults inside the
  crate. `LlmConfigError` (`MissingRequired` / `InvalidValue`) carries
  the failures across the boundary, with a `From` impl into the
  binary's `ConfigError`. `src/config/llm.rs` calls the builders;
  named-string defaults are gone from the binary. The orphaned
  `tests/gemini_oauth_regression.rs` husk is deleted.

* Layer B — `ProviderProtocol` gains four new variants
  (`Bedrock`, `OpenAiCodex`, `GeminiOauth`, `NearAi`) plus a
  `has_dedicated_config()` predicate. The four dedicated-config
  backends (with all aliases) become first-class registry entries in
  `providers.json`, so `is_known()` / `model_env_var()` / the wizard /
  the gateway handler iterate the registry uniformly. The
  `is_nearai`/`is_bedrock`/`is_gemini_oauth`/`is_openai_codex` boolean
  spaghetti collapses to protocol comparisons. `OpenAiCodex` and
  `NearAi` carry explicit `#[serde(rename = "openai_codex" / "nearai",
  alias = ...)]` so the wire-stable adapter strings the gateway and
  frontend already use keep working. `LlmConfig::active_model_name()`
  is now consumed by `cli/doctor.rs` instead of an inlined partial
  dispatch.

* Layer C — `SetupHint` gains four credential-collection variants
  (`AwsCredentials`, `OAuthDeviceCode`, `FileBasedCredentials`,
  `SessionToken`). The wizard's `step_inference_provider` builds its
  menu from a single `registry.selectable()` iteration with generic
  env-detection (declared `api_key_env`, plus an Anthropic-specific
  OAuth fallback). `run_provider_setup` dispatches on the SetupHint
  variant; the remaining `def.id == "..."` checks live inside the
  `ApiKey` arm only because Anthropic and GitHub Copilot present a
  hybrid choice (API key OR OAuth) the simple `ApiKey` hint doesn't
  capture. The synthetic bedrock + nearai entries in
  `handlers/llm.rs::build_llm_providers` are deleted; a single
  registry-driven loop covers both. ADAPTER_LABELS in
  `static/js/surfaces/config.js` gains entries for the new protocols.

* Layer D — `LlmBuiltinOverride` gains a generic
  `extras: HashMap<String, String>` bag with `extra(key)` /
  `set_extra(key, value)` accessors. The bedrock resolver and wizard
  read/write through this bag; `Settings::migrate_legacy_provider_fields()`
  drains the named `bedrock_*` columns into `extras` on
  `Settings::load_from()` so existing `settings.json` files migrate
  losslessly. The named columns are kept (deprecated, marked with
  `#[serde(skip_serializing_if = "Option::is_none")]`) for one
  release; tracked for deletion in #3443.
  `strip_admin_only_llm_keys` and `llm_setting_requires_reload` now
  match dotted-path subkeys under `llm_builtin_overrides.*` so a
  write to e.g. `llm_builtin_overrides.bedrock.extras.region`
  triggers the right gating + chain reload.

Boundary cleanup is behaviour-preserving: 4,933 main-binary unit tests,
739 ironclaw_llm unit tests pass; `cargo clippy --all --benches --tests
--examples --all-features` is clean. New regression tests:
`crates/ironclaw_llm/src/config.rs` (6 builder tests),
`crates/ironclaw_llm/src/registry.rs::dedicated_config_backends_are_in_registry_and_selectable`,
and `src/setup/wizard.rs::legacy_bedrock_fields_migrate_into_extras_on_load`.

Three follow-ups tracked in #3443: delete the deprecated `bedrock_*`
named columns, move `BedrockEmbeddings` out of `src/workspace/` into
the LLM crate (last cargo-feature leak), and drive
`LlmConfig::active_model_name()` off `ProviderProtocol` instead of
backend strings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: add bug-bash regression-snapshot harness

Bug-bash fixtures pin specific open bugs to a deterministic snapshot.
When a bug is fixed, the snapshot diff is the reviewable proof; when
someone reintroduces the bug, the snapshot drifts and CI blocks the
merge.

This commit lands the harness plus the first recorded fixture for
issue #2541 (agent must call a tool, not answer from training data):

  tests/e2e_bug_bash_snapshots.rs
    `snapshot_summarization_uses_tools` replays the fixture, captures
    `ReplayOutcome`, and asserts the YAML snapshot. Gated on
    `feature = "libsql"`, same as other replay-snapshot tests.

  tests/fixtures/llm_traces/bug_bash/summarization_uses_tools.json
    Two-step recorded LLM trace (tool_call -> text) keyed off the
    user prompt via `request_hint.last_user_message_contains`.

  tests/fixtures/llm_traces/bug_bash/README.md
    Coverage map for #2540-#2546 (one recorded, six TODO) plus the
    `IRONCLAW_RECORD_TRACE` recording workflow.

  tests/snapshots/replay__bug_bash_summarization_uses_tools.snap
    Insta YAML snapshot pinning `tool_calls: [echo]`, 2 LLM calls,
    and the event-kind histogram. Drift = regression.

* fix(settings): preserve pre-existing extras during legacy bedrock migration

`migrate_legacy_provider_fields` claimed to be idempotent and to drain
named `bedrock_*` columns into `llm_builtin_overrides["bedrock"].extras`
once on load. The previous implementation drained correctly but used
`HashMap::insert` unconditionally, which means a settings file
carrying BOTH a legacy `bedrock_region` column AND an already-populated
`extras["region"]` (manual hand-edit, or a future writer emitting both
shapes during a transition) would silently downgrade to the legacy
value.

Guard each `set_extra` call with `entry.extra(key).is_none()` so the
new-shape value always wins. Clarify the docstring to state this
explicitly.

Add three regression tests in `settings::tests`:

- `legacy_bedrock_migration_round_trips_through_save` — legacy JSON ->
  load_from -> serialize -> reload, asserts the deprecated columns are
  not re-emitted and extras survive the round trip.
- `legacy_bedrock_migration_preserves_existing_extras` — file with both
  shapes; asserts the pre-existing extras value is kept and absent
  extras are still backfilled from legacy fields.
- `legacy_bedrock_migration_is_idempotent_in_memory` — calling the
  migration twice on the same Settings is a no-op (compares serialized
  shape, since LlmBuiltinOverride does not derive PartialEq).

* fix(pr-3416): address PR review — migration on DB/TOML, admin-key gate, codex login, credential_kind/has_credentials

Addresses comments from gemini-code-assist, Copilot, and serrrfirat on PR #3416.

## Bugs

**Legacy bedrock fields not migrated on DB/TOML loads** (serrrfirat, High).
`Settings::load_from` (JSON) ran `migrate_legacy_provider_fields`, but
`from_db_map` and `load_toml` did not. Existing operators with
`bedrock_*` settings persisted in the DB or `config.toml` would silently
lose their AWS region/profile/cross-region after upgrade because the
resolver now reads only from `llm_builtin_overrides["bedrock"].extras`.
Both loaders now call the migration; added round-trip tests for each.

**Admin-only key write gate had narrower matching than read gate**
(Copilot, High). `strip_admin_only_llm_keys` matches both exact keys
and dotted subpaths under admin-only roots; `is_admin_only_setting_key`
in the web settings handler used `.contains(&key)` only. A non-admin
could write `llm_builtin_overrides.bedrock.extras.region` directly,
bypassing the gate. Promoted `is_admin_only_llm_key` to `pub(crate)`,
made the web write-side gate call it, added regression tests covering
dotted subpaths.

**`ironclaw login --openai-codex` dropped TOML/DB config** (Copilot,
High). The pre-refactor code resolved `Config::from_env` and used
`config.llm.openai_codex` so endpoint / client-id / session-path
overrides committed via TOML or DB stuck. The post-refactor code only
read env vars via `OpenAiCodexLoginOptions::from_env`. Added
`OpenAiCodexLoginOptions::from_resolved_config(&OpenAiCodexConfig)`;
the login command now prefers the resolved config when present and
falls back to env-only when `Config::from_env` itself fails (fresh
machine, no DB).

**Dedicated-auth backends marked configured without credentials**
(serrrfirat, Medium). `nearai` / `gemini_oauth` / `openai_codex` ship
`api_key_required: false` because they don't authenticate via a bearer
API key. The frontend `isProviderConfigured` treated that as "no
credentials needed" and rendered the Use button on a fresh install,
where clicking could trigger an interactive device-code OAuth from
inside a settings request.

Added `credential_kind` (wire-stable snake_case discriminator matching
`SetupHint::kind()`, e.g. `session_token`, `o_auth_device_code`,
`file_based_credentials`, `aws_credentials`) and `has_credentials`
(backend-authoritative; checks AWS env vars for Bedrock, codex session
file existence, file-based credential path expansion + existence) to
the web LLM providers payload. Frontend `isProviderConfigured` /
`providerMissingReason` now gate non-api-key kinds on `has_credentials`.

## Nits

**`fetch_models_for` doc overclaimed "Always returns something"**
(Copilot). The generic openai-compatible branch returns `vec![]` when
`base_url` is empty. Updated the docstring to call this out so callers
know to handle the empty case.

**`AuthError::Other` used for "validation not applicable"** (Gemini
bot). Added a dedicated `AuthError::TokenValidationNotSupported { backend }`
variant; `validate_token` now returns it for Gemini / OpenAiCodex
instead of stringly-formatted `Other`.

**Bug-bash regression-harness URLs pointed at `near/ironclaw`**
(Copilot, x2). The canonical tracker is `nearai/ironclaw`. Rewrote
all seven URLs in `tests/fixtures/llm_traces/bug_bash/README.md` and
the one in `tests/e2e_bug_bash_snapshots.rs`.

## Declined

The Gemini bot's MalformedConfig suggestion at
`crates/ironclaw_llm/src/models.rs:46` was not adopted: the call site is
the openai-compatible model-listing path, not a security-sensitive
request. The fetcher early-returns `vec![]` on empty `base_url` — no
URL parsing happens — and the docstring tightening above covers the
observable surprise. Promoting it to a typed error would change the
public-facing `fetch_models_for` signature for no behavioural gain.

## Tests

- `cargo fmt --check` clean
- `cargo clippy --all --benches --tests --examples --all-features` zero warnings
- `cargo test --lib` 4,941 / 4,941 pass
- `cargo test --features libsql --test e2e_bug_bash_snapshots` 1 / 1 pass
- New regression tests:
  - `settings::tests::legacy_bedrock_fields_migrate_into_extras_on_db_load`
  - `settings::tests::legacy_bedrock_fields_migrate_into_extras_on_toml_load`
  - `channels::web::features::settings::tests::test_admin_only_setting_keys_cover_dotted_subpaths`
  - `channels::web::handlers::llm::tests::test_llm_providers_expose_credential_kind_and_has_credentials`
  - `channels::web::handlers::llm::tests::test_nearai_has_credentials_true_when_session_token_loaded`

* fix(pr-3416): tighten Bedrock/Codex has_credentials probes; collapse set_extra into one .into()

- `backend_has_credentials` for AWS now requires `AWS_PROFILE` OR
  (`AWS_ACCESS_KEY_ID` AND `AWS_SECRET_ACCESS_KEY`). The lone
  `AWS_ACCESS_KEY_ID` / `AWS_SESSION_TOKEN` arms previously flipped
  has_credentials true even though the AWS SDK can't sign without the
  secret key, so the UI was rendering Bedrock as configured on hosts
  that would fail at first call.
- `backend_has_credentials` for OpenAI Codex now honours
  `OPENAI_CODEX_SESSION_PATH` via `read_env` before falling back to
  the default session path under `~/.ironclaw/`. Users with a custom
  session location were seeing "not configured" despite a valid login.
- New regression tests `test_bedrock_partial_aws_env_reports_not_configured`
  and `test_openai_codex_honours_session_path_env` drive the
  `build_llm_providers` call site (not just the helper) so both gaps
  stay closed.
- Tidied `LlmBuiltinOverride::set_extra` to convert the key once and
  reuse it across the remove/insert branches; behaviour identical.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(providers): default nearai model to "auto"

Switch the nearai registry entry's `default_model` from
`claude-sonnet-4-5` to `auto`, NEAR AI's server-side routing alias.
New installs without `NEARAI_MODEL` set now get auto-routed instead
of being pinned to a specific Anthropic model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 14:47:22 -07:00

521 lines
14 KiB
JSON
Raw Blame History

[
{
"id": "nearai",
"aliases": [
"near_ai",
"near"
],
"protocol": "nearai",
"api_key_env": "NEARAI_API_KEY",
"api_key_required": false,
"base_url_env": "NEARAI_BASE_URL",
"model_env": "NEARAI_MODEL",
"default_model": "auto",
"description": "multi-model access via NEAR account",
"setup": {
"kind": "session_token",
"display_name": "NEAR AI",
"key_url": "https://app.near.ai"
}
},
{
"id": "gemini_oauth",
"aliases": [
"gemini-oauth"
],
"protocol": "gemini_oauth",
"api_key_required": false,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
"description": "Official Gemini API via Gemini CLI OAuth",
"setup": {
"kind": "file_based_credentials",
"display_name": "Gemini CLI",
"default_path_hint": "~/.gemini/oauth_creds.json"
}
},
{
"id": "openai_codex",
"aliases": [
"openai-codex",
"codex"
],
"protocol": "openai_codex",
"api_key_required": false,
"model_env": "OPENAI_CODEX_MODEL",
"default_model": "gpt-5.3-codex",
"description": "ChatGPT subscription (Plus/Pro/Max)",
"setup": {
"kind": "o_auth_device_code",
"display_name": "OpenAI Codex",
"backend": "openai_codex"
}
},
{
"id": "openai",
"aliases": [
"open_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "OPENAI_API_KEY",
"api_key_required": true,
"base_url_env": "OPENAI_BASE_URL",
"model_env": "OPENAI_MODEL",
"default_model": "gpt-5-mini",
"description": "OpenAI GPT models (direct API)",
"unsupported_params": ["temperature"],
"setup": {
"kind": "api_key",
"secret_name": "llm_openai_api_key",
"key_url": "https://platform.openai.com/api-keys",
"display_name": "OpenAI",
"can_list_models": true
}
},
{
"id": "anthropic",
"aliases": [
"claude"
],
"protocol": "anthropic",
"api_key_env": "ANTHROPIC_API_KEY",
"api_key_required": true,
"base_url_env": "ANTHROPIC_BASE_URL",
"model_env": "ANTHROPIC_MODEL",
"default_model": "claude-sonnet-4-20250514",
"description": "Anthropic Claude models (direct API)",
"setup": {
"kind": "api_key",
"secret_name": "llm_anthropic_api_key",
"key_url": "https://console.anthropic.com/settings/keys",
"display_name": "Anthropic",
"can_list_models": true
}
},
{
"id": "ollama",
"aliases": [],
"protocol": "ollama",
"default_base_url": "http://localhost:11434",
"base_url_env": "OLLAMA_BASE_URL",
"model_env": "OLLAMA_MODEL",
"default_model": "llama3",
"description": "Local Ollama instance (no API key needed)",
"setup": {
"kind": "ollama",
"display_name": "Ollama",
"can_list_models": true
}
},
{
"id": "openai_compatible",
"aliases": [
"openai-compatible",
"compatible"
],
"protocol": "open_ai_completions",
"base_url_env": "LLM_BASE_URL",
"base_url_required": true,
"api_key_env": "LLM_API_KEY",
"api_key_required": false,
"model_env": "LLM_MODEL",
"default_model": "default",
"extra_headers_env": "LLM_EXTRA_HEADERS",
"description": "Custom OpenAI-compatible endpoint (vLLM, LiteLLM, etc.)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_compatible_api_key",
"display_name": "OpenAI-compatible",
"can_list_models": false
}
},
{
"id": "github_copilot",
"aliases": [
"github-copilot",
"githubcopilot",
"copilot"
],
"protocol": "github_copilot",
"default_base_url": "https://api.githubcopilot.com",
"api_key_env": "GITHUB_COPILOT_TOKEN",
"api_key_required": true,
"model_env": "GITHUB_COPILOT_MODEL",
"default_model": "gpt-4o",
"extra_headers_env": "GITHUB_COPILOT_EXTRA_HEADERS",
"description": "GitHub Copilot Chat API (OAuth token from IDE sign-in)",
"setup": {
"kind": "api_key",
"secret_name": "llm_github_copilot_token",
"key_url": "https://docs.github.com/en/copilot",
"display_name": "GitHub Copilot",
"can_list_models": false
}
},
{
"id": "tinfoil",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://inference.tinfoil.sh/v1",
"api_key_env": "TINFOIL_API_KEY",
"api_key_required": true,
"model_env": "TINFOIL_MODEL",
"default_model": "kimi-k2-5",
"description": "Tinfoil private inference (hardware-attested TEE)",
"unsupported_params": ["temperature"],
"setup": {
"kind": "api_key",
"secret_name": "llm_tinfoil_api_key",
"key_url": "https://tinfoil.sh",
"display_name": "Tinfoil",
"can_list_models": false
}
},
{
"id": "openrouter",
"aliases": [
"open_router"
],
"protocol": "open_router",
"default_base_url": "",
"api_key_env": "OPENROUTER_API_KEY",
"api_key_required": true,
"model_env": "OPENROUTER_MODEL",
"extra_headers_env": "OPENROUTER_EXTRA_HEADERS",
"default_model": "openai/gpt-4o",
"description": "OpenRouter multi-provider gateway (200+ models, preserves reasoning across turns)",
"setup": {
"kind": "api_key",
"secret_name": "llm_openrouter_api_key",
"key_url": "https://openrouter.ai/settings/keys",
"display_name": "OpenRouter",
"can_list_models": false
}
},
{
"id": "groq",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://api.groq.com/openai/v1",
"api_key_env": "GROQ_API_KEY",
"api_key_required": true,
"model_env": "GROQ_MODEL",
"default_model": "llama-3.3-70b-versatile",
"description": "Groq LPU inference (ultra-fast)",
"setup": {
"kind": "api_key",
"secret_name": "llm_groq_api_key",
"key_url": "https://console.groq.com/keys",
"display_name": "Groq",
"can_list_models": true,
"models_filter": "chat"
}
},
{
"id": "nvidia",
"aliases": [
"nvidia_nim",
"nim"
],
"protocol": "open_ai_completions",
"default_base_url": "https://integrate.api.nvidia.com/v1",
"api_key_env": "NVIDIA_API_KEY",
"api_key_required": true,
"model_env": "NVIDIA_MODEL",
"default_model": "meta/llama-3.3-70b-instruct",
"description": "NVIDIA NIM API (high-performance inference)",
"setup": {
"kind": "api_key",
"secret_name": "llm_nvidia_api_key",
"key_url": "https://build.nvidia.com",
"display_name": "NVIDIA NIM",
"can_list_models": true
}
},
{
"id": "venice",
"aliases": [
"venice_ai",
"veniceai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.venice.ai/api/v1",
"api_key_env": "VENICE_API_KEY",
"api_key_required": true,
"model_env": "VENICE_MODEL",
"default_model": "llama-3.3-70b",
"description": "Venice.ai privacy-focused inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_venice_api_key",
"key_url": "https://venice.ai/settings/api",
"display_name": "Venice.ai",
"can_list_models": false
}
},
{
"id": "together",
"aliases": [
"together_ai",
"togetherai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.together.xyz/v1",
"api_key_env": "TOGETHER_API_KEY",
"api_key_required": true,
"model_env": "TOGETHER_MODEL",
"default_model": "meta-llama/Llama-3-70b-chat-hf",
"description": "Together AI inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_together_api_key",
"key_url": "https://api.together.ai/settings/api-keys",
"display_name": "Together AI",
"can_list_models": false
}
},
{
"id": "fireworks",
"aliases": [
"fireworks_ai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.fireworks.ai/inference/v1",
"api_key_env": "FIREWORKS_API_KEY",
"api_key_required": true,
"model_env": "FIREWORKS_MODEL",
"default_model": "accounts/fireworks/models/llama-v3p1-70b-instruct",
"description": "Fireworks AI inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_fireworks_api_key",
"key_url": "https://fireworks.ai/api-keys",
"display_name": "Fireworks AI",
"can_list_models": false
}
},
{
"id": "deepseek",
"aliases": [
"deep_seek"
],
"protocol": "deep_seek",
"default_base_url": "",
"api_key_env": "DEEPSEEK_API_KEY",
"api_key_required": true,
"model_env": "DEEPSEEK_MODEL",
"default_model": "deepseek-chat",
"description": "DeepSeek inference API (preserves reasoning_content for thinking-mode models)",
"setup": {
"kind": "api_key",
"secret_name": "llm_deepseek_api_key",
"key_url": "https://platform.deepseek.com/api_keys",
"display_name": "DeepSeek",
"can_list_models": false
}
},
{
"id": "zai",
"aliases": [
"bigmodel"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.z.ai/api/paas/v4",
"api_key_env": "ZAI_API_KEY",
"api_key_required": true,
"model_env": "ZAI_MODEL",
"default_model": "glm-5",
"description": "Z.AI GLM inference API",
"setup": {
"kind": "api_key",
"secret_name": "llm_zai_api_key",
"key_url": "https://z.ai/manage-apikey/apikey-list",
"display_name": "Z.AI",
"can_list_models": false
}
},
{
"id": "cerebras",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://api.cerebras.ai/v1",
"api_key_env": "CEREBRAS_API_KEY",
"api_key_required": true,
"model_env": "CEREBRAS_MODEL",
"default_model": "llama-3.3-70b",
"description": "Cerebras wafer-scale inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_cerebras_api_key",
"key_url": "https://cloud.cerebras.ai",
"display_name": "Cerebras",
"can_list_models": false
}
},
{
"id": "sambanova",
"aliases": [
"samba_nova"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.sambanova.ai/v1",
"api_key_env": "SAMBANOVA_API_KEY",
"api_key_required": true,
"model_env": "SAMBANOVA_MODEL",
"default_model": "Meta-Llama-3.1-70B-Instruct",
"description": "SambaNova Cloud inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_sambanova_api_key",
"key_url": "https://cloud.sambanova.ai/apis",
"display_name": "SambaNova",
"can_list_models": false
}
},
{
"id": "gemini",
"aliases": [
"google_gemini",
"google"
],
"protocol": "gemini",
"default_base_url": "",
"api_key_env": "GEMINI_API_KEY",
"api_key_required": true,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
"description": "Google Gemini native API (preserves thought_signature on tool calls)",
"setup": {
"kind": "api_key",
"secret_name": "llm_gemini_api_key",
"key_url": "https://aistudio.google.com/app/apikey",
"display_name": "Google Gemini",
"can_list_models": false
}
},
{
"id": "ionet",
"aliases": [
"io_net",
"io.net"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.intelligence.io.solutions/api/v1",
"api_key_env": "IONET_API_KEY",
"api_key_required": true,
"model_env": "IONET_MODEL",
"default_model": "deepseek-coder-v2-instruct",
"description": "io.net Intelligence API",
"setup": {
"kind": "api_key",
"secret_name": "llm_ionet_api_key",
"key_url": "https://cloud.io.net/intelligence",
"display_name": "io.net",
"can_list_models": true
}
},
{
"id": "mistral",
"aliases": [
"mistral_ai",
"mistralai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.mistral.ai/v1",
"api_key_env": "MISTRAL_API_KEY",
"api_key_required": true,
"model_env": "MISTRAL_MODEL",
"default_model": "mistral-large-latest",
"description": "Mistral AI API",
"setup": {
"kind": "api_key",
"secret_name": "llm_mistral_api_key",
"key_url": "https://console.mistral.ai/api-keys",
"display_name": "Mistral",
"can_list_models": true
}
},
{
"id": "yandex",
"aliases": [
"yandex_ai_studio",
"yandexgpt",
"yandex_gpt"
],
"protocol": "open_ai_completions",
"default_base_url": "https://ai.api.cloud.yandex.net/v1",
"api_key_env": "YANDEX_API_KEY",
"api_key_required": true,
"model_env": "YANDEX_MODEL",
"extra_headers_env": "YANDEX_EXTRA_HEADERS",
"default_model": "yandexgpt-lite",
"description": "Yandex AI Studio (YandexGPT)",
"setup": {
"kind": "api_key",
"secret_name": "llm_yandex_api_key",
"key_url": "https://aistudio.yandex.ru/platform/folders/",
"display_name": "Yandex AI Studio",
"can_list_models": true
}
},
{
"id": "minimax",
"aliases": [
"mini_max"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.minimax.io/v1",
"api_key_env": "MINIMAX_API_KEY",
"api_key_required": true,
"base_url_env": "MINIMAX_BASE_URL",
"model_env": "MINIMAX_MODEL",
"default_model": "MiniMax-M2.7",
"description": "MiniMax API (MiniMax-M2.7, MiniMax-M2.7-highspeed, MiniMax-M2.5 and MiniMax-M2.5-highspeed models)",
"setup": {
"kind": "api_key",
"secret_name": "llm_minimax_api_key",
"key_url": "https://platform.minimax.io",
"display_name": "MiniMax",
"can_list_models": false
}
},
{
"id": "cloudflare",
"aliases": [
"cloudflare_ai",
"cf_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "CLOUDFLARE_API_KEY",
"api_key_required": true,
"base_url_env": "CLOUDFLARE_BASE_URL",
"model_env": "CLOUDFLARE_MODEL",
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"description": "Cloudflare Workers AI",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_cloudflare_api_key",
"display_name": "Cloudflare Workers AI",
"can_list_models": false
}
},
{
"id": "bedrock",
"aliases": [
"aws_bedrock",
"aws"
],
"protocol": "bedrock",
"api_key_required": false,
"model_env": "BEDROCK_MODEL",
"default_model": "anthropic.claude-sonnet-4-20250514-v1:0",
"description": "Claude & other models via AWS (IAM, SSO)",
"setup": {
"kind": "aws_credentials",
"display_name": "AWS Bedrock",
"supports_cross_region": true,
"supports_profile": true
}
}
]
Reference in New Issue View Git Blame Copy Permalink