> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Providers

> Every LLM provider the LangWatch AI Gateway can dispatch to.

The gateway delegates provider-specific dispatch to [`bifrost/core`](https://github.com/maximhq/bifrost), which LangWatch embeds as a Go library. This page lists the providers the gateway can talk to today and the path to configuring each.

## Configure a provider credential once, use from many VKs

Every provider credential lives in the LangWatch **Model Providers** table (under **Settings → Model Providers**). It's the same surface that powers evaluators and the playground, no separate "gateway providers" store.

Gateway-only settings (rate limits, fallback priority, providerConfig overrides) live directly on the **ModelProvider** itself, on the **Advanced (Gateway)** tab of the editor. There is no separate binding wrapper. VKs reference ModelProviders by id; the order in the bound routing policy drives primary + fallback selection.

```
ModelProvider (raw credential + Advanced/Gateway settings)
  └── RoutingPolicy.modelProviderIds[] (ordered primary + fallback)
       └── VirtualKey.routingPolicyId (or default policy from the org)
```

This means: rotating the raw OpenAI key rotates it everywhere; changing a rate limit only affects gateway traffic (it's on the Advanced tab, not the basic credential); changing a VK's provider mix is one edit to the routing policy it references.

## Supported providers (v1)

| Provider                                                                   | Routes supported                                        | Auth methods          | Caching passthrough    |
| -------------------------------------------------------------------------- | ------------------------------------------------------- | --------------------- | ---------------------- |
| [OpenAI](/docs/ai-gateway/providers/openai)                                     | chat, responses, embeddings, images, audio, moderations | API key               | auto-prefix            |
| [Anthropic](/docs/ai-gateway/providers/anthropic)                               | messages, chat (translated), streaming                  | API key               | `cache_control` blocks |
| [Azure OpenAI](/docs/ai-gateway/providers/azure-openai)                         | chat, responses, embeddings, images, audio              | API key, AAD          | auto-prefix            |
| [AWS Bedrock](/docs/ai-gateway/providers/bedrock)                               | messages, chat, embeddings (Titan)                      | AWS SigV4, IRSA       | `cachePoint`           |
| [Google Vertex AI](/docs/ai-gateway/providers/vertex)                           | messages, chat                                          | GCP ADC, SA JSON      | implicit context cache |
| [Google Gemini](/docs/ai-gateway/providers/gemini)                              | chat                                                    | API key               | implicit context cache |
| [Custom OpenAI-compatible](/docs/ai-gateway/providers/custom-openai-compatible) | chat, embeddings (depends on upstream)                  | Bearer, custom header | opaque                 |

Behind the scenes bifrost/core also supports Groq, Cohere, Mistral, Ollama, vLLM, SGLang, Perplexity, ElevenLabs, and others. These will light up in the UI as LangWatch ships the per-provider configuration forms.

## Picking a primary + fallback

Most VKs end up with 1-2 fallback providers. Guidelines:

* **Anthropic-first** with Bedrock-Anthropic fallback. Same models on both sides; Bedrock is a warm backup during Anthropic direct outages.
* **OpenAI-first** with Anthropic fallback. Different model families but Claude Haiku can serve `gpt-5-mini` traffic acceptably for coding tasks.
* **Azure-first** with OpenAI direct fallback. Pins traffic inside Azure data-residency most of the time but tolerates regional Azure outages.

See [Fallback Chains](/docs/ai-gateway/providers/fallback-chains) for trigger semantics.

## Multi-region, multi-account

Each ModelProvider row pins a region (Azure endpoint, Bedrock region, Vertex project). If an organisation spans multiple regions, create sibling ModelProvider rows of the same family with a naming convention (e.g. `openai-eu`, `openai-us`) and use VK `model_aliases` to route `gpt-5-mini-eu` vs `gpt-5-mini-us` to the right ModelProvider id.