> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Gateway provider settings

> Gateway-specific settings (rate limits, fallback priority, provider config overrides) now live on each ModelProvider's Advanced tab.

<Note>
  **Renamed in v1.** What used to be called a "provider binding" (`GatewayProviderCredential`) is gone. Gateway-only settings now live directly on the **ModelProvider** record itself, on the **Advanced (Gateway)** tab of the ModelProvider editor.

  There is no separate gateway-binding entity to create, list, edit, or disable. One ModelProvider = one gateway-eligible credential. Virtual keys reference ModelProviders directly.
</Note>

## Why the change

The old binding model had two rows per logical credential: a `ModelProvider` (where the API key lived) and a `GatewayProviderCredential` (where gateway-only metadata like RPM, RPD, slot, fallback priority lived). The split forced operators to navigate two pages, remember which fields belonged where, and pay the cognitive cost of "slot" labels that only mattered for VK fallback-chain composition.

Folding the gateway-specific fields onto the ModelProvider itself collapses the surface to a single editable record. The rest of the system (VKs, routing policies, traces) refers to ModelProviders by id; nothing references a separate binding row.

## What replaced the binding

| Old (v0)                                | New (v1)                                                                                              |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `GatewayProviderCredential` row         | (deleted, fields folded onto `ModelProvider`)                                                         |
| Bound to a project                      | Lives at any of `ORGANIZATION`, `TEAM`, `PROJECT` scope (see [Scope](#scope))                         |
| `slot` (free-form label)                | Removed. VKs now reference ModelProviders by id directly; the routing policy carries the ordered list |
| `rate_limit.rpm`, `.tpm`, `.rpd`        | Same fields, now on **ModelProvider → Advanced (Gateway)** tab                                        |
| `fallback_priority_global`              | Same field, on the Advanced tab. Default-policy ordering uses this then `createdAt`                   |
| `provider_credential_id` (VK reference) | VKs reference `model_provider_id` directly                                                            |

For the new wire shape of the bundle a VK resolves to, see the [`/config/:vk_id` contract](/docs/ai-gateway/api/management#config-vk-id) and the `vk-config-bundle.feature` spec.

## Configuring gateway settings on a ModelProvider

1. Open **Settings → Model Providers**.
2. Pick (or create) the provider you want gateway traffic to use.
3. Switch to the **Advanced (Gateway)** tab inside the editor drawer.
4. Fill in the gateway-only fields (RPM / TPM / RPD / fallback priority / providerConfig JSON overrides). The provider credential itself (API key, region, endpoint) stays on the **Basic** tab and is reused by both the gateway and the rest of LangWatch.
5. Save. The new settings reach the gateway within \~30 s via the `/changes` long-poll.

## Scope

ModelProviders live at one of three scopes:

| Scope          | Visible to VKs scoped at…                  |
| -------------- | ------------------------------------------ |
| `ORGANIZATION` | every team and project in the org          |
| `TEAM`         | projects in that team, and the team itself |
| `PROJECT`      | that project only                          |

A VK's eligible-provider set is the union of every ModelProvider whose scope is at or above one of the VK's scopes (see [vk-scope-inheritance.feature](https://github.com/langwatch/langwatch/blob/main/specs/ai-gateway/governance/vk-scope-inheritance.feature) for the full cascade).

## Fields on the Advanced (Gateway) tab

### Rate limit: RPM

**R**equests **p**er **m**inute the gateway dispatches through this ModelProvider before returning `429 rate_limit_exceeded`. Independent of any VK-level rate limit; combines via token-bucket cross-dimension accounting.

### Rate limit: TPM

**T**okens **p**er **m**inute. Stored in v1, enforcement waits on the streaming-usage accumulator (v1.1).

### Rate limit: RPD

**R**equests **p**er **d**ay rolling window.

### Fallback priority (global)

Lower number = higher priority. Used as the tie-breaker when a routing policy lists multiple ModelProviders and several are simultaneously eligible. Default-policy ordering falls back to `createdAt` when fallback priority is unset.

### Provider config (JSON overrides)

Free-form JSON merged onto the ModelProvider's basic config at gateway-dispatch time. Use for per-credential overrides (e.g. a different Azure deployment name on a sibling row of the same credential family).

### Read-only diagnostic block

The Advanced tab also surfaces health status, circuit-breaker state, and the last `disabledAt` timestamp. These are populated by the gateway; the operator can't write them directly.

## Multiple deployments of the same provider

Create a second ModelProvider row of the same family (e.g. two OpenAI rows for `us` + `eu` regions). VKs and routing policies reference each by id. The old `slot` mechanism is gone; sibling rows with distinct names replace it.

## See also

* [Virtual keys](/docs/ai-gateway/virtual-keys) — multi-scope semantics and how a VK's eligible-provider set is computed.
* [Concepts](/docs/ai-gateway/concepts) — mental model for the gateway primitives.
* [RBAC](/docs/ai-gateway/rbac) — `modelProviders:manage` gates the Advanced tab; the old `gatewayProviders:*` resource is retired.
* [Rate limits](/docs/ai-gateway/rate-limits) — RPM/RPD/TPM dimension model.