Renamed in v1. What used to be called a “provider binding” (GatewayProviderCredential) is gone. Gateway-only settings now live directly on the ModelProvider record itself, on the Advanced (Gateway) tab of the ModelProvider editor.There is no separate gateway-binding entity to create, list, edit, or disable. One ModelProvider = one gateway-eligible credential. Virtual keys reference ModelProviders directly.
Why the change
The old binding model had two rows per logical credential: a ModelProvider (where the API key lived) and a GatewayProviderCredential (where gateway-only metadata like RPM, RPD, slot, fallback priority lived). The split forced operators to navigate two pages, remember which fields belonged where, and pay the cognitive cost of “slot” labels that only mattered for VK fallback-chain composition.
Folding the gateway-specific fields onto the ModelProvider itself collapses the surface to a single editable record. The rest of the system (VKs, routing policies, traces) refers to ModelProviders by id; nothing references a separate binding row.
What replaced the binding
| Old (v0) | New (v1) |
|---|
GatewayProviderCredential row | (deleted, fields folded onto ModelProvider) |
| Bound to a project | Lives at any of ORGANIZATION, TEAM, PROJECT scope (see Scope) |
slot (free-form label) | Removed. VKs now reference ModelProviders by id directly; the routing policy carries the ordered list |
rate_limit.rpm, .tpm, .rpd | Same fields, now on ModelProvider → Advanced (Gateway) tab |
fallback_priority_global | Same field, on the Advanced tab. Default-policy ordering uses this then createdAt |
provider_credential_id (VK reference) | VKs reference model_provider_id directly |
/config/:vk_id contract and the vk-config-bundle.feature spec.
Configuring gateway settings on a ModelProvider
- Open Settings → Model Providers.
- Pick (or create) the provider you want gateway traffic to use.
- Switch to the Advanced (Gateway) tab inside the editor drawer.
- Fill in the gateway-only fields (RPM / TPM / RPD / fallback priority / providerConfig JSON overrides). The provider credential itself (API key, region, endpoint) stays on the Basic tab and is reused by both the gateway and the rest of LangWatch.
- Save. The new settings reach the gateway within ~30 s via the
/changes long-poll.
Scope
ModelProviders live at one of three scopes:
| Scope | Visible to VKs scoped at… |
|---|
ORGANIZATION | every team and project in the org |
TEAM | projects in that team, and the team itself |
PROJECT | that project only |
Fields on the Advanced (Gateway) tab
Rate limit: RPM
Requests per minute the gateway dispatches through this ModelProvider before returning 429 rate_limit_exceeded. Independent of any VK-level rate limit; combines via token-bucket cross-dimension accounting.
Rate limit: TPM
Tokens per minute. Stored in v1, enforcement waits on the streaming-usage accumulator (v1.1).
Rate limit: RPD
Requests per day rolling window.
Fallback priority (global)
Lower number = higher priority. Used as the tie-breaker when a routing policy lists multiple ModelProviders and several are simultaneously eligible. Default-policy ordering falls back to createdAt when fallback priority is unset.
Provider config (JSON overrides)
Free-form JSON merged onto the ModelProvider’s basic config at gateway-dispatch time. Use for per-credential overrides (e.g. a different Azure deployment name on a sibling row of the same credential family).
Read-only diagnostic block
The Advanced tab also surfaces health status, circuit-breaker state, and the last disabledAt timestamp. These are populated by the gateway; the operator can’t write them directly.
Multiple deployments of the same provider
Create a second ModelProvider row of the same family (e.g. two OpenAI rows for us + eu regions). VKs and routing policies reference each by id. The old slot mechanism is gone; sibling rows with distinct names replace it.
See also
- Virtual keys — multi-scope semantics and how a VK’s eligible-provider set is computed.
- Concepts — mental model for the gateway primitives.
- RBAC —
modelProviders:manage gates the Advanced tab; the old gatewayProviders:* resource is retired.
- Rate limits — RPM/RPD/TPM dimension model.
