Skip to main content

Documentation Index

Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

A provider binding is a gateway-specific wrapper around a model provider you’ve already configured in Settings → Model Providers. The LangWatch dashboard already knows how to talk to OpenAI / Anthropic / Gemini / Azure / Bedrock / Vertex — the gateway binding layers gateway-only settings on top of that existing credential:
  • Which slot the binding fills in VK fallback chains
  • Per-binding rate limits (RPM / RPD) applied before provider dispatch
  • Fallback priority for selecting this binding when a VK has multiple bindings eligible
  • (Future) Rotation policy for credential lifecycle
No new API keys here — the provider’s real credential stays in Settings → Model Providers, encrypted at rest with per-organization KMS.

​
Binding a provider to the gateway

  1. Open /gateway/providers → Bind provider.
  2. Select one of the enabled model providers you have access to. The dropdown shows providers from every scope you can reach: the current project, any team containing it, and the org above (see Scope & access below). Providers not yet enabled appear in Settings → Model Providers — enable them there first.
  3. Fill in the gateway-specific fields below.
  4. Click Bind provider. The binding shows up in the VK “Provider fallback chain” drawer for every VK in the project.

​
Scope & access

Model providers live at one of three scopes — org / team / project — mirroring the RBAC principal-scope pattern. A provider binding in a project can reference any model provider that scope-chain walks from the binding’s project up to the owning org.
Provider scopeVisible to bindings in…
ORGANIZATIONevery project in the org (via any team)
TEAMevery project in that team
PROJECTthat project only
The binding picker labels each provider with its scope so you can see where credentials come from at a glance (e.g. OpenAI (org: acme), OpenAI (project: acme-demo)). Why this matters for multi-project teams: one central OpenAI credential at the organization scope means all downstream projects reuse a single API key + billing account + rotation policy, while still letting individual projects layer their own gateway-level rate limits and slot names via the binding. Who can create at each scope — permission follows the standard LangWatch scope ladder:
To create a provider at…Need…
Project scopemodelProviders:manage on the project
Team scopemodelProviders:manage on the team
Organization scopeOrganization admin

​
Fields

​
Model provider

The underlying provider configured in Settings. One binding per provider per project. If a provider is already bound the dropdown omits it (re-edit via the row).

​
Slot

Free-form label used by the VK fallback chain to refer to this binding. Examples: primary, fallback-1, high-volume-openai. Defaults to primary. The slot is cosmetic — the gateway dispatches based on the VK fallback chain order, not the slot name. But slot names make the fallback chain human-readable (“primary → fallback-1 → fallback-2”) rather than showing opaque IDs.

​
Rotation policy

Tracks how the underlying credential rotates. v1 supports MANUAL only — the operator rotates the credential in Settings → Model Providers and the gateway picks up the new value on the next /changes long-poll (≤ 30 s).
The Bind-provider drawer does not surface this field in v1 because a single-option dropdown is worse UX than no field. Every binding is therefore implicitly MANUAL. The Edit drawer still exposes it for forward compat; only MANUAL is accepted. Automatic rotation (on schedule) + external-secret-store integration (HashiCorp Vault, AWS Secrets Manager, etc.) are tracked as v1.1 features and will restore the dropdown when implemented.

​
Rate limit — RPM

Requests per minute that the gateway will dispatch through this binding before returning 429 rate_limit_exceeded. Applied at the binding level; independent of any VK-level rate limit. Use this to protect a single upstream account from an over-eager VK. The gateway token-bucket enforces with cross-dimension accounting so you can combine per-binding + per-VK limits without double-counting. Leave blank to inherit the upstream provider’s limit (i.e., no gateway-side throttle). See Rate limits for the full dimension model.

​
Rate limit — TPM

Tokens per minute. Same shape as RPM but counts token usage instead of request count.
TPM is a v1.1 feature — the streaming-usage accumulator needs to be wired into the limiter before this can be enforced. In v1, a TPM limit is stored but not enforced; requests are accepted and you get the expected trace + budget accounting.

​
Rate limit — RPD

Requests per day. Rolling 24-hour window, resets at first request after the window ticks. Useful for capping long-tail daily spend. Only present on the Edit drawer because RPD doesn’t make sense for the initial bind (you usually set it after observing traffic).

​
Fallback priority (global)

A tiebreaker used when multiple bindings are simultaneously eligible for a VK’s fallback chain. Lower number = higher priority. Leave blank to fall back to the VK’s explicit ordering (normal case). Use when you want a single binding to be the “always-first fallback” across every VK in the project, without editing each VK’s chain. Example: fallback-priority-global: 10 on a cheap secondary makes that binding the first fallback for every VK that doesn’t override.

​
After binding

The new binding shows up in:
  • /gateway/providers list (the page you’re on)
  • VK fallback chain dropdown (add/remove/reorder in the VK drawer)
  • /gateway/usage aggregation (filter by provider slot or provider name)
  • Gateway traces as langwatch.provider = <provider_name> (the slot name is carried on the budget-debit event and in gateway_provider_attempts_total labels, but not emitted as a span attribute in v1)

​
Disabling a binding

Settings → Disable (soft disable, via the row menu):
  • Existing VKs that reference this binding continue to resolve — the binding remains valid on the hot path for VKs that haven’t been edited.
  • New VKs cannot select this binding in the create drawer.
  • Existing bindings can be re-edited to change slot / rate limits without undoing the disable.
Disable is idempotent. To delete, use the Archive flow which hard-removes from the VK fallback chain on next bundle refresh.

​
Permissions

ActionPermission
View bindingsgatewayProviders:view
Bind a new providergatewayProviders:manage
Edit an existing bindinggatewayProviders:update
Disable / archivegatewayProviders:manage
See RBAC for the full permission matrix.

​
See also