Skip to main content

​
Add the credential

Open Settings > Model Providers, click Add Model Provider and select Custom (OpenAI-compatible). Give the row a name, pick the organization, team or project it belongs to, and fill in: Save the row. Every virtual key whose scope can see the row can now dispatch to it, unless the key restricts its providers. See Virtual keys.
The Custom OpenAI-compatible model provider drawer with CUSTOM_API_KEY, CUSTOM_BASE_URL, an Add Header button and the routing handle custom

The Custom (OpenAI-compatible) row: the optional CUSTOM_API_KEY, the required CUSTOM_BASE_URL, Add Header and the routing handle custom.

Add the models the endpoint serves under Custom Models on the row. GET /v1/models lists those names. The endpoint must be reachable from the gateway. The gateway always refuses link-local and cloud metadata addresses, and refuses private and loopback addresses when it runs with BLOCK_LOCAL_HTTP_CALLS=true. On LangWatch Cloud the endpoint must use https: a hosted gateway refuses to start without REQUIRE_HTTPS_CUSTOM_ENDPOINTS=true, and with it an http endpoint is refused. A self-hosted gateway accepts http while that setting stays false.
An http endpoint sends CUSTOM_API_KEY in cleartext, because the gateway puts it in an Authorization: Bearer header on every request. Anyone on the path between the gateway and the endpoint can read the key. Use http only for an endpoint with no CUSTOM_API_KEY. On a self-hosted gateway that dispatches to a row with a key, set REQUIRE_HTTPS_CUSTOM_ENDPOINTS=true as well.

​
Model names

Write custom/<model>, for example custom/qwen3-14b. Give the row a Routing handle when you have several custom endpoints: acme-gpu/qwen3-14b pins that row.

​
Send a request

​
Endpoints

Streaming, tools and other features depend on what the endpoint implements. A feature the endpoint does not support comes back as the endpoint’s own error.
Last modified on September 6, 2026