LangWatch supports connecting to any model that exposes an OpenAI-compatible API, including local inference servers (Ollama, vLLM, TGI), cloud deployments (Databricks, Azure ML), and custom APIs.
Adding a Custom Model
- Navigate to Settings in your project dashboard
- Select Model Provider from the settings menu
- Enable Custom model
- Configure your model:
| Field | Description |
|---|
| Model Name | A descriptive name for your model (e.g., llama-3.1-70b) |
| Base URL | The endpoint URL for your model’s API |
| API Key | Authentication key (if required) |
For local models that don’t require authentication, enter any non-empty string as the API key.
Example Configurations
Ollama
| Field | Value |
|---|
| Base URL | http://localhost:11434/v1 |
| API Key | ollama |
| Field | Value |
|---|
| Base URL | http://localhost:8000/v1 |
| API Key | Your configured token |
| Field | Value |
|---|
| Base URL | https://<workspace>.cloud.databricks.com/serving-endpoints |
| API Key | Your Databricks personal access token |
Using Custom Models
Once configured, your custom models appear in the model selector throughout LangWatch, including the Prompt Playground and when configuring scenarios.
When referencing your custom model in code or API calls, use the format:
For example, if you configured a model named llama-3.1-70b, reference it as custom/llama-3.1-70b.
