Skip to main content
LangWatch supports connecting to any model that exposes an OpenAI-compatible API, including local inference servers (Ollama, vLLM, TGI), cloud deployments (Databricks, Azure ML), and custom APIs.

​
Adding a Custom Model

  1. Navigate to Settings in your project dashboard
  2. Select Model Provider from the settings menu
  3. Enable Custom model
  4. Configure your model:
For local models that don’t require authentication, enter any non-empty string as the API key.

​
Example Configurations

Ollama vLLM Databricks

​
Using Custom Models

Once configured, your custom models appear in the model selector throughout LangWatch, including the Prompt Playground and when configuring scenarios. When referencing your custom model in code or API calls, use the format:
For example, if you configured a model named llama-3.1-70b, reference it as custom/llama-3.1-70b.