Skip to contentSkip to page list

Docs

npx langwatch

Trace any coding agent · Setup guide →

Custom Models

Configure and use custom LLM models in LangWatch, including local inference servers and external endpoints like Databricks.

On this page

LangWatch supports connecting to any model that exposes an OpenAI-compatible API, including local inference servers (Ollama, vLLM, TGI), cloud deployments (Databricks, Azure ML), and custom APIs.

Adding a Custom Model

  1. Navigate to Settings in your project dashboard
  2. Select Model Provider from the settings menu
  3. Enable Custom model
  4. Configure your model:
FieldDescription
Model NameA descriptive name for your model (e.g., llama-3.1-70b)
Base URLThe endpoint URL for your model's API
API KeyAuthentication key (if required)

For local models that don't require authentication, enter any non-empty string as the API key.

Example Configurations

Ollama

FieldValue
Base URLhttp://localhost:11434/v1
API Keyollama

vLLM

FieldValue
Base URLhttp://localhost:8000/v1
API KeyYour configured token

Databricks

FieldValue
Base URLhttps://<workspace>.cloud.databricks.com/serving-endpoints
API KeyYour Databricks personal access token

Using Custom Models

Once configured, your custom models appear in the model selector throughout LangWatch, including the Prompt Playground and when configuring scenarios.

When referencing your custom model in code or API calls, use the format:

example.txtText
custom/<your-model-name>

For example, if you configured a model named llama-3.1-70b, reference it as custom/llama-3.1-70b.

To check it worked, open the Prompt Playground and confirm your model is listed in the model selector, then send it a message to see that the endpoint answers.

Documentation