> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Run Instant Evals on your own decision model > Run Instant Evals on a decision model in your own provider account, set up Azure, and read the rows your model cannot judge. ## LangWatch Instant Preview or your own model Instant Evals run on **LangWatch Instant Preview** by default, billed as in [Limits and cost](https://langwatch.ai/docs/features/instant-evals/limits-and-cost.md). You can pick a decision model in your own provider account instead, per organization, team or project. Your provider then bills you for each row, and LangWatch does not. > [!NOTE] > > This is off by default. Ask LangWatch to turn it on for your organization. Until then, the **Instant Evals** column offers LangWatch Instant Preview only. ## The models you can pick | Model in the picker | Provider to add | What the provider needs | | - | - | - | | OpenAI Decisions API | OpenAI | API key. Runs `gpt-6-luna` | | Azure OpenAI Decisions API | Azure OpenAI | API key, endpoint and a `gpt-6-luna` deployment on your resource. See [Azure](#azure) | | Microsoft Decision 1 (Azure) | Azure OpenAI | API key, endpoint and a Microsoft Decision 1 deployment on Azure AI Foundry. See [Azure](#azure) | | Inception Mercury Decide | Inception | API key | | Perplexity Decider v1.2 | Perplexity | API key | | Perplexity Decider Flash v1.2 | Perplexity | API key | | Cloudflare Clef | Cloudflare Workers AI | Account id and API token | | Cloudflare Clef Flash | Cloudflare Workers AI | Account id and API token | | TypeSafe Jev 1.14 | TypeSafe | API key | | Jev 1.14 (OpenRouter) | OpenRouter | API key | | Microsoft Decision 1 (OpenRouter) | OpenRouter | API key | | Liquid d1.1 (OpenRouter) | OpenRouter | API key | | Fireworks Rune | Fireworks | API key | OpenAI Decisions API and Azure OpenAI Decisions API need `gpt-6-luna` on that account. The other models need no OpenAI account. ## Set it up **1. Add the provider** In **Settings > Model Providers**, click **Add Model Provider**, pick the provider from the table and enter its key. Scope it to each organization, team or project that should use it. See [Model Providers](https://langwatch.ai/docs/platform/model-providers.md). **2. Pick the model** In **Default Models** on the same page, add or edit a config and pick its scopes. In the **Instant Evals** picker, choose your model and save. ![Edit default models drawer for the ACME organization, with the Instant Evals picker open on LangWatch Instant Preview and OpenAI Decisions API](https://mintcdn.com/langwatch/ArGDsvaT_v3TtVxP/images/instant-evals/own-model-picker.png?fit=max\&auto=format\&n=ArGDsvaT_v3TtVxP\&q=85\&s=65bb2c35dff7df5089ff5795b1e2d748) **3. Run** Start a run as usual. The new model judges the next rows within five seconds. The config's **Instant Evals** column then shows your model. ![Default models table with the ACME config, whose Instant Evals column shows OpenAI Decisions API](https://mintcdn.com/langwatch/KLWVLGhFxFzMdL_Y/images/instant-evals/own-model-default-models.png?fit=max\&auto=format\&n=KLWVLGhFxFzMdL_Y\&q=85\&s=3ab5d25a5ed031e1dfb566f85545ee29) The picker offers a model only when every scope of the config holds an enabled key for it. An organization config sees only keys saved on the organization, not a key one project holds. A config on two projects sees only what both hold. As for other [default models](https://langwatch.ai/docs/platform/model-providers.md#default-models), a project config beats a team config, which beats an organization config. A provider key saved on the project beats one on its team or organization. ## Azure Both Azure models use the Azure OpenAI provider, with its API key (`AZURE_OPENAI_API_KEY`) and endpoint (`AZURE_OPENAI_ENDPOINT`). ![Azure OpenAI provider drawer with Use API Gateway off, the endpoint https://my-resource.openai.azure.com and the decisions deployment gpt-6-luna-eu](https://mintcdn.com/langwatch/KLWVLGhFxFzMdL_Y/images/instant-evals/own-model-azure-provider.png?fit=max\&auto=format\&n=KLWVLGhFxFzMdL_Y\&q=85\&s=e6f47738bfe85903f0846cb00cc9bc80) * **Endpoint** (`AZURE_OPENAI_ENDPOINT`): your resource URL, such as `https://my-resource.openai.azure.com`. LangWatch takes the resource name from it and calls `https://my-resource.openai.azure.com/openai/v1/decisions` or, for Microsoft Decision 1, `https://my-resource.services.ai.azure.com/providers/microsoft/v1/systemone`. Hosts on `openai.azure.com`, `cognitiveservices.azure.com` and `services.ai.azure.com` work. Other hosts, `http`, custom ports and API Management gateways are refused, so the key stays inside Azure. * **Decisions deployment** (`AZURE_OPENAI_DECISIONS_DEPLOYMENT`): your `gpt-6-luna` deployment name. Leave empty if it is `gpt-6-luna`. * **Microsoft Decision 1 deployment** (`AZURE_DECISION_1_DEPLOYMENT`): your Microsoft Decision 1 deployment name on Azure AI Foundry. Leave empty if it keeps the model's name. Keep **Use API Gateway** off. With it on, the endpoint and deployment fields are hidden and every row is skipped as `classifier_key_invalid`. Deploy the model on your resource before you pick it, in the Azure portal, Azure AI Foundry or the Azure CLI: ```bash theme={null} az cognitiveservices account deployment create \ --resource-group my-group --name my-resource \ --deployment-name gpt-6-luna --model-name gpt-6-luna --model-format OpenAI --model-version 2026-09-22 \ --sku-name GlobalStandard --sku-capacity 50 ``` Without the deployment, every row is skipped as `classifier_not_configured`. ## When your model cannot answer A run on your own model never falls back to LangWatch Instant Preview. Rows it cannot judge are marked `skipped` with the reason in `error`, and the run carries on. List them with `langwatch instant-eval results --status skipped`. | `error` | Why | Fix | | - | - | - | | `classifier_key_invalid` | The provider refused the key (401, 402 or 403), or the scope has no enabled provider row, no key, or an endpoint outside Azure | Check the row and its key at **Settings > Model Providers** | | `classifier_not_configured` | The provider answered 404: the account does not hold the model, or the Azure deployment does not exist | Deploy the model, or set the deployment name on the Azure row | | `classifier_rate_limited` | The provider answered 429 on every retry | Raise the rate limit or quota on your provider account | | `classifier_provider_unavailable` | The provider answered 5xx on every retry | Try again later | Rate limits and outages are retried before the row is skipped. Refused keys and missing models are not. ## What changes on your own model * **Cost.** Your provider bills you. LangWatch does not, and the [free budget](https://langwatch.ai/docs/features/instant-evals/limits-and-cost.md#the-free-budget) does not apply. * **Limits.** The [row, question and text limits](https://langwatch.ai/docs/features/instant-evals/limits-and-cost.md#limits) still apply. * **Data.** Each row's text and your questions go to your provider, under your agreement with it. LangWatch Instant Preview never sees them. * **Search bar.** Still runs on LangWatch Instant Preview. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.