Testing Agents Behind Authentication

Many production agents sit behind an API that requires authentication — most commonly an OAuth2 client-credentials flow (Auth0 “machine-to-machine” is the canonical example). LangWatch’s HTTP agent type carries a static credential (bearer / api_key / basic), which is not enough when a token has to be exchanged first. A custom code agent closes that gap: a few lines of Python that fetch the token, call your API with it, and keep the credentials in your project’s encrypted secret store — never in the agent’s stored source.

How it fits together

Project secrets hold the client ID and client secret, encrypted at rest. They are exposed to your code agent’s Python as the injected secrets namespace.
The code agent exchanges the credentials for an access token and calls your protected API with it.
A scenario drives the agent like a real user and judges the answers — the agent only passes if it actually got through the auth wall.

1. Store the credentials as project secrets

In Settings → Secrets, create the credentials and the endpoint coordinates — keeping the endpoints in secrets too means the whole agent is configurable without touching its code or any API. Create AUTH0_CLIENT_SECRET through the Settings → Secrets UI, not on a command line: a secret passed as a CLI argument lands in your shell history and the process listing, and can end up in CI logs. The non-sensitive coordinates are fine to create via the CLI:

langwatch secret create AUTH0_CLIENT_ID --value "<your client id>"
langwatch secret create AUTH0_TOKEN_URL --value "https://your-tenant.us.auth0.com/oauth/token"
langwatch secret create AUTH0_AUDIENCE --value "https://api.your-company.internal"
langwatch secret create AUTH0_API_URL --value "https://api.your-company.internal/chat"

Secret names must be UPPER_SNAKE_CASE (^[A-Z][A-Z0-9_]*$). Values are encrypted and never returned by any API.

Never paste the client secret into the agent’s Python. The stored code is readable by anyone with project access; the secrets namespace exists so the credential is stored encrypted at rest and never persisted in the agent’s source. At run time it is injected into the execution’s memory and sent only to your token endpoint.

2. Write the code agent

Create a code agent with a single input message and a single declared output output: the declared output key must match the key the Python returns, or the run fails with missing_output. Use this Python, the reference implementation, exercised continuously against the real code-agent runtime:

import requests


class Code:
    def __call__(self, message: str):
        # Step 1: exchange the client credentials for an access token.
        token_response = requests.post(
            secrets.AUTH0_TOKEN_URL,
            json={
                "grant_type": "client_credentials",
                "client_id": secrets.AUTH0_CLIENT_ID,
                "client_secret": secrets.AUTH0_CLIENT_SECRET,
                "audience": secrets.AUTH0_AUDIENCE,
            },
            timeout=10,
            # A 307/308 redirect would re-send the POST body, credential
            # included, to wherever the response points. Never follow one.
            allow_redirects=False,
        )
        token_response.raise_for_status()
        access_token = token_response.json()["access_token"]

        # Step 2: call the protected API with the minted token.
        api_response = requests.post(
            secrets.AUTH0_API_URL,
            json={"message": message},
            headers={"Authorization": f"Bearer {access_token}"},
            timeout=30,
        )
        api_response.raise_for_status()

        return {"output": api_response.json()["reply"]}

Things the sandbox enforces, in the order people trip on them:

Entry point: class Code with __call__ (no constructor arguments).
secrets is injected into the module globals — it is not the Python stdlib secrets module. Do not import secrets; that would shadow the injected namespace. os.environ is not populated in the sandbox.
Return every declared output key, or the run fails with missing_output.
Available packages: requests, httpx, pydantic, langwatch.
Budget: the token fetch plus your API call must finish inside the runner’s wall-clock limit (60s by default) — keep explicit timeouts on both requests.
Failure behavior: raise_for_status() puts the URL and status code in the error — never the request body — so a rejected credential fails loudly without exposing the secret.

3. Map the agent’s input for scenarios

Because everything else rides the secrets namespace, the agent has a single input — the conversation message — and needs exactly one mapping: message from the scenario’s input. That mapping is creatable in the agent editor today.

Keeping the endpoint coordinates in secrets is deliberate: the alternative — static value mappings on extra inputs — currently cannot be created from the editor UI, only via the agents API. UI support for static value mappings is tracked in #6371.

4. Run a scenario against it

Create a scenario whose criteria can only pass if the agent really got through the auth wall — e.g. facts that only your protected API returns — bundle it with the agent in a suite, and run it. The simulation transcript shows the agent’s answers; the run fails if the exchange breaks.

[user]      where is my order #1042?
[assistant] Order #1042 shipped from the Rotterdam warehouse on Tuesday
            and arrives Thursday before noon.

That answer exists only behind the auth wall — which is the point: the scenario passing is the proof that authentication works.

Troubleshooting

Symptom	Likely cause
Agent answers are empty	The token exchange or API call raised — check the credentials and endpoint URLs. (Improved error surfacing is tracked in #6340.)
`missing_output` error	The Python returned a dict without one of the agent’s declared output keys.
Credential works locally but not on the platform	The secret name in `secrets.NAME` doesn’t match a stored project secret, or the value was stored in a different project.
Timeout	Token fetch + downstream call exceeded the 60s runner budget — check both endpoints’ latency.

Get Started

Agent Simulations

Observability

Evaluations

Prompt Management

Platform

Examples & Cookbooks

Help

Testing Agents Behind Authentication

Testing Agents Behind Authentication

How it fits together

1. Store the credentials as project secrets

2. Write the code agent

3. Map the agent’s input for scenarios

4. Run a scenario against it

Troubleshooting

​Testing Agents Behind Authentication

​How it fits together

​1. Store the credentials as project secrets

​2. Write the code agent

​3. Map the agent’s input for scenarios

​4. Run a scenario against it

​Troubleshooting

Testing Agents Behind Authentication

How it fits together

1. Store the credentials as project secrets

2. Write the code agent

3. Map the agent’s input for scenarios

4. Run a scenario against it

Troubleshooting