Multi-tenant SaaS reseller pattern

If you’re building a SaaS product on top of LLMs, and you want to let your customers bring their own budget without giving them direct provider credentials, this pattern is for you. You become the upstream provider; the LangWatch AI Gateway becomes your provisioning + enforcement + audit layer.

The shape

End user's app
      │
      ▼
YOUR SaaS backend ──provisions VKs via──▶ LangWatch Management API (/api/gateway/v1/*)
      │
      │  issues VK to end-user's runtime
      ▼
End user's runtime ──/v1/chat/completions──▶ LangWatch Gateway ──▶ Upstream provider (your account)

Key properties:

Each end-user has their own VK. One per customer, or per seat, your call.
You set per-customer budgets. Enforced at the gateway. When a customer hits their cap, the gateway returns 402 budget_exceeded. Your code doesn’t need to track spend.
All audit + trace data lands in your LangWatch project. End-users never see LangWatch.
End-users can’t escape your policy. Policy rules, model allowlists, cache rules all attach to the VK.

Step 1: Model the tenancy

Decide your scope granularity:

Model	VK per	Budget scope	Use case
Per customer	one VK per tenant	`virtual_key`	Most SaaS apps, one account = one VK
Per seat	one VK per LangWatch user	`principal`	Internal platform teams where every seat is a real LangWatch user
Per team	one VK per sub-org	`team`-scope budgets	Customers have their own teams/projects

For this walkthrough, assume per-customer model. Note that principal budgets target real LangWatch user accounts, which your external customers are not; for reseller tenancy the per-customer unit is the virtual key itself.

Step 2: Configure the upstream ModelProvider once

Your LangWatch org owns a single ModelProvider row per upstream (OpenAI, Anthropic) at ORGANIZATION scope. End-users never see these. Configure them under Settings → Model Providers → Add Model Provider (Scope = Organization), then open each row’s Advanced (Gateway) tab and set the per-credential gateway caps:

Field	Suggested value
RPM	100000 (your bulk-purchase cap from the upstream)
Fallback priority	10 for primary, 20 for backup
Provider config (JSON)	region/deployment overrides as needed

No separate “binding” entity to mint. The ModelProvider id is what your code references:

# Look up the id once and store it in your backend config
langwatch model-providers list --format json | \
  jq -r '.[] | select(.provider == "openai" and .scope.scope_type == "ORGANIZATION") | .id'
# → mp_01HZX...

Step 3: Provision a VK with its cap when a customer signs up

One call mints the key AND its budget, atomically: the key can never exist for a moment without the cap you asked for. Server-side code (Node.js example):

import { VirtualKeysApiService } from "langwatch";

const langwatch = new VirtualKeysApiService({
  apiKey: process.env.LANGWATCH_BACKEND_TOKEN,
});

export async function provisionCustomer(customerId: string, planLimitUsd: string) {
  const { virtual_key, secret } = await langwatch.create({
    name: `customer-${customerId}`,
    description: `Tenant key for ${customerId}`,
    // scopes omitted: the key defaults to your backend token's project,
    // which is exactly where you want the customer's traces to land.
    budget: {
      limit_usd: planLimitUsd, // from your plans table
      window: "MONTH",
      on_breach: "BLOCK",      // or WARN for grace periods
    },
    config: {
      modelsAllowed: ["gpt-5-mini", "gpt-5", "claude-haiku-4-5-20251001"],
      cache: { mode: "respect" },
    },
  });

  // Persist vk.id; hand the secret to the customer exactly once.
  return { vkId: virtual_key.id, secret };
}

The eligible-provider set is the union of every ModelProvider visible from the key’s scopes (the ones you configured in Step 2). Pin routing_policy_id plus routing_mode: "POLICY" if you run a custom ordering; otherwise leave routing at its default. In a signup webhook:

const { vkId, secret } = await provisionCustomer(newCustomer.id, "25");
await db.customers.update({
  where: { id: newCustomer.id },
  data: { langwatchVkId: vkId },
});
// Send `secret` to the customer via email / dashboard UI / API response
// exactly once. Your system never stores it in plaintext.

Step 4: Change the cap when the plan changes

The budget field on update upserts the key’s own cap, so plan changes are one call keyed by the VK id you stored (no budget id bookkeeping):

export async function setCustomerPlan(vkId: string, planLimitUsd: string) {
  await langwatch.update(vkId, {
    budget: { limit_usd: planLimitUsd, window: "MONTH" },
  });
}

Pass budget: null to remove the cap entirely (the spend history stays). Changes propagate to the gateway within 30 s via the /changes feed, no restart, no customer impact.

Step 5: End-user makes a call

Your customer’s app calls the gateway directly with the VK you gave them:

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.langwatch.ai/v1",
    api_key=os.environ["MY_SAAS_API_KEY"],  # the VK secret you sent them
)

The gateway:

Authenticates the VK.
Checks the customer’s budget.
Applies your modelsAllowed allowlist.
Dispatches to OpenAI using your provider credential.
Meters the cost against the customer’s budget.
Emits a LangWatch trace into your project, attributed to the key.

Step 6: Show the customer their spend and bill them

The read-back half of the loop is one REST call per key:

const spend = await langwatch.spend(vkId); // current UTC month by default
// { virtual_key_id, spent_usd: "12.4531", requests: 1841, window: {...} }

or from a shell:

langwatch virtual-keys spend vk_01HZX... -f json

The number comes from the same cost path the LangWatch dashboard reads, so what you invoice matches what you see in the UI. For the enforcement view (spend against the cap), list VK-scoped budgets:

langwatch gateway-budgets list --scope-type virtual-key -f json | \
  jq '[.budgets[] | {vk: .scope_id, spent: .spent_usd, limit: .limit_usd}]'

spend_available: false in that output means spend could not be totalled server-side; hold billing until it recovers rather than invoicing a zero.

(a) Bill from your pricing table

You define a markup over what the gateway shows you spent, and your Stripe/billing layer reads the spend numbers above and invoices.

(b) Passthrough billing

For passthrough, your UI shows the same numbers you see in LangWatch + a per-customer markup or flat management fee.

Handling over-limit customers

When on_breach: BLOCK, the gateway returns 402 budget_exceeded. Your customer sees:

{
  "error": {
    "type": "budget_exceeded",
    "message": "Budget exceeded for scope=virtual_key window=month",
    "code": "budget_exceeded"
  }
}

Your customer’s app should catch this and either:

Show an upgrade CTA in the end-user’s UI.
Fall back to a free-tier response (“you’ve hit your monthly cap; upgrade for unlimited”).

Because the response is standard OpenAI-compatible, your error handler only needs to switch on error.type.

Rotation & revocation

Customer resets their API key in your UI → you call langwatch.rotate(vkId), get a new secret, send it to the customer. The old secret keeps authenticating for a 24 h grace window, so a client you have not reached yet does not start failing mid-rollout. The gateway’s cache TTL (~60 s) is only how long the new secret takes to be accepted everywhere, not when the old one expires. Customer cancels their subscription → langwatch.revoke(vkId). The next request returns 403 virtual_key_revoked. If you want a grace period, schedule the revoke job for 24–48 h after cancellation instead.

Audit: who did what

Every write through your backend credential is audited with action, target, and before/after metadata. Scoped API keys attribute the write to the key’s owning user; legacy project keys carry no user, so their writes record the machine principal svc_<your_project_id>. Filter the audit log on target virtual_key to get a per-customer provisioning history, under /settings/audit-log in the LangWatch UI. If you need SIEM export: query ONLY the gateway-shape AuditLog rows with a read-only role scoped to that table, and ship the filtered result set, never a whole-database dump (which would carry unrelated, sensitive application data). For example, a daily job running

COPY (SELECT id, "organizationId", "userId", action, "targetKind", "targetId", "createdAt" FROM "AuditLog" WHERE "organizationId" = '<your-org-id>' AND "targetKind" IN ('virtual_key', 'budget', 'model_provider', 'routing_policy', 'cache_rule') AND "createdAt" >= now() - interval '1 day') TO STDOUT WITH CSV

into your SIEM’s ingestion path: an explicit column list pinned to your organization, leaving the before/after payloads out unless your SIEM genuinely needs them. There is no public REST audit-export endpoint in v1, see Audit log → Querying programmatically for the supported paths (UI CSV download, direct SQL).

Gotchas

Never let the customer see your backend API token. It has virtualKeys:create; they’d provision more VKs charged to you.
Rotate VKs when an end-user leaves your customer’s org. Otherwise the ex-user keeps spend access until the budget resets.
Name keys after your tenant id (customer-<id>) at creation time. The key’s name and id are your join keys for spend read-back and audit history; principal_user_id is only for real LangWatch user accounts, not your external customers.
Test the 402 path in your app before go-live. Many apps have unhandled exceptions on budget breach and crash the user’s flow.
Budgets scoped to virtual_key are the right level for per-customer enforcement, and the inline budget field on create/update manages exactly that budget for you.

Rate limits at your gateway level

If you want to throttle a specific customer without moving them to a different plan:

await langwatch.update(vkId, {
  config: {
    rateLimits: { rpm: 60, tpm: 100000 },
  },
});

Gateway enforces at the VK level; customer sees 429 rate_limit_exceeded. Combine with a short-window budget (hour, minute) for finer control.

Get Started

SDK Integration

Coding CLI Integrations

Virtual Keys & Budgets

Governance

Providers

Features

API Reference

Self-Hosting

Cookbooks

Multi-tenant SaaS reseller pattern

The shape

Step 1: Model the tenancy

Step 2: Configure the upstream ModelProvider once

Step 3: Provision a VK with its cap when a customer signs up

Step 4: Change the cap when the plan changes

Step 5: End-user makes a call

Step 6: Show the customer their spend and bill them

(a) Bill from your pricing table

(b) Passthrough billing

Handling over-limit customers

Rotation & revocation

Audit: who did what

Gotchas

Rate limits at your gateway level

See also

​The shape

​Step 1: Model the tenancy

​Step 2: Configure the upstream ModelProvider once

​Step 3: Provision a VK with its cap when a customer signs up

​Step 4: Change the cap when the plan changes

​Step 5: End-user makes a call

​Step 6: Show the customer their spend and bill them

​(a) Bill from your pricing table

​(b) Passthrough billing

​Handling over-limit customers

​Rotation & revocation

​Audit: who did what

​Gotchas

​Rate limits at your gateway level

​See also

The shape

Step 1: Model the tenancy

Step 2: Configure the upstream ModelProvider once

Step 3: Provision a VK with its cap when a customer signs up

Step 4: Change the cap when the plan changes

Step 5: End-user makes a call

Step 6: Show the customer their spend and bill them

(a) Bill from your pricing table

(b) Passthrough billing

Handling over-limit customers

Rotation & revocation

Audit: who did what

Gotchas

Rate limits at your gateway level

See also