Migrating from Portkey

​

Concept mapping at a glance

​

Code migration: the minimum change

import Portkey from "portkey-ai";

const portkey = new Portkey({
  apiKey: process.env.PORTKEY_API_KEY,
  virtualKey: "my-openai-vk",
});

const response = await portkey.chat.completions.create({ ... });

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://gateway.langwatch.ai/v1",
  apiKey: process.env.LANGWATCH_VK,       // e.g. vk-lw-01HZX9K3M...
});

const response = await openai.chat.completions.create({ ... });

1. No Portkey SDK. LangWatch speaks the OpenAI protocol natively, use any official OpenAI client (Node, Python, Go, Java). Same story for Anthropic: new Anthropic({ baseURL: "https://gateway.langwatch.ai" }) with your VK as the apiKey.
2. virtualKey header → apiKey bearer. Portkey carries the VK in a custom header; LangWatch just uses the standard bearer so you don’t need provider-specific plumbing.

​

Config migration: Portkey Config JSON → LangWatch VK + ModelProviders

{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "virtual_key": "openai-vk", "override_params": { "max_tokens": 500 } },
    { "virtual_key": "anthropic-vk" }
  ],
  "cache": { "mode": "simple", "max_age": 3600 },
  "retry": { "attempts": 3 }
}

• Name: whatever you called the Portkey config (e.g. prod-chat)
• Routing policy: the one you just created (or leave unset to use the org default, which orders by fallbackPriorityGlobal then createdAt)
• Cache policy: respect (default, honours Anthropic cache_control passthrough)
• Optional: attach an existing LangWatch evaluator with executionMode=AS_GUARDRAIL as a pre or post hook if you had Portkey guardrails.

• Scope: project (or organization for fleet-wide) + target the new project
• Window: month
• Limit: your Portkey spend cap
• on_breach: block (hard-cap, returns 402) or warn (soft-cap, adds X-LangWatch-Budget-Warning header)

​

Rollout pattern: gradual migration without a big-bang

1. Week 1, shadow traffic. Mint a LangWatch VK with config.shadow_mode: true (v1.1; for v1 just log both upstream responses). Route 1-5% of production traffic through LangWatch alongside Portkey. Compare latency, error rate, and response equality.
2. Week 2, canary by tenant. Route low-risk tenants (internal users, dev environments) fully through LangWatch. Portkey still carries the majority.
3. Week 3, flip customer traffic. Route 50% of customer traffic through LangWatch, with a feature flag that can instantly revert.
4. Week 4, decommission Portkey. Once metrics show parity, null out the Portkey integration. Keep the Portkey org active for 30 days in case of rollback; delete after.

​

Feature-parity gaps (be explicit)

• Loadbalance strategy (sibling-key rotation for rate-limit absorption), LangWatch v1 routes to one credential per provider slot. Sibling-key loadbalancing is on the v1.1 roadmap. Workaround: if you genuinely need it, keep Portkey for those routes, run LangWatch in parallel for governance.
• Semantic cache: LangWatch v1 ships respect, force, disable modes for explicit Anthropic cache_control + a rule engine; semantic similarity lookups are v1.1.
• Prompt management embedded in the gateway: LangWatch has a dedicated Prompt Management product surface that’s more powerful than Portkey’s embedded library, but it’s a separate area rather than a subfolder of the gateway.
• Guardrails catalog: Portkey ships a pre-built library of guardrails; LangWatch reuses the evaluator catalog you already have (and any custom evaluator you’ve built). For most enterprise teams this is a net win (you own the logic); for quick-start it’s a cold start.

​

What to watch during + after migration

• Per-tenant spend: compare your Portkey invoice amount to sum(GatewayBudgetLedger.amountUsd) on the same time window. First-week variance ≤ 3% is normal (different rounding, model-version timings); > 5% warrants investigation.
• p99 latency: LangWatch’s gateway hot path is ~700 ns of gateway-local overhead; upstream provider latency dominates. Compare p99 with + without the gateway (shadow mode is the cleanest A/B). Any regression > 10 ms is worth a ticket.
• 401 rate: during the VK rollout window, any client still hitting Portkey’s endpoint will 404 post-decommission. Set an alert on gateway_http_requests_total{status="401"} rate spikes to catch stragglers before customer escalations.
• Audit log volume: your team will mutate more governance-configuration post-migration (budgets, guardrails, rotation policies). Expect 10-50× more audit rows than Portkey’s minimal audit surface, plan retention accordingly.

​

See also

• Virtual Keys: the LangWatch VK reference.
• Budgets: hierarchical budget semantics.
• Gateway provider settings: the Advanced (Gateway) tab on each ModelProvider.
• Credential-leak incident response: the “oh no it’s on GitHub” playbook.
• Cache-control rollout: how to roll cache rules to production.