The cache became a span, and I didn't write a line of code for it

By Manouk, founder of LangWatch. A companion to the piece Kristiyan from BetterDB and I published together; this one is the view from the observability seat.

The first time I saw a BetterDB cache decision land inside a LangWatch trace, I actually grinned. There was no BetterDB plugin for LangWatch yet, no LangWatch hook inside BetterDB, no shared config, no meeting where our two teams agreed on a payload shape. And yet there it was: a semantic_cache span, nested under an LLM generation, sitting in the same waterfall as the rest of the request, with its cache key, its hit/miss, and its TTL rendered like they'd always belonged there.

This is the thing I keep telling people and they don't quite believe until they see it: it really is that easy. Two independent products, two teams that never coordinated on a format, and the trace just composed itself. That's not luck, and it's not a one-off demo trick; it's exactly what's supposed to happen when both layers speak OpenTelemetry. Understanding why it works so effortlessly is the entire reason I wanted to write this from LangWatch's side of the wire, rather than just co-sign the joint version.

The bet that pays off: be OTel-native in both directions

Most observability platforms are OTel-native in one direction. They'll happily ingest OpenTelemetry, because that's table stakes now. But their own SDKs still emit a proprietary format under the hood, so the moment you instrument with their library, you're back inside a walled garden. Ingest is open; emit is closed.

We built LangWatch to be open in both directions, and I'm genuinely proud of that decision because it's the thing that makes moments like this possible. LangWatch ingests standard OTLP traces with no proprietary agent sitting in your process, and the LangWatch SDKs (TypeScript, Python, Go) are themselves built on OpenTelemetry and emit standard spans. That symmetry is the whole payoff. It means an app that has never heard of LangWatch shows up in LangWatch correctly, and an app instrumented with our SDK exports spans that any OTel backend can read. You're never trapped on either end, and you never have to choose between "instrument for LangWatch" and "instrument for everything else." It's the same instrumentation.

The lovely part is how little the user has to do to get there. Standard spans in, standard spans out, and the integration you were dreading turns out to be nothing you have to build at all.

What "receiving" a trace actually looks like

Here's the part that's specific to being the observability layer, and it's my favorite.

When you're the platform on the receiving end, you don't get to assume anything about who instrumented what. The BetterDB playground app, an open-source RAG chatbot answering questions about Valkey, Redis, and related docs, built on BetterDB's Valkey-native semantic caching, LLM caching, and agent memory packages, was pointed at LangWatch with, by deliberate choice, zero LangWatch SDK calls in its application code. The team put plain OpenTelemetry semantics on standard spans and exported over OTLP/HTTP. That's about the strictest test you can throw at us, because there's no LangWatch-shaped hint anywhere in the payload telling the platform what it's looking at.

And it just works. It renders beautifully, and the reason is a single plain span attribute, langwatch.span.type, that says whether a span is a chain, an LLM call, a RAG step, or a tool. That one hint is enough for the platform to know to show retrieved contexts on the RAG span, and model I/O and cost on the LLM span. Two more attributes, langwatch.thread.id and langwatch.user.id, group a scattered pile of requests back into a single readable conversation. None of that is a proprietary protocol. It's light convention on top of the standard, which means an app can adopt it in an afternoon without adopting us.

And then three independent codebases (the app, the Vercel AI SDK's built-in telemetry, and the @betterdb/* packages) each ask the global OpenTelemetry tracer for a handle and emit spans. None of them imports the others. None of them knows LangWatch exists. When no provider is registered they all get no-op tracers and cost nothing; register one, and context propagation quietly nests all of it into one trace. From where I sit, a fully-formed, multi-layer trace assembled itself out of three strangers, and that's the standard doing exactly what it promised.

Why per-request beats the dashboard

The thing I love most, as someone whose whole job is making AI systems observable, is attribution. Not "the hit rate went up this week." This request, and the specific decision that shaped it.

Ask the playground a fresh question and you get the full pipeline: cache checks, memory recall, embeddings, a vector search, and finally the model actually generating. A few seconds, a fraction of a cent, a dozen-plus spans. Ask the same question again and the shape collapses. A three-millisecond cache check reports a hit, the saved cost is written directly onto that span, and the LLM generation span is simply gone, because the model never ran, so nothing emitted it.

That absence is the part that makes me happy every time. It isn't a filter hiding the LLM span to make a point. The waterfall is showing you the real control flow: the cache short-circuited the expensive work, and the trace reflects that truthfully because the trace is a byproduct of what executed, not a story told after the fact. A cache that used to be an aggregate claim on a separate dashboard becomes a per-request fact you can point at. When the Trace Explorer surfaces duration, cost, and tokens at the list level, you can see which turns were cheap before you open a single one.

Once a decision is a span, you can evaluate it

This is where it stops being about debugging and starts being about something bigger, and it's the reason I think this pattern matters far beyond one integration.

A semantic cache is a probabilistic component. It decides that a new question is "close enough" to a previous one to reuse the answer. That's a judgment call, and judgment calls are worth checking: a served answer might fit the old question but not quite the new one. For most teams that decision is invisible, because it never leaves the cache. It isn't in your traces, so it isn't in anything you can test.

But once the cache decision is a standard span carrying similarity, the served answer, and the new question, it's no longer just debugging material. It's an evaluation dataset. You can run an evaluator over your semantic-cache hits and confirm that each reused answer actually holds up against the question it was served for. With LangWatch Skills, that isn't even a project: Skills give Claude Code the context to derive the criteria from your own codebase, write the eval, and run it in a single session. Traces stop being something you glance at when something breaks and become the raw material for measuring whether your system is doing exactly what you want.

Open and safe at the same time

Being open about telemetry only works if it's also safe, and I care about getting that right. The default here is metadata only: hit or miss, similarity, latency, cost. Prompts, responses, recalled memories, and retrieved documents are attached only when content capture is explicitly turned on. That default is what makes it comfortable to point a production deployment at a shared project. And because both LangWatch and BetterDB are self-hostable, a team in a regulated setting like banking or payments can keep every byte inside its own infrastructure and still get the full-stack view. Turn the payloads on for a demo or a dev environment where seeing them is the whole point; leave them off in production. The visibility and the privacy aren't in tension: you get both.

Why I wanted to write this separately

Kristiyan and I could have just shipped one article, and we did. But I wanted to say the best part out loud from the observability side, because it's easy to read a partner post as a partnership flex: two companies who did a bunch of work to fit together.

We didn't, and that's what makes me so enthusiastic about it. There's nothing in this integration that's specific to LangWatch and BetterDB beyond the fact that both of us build to the standard. Any OTel-native application layer and any OTel-native data layer would compose the same way, with the same amount of glue code, which is none. The integration isn't an artifact of our relationship. It's what falls out for free when everyone builds to the same conventions.

So if you take one thing from the observability seat: full-stack visibility between your application and the infrastructure beneath it is not a feature you buy from a vendor, and it's not a favor two vendors do for each other. It's what you already have the moment both layers speak OpenTelemetry. I got a perfect trace from three codebases that had never heard of me, and you can too, without writing the glue.