Sutrace vs LangSmith: framework-agnostic agent observability without per-trace billing

TL;DR. If you live in LangChain, LangSmith remains the easiest path — it's the same team, the SDK lights up automatically, and the eval tooling integrates with primitives you already use. Don't switch for the sake of switching. If you don't live in LangChain, or your bill is climbing because your agent is chatty (LangSmith Plus starts at $39/seat with per-trace pricing that becomes exponential at scale), Sutrace is worth a look. We're framework-agnostic, OTel-native, and bill on ingest rather than per-trace, so a 200-step agent run isn't 200 line items. EU residency by default. One dashboard for AI agents alongside your hardware, software, and API observability — see the AI agent observability use case for the full picture.

This page is the long version. It covers what LangSmith does best, where Sutrace pulls ahead, the migration path, and the failure modes you should plan for.

What LangSmith gets right

Be honest with yourself. LangSmith is the natural choice for LangChain-native teams for three real reasons:

1. Zero-config tracing inside LangChain. Set LANGCHAIN_TRACING_V2=true and traces appear. No instrumentation, no decorators, no spans-as-context-managers. For teams whose entire agent is built on LCEL and langgraph, this is genuine ergonomic value.
2. Eval primitives that match the framework primitives. LangSmith datasets, runs, and feedback hook into LangChain's Runnable interface natively. You write less glue code than with any other tool.
3. Hub for prompts and chains. Versioned, shareable, callable. The Hub is the closest thing the LLM ecosystem has to a working package registry for prompts.

If those three things describe how your team actually works, you're not the audience for this page. Stay on LangSmith.

The honest LangSmith pricing — from their published page and confirmed in the Helicone comparison — looks like this:

The trap is the trace economics. A single agent run with retrieval, reasoning, and tool calls can easily produce 50–200 spans that LangSmith counts as separate traces (the definition has shifted over releases — it's per-run today, but historically per-LLM-call). Confident AI's LangSmith alternatives breakdown has the receipts. Mirascope's LangSmith alternatives roundup is the other honest read.

When LangSmith still wins

• You live in LangChain. Your prompts are ChatPromptTemplate, your agents are langgraph state machines, your retrieval is LCEL. The native integration is worth real money.
• You use LangChain Hub for prompt management. Sutrace doesn't replicate the Hub. We treat prompts as content; you version them in your repo or your CMS.
• You want a tightly-coupled eval workflow. LangSmith's eval primitives are the strongest in-framework eval toolkit shipping today.
• You're a small team that cares about ergonomics over cost. $39/seat is fine until it isn't. Stay until it isn't.

Where Sutrace pulls ahead

1. Framework-agnostic — OTel-native

Sutrace doesn't care if your agent is LangChain, LlamaIndex, the raw OpenAI SDK, the Anthropic SDK, custom orchestration in Go, or a Bedrock workflow stitched together with Step Functions. We emit and consume the OpenTelemetry GenAI semantic conventions — gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, etc.

If you already have OTel in your stack, point your collector at our endpoint. That's the integration. There is no Sutrace-specific SDK you have to wrap your code in.

LangSmith is moving toward OTel ingest too — but it's secondary. Sutrace is OTel-first.

2. Pricing on ingest, not per-trace

A 200-span agent run is one set of spans, not 200 billable units. We bill on the volume of telemetry ingested (in MB/GB), with cardinality tracked but not billed. The implication: a chatty agent doesn't change your bill more than the data it actually produces.

For teams running long-horizon agents — coding agents, research agents, anything with deep tool-call trees — this is the difference between a flat $200/mo bill and a $2,000/mo bill on the same workload.

3. Hard budget caps that fire synchronously

LangSmith observes spend. Sutrace stops spend. The difference matters when your agent enters a stuck loop and burns $47 in 90 seconds (the RelayPlane runaway documented case). Our budget interlock fires in front of the next provider call. See Hard budget caps for AI agents — the architecture options for the full architecture.

4. On-host prompt redaction

LangSmith stores your prompts. We let you choose. The on-host redactor strips PII, keys, and customer-defined patterns before the trace is exported. Your dashboard sees the redacted version. Your audit log is the redacted version. The original never leaves your VPC.

For EU teams under strict DPO review, this is often the deal-breaker. LangSmith data lives in US infrastructure by default; Sutrace is europe-west3 (Frankfurt) by default. SCC-bound DPA available before sign-up.

5. Prompt-injection signals

Every span carries an injection-detection score. The named 2025/2026 CVEs (EchoLeak CVE-2025-32711, CamoLeak CVE-2025-59145, Tenable's 7-vuln ChatGPT disclosure) are no longer hypotheticals. If you're shipping agents that touch untrusted inputs, you need detection in your telemetry, not as a separate product.

6. Multi-provider routing visibility

If you're using OpenRouter, Bedrock, or any LLM gateway, you need to know which provider actually served each request. LangSmith reports the request as you sent it. Sutrace reports the request and the upstream provider that handled it. See the multi-provider routing post.

7. One dashboard, not seven

Sutrace unifies AI agents with the rest of your stack — hardware (PLC/SCADA), software (OTel/Prometheus), web/APIs, and AI agents. If you're a platform team running a portfolio, you don't want LangSmith for agents, Datadog for APIs, and Grafana for hardware. See the Datadog comparison for the broader unification story.

Side-by-side comparison

Migration playbook

If you're moving off LangSmith, here's the actual sequence — not the marketing one.

Step 1. Stand up an OTel collector. If you already have one, skip this. If not, a contrib distribution with the OTLP receiver and the Sutrace exporter is enough. 50 lines of YAML.

Step 2. Add OTel GenAI auto-instrumentation. For Python, the openinference-instrumentation-langchain package converts LangChain tracing into OTel spans. For raw OpenAI/Anthropic SDKs, the upstream auto-instrumentation packages do the same.

Step 3. Run dual-write for 1–2 weeks. Send to both LangSmith and Sutrace. Compare a few traces side-by-side. The translation is 1:1 for the standard fields; LangSmith-specific concepts (run trees, feedback annotations) become OTel span hierarchies and span events.

Step 4. Move evals. This is the part that takes effort. LangSmith eval datasets export as JSONL; Sutrace ingests them via API. The custom evaluators are usually a langsmith.evaluation.evaluator decorator wrapping a function — you can keep the function, swap the decorator for our @sutrace.eval (or just call our API directly). Plan a day per non-trivial evaluator.

Step 5. Cut over. Keep dual-write for one billing cycle. Then turn LangSmith off.

Most teams complete this in 1–2 weeks of part-time work. The eval migration is the longest pole.

When you should not switch

If any of these are true, stay on LangSmith:

• Your prompt source-of-truth is LangChain Hub.
• Your eval suite is built on langsmith.evaluation and rebuilding it is more expensive than the bill.
• You're a 3-person team and the cost difference is under $300/mo. Engineering time to migrate isn't free.
• You need on-prem deployment today (LangSmith offers an enterprise on-prem tier; Sutrace is cloud-only).

Frequently asked questions

Does Sutrace support LangChain natively?

Yes — via OTel auto-instrumentation. Drop in openinference-instrumentation-langchain, point your OTel exporter at Sutrace, and your LCEL chains and langgraph state transitions show up as nested spans with the GenAI attributes. No code changes.

What about LangChain Hub?

We don't replace it. Most teams move prompts back to their repo (versioned in git), or to a lightweight CMS. The Hub is convenient but not load-bearing.

How does pricing actually compare?

For a typical production agent doing 100k runs/month at ~30 spans per run: LangSmith Plus runs roughly $300–$700 depending on seats and overage. Sutrace ingest at the same volume runs roughly $80–$180 depending on payload size. The crossover point is around 30k runs/month.

Can I run dual-write during migration?

Yes. Both tools accept OTel; you can fan out from one collector to two exporters. Keep dual-write for a billing cycle to verify trace fidelity before cutting LangSmith off.

What about LangGraph state observability?

langgraph state transitions become OTel spans with state-snapshot events. You see the same graph in our trace view that you see in LangSmith — different rendering, same data.

How do you handle evals?

Datasets, LLM-as-judge, custom evaluator functions, regression tracking against baselines. The eval primitives are framework-agnostic — a Python or TS function that returns a score. We don't have a "Hub for evals" the way LangSmith does, but we do have a shared eval library inside your org.

Is Sutrace open source?

No. The on-host redactor library is, the rest is closed-source cloud. If open source / self-host is a hard requirement, Langfuse is the honest recommendation.

Does Sutrace support EU residency?

Default. europe-west3 (Frankfurt). No US replication. SCC-bound DPA available before sign-up. This is non-negotiable for most German, French, and Dutch buyers — and a quiet relief for many US teams selling into Europe.

Get started

Self-serve. Drop the OTel collector config, set a budget cap, and the dashboard fills in within minutes. No sales call. Pricing here.