Skip to content
anythingai.space
Evaluation & Observabilityintermediate13 min · 9 steps

Eval Tooling: LangWatch and Langfuse

What an eval platform gives you (traces, datasets, experiments, online evaluators) and how to keep judge costs and latency in check.

traceTRACE ABC123property assistant…system1 bed, bathroom not connecteduserget_availability(bathroom_connected=None)tool callA101, B205 · connected=Truetool result**A101** and **B205**…reply via SMSYour AI appinstrumentedTrace viewerany platform
dataresponserequest
1 / 9
ProblemStep 1 of 9

Without traces, you can't do evals

A trace records everything the AI did for one request: system prompt, user message, tool calls, tool results and the reply, plus model parameters, tokens, latency and cost.

This one shows two bugs that the final reply alone would hide. The tool was called with bathroom_connected=None, so the user's constraint was dropped, and the SMS reply used markdown.

TipMinimum: input, output, timestamp and a unique ID. Better: prompts, tool calls, tokens, latency, cost. Best: session history, errors, model version and feature flags.
What changed
  • +Instrumentation
  • +One full trace

Lesson 5 of 5 in Evaluation & Observability