“Custom application (OpenAI / Anthropic APIs)”
“LangSmith — Particularly useful for debugging complex chains and agent workflows.”
+ debugging complex chains and agent workflows
“LangSmith — Particularly useful for debugging complex chains and agent workflows.”
“Braintrust — Strong when you also want to measure output quality and compare prompt experiments.”
+ measuring output quality and comparing prompt experiments
“Braintrust — Strong when you also want to measure output quality and compare prompt experiments.”
“Langfuse — Open-source/self-hostable observability, traces, token costs, prompt versions, and evaluations.”
+ open-source/self-hostable observability
“Langfuse — Open-source/self-hostable observability, traces, token costs, prompt versions, and evaluations.”
“Arize Phoenix — Open-source tracing and evaluation, with support for OpenTelemetry.”
+ open-source tracing and evaluation with OpenTelemetry support
“Arize Phoenix — Open-source tracing and evaluation, with support for OpenTelemetry.”
“Existing Datadog/Grafana stack”
“Existing Datadog/Grafana stack”
“Portkey — AI gateway with request logs, routing, and observability.”
+ AI gateway with request logs, routing, and observability
“Portkey — AI gateway with request logs, routing, and observability.”
“LiteLLM — Centralized model proxy, usage tracking, budgets, and routing.”
+ centralized model proxy, usage tracking, budgets, and routing
“LiteLLM — Centralized model proxy, usage tracking, budgets, and routing.”
“Custom application (OpenAI / Anthropic APIs)”
“The benefit is that you can centralize model usage across OpenAI, Anthropic, Google, and other providers.”