“### 1. LangSmith — Best for understanding individual failed runs”
“LangSmith is one of my strongest recommendations for agent debugging.”
“- Visual execution traces.”
“- Helps identify where an agent deviated from expected behaviour.”
“### 2. Braintrust — Best for explaining evaluation failures”
“Braintrust is particularly interesting because it connects testing with evaluation and debugging.”
“- Scorer explanations.”
“### 3. Arize Phoenix — Best for visual debugging and open-source flexibility”
“- Visual traces of agent execution.”
“Phoenix is particularly attractive for engineering teams.”
“Its newer Phoenix Intelligence (PXI) functionality also introduces an AI engineering agent that can investigate traces, help debug failures and create evaluators.”
“### 4. Langfuse — Best open-source all-rounder”
“Langfuse offers a useful combination of testing and observability.”
“One particularly useful capability is connecting evaluation scores to the underlying execution traces.”
“### 5. Comet Opik — Worth investigating for automated failure discovery”
“Opik is interesting because it goes beyond individual trace inspection.”
“- Identify recurring failure patterns.”