We've launched Polly— an AI-powered assistant inside LangSmith that helps you debug, analyze, and improve your agents. Polly, now in beta, understands agent architectures and failure patterns, giving you a faster path to insight and iteration.

Agents behave very differently from simple LLM calls — long prompts, multi-step traces, and multi-turn conversations make debugging slow and painful. Polly tackles this head-on. It helps you:

• Debug faster:
Polly pinpoints failures and surfaces insights from traces and conversations, saving hours of manual inspection.

• Engineer prompts with confidence:
Describe the behavior you want, and Polly helps rewrite prompts, schemas, and examples intelligently.

• See the full picture:
Polly interprets entire execution histories to spot drift, context loss, and behavioral changes over time.

What Polly does

Inside LangSmith, Polly can:

• Answer questions about individual traces

• Analyze full conversation threads

• Identify subtle behavioral issues

• Rewrite system prompts

• Highlight patterns across hundreds of steps

Get started

Enable LangSmith tracing (setup takes minutes)
Open a trace or thread and start chatting with Polly
Iterate on prompts and agent behavior with real production feedback

Read more in the blog announcement: https://blog.langchain.com/introducing-polly-your-ai-agent-engineer/

Try out Polly: smith.langchain.com

Video walkthrough: