LangSmith 🔁 Self-improving LLM evaluators in LangSmith Using an “ LLM-as-a-Judge ” is a popular way to grade outputs from LLM applications. This involves passing the generated output to a separate LLM and asking... June 26, 2024
LangSmith 🧷 Store the model and configuration when saving a prompt When you save a prompt in LangSmith, the model and its configuration will also be stored. You no longer need to reset your preferences in the Playground... June 24, 2024
LangSmith 🛝 Custom models in LangSmith Playground Once you’ve deployed a model server, use it in the LangSmith Playground by selecting either ChatCustomModel or the CustomModel provider for chat-style model... June 24, 2024
LangSmith 🥷 PII Masking in LangSmith LangSmith SDK versions now have enhanced PII masking capabilities. You can easily create anonymizers by specifying a list of regular expressions or providing... June 24, 2024
LangSmith 📁 Workspaces in LangSmith for improved collaboration & organization LangSmith now offers Workspaces to separate resources for different teams or different environments. Admins can add users to Workspaces, granting them... June 13, 2024
LangSmith 🗺️ Variable mapping for online evaluator prompts In LangSmith, you can now use any structured prompt from the LangChain Hub with customizable variables. You can personalize your inputs based on recent runs... June 10, 2024
LangSmith 🛝 Enter the playground from scratch instead of from a trace or a prompt The Playground is now its own tab in the sidebar of LangSmith. To create a new prompt , simply craft a prompt in the empty playground and click "Save As" to... June 10, 2024
LangSmith 💬 Manage private prompts without a handle We’ve improved our SDK and LangSmith Prompts UI to make navigating prompts simpler. Now, only public prompts require a handle on creation. New accounts won’t... May 27, 2024
LangSmith 🔧 Off-the-shelf online evaluator prompts to catch bad retrieval and hallucinations for RAG LangSmith’s online evaluator (LLM-as-judge) automatically runs on production traces with a customizable prompt and automation rules. Our new off-the-shelf... May 27, 2024
LangSmith 🔁 Repetitions to build confidence in your experiment results You can now run multiple repetitions of your experiment in LangSmith. This helps smooth out noise from variability introduced by your application or from... May 27, 2024
