The LangSmith Self-hosted v0.8 release adds a new LangSmith home page, new features including support for custom code evaluators and bulk data export, and improves the reliability and correctness of LangSmith API endpoints.

Breaking Changes

• Removal of the <domain>/api-hub URL path. You should now use the <domain>/api URL path for all api interactions. Continued use of <domain>/api-hub will result in a 404 Not Found error.

• Removal of support for v1 API Keys prefixed with ls__ . Continued use of ls__ prefixed keys will result in a 401 Unauthorized error.

• The platform-backend service now requires access to blob storage if blob storage is enabled for your instance.

New Features since v0.7.0

• Bulk data export from LangSmith for offline analysis

• Enhanced trace comparison view with auto-trace tracking and diff viewer

• Custom code evaluators in LangSmith

• Custom dashboards to monitor LLM app performance

• Resource tags to organize your Workspace in LangSmith

• Prompt tags in LangSmith for version control

•  A new and improved LangSmith home page

• Revamped run over dataset experience for Playground

Performance and Reliability Changes

• Improved performance and reliability of large exports via the LangSmith SDK that contain complex filter statements.

• Implemented retryable response codes (502, 503) instead of 500 for ephemeral errors.

• Increased the frequency with which TTL-driven deletions from Clickhouse will free storage.

• Adjusted inserts of feedback coming from the LangSmith to use asynchronous instead of synchronous inserts to Clickhouse, which should improve concurrency, performance and reliability.

• Improved performance of DELETE FROM queries in Clickhouse which will reduce the execution time for script-driven deletions of traces, workspaces, and organizations.

Infrastructure Changes

• As a reminder, default requirements for all resources were increased as of v0.7. Note that you may need to adjust your cluster settings upward. By default we now require the following minimum sizing:

  • 16 vCPU

  • 64GB RAM

  including at least one node that can fit a pod requesting 4 vCPU/16 GB RAM.

  To override these settings you can manually configure resources requests/limits.

• Updated bundled Clickhouse version to 24.5

• Removal of <domain>/api-hub url

• Changed health checks used for backend pods to check the availability of more services (vs. just the ok endpoint)

• Addition of langsmith-ace-backend service which is used to run custom code in a sandboxed environment

• playground start command/ports have been updated

• Moved expensive blob-storage operations to platform-backend to improve performance

Admin Changes

• Added support for setting Resource Tags for Workspaces

• Removed restriction on number of Run Rules per Workspace