You can now run multiple repetitions of your experiment in LangSmith. This helps smooth out noise from variability introduced by your application or from your LLM-as-a-judge evaluator, so you can build more confidence in the results of your experiment.

In this video, learn how to evaluate on a dataset with repetitions. You can check the mean score across N repetitions, and also compare the outputs for variability across repetitions.