LangSmith's annotation queue now supports allowing multiple people to review an individual run. This makes it easier to coordinate data review across multiple annotators.

• For each annotation queue, you can specify a number of users to review each run.

• If reservations are enabled, runs are "reserved" by the first to view, preventing multiple reviews where they're not wanted

• A run is removed from the queue for everyone when the specified number of reviewers have marked it as done.

Read more in the docs here.