SAN FRANCISCO, July 24, 2026 /PRNewswire/ — Snorkel AI today highlighted the first group of projects supported through Open Benchmarks Grants, a $3 million commitment to support open-source datasets, benchmarks, and evaluation research.
Launched in February 2026, Open Benchmarks Grants has received hundreds of applications from researchers, labs, and engineers working to address a growing challenge: AI systems are advancing faster than the field’s ability to rigorously measure their performance on realistic, consequential work.
“From complex environments and huge autonomy horizons to rich, sophisticated outputs, these projects tackle some of the field’s hardest evaluation challenges,” said Fred Sala, a member of the Open Benchmarks Grants steering committee and assistant professor at the University of Wisconsin–Madison. “I’m excited to see the broader research community use, validate, and build on them.”
Open Benchmarks Grants provides selected teams with funding, expert data development support, research and engineering collaboration, and platform resources. Supported projects include:
- Frontier-Bench (formerly Terminal-Bench 3.0), developed with Laude Institute and the Harbor community, is a harder, more domain-diverse successor to Terminal-Bench 2.1 — built in the open, task by task, under continuous adversarial review.
- Agents’ Last Exam, developed with UC Berkeley RDI and the RDI Foundation, evaluates agents on long-horizon, economically valuable professional workflows. It spans 55 sub-industries and includes more than 1,500 tasks toward a 5,000-task target, sourced and validated by more than 300 industry experts.
- OSWorld 2.0, developed with XLANG Lab, evaluates computer-use agents on 108 long-horizon workflows across 31 self-hosted web environments and professional desktop applications.
- Continual Learning Bench, developed with UC Berkeley SkyLab and the University of Wisconsin–Madison, measures whether agents genuinely improve across sequential, stateful tasks.
- SlopCode Bench, developed with the University of Wisconsin–Madison, measures how code quality degrades as coding agents repeatedly modify and extend their own solutions.
- Terminal-Bench 2.1, developed with Stanford University, Laude Institute and the Harbor community, evaluates agents on challenging work in terminal environments. The release corrected 28 tasks and introduced continuous validation.
With support from Open Benchmarks Grants, Terminal-Bench Science is also now in development, extending the Terminal-Bench framework to computational research workflows across the life, physical, earth, and mathematical sciences.
Beyond the grants program, Snorkel led the development of Senior SWE-Bench with the research teams at Princeton University and the University of Wisconsin–Madison. The benchmark evaluates coding agents on senior-level engineering work, including implementing features from realistic instructions, investigating bugs that require runtime analysis, and producing code that follows existing codebase conventions.
Open Benchmarks Grants was established with support from Hugging Face, Prime Intellect, Together AI, Factory, Harbor, and PyTorch. Applications remain open and are reviewed on a rolling basis.
Learn more and apply for a grant at benchmarks.snorkel.ai.
About Snorkel AI
Snorkel AI is the frontier AI data lab, helping teams build the data and environments behind high-performing frontier and agentic AI. We combine technology with research-driven AI data development to create datasets, benchmarks, evals, and custom solutions for real-world AI systems. Founded out of the Stanford AI Lab in 2019, Snorkel works with leading AI labs and enterprises to move from better data to better outcomes.
SOURCE Snorkel AI