According to research from Mercor, approximately 90% of American working hours are spent on tasks with little or no coverage from existing AI benchmarks. This gap makes it difficult to assess how capable AI models actually are at the work Americans do.
Mercor compiled 29 economically relevant AI benchmarks and mapped them against the U.S. Department of Labor’s O*NET taxonomy, which encompasses 923 occupations and 18,797 tasks across the U.S. workforce. The researchers found that only 208 of 923 occupations have tasks with at least 5 benchmark prompts, and 6 of 22 job families have no such occupations at all.
The coverage gap is not random. Benchmarks disproportionately focus on the largest labor markets and roles most exposed to AI automation. According to Mercor’s analysis, O*NET tasks rated as AI-exposed are about 6 times more likely to be represented on benchmarks than unexposed tasks. Within AI-exposed occupations, tasks in larger wage-bill industries are significantly overrepresented.
For example, while benchmarks heavily cover lawyers’ AI-exposed tasks, they barely cover similar tasks performed by legal secretaries or court clerks. The long tail of smaller, middle-income knowledge work jobs remains largely uncovered. Benchmarks are more densely concentrated on software engineering and financial analysis while remaining sparse on truck driving and gas plant operations.
Mercor notes its findings are provisional, limited by the initial selection of 29 benchmarks and the subjective criteria used to identify economically relevant ones. The researchers acknowledged several limitations: excluding saturated benchmarks risks understating coverage of easier tasks, while including them could overstate performance if saturation reflects lower task realism—a likely scenario given that benchmarks often overstate real-world LLM performance.
The researchers emphasize that measuring model capabilities by job occupation is essential for enterprises selecting models, policymakers forecasting AI’s economic impact, and workers considering their futures. However, the current benchmark landscape creates blind spots in understanding economically valuable model capabilities. Mercor plans to expand and continuously update its Benchmark Map to improve coverage across the broader economy.
Key facts
- 90% of American working hours involve tasks with minimal or no AI benchmark coverage
- Only 208 of 923 occupations have at least 5 benchmark prompts covering their tasks
- AI-exposed tasks are about 6 times more likely to be benchmarked than unexposed tasks
- Legal secretaries and court clerks are underrepresented despite occupying AI-exposed roles
- Benchmarks disproportionately focus on high-wage-bill occupations and the most important tasks within them