Benchmarks Library
Explore comprehensive benchmarks for AI models, code migration tools, and migration performance metrics.
Code Translation Accuracy
Measures how accurately AI models translate code between programming languages
Migration Plan Generation Quality
Evaluates the quality and completeness of AI-generated migration plans
Database Schema Mapping Accuracy
Tests accuracy of AI models in mapping database schemas between different systems
API Translation Benchmark
Measures ability to translate between API specifications (REST, GraphQL, gRPC)
Legacy Code Comprehension
Tests AI ability to understand and document legacy codebases (COBOL, VB6, etc.)
Cloud Configuration Translation
Evaluates accuracy of translating cloud configs (CloudFormation to Terraform, etc.)
Test Generation Coverage
Measures quality and coverage of AI-generated test suites for migrated code
Documentation Generation Quality
Evaluates AI-generated documentation for migrated systems
MMLU (Massive Multitask Language Understanding)
A 57-subject multiple-choice benchmark testing broad academic and professional knowledge across STEM, humanities, social sciences, and law.
MMLU-Pro
A harder, reasoning-focused successor to MMLU with ten answer options and tougher questions designed to separate frontier models that saturated the original.
GSM8K (Grade School Math 8K)
A benchmark of ~8,500 grade-school math word problems that test multi-step arithmetic reasoning with a single numeric answer.
MATH (Competition Mathematics)
A benchmark of 12,500 competition-style math problems across algebra, geometry, number theory, and calculus, graded on exact final-answer match.
HumanEval
A code-generation benchmark of 164 Python programming problems graded by executing unit tests, popularizing the pass@k metric.
MBPP (Mostly Basic Python Problems)
A benchmark of ~1,000 entry-level Python programming tasks with test cases, used to evaluate basic code synthesis from short descriptions.
SWE-bench
A benchmark of real GitHub issues from open-source Python repositories where a model must produce a patch that resolves the issue and passes tests.
SWE-bench Verified
A 500-task, human-validated subset of SWE-bench with clear specifications and reliable tests, used as the standard clean measure of agentic coding.
BIG-bench (Beyond the Imitation Game)
A collaborative suite of 200+ diverse tasks probing reasoning, knowledge, and emergent abilities beyond conventional language benchmarks.
BBH (BIG-bench Hard)
A 23-task subset of BIG-bench focused on challenging multi-step reasoning where chain-of-thought prompting yields large gains.
HellaSwag
A commonsense sentence-completion benchmark where models pick the most plausible continuation among adversarially generated distractors.
ARC (AI2 Reasoning Challenge)
A grade-school science question benchmark split into Easy and Challenge sets, the latter built from questions retrieval methods answer incorrectly.
TruthfulQA
A benchmark measuring whether models avoid generating false answers that mimic common human misconceptions and falsehoods.
GPQA (Graduate-Level Google-Proof Q&A)
A benchmark of expert-written, graduate-level science questions designed to be extremely hard even with web access, testing deep domain reasoning.
MMMU (Massive Multi-discipline Multimodal Understanding)
A multimodal benchmark of college-level questions requiring joint reasoning over text and images such as diagrams, charts, and figures.
MT-Bench
A multi-turn conversational benchmark where a strong LLM judge scores model responses across categories on a 1-10 quality scale.
Chatbot Arena
A live, crowdsourced evaluation where users compare two anonymous model responses and votes are aggregated into Elo-style rankings.
HELM (Holistic Evaluation of Language Models)
A standardized framework evaluating language models across many scenarios and multiple metrics including accuracy, robustness, fairness, and efficiency.
DROP (Discrete Reasoning Over Paragraphs)
A reading-comprehension benchmark requiring discrete operations like addition, counting, sorting, and comparison over passage content.
WinoGrande
A large-scale commonsense benchmark of pronoun-resolution sentence pairs designed to require world knowledge rather than lexical cues.
AGIEval
A benchmark built from human standardized exams such as college entrance, law, and civil-service tests to measure human-centric reasoning.
LiveCodeBench
A contamination-resistant coding benchmark that continuously collects new competitive-programming problems and evaluates by execution over time.
AIME (Competition Math Benchmark)
An olympiad-level math benchmark using American Invitational Mathematics Examination problems with integer answers, a key frontier reasoning test.
tau-bench (Tool-Agent-User Benchmark)
An agentic benchmark testing tool-using models in simulated customer-service dialogues that require following domain policies and calling APIs correctly.
Terminal-Bench
An agentic benchmark evaluating models on completing real command-line tasks inside a sandboxed terminal, verified by automated checks.
RULER (Long-Context Benchmark)
A synthetic long-context benchmark with configurable tasks measuring a model's effective context length beyond simple retrieval.
TPC-C
The classic OLTP benchmark simulating an order-entry warehouse workload, measuring transactions per minute (tpmC) and price/performance.
TPC-H
An ad-hoc decision-support benchmark of 22 complex analytical queries over a star-like schema, reporting query throughput and power at fixed scale factors.
TPC-DS
A modern decision-support benchmark with 99 queries over a multi-snowflake retail schema, designed to stress complex analytics, data loading, and concurrency.
TPC-E
An OLTP benchmark modeling a brokerage firm with realistic data and read-heavy transactions, reporting tpsE as a more representative successor to TPC-C.
YCSB
The Yahoo! Cloud Serving Benchmark for NoSQL and key-value stores, defining standard read/write workload mixes to compare throughput and latency.
sysbench
A scriptable, multi-threaded benchmark tool widely used for MySQL/PostgreSQL OLTP tests as well as CPU, memory, and file I/O microbenchmarks.
pgbench
PostgreSQL's built-in benchmarking tool that runs a TPC-B-like transaction load and custom scripts to measure transactions per second and latency.
HammerDB
An open-source database load-testing tool implementing TPROC-C and TPROC-H (TPC-C/TPC-H-derived) workloads across many relational engines.
BenchBase
An extensible Java framework (successor to OLTPBench) for benchmarking relational databases with many built-in workloads via a common JDBC harness.
ClickBench
An open benchmark for analytical databases using a single wide web-analytics table and 43 queries to compare cold and hot OLAP query latency.
Star Schema Benchmark (SSB)
A simplified TPC-H derivative using a classic star schema and 13 queries in four flights to measure data-warehouse query performance.
TPCx-BB (BigBench)
An end-to-end big-data analytics benchmark mixing SQL, machine learning, and natural-language processing over a retail data set on Hadoop/Spark-style platforms.
TPCx-AI
An end-to-end machine-learning benchmark measuring the full data-science pipeline — ingestion, training, and serving — across multiple AI use cases.
JMH (Java Microbenchmark Harness)
The standard harness for writing reliable JVM microbenchmarks, widely used to measure data-processing and serialization library performance on the JVM.
TeraSort
A distributed sort benchmark that orders one terabyte (or more) of data on a cluster, measuring big-data engine throughput and shuffle efficiency.
Sort Benchmark
A long-running family of competitive sorting benchmarks (GraySort, MinuteSort, JouleSort) that rank systems on speed, cost, and energy for large-scale sorting.
dbt Pipeline Performance Benchmark
A category of benchmarks measuring transformation pipeline performance in dbt — model build time, warehouse compute cost, and incremental run efficiency.
OLTP vs OLAP Latency Benchmark
A comparative benchmark category contrasting transactional (OLTP) point-operation latency with analytical (OLAP) scan-and-aggregate latency to evaluate HTAP systems.
ANN-Benchmarks
The standard open benchmark for approximate nearest neighbor search, plotting recall against queries-per-second across vector index libraries and databases.
BigANN Benchmark
A billion-scale approximate nearest neighbor benchmark testing vector search algorithms on large data sets with constraints on memory, throughput, and recall.
Time-Series Ingestion Benchmark
A benchmark category measuring write throughput, query latency, and compression for time-series databases under high-cardinality metric and event ingestion.
Streaming Throughput Benchmark
A benchmark category for event-streaming platforms measuring producer/consumer throughput and end-to-end latency under sustained load and varying durability settings.
In-Memory Store Benchmark (memtier/redis-benchmark)
Benchmarks for in-memory data stores like Redis and Memcached, measuring operations per second and sub-millisecond latency under varied key/value and pipeline settings.
ETL/ELT Pipeline Throughput Benchmark
A benchmark category measuring data-integration pipeline performance — extraction and load throughput, transformation latency, and end-to-end freshness.
fio (Flexible I/O Tester)
The standard tool for benchmarking storage I/O, measuring IOPS, bandwidth, and latency across configurable read/write patterns, block sizes, and queue depths.
LDBC Social Network Benchmark
The standard benchmark for graph databases, measuring interactive transactional and analytical query performance over a realistic, correlated social-network graph.
SPEC CPU 2017
Industry-standard CPU benchmark suite measuring integer and floating-point compute performance under realistic, compute-bound workloads.
SPECjbb 2015
Java server benchmark modeling a supermarket company's transaction processing to measure JVM and server-side Java throughput and latency.
CoreMark
Compact, portable CPU benchmark from EEMBC designed to measure embedded and microcontroller core performance with a single comparable number.
Dhrystone
Classic synthetic integer benchmark that produces DMIPS, a historical and still-cited measure of general-purpose integer CPU performance.
Whetstone
Historic synthetic floating-point benchmark measuring scientific-style arithmetic performance, reported in MWIPS (millions of Whetstone instructions per second).
STREAM
Simple, portable benchmark measuring sustainable main-memory bandwidth for large vector operations, the standard metric for memory-bound performance.
Geekbench
Cross-platform benchmark measuring single-core and multi-core CPU performance plus GPU compute, widely used to compare phones, laptops, and servers.
LINPACK / HPL
Dense linear-algebra benchmark solving a large system of equations to measure peak floating-point throughput; HPL ranks the TOP500 supercomputers.
TechEmpower Web Framework Benchmarks
Open benchmark suite comparing web frameworks and platforms across standardized request types like JSON, single-query, and plaintext throughput.
wrk HTTP Benchmark
Modern, multithreaded HTTP load-testing tool that generates high request volume from a single machine and reports throughput and latency distribution.
k6 Load Testing
Developer-centric, scriptable load-testing tool using JavaScript scenarios to measure API and web performance with rich thresholds and metrics.
Apache JMeter
Mature, GUI-driven Java load-testing tool for simulating complex multi-protocol user scenarios and measuring throughput, latency, and error rates.
Gatling Load Testing
Scala-based, asynchronous load-testing tool with an expressive scenario DSL and detailed HTML reports for high-concurrency performance testing.
Locust Load Testing
Python-based, distributed load-testing tool where user behavior is defined in code, scaling to many workers for high-concurrency scenario testing.
ApacheBench (ab)
Simple, ubiquitous command-line HTTP benchmarking tool for quick single-endpoint throughput and latency measurement, bundled with Apache.
fio Storage I/O Benchmark
Flexible I/O tester for measuring storage device and filesystem performance across configurable read/write patterns, block sizes, and queue depths.
iperf Network Benchmark
Active network measurement tool that generates TCP, UDP, and SCTP traffic between two hosts to measure achievable bandwidth, jitter, and packet loss.
netperf Network Benchmark
Network performance tool measuring both bulk-transfer throughput and request/response transaction rates, used to characterize latency-sensitive workloads.
Phoronix Test Suite
Open-source, cross-platform benchmarking framework that automates hundreds of real-world tests and aggregates results for reproducible comparison.
Core Web Vitals
Google's set of user-centric web performance metrics, LCP, INP, and CLS, that quantify loading, interactivity, and visual stability of real page loads.
Lighthouse Performance
Google's open-source web auditing tool that runs synthetic page loads and produces a 0-100 performance score from lab metrics like LCP, TBT, and CLS.
Cold-Start Latency Benchmark
Measures the added latency when a serverless function or container must initialize from scratch before serving its first request after being idle.
Build-Time Benchmark
Measures how long it takes to compile and package software, a key developer-productivity and CI-cost metric across clean, incremental, and cached builds.
Container Startup Time Benchmark
Measures how quickly a container goes from launch to ready, covering image pull, runtime creation, and application readiness for scaling and resilience.
API P99 Latency Benchmark
Measures tail latency, the response time at the 99th percentile, to capture worst-case API responsiveness that averages hide and that users feel most.
HTTP/3 and QUIC Protocol Benchmark
Measures how the QUIC-based HTTP/3 transport compares to HTTP/2 over TCP on connection setup, throughput, and latency, especially on lossy networks.
Serverless Cold-Start Latency Benchmark
Measures the added latency when a serverless function must initialize a new execution environment before handling a request, across runtimes and platforms.
Autoscaling Responsiveness Benchmark
Measures how quickly and accurately an autoscaler adds or removes capacity in response to load changes, including reaction time, overshoot, and stabilization.
Cost-per-Request Benchmark
Measures the fully loaded cloud cost of serving a single unit of work, attributing compute, memory, network, and storage spend to throughput.
Cloud Storage Throughput Benchmark
Measures sustained read and write throughput, IOPS, and latency of cloud object, block, and file storage under varied access patterns and concurrency.
CDN Latency Benchmark
Measures content delivery network performance from many geographic vantage points, including edge latency, cache hit ratio, and origin offload.
Kubernetes Scheduling Latency Benchmark
Measures how quickly the Kubernetes scheduler places pods on nodes and how the control plane scales as pod, node, and churn counts grow.
CIS Benchmark Compliance Score
Measures a system's adherence to CIS Benchmarks, prescriptive secure-configuration baselines, reported as the share of passing controls by profile level.
Vulnerability Scan Coverage Benchmark
Measures how completely a vulnerability management program scans its asset estate, reporting asset coverage, scan freshness, and authenticated-scan ratio.
SAST/DAST Detection Rate Benchmark
Measures how accurately static and dynamic application security testing tools find real vulnerabilities, reporting true-positive, false-positive, and recall rates.
Security MTTR Benchmark
Measures mean time to remediate security findings, from detection to fix verification, segmented by severity and asset criticality.
Mobile App Startup Time Benchmark
Measures how quickly a mobile app becomes usable, distinguishing cold, warm, and hot starts and the time to first frame and first interaction.
Mobile Frame Rate and Jank Benchmark
Measures UI rendering smoothness on mobile, quantifying dropped frames, jank, and frame-time consistency against the display refresh budget.
Mobile Battery Usage Benchmark
Measures the energy a mobile app consumes per task and over time, attributing drain to CPU, network, GPU, location, and wakelocks.
Mobile App Bundle Size Benchmark
Measures the download and installed size of a mobile app and its components, tracking size by architecture, resources, and code to control bloat.
Geekbench Mobile Benchmark
A cross-platform CPU and compute benchmark widely used to compare mobile device processor performance via standardized single- and multi-core workloads.
DORA Metrics Benchmark
Measures software delivery and operational performance via four key metrics: deployment frequency, lead time, change failure rate, and time to restore.
Code Coverage Benchmark
Measures the proportion of code exercised by automated tests, across line, branch, statement, and function coverage, as a test-completeness indicator.
Mutation Testing Score Benchmark
Measures test-suite effectiveness by injecting small faults into code and checking how many are detected, reported as the mutation score.
Flaky Test Rate Benchmark
Measures the prevalence and impact of nondeterministic tests that pass and fail without code changes, eroding trust in the test suite.
CI Build Time (p95) Benchmark
Measures continuous-integration pipeline duration, focusing on the p95 build time and its phases, to quantify developer feedback latency.
Lighthouse Accessibility Benchmark
Measures a web page's automated accessibility conformance using Lighthouse audits mapped to WCAG, producing a 0-100 accessibility score.
Cloud Egress Cost Efficiency Benchmark
Measures the cost and volume of data leaving cloud environments, attributing egress spend to flows, regions, and services to surface optimization targets.
Container Image Vulnerability Density Benchmark
Measures the count and severity of known vulnerabilities per container image, normalized by size or package count, to compare image security posture.
Defect Escape Rate Benchmark
Measures the proportion of defects that reach production undetected by pre-release testing, indicating the effectiveness of quality gates.
CI/CD Pipeline Reliability Benchmark
Measures how dependably a delivery pipeline succeeds, reporting pass rate, infrastructure-failure share, and mean time to recover a broken pipeline.
Secrets Detection Accuracy Benchmark
Measures how accurately tools find leaked credentials in code and history, reporting recall, precision, and false-positive rate across secret types.
MTEB (Massive Text Embedding Benchmark)
A broad benchmark for text embedding models spanning classification, clustering, retrieval, reranking, and semantic similarity across many datasets and languages.
BEIR (Benchmarking Information Retrieval)
A heterogeneous zero-shot retrieval benchmark that tests how well a single retrieval model generalizes across diverse domains and query types without task-specific training.
IFEval (Instruction-Following Eval)
A benchmark that measures whether LLMs follow precise, verifiable formatting and content instructions using automatic checks rather than subjective judgment.
BFCL (Berkeley Function-Calling Leaderboard)
A benchmark for evaluating how accurately LLMs select, call, and parameterize functions and tools, including parallel, multiple, and multi-turn calling scenarios.
MGSM (Multilingual Grade School Math)
A multilingual extension of grade-school math word problems that tests whether LLMs can reason through arithmetic in many languages, not just English.
SimpleQA
A factuality benchmark of short, fact-seeking questions with single verifiable answers, designed to measure how often LLMs are correct, wrong, or appropriately abstain.
FRAMES (Factuality, Retrieval, And reasoning MEasurement Set)
A benchmark for retrieval-augmented generation that tests end-to-end factuality, multi-document retrieval, and multi-hop reasoning on questions needing several sources.
MuSR (Multistep Soft Reasoning)
A benchmark of long natural-language narratives requiring multistep commonsense and logical reasoning, such as murder mysteries and object-placement puzzles.
LiveBench
A contamination-resistant benchmark that continuously refreshes questions from recent sources and grades automatically against objective ground truth across many task categories.
BigCodeBench
A code-generation benchmark for realistic programming tasks that require composing many library calls, evaluated with rigorous test suites and high branch coverage.
CRUXEval (Code Reasoning, Understanding, and Execution)
A benchmark that tests whether models can reason about code execution by predicting function inputs from outputs and outputs from inputs.
RewardBench
A benchmark for reward models and LLM judges that measures how well they prefer better responses over worse ones across chat, reasoning, safety, and refusal cases.
AlpacaEval
An automated LLM-as-judge benchmark that estimates a model's win rate against a reference model on open-ended instructions, with a length-controlled variant to reduce verbosity bias.
Arena-Hard
An automatic benchmark of challenging, real-user-derived prompts graded by an LLM judge, built to align closely with human preference rankings and separate strong models.
ImageNet (ILSVRC Classification)
The foundational large-scale image classification benchmark covering 1,000 object categories, long used to track progress in computer vision and pretraining.
COCO (Object Detection and Segmentation)
A large-scale benchmark for object detection, instance segmentation, and keypoints in complex everyday scenes, evaluated with mean average precision across IoU thresholds.
VQAv2 (Visual Question Answering)
A benchmark that tests whether models can answer open-ended natural-language questions about images, balanced to reduce language-only shortcuts.
MMBench (Multimodal Benchmark)
A systematic multimodal benchmark that evaluates vision-language models across many fine-grained ability dimensions using a robustness-checked multiple-choice protocol.
DocVQA (Document Visual Question Answering)
A benchmark for answering questions about document images, testing OCR, layout understanding, and reasoning over text, tables, and forms.
ChartQA
A benchmark for answering questions about charts and plots that require visual data extraction plus arithmetic and logical reasoning over the extracted values.
LibriSpeech (ASR Word Error Rate)
A widely used benchmark for automatic speech recognition built from read English audiobooks, measured primarily by word error rate on clean and noisy splits.
HarmBench
A standardized red-teaming benchmark that measures how often automated attacks elicit harmful behaviors from LLMs and how well refusal and defenses hold up.
MLPerf Training
Industry-standard benchmark suite measuring how fast hardware and software systems train machine-learning models to a fixed target quality.
MLPerf Inference
Benchmark suite measuring how fast and efficiently systems serve trained ML models under realistic latency and throughput constraints.
MLPerf Tiny
Benchmark suite for ultra-low-power machine learning on microcontrollers and embedded devices, measuring latency, energy, and accuracy.
DAWNBench
Stanford benchmark that measured end-to-end deep-learning training and inference by time-to-accuracy and cost, popularizing those metrics.
SPECjbb 2015
Standard Java server benchmark modeling a supermarket company's business logic to measure throughput and critical response-time performance.
SPECpower_ssj2008
Benchmark measuring server energy efficiency by reporting performance per watt across graduated load levels from idle to peak.
SPECjvm 2008
Benchmark suite measuring core Java Virtual Machine performance across compute-intensive workloads independent of application or hardware tuning.
SPEC OMP 2012
Benchmark suite measuring shared-memory parallel performance of OpenMP applications across scientific and engineering workloads.
HPCG
High Performance Conjugate Gradients benchmark measuring HPC system performance on memory-bound, sparse computations that mirror real applications.
HPL-AI / HPL-MxP
Mixed-precision LINPACK variant measuring supercomputer performance using low-precision arithmetic refined to full accuracy, reflecting AI hardware.
Graph500
Benchmark ranking supercomputers on data-intensive graph processing, measuring traversed edges per second instead of floating-point throughput.
GAP Benchmark Suite
Reference graph-algorithm benchmark suite providing optimized kernels and standard graphs to fairly compare graph-processing performance.
Computer Language Benchmarks Game
Long-running comparison of programming-language implementations on small algorithmic tasks, measuring runtime, memory, and code size.
Renaissance JVM Benchmark Suite
Modern JVM benchmark suite using real-world concurrent and parallel workloads to stress runtime optimization, GC, and JIT compilers.
DaCapo JVM Benchmark Suite
Long-established Java benchmark suite using real open-source application workloads to evaluate JVM, JIT, and garbage-collection performance.
SPECviewperf
Standard benchmark measuring professional graphics-workstation performance running real CAD, visualization, and content-creation application viewsets.
3DMark
Cross-platform graphics benchmark suite measuring GPU and gaming performance through standardized rendering tests and synthetic feature tests.
Unigine Superposition
GPU stress and benchmark tool using a detailed real-time scene to measure graphics performance, stability, and thermal behavior.
PassMark PerformanceTest
Whole-system PC benchmark suite measuring CPU, memory, disk, 2D, and 3D performance and aggregating them into comparable component scores.
UnixBench
Classic Unix and Linux system benchmark measuring overall performance through CPU, process, file I/O, and system-call tests aggregated into an index.
sysbench CPU Benchmark
CPU test mode of the sysbench tool measuring processor throughput via prime-number computation across single and multiple threads.
stress-ng
Configurable stress-test and micro-benchmark tool that loads CPU, memory, I/O, and kernel subsystems through hundreds of targeted stressors.