YCSB

The Yahoo! Cloud Serving Benchmark (YCSB) is an open-source framework for evaluating the performance of NoSQL databases, key-value stores, and cloud serving systems. Released by Yahoo! in 2010, it became the standard way to compare systems such as Cassandra, MongoDB, HBase, DynamoDB, Redis, and many others on equal terms. Its appeal is a simple, extensible workload model and a pluggable adapter layer.

What It Measures

YCSB measures throughput (operations per second) and latency distributions (average, 95th, and 99th percentile) for basic record operations: read, insert, update, scan, and read-modify-write. It deliberately focuses on the serving tier — single-record operations at high concurrency — rather than complex queries or transactions.

Methodology

YCSB ships with a set of core workloads labeled A through F that define different mixes: A is read-heavy update (50/50 read/update), B is read-mostly (95/5), C is read-only, D inserts and reads recent records, E is short scans, and F is read-modify-write. You choose a request distribution (uniform, Zipfian, or latest) to control hot-key skew, a record count, and a target throughput. The framework loads a data set, then runs the operation mix while measuring per-operation latency and aggregate throughput. Database-specific adapters translate the generic operations into native client calls, so results depend heavily on the adapter and client configuration. Many forks (e.g., for specific vendors) extend the core set.

How to Interpret Results

Always report the workload (A–F), the request distribution, the record and operation counts, the consistency level, and the client/thread configuration — changing any of these changes the numbers dramatically. Latency and throughput trade off, so look at the latency at a fixed throughput rather than peak throughput alone. Zipfian distributions expose caching and hot-key handling. Because adapters vary in quality and maturity, treat cross-vendor comparisons with caution and prefer results you reproduce yourself with a consistent setup.

Limitations

YCSB models only simple single-record operations with no joins, secondary-index queries, or multi-key transactions, so it understates the differences between richer data models. Results are sensitive to consistency settings, which are easy to misconfigure, leading to apples-to-oranges comparisons. The default data is synthetic and uniform in size. Use YCSB to characterize serving-tier throughput and tail latency, not to judge a database's full feature set. Its enduring popularity comes from how easy it is to run and extend, which made it the common language for early NoSQL performance debates and the basis for many vendor and academic forks.