sysbench

sysbench is an open-source, scriptable benchmarking tool best known for stress-testing relational databases, especially MySQL and PostgreSQL. Originally a fixed-test tool, modern sysbench (0.5+) is driven by Lua scripts, which makes its OLTP workloads flexible and easy to customize. It is one of the most commonly used tools for quick, repeatable database performance comparisons.

What It Measures

For database tests, sysbench reports transactions per second (TPS), queries per second (QPS), and latency statistics including average and 95th-percentile values. It also includes non-database modes for CPU (prime-number computation), memory bandwidth, mutex contention, threads, and file I/O, making it a general-purpose system microbenchmark as well.

Methodology

The bundled OLTP Lua scripts — such as oltp_read_only, oltp_write_only, oltp_read_write, and oltp_point_select — create one or more tables of a chosen row count, then run a configurable number of client threads issuing the scripted mix of point selects, range scans, updates, inserts, and deletes inside transactions. You control thread count, table size and number, test duration, and the report interval. sysbench prepares the data, runs the load, and prints aggregate throughput and a latency histogram. Because the workload is defined in Lua, teams often write custom scripts to mirror their own query patterns, which is a major reason for its popularity.

How to Interpret Results

Report thread count, table size, the specific Lua workload, and database configuration alongside any TPS or QPS number — all strongly affect the result. Sweep the thread count to find where throughput saturates and where latency rises, rather than quoting a single peak. Compare read-only, write-only, and mixed runs separately, since they exercise different bottlenecks (buffer pool, redo log, locking). Ensure the data set is large enough to exceed cache if you want to test storage rather than memory.

Limitations

The default OLTP scripts are simple and uniform, with no complex joins or business logic, so they characterize raw transaction processing rather than application performance. Results depend heavily on configuration and can be tuned to flatter a system, so always publish the full settings. sysbench is a microbenchmark and load tool, not an audited, standardized benchmark like TPC, so treat its numbers as relative measurements within a controlled comparison. Because it is fast to set up and easy to script, sysbench is most valuable for tuning experiments, regression detection, and quick relative comparisons rather than as a definitive cross-vendor verdict. A disciplined approach sweeps thread counts, fixes the data size relative to memory, and records the full configuration so that later runs can be compared meaningfully.