fio (Flexible I/O Tester)

fio (Flexible I/O Tester) is the de facto standard tool for benchmarking storage performance on Linux and other platforms. For data engineering it matters because storage I/O often determines database and data-pipeline performance: a warehouse, log-structured store, or shuffle stage is frequently bound by the underlying disk or network storage rather than CPU. fio lets engineers characterize a device or filesystem precisely before relying on it.

What It Measures

fio reports IOPS (I/O operations per second), bandwidth (throughput in MB/s), and latency distributions (average and percentiles, separately for reads and writes). It can isolate the four canonical patterns — sequential read, sequential write, random read, random write — and mixes of them, because devices behave very differently across these. SSDs excel at random IOPS while spinning disks favor sequential bandwidth, and fio exposes exactly where a device stands.

Methodology

fio runs jobs defined either on the command line or in a job file. Key parameters are the read/write mix (rw=read, write, randread, randwrite, randrw), the block size (small blocks like 4 KB stress IOPS; large blocks like 1 MB stress bandwidth), the I/O depth (queue depth, how many requests are outstanding), the number of parallel jobs, the I/O engine (such as libaio or io_uring for asynchronous I/O), and whether the OS page cache is bypassed with direct I/O. You set a file or device, a size larger than RAM to defeat caching when testing the device itself, and a duration. fio issues the configured pattern and reports per-job and aggregate IOPS, bandwidth, and latency histograms, including useful tail percentiles.

How to Interpret Results

Match the test pattern to your workload: databases doing random point lookups care about 4 KB random-read IOPS and latency, while sequential scan or log workloads care about large-block bandwidth. Always use direct I/O and a working set larger than RAM when measuring the device rather than the cache. Read latency percentiles, not just averages, because tail latency drives tail query times. Report block size, queue depth, and I/O engine, since changing any of them changes results substantially. Higher queue depth raises IOPS but also latency.

Limitations

fio measures raw storage, not the database or application layered on it, so good fio numbers do not guarantee good query performance. Results depend heavily on caching, file-system, and cloud-volume settings (such as provisioned IOPS), which are easy to misconfigure. Synthetic patterns may not match real access mixes. Use fio to characterize and validate storage, choosing patterns and parameters that mirror your actual data workload.