SPECjbb 2015

SPECjbb 2015 measures the performance of a Java Virtual Machine and the underlying server running a realistic business application. It models a worldwide supermarket company with point-of-sale requests, online purchases, and data-mining operations, exercising the full Java middle tier rather than a microbenchmark. Because it stresses object allocation, garbage collection, and concurrent transaction processing together, it reflects how enterprise Java services actually behave under load.

It is the standard reference for comparing JVMs, garbage collectors, and server hardware on enterprise Java workloads, and vendors routinely publish SPECjbb numbers when launching new JVM releases or server platforms.

What It Measures

The benchmark reports two headline metrics. max-jOPS is the maximum throughput in Java Operations Per Second the system can sustain. critical-jOPS is throughput measured under strict response-time service-level agreements, rewarding systems that stay fast under load rather than only fast on average. Together they capture both raw capacity and latency discipline. The gap between them is itself informative: it exposes how gracefully a system holds latency as it approaches saturation.

Methodology

SPECjbb 2015 runs as one or more backends, transaction injectors, and a controller. It ramps load gradually, finding the injection rate at which the system saturates. Response-time percentiles are sampled throughout, and critical-jOPS is derived from points where latency bounds (such as 10ms to 100ms) are honored. Runs can be configured as a single JVM, multiple JVMs, or a distributed cluster, letting teams test scale-up and scale-out designs. Compliant results require full disclosure of hardware, JVM version, heap settings, and garbage-collector flags, and are reviewed before publication so results remain reproducible and comparable.

How to Interpret Results

Use max-jOPS to gauge peak capacity and critical-jOPS to gauge usable capacity under SLAs; the latter is usually the more honest number for production planning. A large gap between the two signals latency problems, often from garbage-collection pauses or lock contention. Compare configurations only when heap size, core count, and JVM build are documented. Higher is better for both metrics. The benchmark is especially useful for tuning GC algorithms such as G1, ZGC, or Shenandoah, where critical-jOPS exposes pause behavior that averages hide, and for sizing heap against throughput targets.

Limitations

Results are sensitive to JVM tuning, so an out-of-the-box score can differ sharply from a hand-tuned one; published numbers often reflect expert tuning unavailable in practice. The supermarket model is one workload shape and may not match streaming, batch, or reactive services. It does not test external databases, network fan-out, or persistence layers, so it isolates the JVM and CPU rather than a full distributed system. Use it to compare JVMs and server compute, then validate with your own application load and realistic data access.