MLPerf Tiny

MLPerf Tiny, part of the MLCommons MLPerf family, benchmarks machine learning on the smallest, most power-constrained hardware: microcontrollers, DSPs, and embedded accelerators that run on milliwatts. It targets the TinyML domain, where models must fit in kilobytes of memory and run on always-on, battery-powered sensors.

The suite defines four reference tasks representative of embedded use: keyword spotting (small audio command recognition), visual wake words (person detection from low-resolution images), image classification (CIFAR-10 scale), and anomaly detection (industrial machine sound). Each task fixes a model and dataset so submitters compete on hardware and software efficiency.

What It Measures

MLPerf Tiny reports inference latency per single sample, energy consumed per inference, and the model's accuracy on the reference task. Latency and energy are the headline numbers because TinyML deployments are dominated by battery life and real-time response. Accuracy ensures aggressive optimization does not degrade the model below a usable threshold.

Methodology

Submitters run the reference models on their target board using the EEMBC EnergyRunner harness, which drives inputs and measures power with calibrated instrumentation. Performance runs measure latency; energy runs measure joules per inference under controlled voltage. Closed division requires the reference model architecture; Open division allows alternative models and quantization. Accuracy is validated against held-out data to keep submissions honest.

How to Interpret Results

For battery-powered designs, energy per inference is usually the deciding metric — a device that is slightly slower but far more efficient lasts longer in the field. Latency matters for real-time triggers like wake-word detection. Compare boards within the same task and division, and weigh accuracy alongside speed, since a heavily quantized model may save energy at the cost of missed detections. Normalize by clock speed or price to judge silicon efficiency.

Limitations

The four tasks are narrow and may not represent a specific embedded workload. Results depend heavily on the toolchain and compiler, so software maturity can dominate hardware capability. Energy measurement requires specialized equipment, limiting reproducibility outside vendor labs. The suite does not cover on-device training or larger embedded models, and real deployments face thermal, sensor, and integration constraints the benchmark abstracts away.