HarmBench

HarmBench is a standardized evaluation framework for automated red teaming of large language models. Red teaming probes whether a model can be induced to produce harmful content; HarmBench makes this measurable and comparable by fixing the harmful behaviors, the attack methods, and the judge used to decide whether an attack succeeded.

What It Measures

The benchmark defines a set of disallowed behaviors spanning categories such as cybercrime, illegal weapons, dangerous chemicals, misinformation, harassment, and privacy violations. For each behavior, automated attacks attempt to make the model comply. The core metric is attack success rate (ASR): the fraction of attempts that produce genuinely harmful, on-target output. Its complement, the robust refusal rate, captures how reliably the model declines. Reporting ASR per category shows which harm types a model resists or fails to resist.

Methodology

HarmBench pairs a library of attack algorithms, ranging from prompt-engineering jailbreaks to optimization-based adversarial suffixes, with a standardized classifier that judges whether a response actually exhibits the targeted harmful behavior rather than merely mentioning the topic. Holding the behaviors, attacks, and judge constant lets different models and different defenses be compared on equal footing. The framework also supports evaluating defenses, measuring how much a mitigation lowers ASR, and distinguishes harmful completions from refusals or off-topic responses to avoid crediting safe-but-unhelpful answers.

How to Interpret Results

Lower ASR is better and indicates stronger safety, but read it by category, since a model may resist most harms yet remain vulnerable in one area that matters for your deployment. Compare ASR across attack types: robustness to simple jailbreaks but failure under optimization attacks reveals shallow defenses. When evaluating mitigations, weigh the ASR reduction against any drop in helpfulness, because aggressive refusal can harm legitimate use. Treat ASR as a relative safety signal, not an absolute guarantee.

Limitations

Results depend heavily on the classifier judging harm, which can misclassify borderline outputs, and on the specific attack set, which cannot cover every real adversary. A low ASR against known attacks does not prove safety against novel ones. The fixed behavior taxonomy may omit harms relevant to a given context, and overly cautious models can score well by refusing broadly at the cost of usefulness. HarmBench measures susceptibility under defined conditions, not comprehensive real-world safety.