BFCL (Berkeley Function-Calling Leaderboard)

The Berkeley Function-Calling Leaderboard (BFCL) evaluates a capability central to AI agents: turning a natural-language request into correct, well-formed calls to external functions or tools. As LLMs increasingly act through APIs, the accuracy of function selection and argument construction often matters more than fluent prose.

What It Measures

BFCL spans several calling patterns. Simple calls require one function with correct arguments. Parallel calls require several functions invoked at once. Multiple-function tasks require choosing the right function from a set of candidates. Multi-turn and multi-step tasks require maintaining state across a conversation while calling tools. The benchmark also tests relevance detection: recognizing when no available function fits and the model should abstain rather than hallucinate a call.

Methodology

BFCL grades responses two ways. Abstract syntax tree (AST) evaluation parses the proposed call and checks the function name, argument names, types, and values against a reference without running anything, which works across languages and avoids flaky execution. Executable evaluation actually runs the call against real or mocked APIs and verifies the result, catching errors that look syntactically valid but behave wrongly. Separate categories score irrelevance detection so models are penalized for inventing calls when they should decline.

How to Interpret Results

Look beyond the overall average to the category breakdown. Strong simple-call accuracy with weak multi-turn or parallel scores signals an agent that handles toy cases but struggles with real workflows. High irrelevance scores matter for safety, since an agent that fabricates tool calls can take wrong actions. Compare AST and executable accuracy: a large gap means the model produces plausible-looking calls that fail at runtime.

Limitations

BFCL relies on a curated set of functions and schemas that may not match a given application's tools, so absolute scores transfer imperfectly. AST checking can be strict about argument formatting in ways that penalize harmless variation, while executable checking depends on stable test environments. Like other public leaderboards, it can be gamed by training on similar schemas. Use BFCL to compare tool-using models, then validate against your own tool catalog.