ArXiv

The Bitter Lesson of Tool Calling

2026-08-06 · 17:58 UTC ·Ishan Patel, Sahil Sen, Elias Lumer... ·1 min read

Authors: Ishan Patel, Sahil Sen, Elias Lumer...
Categories: cs.CL
arXiv: https://arxiv.org/abs/2608.06370v1
PDF: https://arxiv.org/pdf/2608.06370v1

Brief

Programmatic tool calling (PTC) exposes tools as typed Python stubs that models invoke via code; execution and results occur within one agent turn. Evaluated on BFCL v4 over 14 models, PTC matched or improved performance versus native JSON tool calling (GPT-5.6 family +10.6%) and showed better robustness under parallel fan-out and context-rot. Full text not available (abstract-only).

Why it matters

On BFCL v4 across 14 language models, programmatic tool calling (PTC) matched or exceeded native JSON tool calling in 11 of 14 models; the GPT-5.6 family showed the largest gain, +10.6% over the JSON tool-calling baseline.

Key details

PTC matched or outperformed the baseline in 13 of 14 models under parallel fan-out and stayed stable under context-rot conditions where the JSON baseline degraded 2.3% on average; PTC exposes tools as typed Python stubs invoked in a single agent turn.

Cleaned source text

Abstract