ArXiv

Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

2026-07-29 · 12:47 UTC ·Amirmohammad Farzaneh, Osvaldo Simeone ·1 min read
Authors
Amirmohammad Farzaneh, Osvaldo Simeone
Categories
stat.ML, cs.AI, cs.IT, cs.LG
arXiv
https://arxiv.org/abs/2607.26865v1
PDF
https://arxiv.org/pdf/2607.26865v1

Brief

TSDS addresses edge deployment of ReAct LLM agents by stopping on-device reasoning early with a convergence probe and deferring high-perplexity actions to a cloud model. The paper introduces a joint Learn-Then-Test (LTT) calibration giving finite-sample guarantees on episode reward and cloud-call rate, and reports 43%–73% thinking-compute reductions on HotpotQA, MBPP, and a household-robot task versus deferral-only baselines.

Why it matters

TSDS (Think Short, Defer Smart) combines a lightweight convergence probe that halts on-device ReAct reasoning once the intended action stabilizes with a perplexity-based deferral rule that escalates uncertain actions to a cloud model; both mechanisms are jointly calibrated via a multi-objective Learn-Then-Test (LTT) procedure that gives finite-sample guarantees on expected episode reward and cloud-call rate.

Key details

  • Evaluated on four benchmarks (GSM8K, HotpotQA, MBPP, household robot), TSDS reduced per-episode thinking compute by 43%–73% versus deferral-only baselines on HotpotQA, MBPP, and the household robot task while maintaining certified reward and cloud-call-rate guarantees; comparisons include thought-calibration-only and calibrated-deferral-only baselines (arXiv preprint 2026-07-29).
Source evidence

Abstract

LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.