ArXiv

Diagnosing Compositional Generalization in Sequential Robot Tasks

2026-07-31 · 17:59 UTC ·Yixiao Wang, Cheng-En Wu, Lingfeng Sun... ·1 min read
Authors
Yixiao Wang, Cheng-En Wu, Lingfeng Sun...
Categories
cs.RO
arXiv
https://arxiv.org/abs/2607.29687v1
PDF
https://arxiv.org/pdf/2607.29687v1

Brief

Diagnosing Compositional Generalization in Sequential Robot Tasks (Wang et al., arXiv 2026-07-31) studies how instruction-space coverage affects OOD performance. The authors propose a three-way decomposition of the generalization gap, show that carefully chosen subsets (≈25% of task tuples) can recover strong OOD behavior, and report that one-demo finetuning boosts OOD success from 0.4% to 54.7%, suggesting data collection should prioritize dependency coverage.

Why it matters

Decomposes the compositional generalization gap into three concrete sources—marginal instruction shift, instruction-compositional shift, and context–action shift—providing a diagnostic framework for when sparse instruction coverage will succeed.

Key details

  • Demonstrates exhaustive tuple enumeration is unnecessary: a structured subset as small as one quarter of the full task space can recover strong out-of-distribution performance when it preserves action-relevant dependencies; semantically dependent tasks require relational-structure coverage rather than mere factor diversity.
  • Finds sparse-training failures are often due to instruction steering rather than missing low-level skills: finetuning with a single demonstration per task raises OOD success from 0.4% to 54.7%.
Source evidence

Abstract

Sequential robot manipulation requires policies to execute novel combinations of familiar instruction components. However, collecting demonstrations for all possible instruction tuples is combinatorially expensive, while sparsely covered datasets often fail under out-of-distribution recombination. This paper studies compositional generalization through the lens of instruction-space coverage. We decompose the generalization gap into three sources: \textit{marginal instruction shift}, \textit{instruction-compositional shift}, and \textit{context--action shift}. This decomposition allows us to diagnose when sparse training coverage is sufficient, and what structure the training set must preserve for reliable action prediction. Our results show that exhaustive tuple enumeration is unnecessary: a structured subset, as small as one quarter of the full task space, can recover strong out-of-distribution performance when it covers action-relevant dependencies. We further find that sparse training often fails due to instruction steering rather than missing low-level skills; finetuning only one demonstration per task improves OOD success from (0.4\%) to (54.7\%). For semantically dependent tasks, effective coverage must capture relational structure rather than only factor diversity. These findings suggest that efficient robot data collection should prioritize dependency coverage in instruction space over exhaustive task expansion. More results are available in the supplementary material. Project website: https://yixiaowang7.github.io/DiagnosingCompositionalGeneralizationRobotPage/.