Abstract
Comment: Project page: https://skevinci.github.io/coordex/
CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation
- Authors
- Sikai Li, Shuning Li, Zhenyu Wei...
- Categories
- cs.RO, cs.AI, cs.LG
Brief
CoorDex presents a pipeline that distills simulated whole-body and hand demonstrations into proprioception-conditioned latent priors, freezes those priors as the action space, and trains a coordinated latent residual policy with separate body/hand residual heads. This enables continuous, contact-rich loco-manipulation on a Unitree G1 with a 20-DoF WUJI hand (bottle grasp/carry, fridge opening, cube manipulation); ablations show the coordinated latent approach is necessary where joint-space and monolithic baselines fail.
Why it matters
CoorDex converts high-DoF whole-body and a 20-DoF WUJI hand into coordinated latent residual control on a Unitree G1 humanoid, demonstrating continuous on-the-move dexterous tasks (non-stop bottle grasp-and-carry, fridge-door opening while walking, cube pick-and-turn).
Key details
- Training pipeline: simulated whole-body/hand demonstrations -> privileged motion-tracking teachers -> distilled proprioception-conditioned latent priors (frozen as the RL action space) -> coordinated latent residual policy with shared task context and separate body/hand residual heads.
- Ablations on the walk-grasp-carry task show joint-space PPO, joint-space hand control, and monolithic latent prediction fail under the same reward budget, while the latent-prior interface plus coordinated residual structure enables successful training.