ArXiv

FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation

2026-07-20 · 17:58 UTC ·Ruicheng Li, Qixiu Li, Ruichun Ma... ·1 min read
Authors
Ruicheng Li, Qixiu Li, Ruichun Ma...
Categories
cs.RO
arXiv
https://arxiv.org/abs/2607.18231v1
PDF
https://arxiv.org/pdf/2607.18231v1

Brief

FM-VLA augments vision-language-action models with a force-based memory to handle non-Markovian, contact-rich manipulation where visual cues are ambiguous. The method compresses force histories into VAE latent tokens and projects them (with short state history) into the action expert. Evaluated on three memory-dependent tasks, FM-VLA achieves >80% success with low inference cost; summary based on the paper abstract.

Why it matters

FM-VLA encodes force histories into compact force-memory tokens using a variational autoencoder (VAE) pretrained on force time-series reconstruction, and feeds those latent force representations plus short state history as conditioning tokens to the action expert.

Key details

  • On three memory-dependent, contact-rich tasks—finding a hidden block, pressing a button, and wiping a dish for a specific number of times—FM-VLA attains over 80% success with minimal inference overhead, significantly outperforming vision-based memory baselines.
  • Preprint posted to arXiv on 2026-07-20 (authors: Ruicheng Li et al.); project page: https://qft-333.github.io/FM-VLA-Page/.
Source evidence

Abstract

Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches. Project page: https://qft-333.github.io/FM-VLA-Page/