Twitter/X

PTLD (Sim-to-Real Privileged Tactile Latent Distillation for Dexterous…

2026-07-13 · 02:21 UTC ·@JitendraMalikCV ·0 min read

Brief

PTLD (Sim-to-Real Privileged Tactile Latent Distillation) argues that a small amount of real data lets researchers train rich tactile sensorimotor policies that outperform prior zero-shot sim-to-real methods limited to simple sensors like proprioception. The paper (arXiv:2603.04531) is by Rosy Chen, Mukadam, Michael Kaess, Tingfan Wu, Francois Hogan, Jitendra Malik and Akash Sharma (CMU, UW, UC Berkeley, FAIR).

Why it matters

PTLD (Sim-to-Real Privileged Tactile Latent Distillation for Dexterous Manipulation) is published on arXiv as 2603.04531; authors include Rosy Chen, Mukadam, Michael Kaess, Tingfan Wu, Francois Hogan, Jitendra Malik, and Akash Sharma (affiliations: CMU, UW, UC Berkeley, FAIR at Meta).

Key details

  • The authors claim prior zero-shot sim-to-real approaches forced compromises such as using only simple sensors like proprioception; PTLD shows that if you collect a small amount of real data, you can deploy rich tactile sensorimotor policies with significantly stronger performance.
  • This work was Akash Sharma's final Ph.D. project co-led by Rosy Chen, announced 2026-07-13, and includes a project page and demo video linked from the post.
Source evidence

Novel approach to training tactile policies..

Akash Sharma (@akashshrm02)

This was my final Ph.D. work co-led by Rosy Chen and I.

Prior work fixates on zero-shot sim-to-real resulting in compromises such as the choice of simple sensors like proprioception.
With PTLD we show that if you're allowed to collect a little real data, you can actually deploy rich sensorimotor policies with significantly stronger performance.

Paper: PTLD: Sim-to-Real Privileged Tactile Latent Distillation for Dexterous Manipulation.
@rosychen0501, @mukadammh, @michaelkaess, Tingfan Wu, Francois Hogan,@JitendraMalikCV, @akashshrm02

CMU · UW · UC Berkeley · FAIR at Meta.

📄 arxiv.org/abs/2603.04531
🌐 akashsharma02.github.io/ptld…
🎥 piped.video/YVFZt2YAS3Y

Video

— https://nitter.net/akashshrm02/status/2076005141745147916#m