Vision-Language-Action (VLA) models have shown remarkable generalization by mapping web-scale knowledge to robotic control, yet they remain blind to physical contact. Consequently, they struggle with contact-rich manipulation tasks that require reasoning about force, texture, and slip. While some approaches incorporate low-dimensional tactile signals, they fail to capture the high-resolution dynamics essential for such interactions.
To address this limitation, we introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future. Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs, coupled with wrist-camera local vision and third-person macro vision. We first train a unified policy with a Hierarchical Spatial Alignment (HSA) loss that aligns tactile tokens with their spatial counterparts in the wrist and third-person views. We then finetune the system with a tactile world model that predicts future tactile signals. Across contact-rich manipulation tasks, DreamTacVLA outperforms state-of-the-art VLA baselines, achieving up to 95% success.
We align macro (third-person), local (wrist), and micro (tactile) views into a single latent space using an InfoNCE-style loss over activated tactile pixels and their visual counterparts.
A frozen V-JEPA2 backbone, finetuned on our tactile data, predicts future tactile embeddings conditioned on a draft action — an implicit physics engine for contact.
The policy first thinks a draft action, dreams its tactile future, and acts on a refined command — producing fine-grained corrections without an explicit planner.
We evaluate DreamTacVLA on four tasks demanding precise, contact-rich manipulation — tight tolerances, slip detection, and stable tool contact.
assets/videos/peg_in_hole.mp4 or .gif
Classic high-precision insertion. The port is partially occluded, forcing the policy to rely on tactile feedback for the final alignment.
assets/videos/usb_insert.mp4 or .gif
Inserting a USB-A plug into a port with sub-millimeter tolerances that are ambiguous from vision alone.
assets/videos/gear_assembly.mp4 or .gif
Sliding a small gear onto a shaft. Requires aligning the gear's hole with the shaft — easily failed by misalignment.
assets/videos/tool_stabilize.mp4 or .gif
Grip a cube and use one of its vertices to support a thin vertical cylinder, maintaining its upright pose under small disturbances.
Real-world success rates over 100 trials per task (mean ± std across 3 runs).
| Method | Peg-in-Hole | USB Insert | Gear Assembly | Tool Stabilize |
|---|---|---|---|---|
| ACT | 35.2 ± 0.7 | 62.6 ± 0.5 | 22.4 ± 0.8 | 19.3 ± 0.6 |
| Diffusion Policy | 35.5 ± 0.9 | 56.3 ± 0.8 | 33.1 ± 0.7 | 30.4 ± 0.9 |
| π0 | 48.7 ± 1.0 | 59.4 ± 0.9 | 45.2 ± 1.1 | 41.0 ± 0.8 |
| ACT (w/ Tactile) | 45.6 ± 0.3 | 63.1 ± 0.4 | 40.2 ± 0.2 | 39.1 ± 0.7 |
| Ours (HSA only) | 60.8 ± 0.9 | 63.7 ± 0.8 | 51.5 ± 1.0 | 42.9 ± 0.7 |
| Ours (Dream only) | 75.4 ± 0.8 | 75.2 ± 0.7 | 64.9 ± 0.6 | 68.5 ± 0.9 |
| Ours (HSA & Dream) | 95.0 ± 0.2 | 85.7 ± 0.6 | 81.1 ± 0.4 | 74.6 ± 0.5 |
Available on Hugging Face: michaelyeah7/dobot_xtrainer_tac_1107.
@article{ye2025learning,
title={Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation},
author={Ye, Guo and Zhang, Zexi and Zhao, Xu and Wu, Shang and Lu, Haoran and Lu, Shihan and Liu, Han},
journal={arXiv preprint arXiv:2512.23864},
year={2025}
}