DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control

IEEE International Conference on Robotics and Automation (ICRA), 2026
Yuanchuan Lai1, Qing Gao1,*, Ziyan Liang1, Xianfeng Cheng1, Junjie Hu2, Zhaojie Ju3
Sun Yat-sen University, The Chinese University of Hong Kong, University of Portsmouth

Abstract

In dual-arm dexterous teleoperation, crossplatform generalization of motion retargeting and interactivity of grasping are crucial. However, the heterogeneity of robotic architectures and the wide variety of grasping objects pose significant challenges to achieving precise motion retargeting and compliant grasping in dual-arm dexterous teleoperation. To address these challenges, a dual-arm dexterous teleoperation system (DexTele) is proposed based on motion retargeting and adaptive force control. First, a vision-based motion retargeting module is designed to generate preliminary robot motions from human images. In this module, a motion-graph encoder and latent optimization are proposed for precise and convenient cross-platform motion retargeting. Second, an adaptive grasping module is designed to achieve compliant grasping. This module combines a vision-language model (VLM) with model predictive control (MPC), allowing the system to predict the required grasping force for a target object and perform gradient-based online optimization. Finally, extensive experiments demonstrate that the DexTele achieves precise motion retargeting and compliant grasping with generalization across multiple robot platforms.

Motivation of our model

Demonstration of motion retargeting across multiple robot platforms.

From top to bottom, the six rows represent: the human demonstrator, the mesh rendering of the human demonstrator, the YuMi robot in simulation, the Unitree H1 robot in simulation, the RMC-DA robot in simulation, and the RMC-DA robot in a real-world environment.



Architecture of our I-CTRL

Overview of the dual-arm dexterous teleoperation system.

Human motions are first captured via FrankMocap and processed into 3D body and hand poses. The extracted hand and arm motions are retargeted to the corresponding movements of the robot’s hand and arm. The robot’s arm angles are directly mapped to the robot, while the hand angles are adjusted by the adaptive grasping module before executing the physical grasping.

BibTeX


@article{
  title={DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control},
  author={Yuanchuan Lai, Qing Gao, Ziyan Liang, Xianfeng Cheng, Junjie Hu, Zhaojie Ju},
  journal={ICRA},
  year={2026}
}