University of Macau Team Tackles Robotic Grasping: New Framework Enables Zero-Shot Stable Grasping Across Robotic Hands, Published in Nature Machine Intelligence
A research team led by Professor Xu Qingsong in the Faculty of Engineering at the University of Macau (UM) has achieved a breakthrough in the field of universal robotic manipulation. The team has developed a comprehensive framework for universal robotic grasping that can be adapted to complex scenarios, enabling the rapid deployment of customised and commercial robotic hands, and offering an innovative technical solution for advancing flexible automated production, medical assistive robots and service robotics. The research findings have been published in Nature Machine Intelligence.
As intelligent service robots are deployed in an increasing number of scenarios, two issues with existing manipulation algorithms are hindering their large-scale commercial application: insufficient adaptability across hardware platforms, and poor robustness in complex environments with limited sensory perception. Traditional deep learning models that rely on monocular depth vision struggle to accurately characterise the physical contact between robotic hands and objects, and show significant limitations in generalisation when performing grasping tasks with multi-fingered, multi-degree-of-freedom robotic manipulators of varying models.
To address these challenges, the team decoupled the kinematic model of robotic hands from the interactive contact states during manipulation, establishing robust and deployable contact state features that are independent of specific robotic hand models. Using existing paired grasping datasets, the team constructed a large-scale, device-agnostic database of decoupled contact features and trained neural networks to restore contact states from visual scene observations. By integrating a contact feature neural network as the environmental perception module with a universal analytical optimisation grasping framework based on contact feature reconstruction, the system achieves zero-shot adaptation, enabling all two- to five-fingered hinge-type robotic hands to perform stable grasping operations in cluttered, unstructured and densely stacked three-dimensional environments.
The study proposes the first learning-optimisation hybrid algorithm framework to enable zero-shot robust grasping generalisation across different robotic hands in highly cluttered scenarios, achieves state-of-the-art grasping performance in three-dimensional space, and uses a low-cost sensing scheme with great potential for practical deployment.
The corresponding author of the study is Xu Qingsong. The first author is Wang Xianli, a postdoctoral fellow in the UM Faculty of Engineering, and Tam Lap Mou, professor in the UM Faculty of Engineering, is also an author. The research was supported by the National Natural Science Foundation of China, the Science and Technology Development Fund of the Macao SAR, and the University of Macau.
(Source: University of Macau)
Source: University of Macau
Published by GSM · USA (original Chinese edition)
Photo: 图：配备先进抓取手的 PR2 机器人（机器人抓取研究资料图）。Oleg Alexandrov 摄，CC BY-SA 3.0，经 Wikimedia Commons。 / Photo: A PR2 robot with advanced grasping hands (file photo on robotic grasping research). Photo by Oleg Alexandrov, CC BY-SA 3.0, via Wikimedia Commons.
