An end-to-end computationally lightweight vision-based grasping system for grocery items - Mansakul, Thanavin; Tang, Gilbert; Webb, Phil; Rice, Jamie; Oakley, Daniel; Fowler, James (MDPI, 2025)
Vision-based grasping for mobile manipulators poses significant challenges in machine perception, computational efficiency, and real-world deployment. This study presents a computationally lightweight, end-to-end grasp detection framework that integrates object detection, object pose estimation, and grasp point prediction for a mobile manipulator equipped with a parallel gripper. A transformation model is developed to map coordinates from the image frame to the robot frame, enabling accurate manipulation. To evaluate system performance, a benchmark and a dataset tailored to pick-and-pack grocery tasks are introduced. Experimental validation demonstrates an average execution time of under 5 s on an edge device, achieving a 100% success rate on Level 1 and 96% on Level 2 of the benchmark. Additionally, the system achieves an average compute-to-speed ratio of 0.0130, highlighting its energy efficiency. The proposed framework offers a practical, robust, and efficient solution for lightweight robotic applications in real-world environments.
Keywords: 4605 Data Management and Data Science; 46 Information and Computing Sciences; 4007 Control Engineering, Mechatronics and Robotics; 40 Engineering; Networking and Information Technology R&D (NITRD); 7 Affordable and Clean Energy; Analytical Chemistry; 3103 Ecology; 4008 Electrical engineering; 4009 Electronics, sensors and digital hardware; 4104 Environmental management; 4606 Distributed computing and systems software; vision-based grasping system; end-to-end grasp detection; mobile manipulator; lightweight computation; object detection; object pose estimation; machine vision