CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts
Published:
F. Tokuda1, 2, A. Seino1, 2, A. Kobayashi1, 2, K. Kosuge1, 3
1. Centre for Transformative Garment Production, Unites 1215 to 1220, 12/F, Building 19W, SPX1, Hong Kong Schience Park, Pak Shek Kok, N. T., Hong Kong SAR
2. Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR
3. Director of the JC STEM Lab of Robotics forSoft Materials, Department of Electrical and Electronic Engineering, Faculty of Engineering, The University of Hong Kong, Hong Kong SAR
Published in 2023 IEEE International Conference on Robotics and Automation (ICRA), (2023)
DOI: 10.1109/ICRA48891.2023.10160635
Download here.
- Abstract
- This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator’s motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme.