Learning Keypoints from Synthetic Data for Robotic Cloth Folding
Robotic cloth manipulation is challenging due to its deformability, which
makes determining its full state infeasible. However, for cloth folding, it
suffices to know the position of a few semantic keypoints. Convolutional neural
networks (CNN) can be used to detect these keypoints, but require large amounts
of annotated data, which is expensive to collect. To overcome this, we propose
to learn these keypoint detectors purely from synthetic data, enabling low-cost
data collection. In this paper, we procedurally generate images of towels and
use them to train a CNN. We evaluate the performance of this detector for
folding towels on a unimanual robot setup and find that the grasp and fold
success rates are 77% and 53%, respectively. We conclude that learning keypoint
detectors from synthetic data for cloth folding and related tasks is a
promising research direction, discuss some failures and relate them to future
work. A video of the system, as well as the codebase, more details on the CNN
architecture and the training setup can be found at
https://github.com/tlpss/workshop-icra-2022-cloth-keypoints.git.