Human Point Cloud Inpainting for Industrial Human-Robot Collaboration Using Deep Generative Model

In complex industrial human-robot collaboration (HRC) environment, obstacles in the shared working space will occlude the operator, and the industrial robot will threaten the safety of the operator if it is unable to get the complete human spatial point cloud. This paper proposes a real-time human point cloud inpainting method based on the deep generative model. The method can recover the human point cloud occluded by obstacles in the shared working space to ensure the safety of the operator. The method proposed in this paper can be mainly divided into three parts: (i) real-time obstacles detection. This process can detect obstacle locations in real time and generate the image of obstacles. (ii) the application of the deep generative model algorithm. It is a complete convolutional neural network (CNN) structure and introduces advanced generative adversarial loss. The model can generate the missing depth data of operators at arbitrary position in the human depth image. (iii) spatial mapping of the depth image. The depth image will be mapped to point cloud by coordinate system conversion. The effectiveness of the method is verified by filling hole of the human point cloud occluded by obstacles in industrial HRC environment. The experiment results show that the proposed method can accurately generate the occluded human point cloud in real time and ensure the safety of the operator.