A deep learning based hazardous materials (HAZMAT) sign detection robot with restricted computational resources

One of the most challenging and non-trivial tasks in robot-based rescue operations is the Hazardous Materials (HAZMAT) sign detection in dangerous operation fields, in order to prevent further unexpected disasters. Each HAZMAT sign has a specific meaning that the rescue robot should detect and interpret it to take a safe action, accordingly. Accurate HAZMAT detection and real-time processing are the two most important factors in such robotics applications. Furthermore, the rescue robot should cope with some secondary challenges such as image distortion and restricted CPU and computational resources, embedded in the robot. In this research, we propose a CNN-Based pipeline called DeepHAZMAT for HAZMAT sign detection and segmentation in four steps: (1) Input data volume optimisation before feeding into the CNN network, (2) Application of a YOLO-based structure to collect the required visual information from the hazardous areas, (3) HAZMAT sign segmentation and separation from the background using adaptive GrabCut technique, and (4) Post-processing optimisation using morphological operators and convex hull algorithms. In spite of the utilisation of a very limited CPU and memory resources, the experimental results show the proposed method has successfully maintained a better performance in terms of detection-speed and detection-accuracy, compared to classical and modern state-of-the-art methods.