A Fast Inference Framework for Medical Image Semantic Segmentation Tasks Using Deep Learning Framework

Deep neural network powered semantic segmentation implementation has great advantages of providing accurate object detection using pixel-based classification; however, when this technique is applied within resource-constrained platforms, such as mobile medical devices and surgery robotic platforms, it faces great challenges in terms of limited memory and computational power. In this chapter, we explore the possibility of training neural networks for semantic segmentation task with small memory requirements (1/4 to full scale of image in PASCAL VOC 2012, Cityscape datasets and Endoscopic artefact database), while maintaining performance as the same as the training results with large memory footprint using full scale images. Our proposed method provides a visual memory unified framework, where global semantic information for local feature extraction is combined at the training stage. We demonstrated the possibility of training a deep neural network with a pixel accuracy of 91.5% for 32G memory systems; furthermore, our improved visual memory unified model can achieve a 2% improvement in performance compared with other semantic segmentation networks. The framework has been implemented based on a Xilinx evaluation board ZCU102, achieving 10× faster inference time compared with other embedded platforms.