3D-PipeCLIP: Leveraging geometric-language alignment for sewer defect classification from point cloud data

Automated 3D inspection of sewer pipes using LiDAR or Time-of-Flight sensors offers advantages for defect detection. However, geometric point cloud classifiers often fail to generalize from synthetic training data to noisy real-world scans. To overcome the need for very large, expert-annotated physical datasets, we propose a framework that anchors 3D feature extractors to geometric language descriptions. In this approach, a dynamic graph convolutional neural network (DGCNN) is used to extract embedding features from point clouds. During inference, the system compares these point cloud embeddings to text embeddings of geometric descriptors using cosine similarity. This differs substantially from standard hard classification, which would use a fully connected layer to a softmax head. The system is trained by minimising the contrastive language-image pretraining (CLIP) loss. In a zero-shot transfer setting from synthetic to physical laboratory scans, our method gives an F1-score of 37.86% on the real data, which doubles the performance of a baseline previously published using the same DGCNN without language anchors. When we apply a few-shot fine-tuning protocol based on real data, the model produces a weighted F1-score of 57.81% on independent real data. This places our approach near state-of-the-art methods (62.58%) that have also been trained with cross-domain knowledge of one million plus images. Our approach, therefore, is appealing for its data efficiency and also highlights for the first time the utility of geometric text anchors in this domain.