FuSC: fusing superpixels for improved semantic consistency

Open-set segmentation has caught the community's attention only in the last few years, and it is a growing and active research area with many challenges ahead. To better identify open-set pixels, we address two known issues by improving data representation and ensuring semantic consistency in open-set predictions. First, we present a method called Open Gaussian Mixture of Models (OpenGMM) that allows for multimodal statistical distributions in known class pixels using a Gaussian Mixture of Models instead of unimodal approaches, like Principal Component Analysis. The second approach improved semantic consistency by applying a post-processing technique that uses superpixels to enforce homogeneous regions to have similar predictions, rectifying erroneously classified pixels within these regions and providing better delineation of object borders. We also developed a novel superpixel method called Fusing Superpixels for Improved Semantic Consistency (FuSC) that produced more homogeneous superpixels and enhanced, even more, the open-set segmentation prediction. We applied the proposed approaches to well-known remote sensing datasets with labeled ground truth for semantic segmentation tasks. The proposed methods improved the highest AUROC quantitative results for the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets. Using FuSC, we achieved novel open-set state-of-the-art results for both datasets, improving AUROC results from 0.850 to 0.880 (3.53%) for Vaihingen and 0.764 to 0.797 (4.32%) for Potsdam datasets. The official implementation is available at: https://github.com/iannunes/FuSC.