Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation

Liu Y, Ding C, Tian Y, Pang G, Belagiannis V, Reid I, Carneiro G (2024)

Publication Language: English

Publication Type: Conference contribution

Publication year: 2024

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 1151-1161

Conference Proceedings Title: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

Event location: Paris FR

ISBN: 979-8-3503-0719-1

DOI: 10.1109/ICCV51070.2023.00112

Abstract

Semantic segmentation models classify pixels into a set of known ("in-distribution") visual classes. When deployed in an open world, the reliability of these models depends on their ability to not only classify in-distribution pixels but also to detect out-of-distribution (OoD) pixels. Historically, the poor OoD detection performance of these models has motivated the design of methods based on model re-training using synthetic training images that include OoD visual objects. Although successful, these re-trained methods have two issues: 1) their in-distribution segmentation accuracy may drop during re-training, and 2) their OoD detection accuracy does not generalise well to new contexts outside the training set (e.g., from city to country context). In this paper, we mitigate these issues with: (i) a new residual pattern learning (RPL) module that assists the segmentation model to detect OoD pixels with minimal deterioration to inlier segmentation accuracy; and (ii) a novel context-robust contrastive learning (CoroCL) that enforces RPL to robustly detect OoD pixels in various contexts. Our approach improves by around 10% FPR and 7% AuPRC previous state-of-the-art in Fishyscapes, Segment-Me-If-You-Can, and RoadAnomaly datasets.

Authors with CRIS profile

Vasileios Belagiannis Professur für Maschinelles Lernen in der Signalverarbeitung

Involved external institutions

University of Adelaide AU Australia (AU) Harvard University US United States (USA) (US) National University of Singapore (NUS) SG Singapore (SG) University of Surrey GB United Kingdom (GB)

How to cite

APA:

Liu, Y., Ding, C., Tian, Y., Pang, G., Belagiannis, V., Reid, I., & Carneiro, G. (2023). Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 1151-1161). Paris, FR: Institute of Electrical and Electronics Engineers Inc..

MLA:

Liu, Yuyuan, et al. "Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation." Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris Institute of Electrical and Electronics Engineers Inc., 2023. 1151-1161.

BibTeX: Download