An Adaptive Pixel Affinity Propagation Framework for Unsupervised Image Segmentation Using Structural Consistency Learning and Progressive Region Optimization

Abstract

Robust unsupervised image segmentation has been a long standing and fundamental problem in computer vision that still imposes many restrictions and challenges in computer vision when dealing with different heterogeneous scenes. Introduce the Adaptive Pixel Affinity Propagation (APAP) framework which combines three complementary building blocks: (1) an adaptive affinity propagation module, that can calculate pairwise affinity between pixels using adaptive kernels with a spatial variation, versus neighbourhoods windows; (2) a structural consistency learning objective, enforcing the consistency between segmentation predictions under geometric and photometric perturbation; and (3) a progressive region optimization scheme that progressively coarsens and refines regions using a graph coarsening procedure, weighted by confidence. The key idea of APAP is to simultaneously identify the local affinity and enforce a global region-consistency prior, thereby overcoming the problems of over-segmentation shown by previous clustering-based and/or contrastive methods in regions which have high texture content and low contrast. Inspect APAP on 3 representative benchmarks (BSDS500, PASCAL VOC 2012, Cityscapes) and achieve regular improvements on each of the 3 evaluations: mean Intersection-over-Union (mIoU), Boundary F-score and Adjusted Rand Index (ARI). The results of the ablation studies are confirmed that each proposed ablation component has an important impact on the overall performance, with the progressive region optimization stage exhibiting the highest performance gains in boundary precision. The analysis of the results indicate that the simultaneous use of adaptive affinity modeling and explicit consistency constraints of the structure is a good direction for label-free segmentation.