Webly Supervised Semantic Segmentation

Bin Jin1    Maria V. Ortiz Segovia2    Sabine Süsstrunk1

1Image and Visual Representation Lab
Ecole Polytechnique Fédérale de Lausanne
2 Océ, A Canon Company

Abstract


We propose a weakly supervised semantic segmentation algorithm that uses image tags for supervision. We ap- ply the tags in queries to collect three sets of web images, which encode the clean foregrounds, the common back- grounds, and realistic scenes of the classes. We introduce a novel three-stage training pipeline to progressively learn semantic segmentation models. We first train and refine a class-specific shallow neural network to obtain segmentation masks for each class. The shallow neural networks of all classes are then assembled into one deep convolutional neural network for end-to-end training and testing. Experiments show that our method notably outperforms previous state-of-the-art weakly supervised semantic segmentation approaches on the PASCAL VOC 2012 segmentation bench- mark. We further apply the class-specific shallow neural networks to object segmentation and obtain excellent results.


Results









Materials





[ Paper ]      [ Poster ]      [ The supplementary material ]      [ Code ]     

Citation

Bin Jin, Maria V. Ortiz Segovia, Sabine Süsstrunk
Webly Supervised Semantic Segmentation
IEEE Conference on Computer Vision and Patten Recognition (CVPR), 2017.

          
  @INPROCEEDINGS{7532767, 
    author={B. Jin and M. V. O. Segovia and S. Süsstrunk}, 
    booktitle={2017 IEEE Conference on Computer Vision and Patten Recognition (CVPR)}, 
    title={Webly Supervised Semantic Segmentation}, 
    year={2017},}

Acknowledgements


The authors thank the anonymous reviewers for their useful comments and suggestions.