Image Aesthetics Predictors Based on Weighted CNNs

Bin Jin¹ Maria V. Ortiz Segovia² Sabine Süsstrunk¹

¹Image and Visual Representation Lab
Ecole Polytechnique Fédérale de Lausanne
² Océ, A Canon Company

Abstract

Convolutional Neural Networks (CNNs) have been widely adopted for many imaging applications. For image aesthetics prediction, state-of-the-art algorithms train CNNs on a recently-published large-scale dataset, AVA. However, the distribution of the aesthetic scores on this dataset is extremely unbalanced, which limits the prediction capability of existing methods. We overcome such limitation by using weighted CNNs. We train a regression model that improves the prediction accuracy of the aesthetic scores over state-of-the-art algorithms. In addition, we propose a novel histogram prediction model that not only predicts the aesthetic score, but also estimates the difficulty of performing aesthetics assessment for an input image. We further show an image enhancement application where we obtain an aesthetically pleasing crop of an input image using our regression model.

Application

Materials

[ Paper ] [ Poster ] [ The supplementary material ] [ Code ] [ Model Weights (score prediction 500 MB) (histogram prediction 500 MB) ]

Citation

Bin Jin, Maria V. Ortiz Segovia, Sabine Süsstrunk
Image Aesthetics Predictors Based on Weighted CNNs
IEEE International Conference on Image Processing (ICIP), 2016.

          
  @INPROCEEDINGS{7532767, 
    author={B. Jin and M. V. O. Segovia and S. Süsstrunk}, 
    booktitle={2016 IEEE International Conference on Image Processing (ICIP)}, 
    title={Image aesthetic predictors based on weighted CNNs}, 
    year={2016}, 
    pages={2291-2295}, 
    keywords={Computational modeling;Histograms;Image enhancement;Neural networks;Predictive models;Reliability;Training;Aesthetics;CNN;sample weights}, 
    doi={10.1109/ICIP.2016.7532767}, 
    month={Sept},}

Acknowledgements

The authors thank the anonymous reviewers for their useful comments and suggestions.