Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature
Identifiers (Article)
Identifiers (Files)
Abstract
In the past few years, the number of fine-art collections that are dig- itized and publicly available has been growing rapidly. With the availability of such large collections of digitized artworks comes the need to develop multime- dia systems to archive and retrieve this pool of data. Measuring the visual similar- ity between artistic items is an essential step for such multimedia systems, which can benefit more high-level multimedia tasks. In order to model this similarity between paintings, we should extract the appropriate visual features for paintings and find out the best approach to learn the similarity metric based on these fea- tures. We investigate a comprehensive list of visual features and metric learning approaches to learn an optimized similarity measure between paintings. We de- velop a machine that is able to make aesthetic-related semantic-level judgments, such as predicting a painting’s style, genre, and artist, as well as providing simi- larity measures optimized based on the knowledge available in the domain of art historical interpretation. Our experiments show the value of using this similarity measure for the aforementioned prediction tasks.
Statistics
References
A. E. Abdel-Hakim and A. A. Farag. Csift: A sift descriptor with color invariant characteristics. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2006.
R. Arnheim. Visual thinking. Univ of California Press, 1969.
R. S. Arora and A. M. Elgammal. Towards automated classification of fine-art
painting style: A comparative study. In ICPR, 2012.
Y. Bar, N. Levy, and L. Wolf. Classification of artistic styles using binarized
features derived from a deep neural network. 2014.
A. Bentkowska-Kafel and J. Coddington. Computer Vision and Image Analysis of
Art: Proceedings of the SPIE Electronic Imaging Symposium, San Jose Conven-
tion Center, 18-22 January 2010. PROCEEDINGS OF SPIE. 2010.
I. E. Berezhnoy, E. O. Postma, and H. J. van den Herik. Automatic extrac- tion of brushstroke orientation from paintings. Machine Vision and Applications,
(1):1–9, 2009.
A.BergamoandL.Torresani.Classemesandotherclassifier-basedfeaturesforef-
ficient object categorization. IEEE Transactions on Pattern Analysis and Machine
Intelligence, page 1, 2014.
A. Bergamo, L. Torresani, and A. W. Fitzgibbon. Picodes: Learning a compact
code for novel-category recognition. In Advances in Neural Information Process-
ing Systems, pages 2088–2096, 2011.
G. Carneiro, N. P. da Silva, A. D. Bue, and J. P. Costeira. Artistic image classifi-
cation: An analysis on the printart database. In ECCV, 2012.
N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In International Conference on Computer Vision & Pattern Recognition, volume 2,
pages 886–893, June 2005.
J.V.Davis,B.Kulis,P.Jain,S.Sra,andI.S.Dhillon.Information-theoreticmetric
learning. In ICML, 2007.
M.V.FahadShahbazKhan,JoostvandeWeijer.Whopaintedthispainting?2010.
L. Fichner-Rathus. Foundations of Art and Design. Clark Baxter, 2008.
J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood com-
ponents analysis. In NIPS, 2004.
C. R. Johnson, E. Hendriks, I. J. Berezhnoy, E. Brevdo, S. M. Hughes,
I. Daubechies, J. Li, E. Postma, and J. Z. Wang. Image processing for artist iden-
tification. Signal Processing Magazine, IEEE, 25(4):37–48, 2008.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing sys-
tems, pages 1097–1105, 2012.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied
to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
J. Li and J. Z. Wang. Studying digital imagery of ancient paintings by mixtures of stochastic models. Image Processing, IEEE Transactions on, 13(3):340–353,
J. Li, L. Yao, E. Hendriks, and J. Z. Wang. Rhythmic brushstrokes distinguish van gogh from his contemporaries: Findings via automated brushstroke extraction. IEEE Trans. Pattern Anal. Mach. Intell., 2012.
T. E. Lombardi. The classification of style in fine-art painting. ETD Collection for Pace University. Paper AAI3189084., 2005.
D. G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 2004.
S. Lyu, D. Rockmore, and H. Farid. A digital technique for art authentication. Proceedings of the National Academy of Sciences of the United States of America, 101(49):17006–17010, 2004.
A. Oliva and A. Torralba. Modeling the shape of the scene: A holistic representa- tion of the spatial envelope. IJCV, 2001.
G.Polatkan,S.Jafarpour,A.Brasoveanu,S.Hughes,andI.Daubechies.Detection of forgery in paintings using supervised learning. In 16th IEEE International Conference on Image Processing (ICIP), 2009.
R. Sablatnig, P. Kammerer, and E. Zolda. Hierarchical classification of paintings using face- and brush stroke models. 1998.
B. Saleh, K. Abe, and A. Elgammal. Knowledge discovery of artistic influences: A metric learning approach. In ICCC, 2014.
C. Shen, J. Kim, L. Wang, and A. van den Hengel. Positive semidefinite metric learning using boosting-like algorithms. Journal of Machine Learning Research, 13:1007–1036, 2012.
D. G. Stork. Computer vision and computer graphics analysis of paintings and drawings: An introduction to the literature. In Computer Analysis of Images and Patterns, pages 9–24. Springer, 2009.
L. Torresani, M. Szummer, and A. Fitzgibbon. Efficient object category recogni- tion using classemes. In ECCV, 2010.
A. Vedaldi and K. Lenc. Matconvnet – convolutional neural networks for matlab. CoRR, abs/1412.4564, 2014.
K. Weinberger and G. Tesauro. Metric learning for kernel regression. In Eleventh international conference on artificial intelligence and statistics, pages 608–615, 2007.
K.Q.WeinbergerandL.K.Saul.Distancemetriclearningforlargemarginnearest neighbor classification. JMLR, 2009.
