This paper describes the system developed by team MLPBOON for MediaEval 2016 Predicting Media Interestingness Image Subtask. After experimenting with various features and classi ers on the development dataset, our nal system involves use of CNN features (fc7 layer of AlexNet) for the input representation and logistic regression as the classi er. For the proposed method, the MAP for the best run reaches a value of 0.229.
The MediaEval 2016 Predicting Media Interestingness Task
[
The solution should essentially involve encoding into
features many generic factors that are taken into account by
humans while judging interestingness of an image [
Therefore, while taking into account subjectivity, we wish
to determine features good for satisfactorily solving the task.
In this context, several e orts have been made to understand
factors that a ect, or cues that contribute to interestingness
of an image, even at an individual level. Katti et al. [
Our approach was inspired by the following line of thought: if the right set of features are identi ed then any simple classi er should produce good results. Thus, we decided upon the proposed system after experimenting with di erent feature sets. 2.
We have opted for a more traditional machine learning pipeline involving - feature selection & preprocessing, training of classi cation model and then the predictions.
Given the training data feature matrix X 2 RN F consisting of N examples, each described by a F -dimensional vector, we rst standardize it and apply principal component analysis (PCA) to reduce its dimensionality. The transformed feature matrix Z = (zi)i 2 RN M is used to experiment with various classi ers. Here M depends on the number of top eigenvalues we wish to consider.
After preliminary testing (discussed in section 3), we
decided to move ahead with logistic regression as our classi er.
Logistic regression minimizes the following cost function [
N Cost (w) = C X log(1 + e yiwT zi ) + 1 wT w; 2 where w denotes the weight vector, C > 0 denotes penalty parameter, zi denotes feature vector for the ith instance of training data, while yi denotes its label (0 if non-interesting and 1 if interesting). Note that a column of ones is appended to Z to include the hyperplane intercept as a coe cient of w. Now given a test data instance t, its label y is assigned according to equation (2). (1) (2) y = (1; if wT t
0 0; otherwise After experimenting with various descriptors (as discussed later in Section 3.1), we use CNN features extracted from fc7 layer of the AlexNet as our input feature representation for building X. In the following section we discuss our experimental results obtained by varying di erent parameters of the above stated system. 3.
The training dataset consised of 5054 images extracted
from 52 movie trailers, while the test data consisted of 2342
images extracted from 26 movie trailers. [
The run submission results are given in Table 1. The table gives the mean average precision (MAP) - the o cial metric, precision and recall on the interesting images of di erent runs corresponding to the respective penalty parameter and number of transformed features retained after PCA. The general strategy for the run submissions was to rst decide and x the number of PCA features and subsequently tune C for best MAP on development data.
As observed, C decreases with increasing PCA features. This trend can be possibly explained as a way to avoid overtting. The 5th run gives the best MAP, however, the MAP for all the runs seems comparable. This points towards the utility of dimensionality reduction which signi cantly reduces the redundancy without a ecting the results much. It was observed that 400 and 780 transformed features capture about 95% and 98% variance of the data, respectively. The di erence between MAP on development and test data for all the runs was very small and lied between 0.01-0.03. The maximum MAP on development data was 0.24 with 1st run's system con guration.
System Design Decisions
We experimented with the following features provided by the
task [
Features: The results on the development data using the GIST (512 dimensional feature vector) and ColorHistogram (128 dimensional feature vector) features were not very positive over any classi er. The use of CNN features
No. of features 780 700 700 400 2016 (4096-dimensional fc7 & 1000-dimensional prob layers) did show signi cant improvements with fc7 features in particular performing better over the prob features. We also observed that combination of CNN features with GIST and ColorHistogram features gave similar performance to the case when we use just CNN features. Hence we went forward with using just CNN features, in particular from fc7 layer.
Classi er: After selecting CNN features we experimented
with various classi ers with di erent parameters.
Specifically, we tried (1) SVM with linear, polynomial and rbf
kernels (2) ridge regression classi er (3) stochastic
gradient descent classi er with hinge, log, modi ed-huber and
squared-hinge loss functions (4) logistic regression [
In summary, we have presented a system for interestingness prediction in images. Despite its simplicity, we obtain reasonable mean average precision values with the maximum being 0.229. From an analysis of the system's development history we think that selection of features was more important than the selection of the classi er. We believe it would be useful to identify and incorporate high level features describing image composition and object expressivity such as facial expressions. Moreover, to analyze the issue of subjectivity, it would be interesting to check inter-annotator agreement over test images. 5.