In this paper, we describe our model designed for automatic prediction of media interestingness. Specifically, a two-stage learning framework is proposed. In the first stage, supervised dimensionality reduction is employed to discover the key discriminant information embedded in the original feature space. We present a new algorithm dubbed biased discriminant embedding (BDE) to extract discriminant features with discrete labels and use supervised manifold regression (SMR) to extract discriminant features with continuous labels. In the second stage, SVM is utilized for prediction. Experimental results validate the efectiveness of our approaches.
Predicting the interestingness of multimedia content has long
been studied in the psychology community [
In this paper, we propose to use dimensionality reduction
to extract low-dimensional features for MediaEval 2017
Predicting Media Interestingness Task. Specifically, we propose a
new algorithm called biased discriminant embedding (BDE)
for discrete labels and utilize supervised manifold regression
(SMR) [
DIMENSIONALITY REDUCTION
Biased Discriminant Embedding Given the data matrix X = [x1, x2, ..., x], where x ∈ R denotes the feature vector of the -th image or video, and label vector l = [1, 2, ..., ], where ∈ {0, 1} denotes the corresponding label of x, with 1 for interesting and 0 for non-interesting, biased discriminant embedding (BDE) aims to learn a × transformation matrix W, which maximizes the biased discriminant information in the reduced subspace. The motivation for proposing the biased discrimination is that in media interestingness prediction, we are probably more interested in the interesting class than the non-interesting Copyright held by the owner/author(s). W = arg min ∑︁
W
,=1 interestingness level of x and that of x. where = | − | measures the similarity between the
For each high-dimensional data point x, we can obtain its low-dimensional representation by y = apply SVM to y for interestingness prediction.
W x. Then we 3
For each image data sample, we construct a 1299-D feature
vector by selecting features from the feature set provided
by the task organizers, including 128-D color histogram
features, 300-D denseSIFT features, 512-D gist features, 300-D
hog2× 2, and 59-D LBP features. For the video data, we treat
each frame as a separate image, and calculate the average
and standard deviation over all frames in this shot, and thus
we have a 2598-D feature set for each video. We
normalize each dimension of the training data to the range [
For Run 1 of image data, we use the normalized 1299-D feature vector as the input of SVM. For Runs 2-5 of image data, we reduce the original data to the 23-D, 25-D, 26-D, 27-D subspaces via BDE (for discrete labels) and SMR (for continuous labels), respectively. For Run 1 of video data, we one. The objective function of BDE is given as follows: W = arg max
W ︃(
W SW )︃ notes the biased within-class scatter, S = ∑︀ x)(x − x) de,=1( × | − |)(x − x)(x − scatter, and = (−|| x − x) denotes the biased between-class x||2/2 ) measures the closeness between two data samples x and x. The optimization problem could be solved by generalized eigen-decomposition. 2.2
Supervised Manifold Regression
Supervised manifold regression (SMR) [
Run 1 Run 2 Run 3 Run 4 Run 5
Images MAP@10 0.1184 0.132 0.1332 0.1315 0.1369
MAP 0.2812 0.2916 0.2898 0.2884 0.291
Videos MAP@10 0.0556 0.0468 0.0468 0.0463 0.0445
MAP
0.1813
0.1761
0.1761
0.1742
0.1746
use the normalized 2598-D feature vector as the input of
SVM. For Runs 2-5 of video data, we reduce the original
data to the 23-D, 25-D, 26-D, 27-D subspaces via BDE (for
discrete labels) and SMR (for continuous labels), respectively.
To predict the binary interestingness labels, we use -SVC [
We further analyze the contribution of each dimension in the original feature space. The contribution of the -th dimension is defined as = ∑︀ | |, where denotes the -th eigenvalue, denotes the (, )-th element of W, and | · | denotes the absolute value operator. From Figures 1(a) and 1(c), we can observe that color histogram and LBP features contribute more than the others while the GIST features contribute the least in the discrete prediction task. In continuous prediction (Figures 1(b) and 1(d)), the color histogram and GIST features contribute the most among the five feature sets. 4
DISCUSSION AND OUTLOOK This paper introduces our model designed for media interestingness prediction. For the future work, we aim to improve the performance of video interestingness prediction by incorporating the video temporal information. Moreover, as the ground truth (labels) of interestingness are provided by human beings, they generally vary with each individual and are somewhat subjective. We are therefore particularly interested in refining the human labeled ground truth (especially for continuous case) via machine learning technologies.
This work was supported in part by the National Natural Science Foundation of China under Grant 61503317, and in part by the Faculty Research Grant of Hong Kong Baptist University (HKBU) under Project FRG2/16-17/032.