This paper describes our approach designed for the MediaEval 2018 Predicting Media Memorability Task. First, a subspace learning method called Memorability Preserving Embedding (MPE) is proposed to learn discriminative subspace from the original feature space according to the memorability scores. Then the Support Vector Regressor (SVR) is applied to the learned subspace for memorability prediction. The prediction performance demonstrates that SVR can achieve good performance even in a very low-dimensional subspace, which implies that the subspace learned by the MPE is capable of preserving important memorability information. Moreover, the results indicate that the short-term memorability is more predictable than the long-term memorability.
Predicting media memorability plays a key role in many
realworld applications such as media retrieval and
recommendation, and has attracted much attention recently [
Images and videos often have very high dimensionality,
which brings computational challenges to the analysis tasks.
To solve the memorability prediction task in an eficient way,
in this paper, we propose a supervised subspace learning
method called Memorability Preserving Embedding (MPE).
The motivation of designing such a subspace learning method
for the task rather than directly performing the prediction
is that we believe most of the discriminative information of
the high-dimensional media data is actually embedded in
a relatively low-dimensional subspace and discovering such
a subspace could enhance the performance of prediction.
Therefore, the proposed MPE aims to learn a
transformation matrix to project the high-dimensional training data
to a low-dimensional subspace, in which the memorability
information and manifold structure of the dataset are well
preserved. In the test stage, we use the learned
transformation matrix to map the test data to the subspace, and apply
a Support Vector Regressor (SVR) [
Given the training set = {(x1, 1), (x2, 2), ..., (x, )},
with x ∈ R ( = 1, · · · , ) being the visual feature vector
of the -th video and ∈ [
W ,=1
where = (− ( − )2/2 2) measures the similarity
between the memorability score of x and that of x, =
(−|| x − x||2/2 2) measures the closeness between x and
x, and ∈ [
Eq. (1) could be equivalently rewritten as follows: W = arg min (W XLX W),
W
where X = [x1, x2, ..., x] ∈ R× is the data matrix, L =
D − A is the × Laplacian matrix [
XLX w = w.
After obtaining W, for each high-dimensional data sample x in the development and test sets, we can obtain its lowdimensional representation by y = W x. Then we apply SVR to y for memorability prediction. (2) (3)
In this section, we report our experimental results on the
MediaEval 2018 Predicting Media Memorability Task [
We use both video specialized features and image features,
which are provided by the task, to construct the original
feature space. For the video features, we use the 101-D C3D
feature vector. For the image features, we use the 122-D
local binary pattern (LBP) feature vector and the 768-D
color histogram feature vector. We select these features as
they have demonstrated good performance in visual analysis
tasks [
We use all 8000 video data samples in the development
set for training. Before subspace learning, we normalize the
values of diferent features to [
∙ For Run 1, we set the reduced dimension = 4. Then
we learn the × (i.e., 2771× 4 in this case)
transformation matrix W via MPE using the development
set, and utilize W to map both development and
test data onto the 4-D subspace. Finally, we train
the -SVR [
The remaining procedure and the parameter setting in Runs 2, 3, and 4 are the same as those in Run 1.
Table 1 shows the performance (in terms of Spearman Correlation and MSE) of our approach. From the results, we have several observations. First, we observe that the results (both Spearman and MSE) on the short-term subtask are better than those on the long-term subtask, which indicates that the short-term memorability is more predictable than the long-term memorability. Besides, by comparing the MSE of runs 1 and 2 ( = 4, 5) and that of runs 3 and 4 ( = 9, 10),
Short
Short = 4 we notice that runs 1 and 2 are better than runs 3 and 4 in terms of Spearman, and are comparable in terms of MSE. This fact may imply that most of the discriminative information is embedded in a very low-dimensional subspace and increasing more dimensions may not necessarily improve the performance.
To further validate the efectiveness of subspace learning, we compare the performance of SVR on the learned subspace and that on the original 2771-D space using the development set. We use 5-fold cross validation and average the results. The Spearman coeficient and MSE in Table 2 show that the performance on the original space is slightly worse than that on learned subspaces, supporting our assumption that the original high-dimensional space may contain redundant or even noisy information, and reducing the dimensionality with supervised information could improve the subsequent learning performance. However, the results in terms of Spearman coeficient is far from satisfactory. The reason might be that MPE is a linear mapping method, which is not suficient to capture the complex discriminant information embedded in the high-dimensional feature space. This motivates us to consider extending our method to the nonlinear case to improve the performance. 4
This paper describes our approach designed for memorability prediction. A subspace learning method, MPE, is proposed to learn the subspace that preserves the memorability information. After that, SVR is utilized for memorability prediction in the learned subspace. The results on the MediaEval 2018 Predicting Media Memorability Task validate the efectiveness of our approach. Our future work will focus on exploring the physical meaning of the learned subspace, as this could improve the interpretability of our approach. Moreover, we plan to generalize our method to nonlinear scenario to enhance its data representation ability.
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61503317 and in part by the General Research Fund (GRF) from the Research Grant Council (RGC) of Hong Kong SAR under Project HKBU12202417.