=Paper= {{Paper |id=Vol-1345/gamifir15_3 |storemode=property |title=High Quality Photo Collection via Gamification |pdfUrl=https://ceur-ws.org/Vol-1345/gamifir15_3.pdf |volume=Vol-1345 |dblpUrl=https://dblp.org/rec/conf/ecir/RaduAIG15 }} ==High Quality Photo Collection via Gamification== https://ceur-ws.org/Vol-1345/gamifir15_3.pdf
                    High Quality Photo Collection via Gamification

                            Anca Livia Radu                                     Aliaksandr Autayeu
                     DISI, University of Trento, Italy                    DISI, University of Trento, Italy
                        ancalivia.radu@unitn.it                                autayeu@disi.unitn.it
                            Bogdan Ionescu                                      Fausto Giunchiglia
                     LAPI, University Politehnica of                      DISI, University of Trento, Italy
                           Bucharest, Romania                                   fausto@disi.unitn.it
                         bionescu@imag.pub.ro



                                                                             Further, the amount of low quality images on the so-
                                                                         cial online platforms like Panoramio1 , Picasa2 or Flickr3
                            Abstract                                     is overwhelming so that even effective retrieval techniques
                                                                         have difficulties to filter them out. Furthermore, the current
     This paper addresses the issue of gathering high
                                                                         retrieval techniques are also limited since they rely mainly
     quality photographic representations of the world
                                                                         on text, image, or more recently on GPS coordinates. Tex-
     by acquiring representative and diverse images
                                                                         tual tags tend to be noisy or inaccurate, automatic content
     with locations. These locations refer to differ-
                                                                         descriptors fail to provide high-level understanding of the
     ent public spaces such as parks, museums, ad-
                                                                         scene while GPS coordinates capture the position of the
     ministrations, monuments, etc. The representa-
                                                                         photographer and not necessarily the position of the query
     tions should be clear and focused so anyone see-
                                                                         object. This limits significantly the quality of the retrieval
     ing them will easily understand what they de-
                                                                         results [2].
     pict. This work comes in the context when the
     typical users are interested in receiving accurate                      The current solutions combine image retrieval from the
     relevant-to-the-query and non-redundant images                      online platforms with media analysis techniques. These
     so they can build a correct exhaustive perception                   techniques are limited since semantic notions cannot be ef-
     of the query. We propose to tackle this issue by                    ficiently translated for media. Moreover, these solutions
     combining two approaches previously considered                      aiming at improving the retrieval focused until recently
     disjoint: gamification, i.e., involving people for                  mainly on the relevance. However, an efficient system
     photo generation and media analysis techniques                      should provide results that are relevant, but that also cover
     for participants reputation assessment and photo                    different aspects of the query and not duplicates of the same
     filtering. This paper presents our preliminary re-                  perspective, so the user can build a correct and complete
     sults achieved for implementing the gamification                    view of the query. In this paper we focus more on the di-
     strategy.                                                           versity issue.
                                                                             Research on media analysis reached the point where
1    Introduction                                                        quality photo collection requires the use of user expertise.
                                                                         Thus, we try to overcome these shortcomings, by propos-
The online image collections increased continuously and                  ing a new hybrid mechanism combining two fields: gam-
the multimedia resources are various. As a global effort,                ification as human intelligence and proved-to-work media
the web is already populated with many images of entities                analysis techniques.
in general and locations in particular. However, there are                   The remainder of the paper is organized as follows: Sec-
not enough high quality images to represent the uniqueness               tion 2 summarises the related work. Section 3 details our
of these entities [1].                                                   contribution as a hybrid mechanism which implies gam-
Copyright c 2015 for the individual papers by the paper’s authors.       ification for image generation and then automated media
Copying permitted for private and academic purposes. This volume is      analysis techniques for computing the reputation of the
published and copyrighted by its editors.
                                                                           1 http://panoramio.com
In: F. Hopfgartner, G. Kazai, U. Kruschwitz, and M. Meder (eds.): Pro-
                                                                           2 http://picasaweb.google.com
ceedings of the GamifIR’15 Workshop, Vienna, Austria, 29-March-2015,
published at http://ceur-ws.org                                            3 http://flickr.com
players. Section 4.1 presents the studies conducted for
evaluating the structure of the gamification part and also
the definition of representative and diverse images set. In
Section 4, the first pilot test of the photo hunting festival is
presented and the results we obtained. Section 5 concludes
the paper.

2   State of the art
                                                                               Figure 1: Proposed hybrid mechanism.
Various approaches focused on improving search capabili-
ties on the current social media platforms and on providing        ers, taken from Open Street Map4 .
users both representative and diverse results. Some of the
most successful approaches on this direction are related to           Each user has an initial budget to buy venues. An avail-
re-ranking [3].                                                    able venue can be bought by a user with enough money
    Re-ranking techniques attempt to refine the initial re-        by simply paying and providing some information about
sults retrieved from the online platforms using the visual         the venue (name and category). If the venue is not avail-
content of the images. An example is the approach in [4].          able, the user can spin a “wheel of fortune”. The possible
The paper defines a retrieved image as being representative        outcomes are: money, request to provide a poster for that
and diverse if it is representative for a local group in the       venue (picture and information) in exchange for money or
set, it covers many distinct groups and incorporates an ar-        request to verify your own venue’s poster provided by an-
bitrary pre-specified ranking as prior knowledge. To deter-        other player.
mine these properties, authors propose a unified framework            Some location-based applications like Foursquare5 and
of absorbing Markov chain random walks. A different ap-            Brightkite have included the process of collecting urban in-
proach [5] estimates the relevance scores of images with           formation and images with real world locations [8]. Users
respect to the query based on both the visual information          can check-in to the service with a status and a location and
of images and the semantic information of associated tags.         establish friend relationships to view each other’s updates.
Further, it estimates the semantic similarities of the images      These applications help people find out when friends are
based on their tags. Using the two estimations, the rank-          nearby or to decide where to go out.
ing list is generated by a greedy ordering algorithm which             A photographic marathon is a fun and challenging com-
optimizes average diverse precision. However, these tech-          petition organised for collecting images with locations and
niques are limited, because they are not capable of “under-        events. For example, Maratona Fotografica6 attracts peo-
standing” the world and the content of the images, since           ple for collecting images of historical centres of different
automatic content descriptors fail to provide high-level un-       Italian cities. Participants receive a list with locations to
derstanding of images.                                             reach and they are rewarded for the fastest tour or the most
    Gamification is another field with potential, but with-        interesting set of pictures.
out concrete approaches in collecting representative and
diverse images. Games with a purpose are a narrow e-                  Photo contest sites became more and more popular due
xample of gamified activities where humans become part             to fast development of technology, when almost all of
of collective computation. They are used in different ar-          us have a camera in our pocket. ViewBug7 , Pixoto8 or
eas, including computer vision and Internet retrieval. For         PHOTOBraniac9 are three examples of photo contest sites
example, ESP Game aims at collecting correct labels for i-         where participants can win cash and organizers collect im-
mages. Two players are asked to provide the same word or           ages of different types.
phrase without communicating while an image is the only               Re-ranking techniques are incapable of understanding
thing they have in common. The only modality of provid-            the content of the images and thus are limited in refining the
ing the same output is to write down a correct label to the        results retrieved from online platforms. In contrast, our ap-
image [6].                                                         proach takes advantage of the collective human intelligence
    In time, games evolved and now different location-based        through gamification for quality photo generation and fur-
games with a purpose are also used for content generation.         ther applies only proven-to-work automated techniques.
Urbanopoly [7] is a social mobile and location-based game
with a purpose inspired from the famous Monopoly real
                                                                     4 http://www.openstreetmap.org/
board game. It aims at collecting, verifying and correcting
                                                                     5 http://www.foursquare.com
data about “venues” in urban environment. As in the origi-           6 http://www.disturbo.net/
nal version, the players are landlords that aim to collect as        7 http://www.viewbug.com/
many venues as possible. But in the case of Urbanopoly,              8 http://www.pixoto.com//

the venues are real places in the surroundings of the play-          9 http://www.photobrainiac.com/
3     A photo hunting festival                                     views) as the meaning of the image is not affected by the
                                                                   manipulation and its only purpose is to improve the appea-
In this paper we address the problem of constructing a large
                                                                   rance of the image.
database with high quality images of locations, both repre-
                                                                        A set of images is considered to be diverse if each image
sentative and diverse. Our approach is designed to avoid
                                                                   depicts different visual characteristics of the target with a
the limitations of the online platforms and re-ranking tech-
                                                                   certain degree of complementarity. The whole obtained
niques by combining two fields previously considered dis-
                                                                   from the entire set of images creates a complete, clear and
joint:
                                                                   meaningful vision of that specific location.
1. gamification: human expertise employed for improved                  The diversity focuses only on the aspects that directly
photo generation;                                                  refer to the location itself. Therefore, even if light ma-
                                                                   nipulations and temporal variations (dawn-day-dusk-night
2. automated media analysis with location tracking and             and spring-summer-autumn-winter) with their correspond-
tags analysis used for filtering the images and assessing the      ing weather conditions are accepted and contribute to build-
reputation of the participants. More precisely, automated          ing a large overview of the location’s background, they
media analysis plays the role of detecting undesired im-           are not sufficient for building a complete overview over
ages: low quality (e.g., blurred or unfocused), containing         the concrete location itself. Instead, in order to build a
faces in focus and duplicate images.                               complete vision of the location, the following variations
                                                                   are accepted: outside/inside, frontal/side/back, top/bottom,
2. automated media analysis with location tracking and             large/narrow, distant/ close, entire/partial views and also
tags analysis that support gamification in returning qual-         typical aspects that are specific to that certain location, e.g.,
ity results by filtering the images and assessing the reputa-      objects, paintings, animals, plants, etc. from the inside or
tion of the participants. More precisely, automated media          outside of the location.
analysis plays the role of detecting undesired images: low         – ik : one given item k in the challenge chosen by the team;
quality (e.g., blurred or unfocused), containing faces in fo-      – qii k : the number of quality images uploaded by the team
cus and duplicate images.                                          for a certain item ik ;
                                                                   – pii k : the number of images uploaded by the team for item
   The gamification part of the hybrid mechanism was im-           ik ;
plemented and thus tested in a preliminary validation iter-        – I: the number of items in the challenge chosen by the
ation. The second part is to be implemented in the next            team;
larger iterations.                                                 – riik :the number of reliable images uploaded by the team
                                                                   for a certain item IT i ;
3.1   Structure of the festival
                                                                   – ciik : the number of correct images uploaded by the team
The paper uses the following concepts and notations:               for a certain item IT i .
– Festival: event that bundles together different sub-events            We propose the implementation of the gamification part
sharing the same organization and the same period and              as a photo hunting festival where participants have the po-
area.                                                              ssibility to choose among several challenges. We are inter-
– Challenge: sub-division of the festival that stands as a         ested in attracting different types of audience. Therefore,
separate competition and consists of an individualized list        each challenge covers various activities and requires differ-
of items that participants need to solve.                          ent skills. The simplest challenges require participants to
– Item: description of a location that participants are re-        go to certain locations in the city and take pictures. The
quired to capture through more pictures.                           most difficult ones add a degree of creativity, knowledge of
– Checked item: an item for which the participants com-            history and physical skills.
pleted all the requirements (i.e., the route of the participants        Motivating people to participate at the festival is done
includes the location of that item and they also provided          both implicitly through the photo taking action, the me-
pictures of it).                                                   chanics of the festival and the social aspect (getting to-
– Representative and diverse images: A photo is consid-            gether, competing, entertainment, etc.), but also explicitly
ered to be representative for the location if it depicts it in a   using clear incentives. The latter is achieved through a mas-
clear, distinctive and focused manner the uniqueness of that       ter class on photography that is held before the opening of
location. Anybody knowing the appearance of the location           the festival by an expert team. They will explain the par-
and seeing the image should easily recognize it.                   ticipants how to take quality images with different devices
   Other images are also accepted as being representative:         and also which are the expectations regarding their images.
images taken in different moments of the year (long-term                All challenges run against time (typically, 2-3 hours).
temporal variation) and/or under different weather condi-          Also, the teams have the freedom of creating their own
tions; images taken during different moments of the day            path, as long as they approach and capture the required lo-
(short-term temporal variation); edited images (creative           cations. The physical approach of locations is verified u-
sing a smartphone application that participants are required        3.2   Players’ reputation
to install prior to the beginning of the challenge. All chal-
                                                                    A large number of participants and the resulting compe-
lenges are considered complete only if a given number of
                                                                    tition may attract low quality results and inadequate be-
locations in the list are captured (e.g., 9 out of 10 locations).
                                                                    haviour, like cheating. Given both our purpose of gathering
    The challenges running in the festival are:                     high quality diverse photos and the expected large amount
                                                                    of user generated content, it is mandatory to find a mecha-
1. Walking Challenge – can be seen as urban hiking. Par-            nism to moderate the festival. The moderation refers to a-
ticipants are given a list with the exact name and address          ssessing the quality of the uploads and further pre-filtering
of different locations to take pictures of. No vehicle is al-       them and computing the reputation of the participants. In
lowed.                                                              our case, the entire process is performed using the auto-
                                                                    mated media analysis as described in Section 3.
2. Creativity Challenge – participants will receive a list              Thus, the criteria that we are searching for in the teams’
of riddles with locations. Neither the name, nor the address        uploads that further allow us to extend the reputation of
of the locations will be provided. They are further asked           the participants are quality, correctness and reliance. Apart
to guess the locations behind the riddles and show their            from contributing to the reputation computation, those ima-
answers through physical approach and pictures of those             ges that do not follow the criteria of quality and correctness
locations. No vehicle is allowed.                                   are automatically filtered out.
                                                                        Quality in images, as computed in equation (1), refers
3. Complete the Picture Challenge – participants will re-           to a set of requirements with a big impact on images a-
ceive a list of photo-riddles. A photo-riddle is an incom-          ppearance. The requirements relate to the quality of the
plete picture from which the participants must figure out           image itself, disregarding the content significance: sharp-
the concrete location and take other pictures. For example,         ness, alignment, no faces in focus. Thus, we accept only
they have to take the whole picture given a certain part, to        sharp, aligned and with-no-faces-in-focus images.
take the right part given the left, take the front given the
back, etc. Neither the name, nor the address of the loca-                                          1 X qiik
                                                                                       quality =                              (1)
tions will be provided. No vehicle is allowed.                                                     I i piik

While Creativity and Complete the Picture Challenges are            The value of correctness as computed in equation 2 judges
active, the participants have also the possibility to create        the capture of the images during the challenges. The par-
and upload riddles for the other participants. In order to          ticipants might try to cheat by uploading images that do
ensure that the riddles uploaded by the participants are co-        not belong to them. Thus, we verify both the list of loca-
rrect and solvable, the organizers check them before releas-        tions approached by the teams and the EXIF in the photos.
ing. However, in case the number of riddles uploaded by             In case they upload an image with a certain location they
the participants cannot be handled by the organizers, they          “pretend” to have reached, but which does not belong to
will be uploaded unchecked to the application. Participants         their tracked path, it means they are trying to “game” the
will be warned about the unchecked riddles.                         challenge.
                                                                                                    1 X ciik
                                                                                    correctness =                          (2)
4. Time Travel Adventure Challenge – participants will                                              I i piik
be provided historical pictures with locations as photo-
                                                                    Asking participants to tag the images they provide is a
clues and a time-slot referring to the building time. Teams
                                                                    modality of collecting additional information and, also,
will have to recognize the locations and to bring new ones.
                                                                    judging the reliance of their images (see equation 3). Thus,
Neither the name, nor the address of the locations are pro-
                                                                    for each location, the teams are asked to judge a series of
vided. No vehicle is allowed.
                                                                    mandatory suggested tags and further to freely add, if they
                                                                    want, some extra-tags. The mandatory tags describe the
5. Bike Challenge – is similar to the walking challenge,            name of a location and will be provided according to the
but the participants are allowed to use bicycles, as a non-         position of a team (recorded by the smartphone applica-
polluting vehicle. The locations cover larger areas com-            tion). They are of two types:
pared to the other challenges.                                      – checked tags: tags specifically chosen incorrect (i.e., de-
                                                                    scribing a location that is obviously too far from the actual
6. Journalism Challenge – allows participants to act like           position of the team). They are meant to test the credibility
observers and take pictures of the most important moments           of the team and will not be attached to the images.
and aspects of the festival. Part of these pictures will be         – unchecked tags: tags describing a location close to the ac-
used for advertising the event. Funny pictures, with as             tual position of the team. However, the localization system
many participants as possible are favoured.                         is sometimes not accurate enough to distinguish between
locations close one to each other. Thus, the team might
need to select one tag among several located in the same
small area. The tags provided by the teams will or will not
be accepted, according to how well they succeed to filter
the checked tags.
                                                                       1) 1893             2) 1923              3) 1864
                               1 X riik
                  reliance =                             (3)          Figure 2: Items for Time Travel Adventure Challenge.
                               I i piik
                                                                Challenge (46.15%), then Walking, Creativity, Bike Chal-
All three variables range from 0 to 1. The three variables      lenges (30.76%), further to Complete the Picture Challenge
have equal contribution in computing the reputation of each     (26.92%) and lastly to Journalism Challenge (3.84%). In
team:                                                           the same time, we collected the contributions of the parti-
                                                                cipants that had the possibility to provide ideas about new
                                                                challenges or to modify the current ones.
 reputation = (quality+correctness+reliance)/3 (4)
                                                                   Finally, contributors were questioned about the addi-
                                                                tional activity of creating new riddles for other partici-
4     Validation results                                        pants in Creativity and Complete the Picture Challenges.
4.1   Validation of festival’s structure                        Thus, 46.15% of the contributors for the first, respectively
                                                                57.69% for the second one declared this activity makes
Optimal results collected through the festival require a fine   them wanting more to participate.
structure tuning. Thus, we conducted a study that helps to         For the second study, a majority of 56% agreed with the
identify and build the best structure for each challenge and    definition of representativeness, while 56% strongly agree
of the entire festival.                                         with the diversity.
    Another study was conducted with the purpose of eva-
luating and improving the representativeness and diversity
                                                                4.2   Description of the implementation
definitions presented in Section 3.1.
Both studies were conducted between 24th September and          A first pilot test was organized in Bucharest, Romania for
24th October 2014 and they were distributed online to 34        testing all the requirements and problems that may occur in
people. In total, 26 contributors responded for the first       large scale festivals. In this iteration we only implemented
study and 25 for the second one. All of them were located       the gamification part from the hybrid approach described in
in Romania.                                                     Section 3. Thus, we launched 3 out of 6 challenges as pre-
    The questions requiring the contributors’ agreement to-     sented in Section 3.1: Walking, Creativity and Time Travel
ward a certain idea/definition use a 7 points Likert scale      Adventure Challenges (see samples below).
with the options: “Strongly disagree”, “Disagree”, “Mildly          Each of the three challenges contained 7 items, thus a to-
disagree”, “Neither agree, nor disagree”, “Mildly agree”,       tal of 21 locations to be captured. The locations referred to
“Agree”, “Strongly agree”.                                      different museums, administrative buildings, monuments,
    For the first study, contributors were initially asked to   squares, etc. from Bucharest, more or less known to the
present their level of familiarity with the concept of sca-     public. 7 mixed teams (individual or with 2 people) parti-
venger hunt. Results indicate that the majority (42.30%)        cipated at all three challenges after being presented a set of
of contributors never heard of scavenger hunt. Further,         participation rules. Samples from the three challenges are
contributors either have heard, but know nothing about          presented below.
it (15.38%), or know something, but never participated
                                                                Challenge 1: Walking Challenge
(26.92%). Finally, only 15.38% of the contributors par-
                                                                1. Old Princely Court and Church (Palatul şi Biserica
ticipated to a scavenger hunt.
                                                                Curtea Veche), str. Franceză 25-31.
    Further, we asked the contributors about their level of
                                                                2. The Linden Tree Inn (Hanul cu Tei), str. Lipscani 63-65.
agreement towards different scenario elements, like the
                                                                3. Curtea Berarilor, str. Şelari 9-11, Bucureşti.
idea of team participation (50% Agree), photography tu-
torial before the opening of the festival (69.23% Strongly      Challenge 2: Creativity Challenge
Agree) and also for the installation of a smartphone appli-     1. Find a very touristic pedestrian street named after
cation prior the beginning of the festival (50% Agree that      Leipzig.
the application is useful).                                     2. Find the oldest “recently in use” hotel building in Bucha-
    Further, we presented each of the six available cha-        rest that kept the initial 19th century inns structure, despite
llenges for which we asked them to rate to which extent         the repeated restorations.
they would be interested to participate. Thus, they showed      3. Find “the City of the Cross”, patronised by the saints
their interest in competing first at Time Travel Adventure      that orthodoxism celebrates in one week from today.
         a - from challenges              b - from Flickr                c - from challenges          d - from Google Images
                                    Figure 3: a), b) The Comedy Theatre; c), d) Curtea Berarilor.
           Table 1: Results collected per challenge.                                  Table 2: Results per challenge.
                        av. nb. of           av. nb. of                                                                  Time
                                                                                                   Walking Creativity
                    collected images       correct images                                                               Tr. Adv.
  Challenge 1              8.3                  8.2                  nb. of imgs                     407         375      388
  Challenge 2             7.65                  6.5                  imgs/team/item                  8.3         7.65    7.91
  Challenge 3             7.91                 7.53                  correct imgs/team/item          8.2          6.5    7.53
                                                                     imgs/team/reached item          8.3         8.62    8.88
Challenge 3: Time Travel Adventure Challenge                         correct items                   6.85        5.14    5.71
Figure 2 depicts items for the Time Travel Adventure Cha-
llenge (including the building year).                               average number of correctly collected images.
                                                                        As expected, challenges requiring additional skills (Cre-
4.3   Results of the pilot test                                     ativity and Time Travel Adventure Challenges) collected a
During the festival, we collected a total number of 1170            lower average number of images compared to the simpler
images for all 21 locations. We want to check the moti-             ones (Walking Challenge). Moreover, the average num-
vation in the gamification part, before spending time on            ber of correctly identified and captured location is 5.9 out
implementing the entire application and the media analy-            of 7, given the fact that each of the three challenges was
sis techniques. Thus, for validation and analysis purposes          considered to be finished if 6 out of 7 locations were co-
we manually checked the results. We discovered a major-             rrectly captured. Additionally, Table 2 presents the results
ity of 1089 images were correct. Apart from being both              obtained individually by each of the three challenges.
representative and diverse images, the images are of high               To have a subjective measure of performance, two pairs
quality, despite the non-professional status of the partici-        of image-sets are illustrated in Figure 3 as a visual com-
pants. The rest of 81 images were classified as being inco-         parison between the results obtained through our approach
rrect, not because they were not representative and diverse         (Figures 3a and 3c) and the first results retrieved from
for the captured locations, but because it happened that the        Flickr10 and Google Images11 (Figures 3b and 3d) in Ja-
participants mixed some locations and they captured dif-            nuary 18th 2015 using as keywords the names of two lo-
ferent locations than those required in the challenges. That        cations (The Comedy Theatre Bucharest and Curtea Bera-
happened partly because of their low experience with the            rilor) captured during the festival. The images collected
locations and partly because of the difficulty and even am-         through our Festival are of high quality, from both repre-
biguity mostly in the Creativity and Time Travel Adventure          sentative and diverse point of view. On the other hand, the
Challenges.                                                         retrieved images depicted in Figure 3b are diverse enough,
   As mentioned in Section 4.2, we asked the participants           but mostly obviously do not depict the query location. In-
to collect a minimum number of 5 pictures per location.             stead, the images from Figure 3d are representative for the
The results show an average number of 7.95 images per               query location, but lack diversity.
location, varying from 0 (for locations not reached by the
teams) to 26 images.                                                5     Conclusions
   Further, eliminating the locations that teams did not
                                                                    In this paper we proposed and evaluated a first part of a
reach, the average number of collected images rises to 8.60.
                                                                    replicable novel hybrid mechanism that combines a photo
From this fact, we can conclude that participants were will-
                                                                    hunting festival for photo generation and media analysis
ing to capture a higher diversity than the required mini-
                                                                    techniques for photo filtering and assessing participants
mum. From all images, an average number of 7.41 co-
                                                                    reputation. Different variations of the approach can be
rrect images were collected per location. The rest of in-
                                                                    run on large scale even with online coordination. In this
correct images comes from those locations confounded or
                                                                    way, many images of high quality can be collected, wi-
not reached. Thus, we were successful in collecting the
                                                                    thout relying on the limited image retrieval and media a-
desired high quality images using our approach.
   Further, Table 1 presents individually for each challenge            10 http://flickr.com

both the average number of totally collected images and the             11 http://images.google.com/
nalysis techniques. The target are locations from all over     Location Sharing Application, SIGCHI Conf. on
the world, from very popular to less known for which there     Human Factors in Computing Systems, 2011.
are not enough online resources. Close future work con-
sists in organizing a large scale version of the festival in
Trento (spring) accompanied by the automated media ana-
lysis technique.

6   Acknowledgments
This research was supported by the Romanian Sectoral
Operational Programme POSDRU/89/1.5/S/62557 and the
Collaborative Project Smart Society: “Hybrid and Di-
versity Aware Collective Adaptative Systems: When
people Meet Machines to Build a Smarter Society”
(http://www.smart-society-project.eu/), funded by the Eu-
ropean Commission’s 7th Framework ICT Programme for
Research and Technological Development under the Grant
Agreement no. 600854.

References
      [1] F. Schroff, A. Criminisi, A. Zissermann. Har-
          vesting Image Databases from the Web, IEEE
          Transactions on PAMI, 2011.

      [2] A. Dia Miron, B. Moisuc, J. Gensel, M.
          Villanova-Oliver, H. Martin. Towards the se-
          mantic and context-aware management of mo-
          bile multimedia, Journal MTAP 53(2):391-429,
          2011.

      [3] B. Ionescu, A. Popescu, M. Lupu, A. L. Gı̂nscâ,
          H. Müller. Retrieving Diverse Social Images at
          MediaEval 2014: Challenge, Dataset and Eval-
          uation, MediaEval 2014.

      [4] X. Zhu, A. Goldberg, J.V. Gael, D. Andrzejew-
          ski. Improving Diversity in Ranking using Ab-
          sorbing Random Walks, Int. Conf. HLT-NAACL,
          2007.

      [5] W. Meng, Y. Kuiyuan, H. Xian-Sheng, Z. Hong-
          Jiang. Towards a Relevant and Diverse Search of
          Social Images, IEEE Transactions on Multime-
          dia 12(8):829-842, 2010.

      [6] L. von-Ahn. Games with a purpose, IEEE Com-
          puter, vol. 39, no. 6, pp. 92-94, 2006.

      [7] I. Celino, D. Cerizza, S. Contessa, D. Dell’Aglio,
          E. Valle, S. Fumeo. Urbanopoly – a social and
          location-based game with a purpose to crowd-
          source your urban data, ASE/IEEE Int. Conf. on
          Social Computing, USA, 2012.

      [8] J. Lindqvist, J. Cranshaw, J. Wiese, J. Zimmer-
          man. I’m the Mayor of My House: Examining
          Why People Use Foursquare - a Social-driven