<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>ACM RecSys</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>You are What You Eat! Tracking Health Through Recipe Interactions</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Alan Said</string-name>
          <email>alansaid@acm.org</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alejandro Bellogín</string-name>
          <email>alejandro.bellogin@uam.es</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>TU-Delft</institution>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Universidad Autónoma de Madrid</institution>
          ,
          <country country="ES">Spain</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2014</year>
      </pub-date>
      <volume>10</volume>
      <abstract>
        <p>On today's World Wide Web, social recommender systems have become a commodity regardless of application domain. Even tangible items such as food and clothes have become social. Together with a seemingly endless amount of personalization and recommender systems ranging from movies, music, or consumer products, recipe recommender systems are attracting many users looking for inspiration on the next thing to purchase or cook. There is however a conceptual difference between recommending consumer goods for leisure and entertainment, and recommending food. What people eat has a direct effect on their health, an aspect commonly overlooked in the context of recommendation. In this work, we present an early analysis of users' interactions with recipes (ratings) on the online social network Allrecipes.com. We compare the interaction patterns of users from locations known to have poor health to users from locations known to have good health in order to identify whether there is an observable difference between the two populations. Our results point to a statistically significant difference between the healthy and unhealthy groups, a difference that could potentially be used to create health-conscious, personalized, recommendation services to aid people in their daily lives.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Categories and Subject Descriptors</title>
      <p>1.</p>
    </sec>
    <sec id="sec-2">
      <title>INTRODUCTION</title>
      <p>Today, Internet users turn to the Web for help with the planning and
selection of many daily tasks; whether what music to listen to
(Spotify), what consumer products to purchase (Amazon), what movies
to watch (Netflix), or what food to prepare (Allrecipes). Consumers
put a considerable amount of trust into systems which are able to
simplify their information needs, no matter the type of information
(or products) sought for. Often, these online services implement
persuasion systems telling the users to buy, listen to, watch, or even
eat items or products that their peers have interacted with. It should
however be noted that there is a distinct conceptual difference in
recommending a piece of information to be consumed online, e.g.
a news article or a song, and a tangible object, e.g. a computer or
a car. Among the differences between the types of objects, we find
aspects such as consumption cost (in terms of money, time, effort),
the expected longevity of a product (a music track lasting a few
minutes, a book lasting a week, a car lasting several years), etc.
These aspects need to be accounted for when creating a
personalized experience, whether for an online consumption case, or for a
real-world product.</p>
      <p>In turn, when recommending food and recipes, there is an
additional dimension of the recommendation that needs to be
considered: the health aspect of what is being recommended to a specific
user. A personalization system which has a (more or less) direct
effect on the user’s daily life and health, such as a recipe
recommender, needs to be aware of the potential outcome of the
recommendation, not only in terms of increased business value for the
vendor and the general utility as experienced by the consumer, but
also of the well-being of the consuming user.</p>
      <p>It is because of the above stated aspect that we, in this paper,
focus on health aspects involved in personalizing users’ experiences
in a food-related online social network. We do so by taking into
account the general health in the area where the user lives. By
using data from County Health Rankings &amp; Roadmaps1 in
combination with data from the recipe-focused online social network
Allrecipes2 we are able to show that there is a significant
difference in consumption patterns between users from counties with a
high health ranking and users from counties with a low health
ranking. Our motivation is that these differences can be used to identify
users with higher health risks, even in cases where the geographical
location is not known.</p>
      <p>The main contribution of our work is to show a significant
correlation between recipe usage on an online social network and the
reported health in users’ geographic locations.
1www.countyhealthrankings.org
2www.allrecipes.com</p>
    </sec>
    <sec id="sec-3">
      <title>RELATED WORK</title>
      <p>
        Over the last decade, a massive body of work on multimedia
recommender systems has been accumulated, e.g. movies [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]
music [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], online news [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], and practically any other type of consumer
products [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Food recommendation on the other hand, which also
has been an online phenomenon for a long time, has only recently
started gaining attraction from information system and
personalization researchers and practitioners, e.g. improving the food
preparation competence of cooks [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], dinner planning for groups [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ],
educating potential cooks on healthy foods [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] or diversifying the
meals served in care facilities [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        When personalizing the culinary experience, it is important to be
aware of the conceptual difference between recommending a movie
to watch or a song to listen to, compared to recommending a dish
to eat or cook. The movies one watches and songs one listens to
have no direct effect on the health of the subject receiving
recommendations. Recommending food on the other hand, as mentioned
in Section 1, means that the recommendation will indeed have an
effect on the user’s health, either by simply proposing the user to
eat something unhealthy directly, or, by attempting to altering a
user’s (long term) food habits – which might remain even after the
user is no longer using the service. However, there exists only a
limited body of work on food recommendation and personalization
from a health-oriented aspect, e.g. Hsiao and Chang [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] show that
by aiding in planning meals it is possible to improve the health of
a system’s users. Some research approaches food recommendation
from the perspective of diet and exercise [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], attempting to
understand the users’ reasoning around recipes. More recently, Harvey
et al. [
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ] reported on a study attempting to identify the factors
that affect the ratings given to recipes in order to leverage this
information in a recipe recommender system able to recommend recipes
which are not only nutritional, but also well-liked by the users.
      </p>
      <p>
        In this work, we base our finding on geographical areas with
good or bad health, inspired by the line of research known as Health
Geography [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Here, Dummer showed that “Geography and health
are intrinsically linked" [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. With this in mind, we attempt to find
whether it is possible to use concepts from information
management and human-computer interaction to alleviate potential health
effects in online recommendation services even when the location
of the user is not known.
      </p>
    </sec>
    <sec id="sec-4">
      <title>RECIPES &amp; HEALTH DATA</title>
      <p>To perform our analysis, we scraped the recipe-related social
network Allrecipes.com. In this process, we collected user profiles,
recipes, ingredients, recipe boxes (users collect and rate their recipes
in virtual recipe boxes making them easily accessible at later points
in time), social connections, and demographic information on users
(location, interests, hobbies, etc.). This data collection3 was
performed during October 2013, and resulted in a dataset containing
information on more than 170 thousand users, 54 thousand recipes,
8; 400 ingredients, and 17 million recipe box assignments (which
we refer to as ratings4).</p>
      <p>Having collected the data, we used health rankings by county
from County Health Rankings to identify users living in healthy and
unhealthy counties. Our health focus was specifically on obesity,
i.e. the percentage of adults suffering from obesity in each county.
3The scripts used to scrape the data from the Allrecipes website are
available at github.com/alansaid/RecipeCrawler
4Even though users can rate the recipes they put in their recipe
boxes (if they wish), in the scope of this paper we have only
analyzed the binary relationships between users and recipes.</p>
      <p>The health ranking dataset contains data for more than 3; 400 US
counties, including the percentage of obese adults.</p>
      <p>The dataset collected from Allrecipes does not contain the
counties where users live in. In order to connect users to counties, we
used a mapping of 42; 000 US cities to 3; 200 US counties5. This
allowed us to link the recipe and health datasets to each other. It
should be noted that users of the Allrecipes social network do not
have to state their hometown, and when they choose to do so, this
is done in free text. The implication is that it is not possible to
automatically map all users to counties, e.g. some users state made up
cities, or local slang names (Chicagoland for Chicago, The Big
Apple for New York, etc.), or simply misspell the name of their
hometown. Additionally, large cities (e.g. Dallas, TX) may be
composed of several counties, making the mapping of these cities onto
distinct counties problematic unless additional information is
available or manual mapping is performed. Furthermore, the counties
in the county health ranking dataset and the city-to-county
mapping dataset do not overlap perfectly, as noted above the county
health data contains 3; 400 counties whereas the county mapping
data contains 3; 200. However, with some manual tuning
(replacing e.g. Hollywoodland with Hollywood, The Big Apple with New
York City, etc.) we were able to infer the counties for the majority
of the users.
4.</p>
    </sec>
    <sec id="sec-5">
      <title>MAPPING UNHEALTHY INGREDIENTS</title>
    </sec>
    <sec id="sec-6">
      <title>TO HEALTH DATA</title>
      <p>In order to analyze whether it is indeed possible to use the county
health ranking data in combination with food-oriented websites,
e.g. Allrecipes, we focused on a relatively small number of healthy
and unhealthy counties.</p>
      <p>As a first step, we identified how often a certain ingredient is
used by users in a certain county. This was accomplished by
mapping each recipe onto its composing ingredients, and
correspondingly mapping all ratings given by users (per county) on the recipes
onto the ingredients of the recipes. This process war repeated for
the one hundred and ten most used ingredients in each county.
Following this, we calculated the percentage of how often an
ingredient was used in average in the counties with low obesity and high
obesity separately. This information allowed us to identify the five
5www.farinspace.com/us-cities-and-state-sql-dump
most obese and five least obese counties with available
ingredient data. Due to the mapping procedure and dataset described in
the previous section, the five counties with the lowest percentage
of obese adults selected were within the top 15 of the least obese
counties. Similarly, the counties with the highest percentage of
obesity were within the top 100 of the most obese counties. The
top counties together with statistics for each are shown in Table 1.</p>
      <p>
        It should be noted that the geographic distribution of the counties
is not limited to an isolated geographical location within the US,
instead the counties are spread throughout the country, as shown
in Fig. 1. This should further strengthen the health aspect of the
analysis, while minimizing potential effects of local food trends
found in isolated geographical locations [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
    </sec>
    <sec id="sec-7">
      <title>ANALYSIS &amp; RESULTS</title>
      <p>For each group of counties, i.e. with high and low obesity
percentage, we identified the top 110 most popularly used ingredients in
both types of counties, i.e. the top intersecting ingredients used
by users in both types of counties. Table 2 shows the 20
ingredients used most often in counties with high (") obesity and the
corresponding percentage in counties with low (#) obesity. Having
this information, we performed a statistical significance analysis
(t-test) on the vectors containing the percentages of how often the
ingredients were used in both type of counties (the same
ingredients appearing in the same places in both vectors). The justification
of this is that, if the ingredients were in fact used differently in the
two types of counties, we should be able to distinguish between
high and low risk users independent of their geographical location.
Thus ensuring that high/low-risk users can be identified by their
online recipe interaction patterns.</p>
      <p>The obtained p-value from the t-test (p &lt; 0:05) confirms that the
ingredient usage in counties with high obesity is in fact different
from that of counties with low obesity. The implication of this is
that high-risk/low-risk users can be identified simply by their recipe
interactions in an online social network. This information can in
turn be used to personalize a food recommendation system based
on the recorded interactions of a user.
6.</p>
    </sec>
    <sec id="sec-8">
      <title>DISCUSSION</title>
      <p>In the previous sections, we have described our analysis of a
healthrelated dataset and an analysis of a real-world recipe-focused online
social network. Our results point to that it is possible to identify
users from high-risk (poor health) areas just from their recipe
interactions. This suggests that, should a recommendation system be
employed, it can be tailored to not only provide high-quality recipes
to the user, but also take into consideration the potential health
aspects of the user. The health effects can be mitigated by either
filtering out recipes which can be deemed unhealthy, or to create
personalized recipes – by altering the doses of certain ingredients
– and still fulfilling the users’ expectations. This needs however
be done in such a way as to not lower the usability and quality of
the system, as perceived by the user. A personalization approach
of this type would serve as an insurance that the service would not
be the cause of, or aiding to, any detrimental effects on the users’
health. Given the increasing quality of recommender systems, a
system being conscious of the (inferred) health of its users appears
as a plausible next step.</p>
      <p>We are aware of the limits of our analysis, e.g. only analyzing the
binary connections between a recipe and an ingredient – not taking
into consideration the amount of the ingredient used. Nevertheless,
we believe our results to be indicative of what can be attained when
using the ingredient amount as well. This is currently the focus of
our ongoing work, however, the ambiguous and non-standardized
unit and ingredient declaration in recipes, e.g. one cucumber, half
a cup of sugar, one glass of water, two crackers, etc., makes this a
non-trivial task.</p>
      <p>It should be noted that the results obtained in our analysis are the
result of early work, we do however believe that this is a feasible
approach to proactively care for the users of similar food- or
otherwise health-oriented services. As mentioned in Section 1, there is
a conceptual difference between recommending an
entertainmentfocused item (song, movie) compared to domains where the
personalization system has a direct effect on the user’s health.</p>
    </sec>
    <sec id="sec-9">
      <title>CONCLUSION &amp; FUTURE WORK</title>
      <p>In this work, we have analyzed a recipe dataset and combined it
with data reporting health aspects in US counties. We have
identified counties that suffer from poor health (large percentage of adults
suffering from obesity) and found that there exist statistically
significant differences in how users from poor health counties interact
with recipes compared to users from counties with good health (low
percentage of adults suffering from obesity). Our work suggests a
potential approach to health-oriented recommender systems which
takes into account the possible adverse effects on a user, based on
demographic information as well as through information on the
recorded interactions (ratings) with the system.</p>
      <p>As for future (and current) work paths, we are currently
investigating whether there are other user-related features that also
correlate to health aspects, e.g. inferring health through stated interests
and hobbies. Similarly, we intend to investigate whether the social
ties (follower/followee relationships) between users, a concept that
has been proven to be useful in personalization and
recommendation approaches in other domains, hold similar health-related
information. Additionally, we plan to study whether the nutritional
aspects of ingredients can help in identifying health-oriented aspects
in individual users.</p>
    </sec>
    <sec id="sec-10">
      <title>ACKNOWLEDGMENTS</title>
      <p>This work was in part carried out during the tenure of an ERCIM
“Alain Bensoussan” Fellowship Programme. The research leading
to these results has received funding from the European Union
Seventh Framework Programme (FP7/2007-2013) under grant
agreement no.246016.</p>
      <p>The authors would like to thank Arjen P. de Vries and Jacco van
Ossenbruggen from CWI for feedback during the work resulting in
this paper.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>X.</given-names>
            <surname>Amatriain</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Basilico</surname>
          </string-name>
          .
          <article-title>Netflix recommendations: Beyond the 5 stars (part 1) - the netflix tech blog</article-title>
          . http://techblog.netflix.com/
          <year>2012</year>
          /04/ netflix-recommendations
          <article-title>-beyond-5-stars</article-title>
          .
          <source>html (retrieved May 12</source>
          ,
          <year>2012</year>
          ),
          <year>April 2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>C.</given-names>
            <surname>Anderson</surname>
          </string-name>
          .
          <article-title>The Long Tail: Why the Future of Business Is Selling Less of More</article-title>
          . Hyperion,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>S.</given-names>
            <surname>Berkovsky</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Freyne</surname>
          </string-name>
          .
          <article-title>Group-based recipe recommendations: Analysis of data aggregation strategies</article-title>
          .
          <source>In Proceedings of the Fourth ACM Conference on Recommender Systems, RecSys '10</source>
          , pages
          <fpage>111</fpage>
          -
          <lpage>118</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Ò.</given-names>
            <surname>Celma</surname>
          </string-name>
          .
          <article-title>Music Recommendation and Discovery in the Long Tail</article-title>
          .
          <source>PhD thesis</source>
          , Universitat Pompeu Fabra, Barcelona,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>T. De Pessemier</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Dooms</surname>
            , and
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Martens</surname>
          </string-name>
          .
          <article-title>A food recommender for patients in a care facility</article-title>
          .
          <source>In Proceedings of the 7th ACM Conference on Recommender Systems, RecSys '13</source>
          , pages
          <fpage>209</fpage>
          -
          <lpage>212</lpage>
          , New York, NY, USA,
          <year>2013</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>T. J.</given-names>
            <surname>Dummer</surname>
          </string-name>
          .
          <article-title>Health geography: supporting public health policy and planning</article-title>
          .
          <source>Canadian Medical Association Journal</source>
          ,
          <volume>178</volume>
          (
          <issue>9</issue>
          ):
          <fpage>1177</fpage>
          -
          <lpage>1180</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>J.</given-names>
            <surname>Freyne</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Berkovsky</surname>
          </string-name>
          .
          <article-title>Intelligent food planning: Personalized recipe recommendation</article-title>
          .
          <source>In Proceedings of the 15th International Conference on Intelligent User Interfaces</source>
          ,
          <source>IUI '10</source>
          , pages
          <fpage>321</fpage>
          -
          <lpage>324</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J.</given-names>
            <surname>Freyne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Berkovsky</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Smith.</surname>
          </string-name>
          <article-title>Recipe recommendation: Accuracy and reasoning</article-title>
          .
          <source>In Proceedings of the 19th International Conference on User Modeling</source>
          , Adaption, and Personalization,
          <source>UMAP'11</source>
          , pages
          <fpage>99</fpage>
          -
          <lpage>110</lpage>
          , Berlin, Heidelberg,
          <year>2011</year>
          . Springer-Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Garcin</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Faltings</surname>
          </string-name>
          .
          <article-title>Pen recsys: A personalized news recommender systems framework</article-title>
          .
          <source>In Proceedings of the 2013 International News Recommender Systems Workshop and Challenge</source>
          ,
          <source>NRS '13</source>
          , pages
          <fpage>3</fpage>
          -
          <lpage>9</lpage>
          , New York, NY, USA,
          <year>2013</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>M.</given-names>
            <surname>Harvey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Ludwig</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Elsweiler</surname>
          </string-name>
          .
          <article-title>Learning user tastes: a first step to generating healthy meal plans</article-title>
          ?
          <source>In Proceedings of the ECIR Workshop on Searching4Fun, Searching4Fun '12'</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M.</given-names>
            <surname>Harvey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Ludwig</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Elsweiler</surname>
          </string-name>
          .
          <article-title>You are what you eat: Learning user tastes for rating prediction</article-title>
          .
          <source>In Proceedings of the 20th International Symposium on String Processing and Information Retrieval</source>
          , SPIRE, pages
          <fpage>153</fpage>
          -
          <lpage>164</lpage>
          . Springer,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>J.-H.</given-names>
            <surname>Hsiao</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Chang</surname>
          </string-name>
          .
          <article-title>Smartdiet: A personal diet consultant for healthy meal planning</article-title>
          .
          <source>In Proceedings of the 2010 IEEE 23rd International Symposium on Computer-Based Medical Systems, CBMS '10</source>
          , pages
          <fpage>421</fpage>
          -
          <lpage>425</lpage>
          , Washington, DC, USA,
          <year>2010</year>
          . IEEE Computer Society.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>G.</given-names>
            <surname>Moon</surname>
          </string-name>
          .
          <article-title>Health geography</article-title>
          . In R. Kitchin and N. Thrift, editors,
          <source>International Encyclopedia of Human Geography</source>
          , volume
          <volume>5</volume>
          , pages
          <fpage>35</fpage>
          -
          <lpage>55</lpage>
          . Elsevier,
          <year>July 2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>J.</given-names>
            <surname>Wagner</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Geleijnse, and</article-title>
          <string-name>
            <surname>A. van Halteren. Guidance</surname>
          </string-name>
          <article-title>and support for healthy food preparation in an augmented kitchen</article-title>
          .
          <source>In Proceedings of the 2011 Workshop on Context-awareness in Retrieval and Recommendation</source>
          ,
          <source>CaRR '11</source>
          , pages
          <fpage>47</fpage>
          -
          <lpage>50</lpage>
          , New York, NY, USA,
          <year>2011</year>
          . ACM.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>