<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Use Visual Features From Surrounding Scenes to Improve Personal Air Quality Data Prediction Performance</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Trung-Quan Nguyen</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Dang-Hieu Nguyen</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Loc Tai Tan Nguyen</string-name>
          <email>locntt.12@grad.uit.edu.vn</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Vietnam National University</institution>
          ,
          <addr-line>Ho Chi Minh City</addr-line>
          ,
          <country country="VN">Vietnam</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <fpage>14</fpage>
      <lpage>15</lpage>
      <abstract>
        <p>In this paper, we propose a method to predict the personal air quality index in an area by using the combination of the levels of the following pollutants: PM2.5, NO2, and O3, measured from the nearby weather stations of that area, and the photos of surrounding scenes taken at that area. Our approach uses the Inverse Distance Weighted (IDW) technique to estimate the missing air pollutant levels and then use regression to integrate visual features from taken photos to optimize the predicted values. After that, we can use those values to calculate the Air Quality Index (AQI). The results show that the proposed method may not improve the performance of the prediction in some cases.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>INTRODUCTION</title>
      <p>
        The need to know the personal air pollution data is vital because it
is better to provide each individual with regional air quality data,
which seems to be more accurate than the global data measured
from far away weather stations. The problem is that the
performance of personal air quality prediction mainly interpolated from
public weather data is not good. This paper reports our solution to
tackle this challenge by finding out whether pictures of places can
improve the prediction results. To know more about this challenge
and the dataset that we will use, you can refer to the overview paper
of MediaEval 2020 - Insight for Wellbeing: Multimodal personal
health lifelog data analysis [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
    </sec>
    <sec id="sec-2">
      <title>RELATED WORK</title>
      <p>
        The experiment on using surrounding images to predict the air
quality has been conducted in several projects. For instance,
analyzing the sky images [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] and integrating visual features [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] into
the prediction model to predict the air quality rank are the most
significant projects. Those two projects used neural network
models to perform air quality rank prediction, which is a categorical
variable. Unlike them, this paper will use the IDW method and
the regression model to predict the numerical values of these air
pollutants levels: PM2.5, NO2, and O3.
      </p>
    </sec>
    <sec id="sec-3">
      <title>APPROACH</title>
      <p>
        Because of the time limitation, we have to propose a method that
does not require an incredible training time. At first, we will use
the pure form of IDW technique [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] to predict pollutant levels.
Then, the multiple linear regression will help us to combine these
      </p>
      <sec id="sec-3-1">
        <title>Description</title>
      </sec>
      <sec id="sec-3-2">
        <title>Waterway Sky Morning Tree</title>
        <p>Road surface
Road
River
Walkway
Architecture
Thoroughfare</p>
        <p>Confidence Score
predicted values with an additional visual feature to produce new
pollutant levels.
3.1</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Extract visual features</title>
      <p>We use Google Cloud’s Vision API to extract information about
entities in images. Each image will have a maximum of 10 labels that
have the highest confidence score. For example, Table 1 shows labels
of the image GH030011_005250.jpg. We create a boolean feature from
those labels to define whether that location is an open space or not.
It means that if an image has one of the labels in Table 2, it will be
a picture of an open space area, and therefore, the __
feature has the value of 1 and vice versa. We believe that those
areas usually have better air quality, so it is the reason why we use
the __ attribute as a supplemental input.
3.2</p>
    </sec>
    <sec id="sec-5">
      <title>Produce the prediction</title>
      <p>The first step is to use the IDW to predict pollutant levels of PM2.5,
NO2, and O3 for each hourly time frame from the known values of
pollution data provided by 26 weather stations surrounding Tokyo.
These predicted values will be the first input of our regression
model and the second one is the __ attribute created
when we extract visual features in section 3.1. We continue to fit
the regression model with these two independent variables to make
the prediction.</p>
      <p>Our linear regression model has the following formula:
 = 1 + 2
with  is the value of the pollutant level needs to be predicted,
1 is the value of the pollutant level predicted by IDW, 2 is the
__ attribute, and  ,  are the coeficients. Finding those
coeficients means that the regression model will be fitted.
(1)</p>
    </sec>
    <sec id="sec-6">
      <title>RESULTS AND ANALYSIS</title>
      <p>The evaluation of PM2.5, NO2, and O3 prediction in the case of not
using visual features and vice versa, provided by MediaEval task
organizers are shown in Table 3, Table 4, Table 5, Table 6, Table 7,
Trable 8, respectively.</p>
      <p>In general, PM2.5, O3, and NO2 prediction results are improved,
except for the case of NO2 levels of the two running courses course1test1, REFERENCES
course1test2. The reason behind this could be because we did not
cluster the images of each course separately.
5</p>
    </sec>
    <sec id="sec-7">
      <title>DISCUSSION AND OUTLOOK</title>
      <p>
        We are currently investigating more advanced algorithms, such as
implementing the combination of IDW with multiple regression [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]
and neural network models. Also, we plan to enrich our models with
more weather data, such as wind direction, wind speed, temperature,
to improve accuracy.
Insight for Wellbeing: Multimodal personal health lifelog data analysis
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Zhao P. J Nguyen N.T. Nguyen T.B. Dang-Nguyen D. T. Gurrin C. Dao</surname>
            ,
            <given-names>M. S.</given-names>
          </string-name>
          <year>2020</year>
          .
          <article-title>Overview of MediaEval 2020: Insights for Wellbeing Task - Multimodal Personal Health Lifelog Data Analysis</article-title>
          .
          <source>In MediaEval Benchmarking Initiative for Multimedia Evaluation, CEUR Workshop Proceedings.</source>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>V.</given-names>
            <surname>Roshan</surname>
          </string-name>
          Joseph and
          <string-name>
            <given-names>Lulu</given-names>
            <surname>Kang</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Regression-Based Inverse Distance Weighting With Applications to Computer Experiments</article-title>
          .
          <source>Technometrics</source>
          <volume>53</volume>
          ,
          <issue>3</issue>
          (
          <year>2011</year>
          ),
          <fpage>254</fpage>
          -
          <lpage>265</lpage>
          . http://www.jstor.org/stable/23210401
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Donald</given-names>
            <surname>Shepard</surname>
          </string-name>
          .
          <year>1968</year>
          .
          <article-title>A Two-Dimensional Interpolation Function for Irregularly-Spaced Data</article-title>
          .
          <source>In Proceedings of the 1968 23rd ACM National Conference (ACM '68)</source>
          .
          <article-title>Association for Computing Machinery</article-title>
          , New York, NY, USA,
          <fpage>517</fpage>
          -
          <lpage>524</lpage>
          . https://doi.org/10.1145/800186.810616
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Mohammadsaleh</given-names>
            <surname>Vahdatpour</surname>
          </string-name>
          , Hedieh Sajedi, and
          <string-name>
            <given-names>Farzad</given-names>
            <surname>Ramezani</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>Air pollution forecasting from sky images with shallow and deep classifiers</article-title>
          .
          <source>Earth Science Informatics</source>
          <volume>11</volume>
          (09
          <year>2018</year>
          ). https://doi.org/ 10.1007/s12145-018-0334-x
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>P.</given-names>
            <surname>Vo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Phan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Dao</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Zettsu</surname>
          </string-name>
          .
          <year>2019</year>
          .
          <article-title>Association Model between Visual Feature and AQI Rank Using Lifelog Data</article-title>
          .
          <source>In 2019 IEEE International Conference on Big Data (Big Data)</source>
          .
          <fpage>4197</fpage>
          -
          <lpage>4200</lpage>
          . https://doi.org/10.1109/BigData47090.
          <year>2019</year>
          .9005636
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>