<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Theory</journal-title>
      </journal-title-group>
      <issn pub-type="ppub">1569-190X</issn>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.1109/BigData.2014.7004425</article-id>
      <title-group>
        <article-title>Distributed Traffic Flow Prediction with Label Proportions: From in-Network towards High Performance Computation with MPI</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Thomas Liebig</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marco Stolpe</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Katharina Morik</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Therefore the</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Dortmund</institution>
          ,
          <addr-line>44221 Dortmund</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2014</year>
      </pub-date>
      <volume>47</volume>
      <issue>0</issue>
      <fpage>10</fpage>
      <lpage>14</lpage>
      <abstract>
        <p>Modern traffic management should benefit from the diverse sensors, smart phones, and social networks data that offer the potential of enhanced services. In disaster scenarios, it is no longer guaranteed that a central server and reliable communication is always available. This motivates a distributed computing setting with restricted communication. Also in distributed High Performance Computing communication costs have to be reduced to the minimum and costly broadcast to all compute nodes hould be avoided. We want to learn local models with high communication efficiency. They still require the exchange of label information in a setting of supervised learning. The transmission of all labels among the nodes can be as costly as communicating all observations. Sophisticated methods are required to trade-off prediction performance against communication costs. We hereby present an in-network algorithm based on local models that only sends label counts to neighboring nodes. method is a novel approach that transfers no data about individual observations, but just agProceedings of the 2 nd International Workshop on Mining Urban</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>gregated label information. We outline its MPI
implementation. And evaluate our approach on
real world data in a traffic monitoring scenario.
Tests reveal that in comparison to sending all
labels, the algorithm is scalable.
authors. Copying permitted for private and academic purposes.</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>Traffic flow prediction is an important task for traffic
managers. It allows performance assessment of major traffic
infrastructure, like roads and junctions. Individual mobility
benefits from predictions, as they provide necessary data
for proactive, smart decisions on individual travel plans,
e.g. predictive situation-aware trip planning by avoidance
of likely traffic hazards (Niu et al., 2015; Liebig et al.,
2014). Traffic flow models are based on sensor
observations of current traffic gained by a mesh of (mostly static)
presence sensors. While existing learning methods
centralize and process measurements on a dedicated traffic
management server, they have some major drawbacks,
particularly in cases of disaster: The need for a reliable
communication infrastructure reduces sustainability in case of
natural hazards.</p>
      <p>First, the server-side collection causes
high communication costs, decreasing the system’s
ability to process all sensor data, in time. Second, the area of
traffic prediction systems is limited by the political area of
homogeneous regulations for sending the data through the
network. Third, increasing the network’s density bares the
risk of re-identification of individual persons and tracking
them throughout the network. Existing systems are
therefore limited by communication bandwidths, processing
capabilities and political regulations.</p>
      <p>We tackle these limitations by a distributed spatio-temporal
in-network learning algorithm, where sensors compute
local models and efficiently communicate label counts
with their topological neighbors.</p>
      <p>Our approach sends
space-time aggregated values that, by design, provide
kanonymity. Hence, our method is privacy preserving and
can be applied for large-scale traffic management
scenarios. Our particular focus is on the prediction of future
traffic flow at junctions throughout the region of interest (e.g.
a city, a state or even areas at European scale). Possible
applications comprise, for instance,
• distributed car-to-car scenarios where cars or trucks
communicate at junctions the number of observed
vehicles at the road to estimate traffic flow and
alter their individual transportation plans based on
predicted traffic conditions, or,
• large scale traffic flow prediction that processes
massive local observations on a high performance
computer.</p>
      <p>Scalable in-network algorithms belong to the field of
distributed data mining. Existing work mostly focuses on
horizontally partitioned data. There, full observations, i.e. all
features and labels, are stored on different nodes in a
network. However, network states representing the current
traffic flow are vertically partitioned. Here, only partial
information about observations is stored on different nodes.
Learning and prediction therefore either require the
transmission of observations or labels to other nodes.
Previous work (Das et al., 2011; Lee et al., 2012; Stolpe et al.,
2013) has focused on sending less information about
observations to a central coordinator. Here, we deal with
reducing the amount of labels sent to neighboring peer nodes.
Communication-efficient algorithms for vertical distributed
learning are not just relevant for traffic flow prediction, but
for applications as diverse as intrusion detection,
monitoring production processes or smart grid management. The
main contributions of our work are the following:
1. We introduce a privacy-preserving approach for the
distributed learning of spatio-temporal prediction
models which transfers only aggregated label
information, but no data about individual observations.
2. A connection is drawn between the task of learning
from label proportions and reducing communication
costs in distributed environments, and it is evaluated
on real-world data.
3. We introduce a fast search strategy for the LLP
algorithm (Stolpe &amp; Morik, 2011) and demonstrate its
prediction performance in the context of traffic flow
prediction.</p>
      <p>The next section reviews related work. Section 3 details our
problem setting and introduces a novel approach for the
innetwork training of local models. Section 4 discusses
learning from aggregated label information, discusses its
implementation in using message passing interface (MPI),
analyses its communication cost and aspects of privacy.
Evaluations of our approach can be found in Sect. 5. We finish
with conclusions and outlook on future work.</p>
    </sec>
    <sec id="sec-3">
      <title>2. Related Work</title>
      <p>Many distributed data mining algorithm learn from
horizontally partitioned data, whereas our data is vertically
partitioned.</p>
      <sec id="sec-3-1">
        <title>In this context, privacy-preserving SVMs</title>
        <p>
          like (Yunhong et al., 2009) are not scalable, since they
send quadratic kernel matrices to a central server.
Distributed optimizati
          <xref ref-type="bibr" rid="ref2">on algorithms (Bellet et al., 2014</xref>
          )
exchange predictions for each observation per iteration,
potentially sending more than the entire dataset. So does a
co-regularized least squares regression in (Brefeld et al.,
2006). Communication-efficient anomaly detection
algorithms (Das et al., 2011; Stolpe et al., 2013) combine local
and global models, but are 1-class algorithms reducing data
sent about observations, not labels. In (Lee et al., 2012),
local support vector machine (SVM) models are trained, but
all labels are sent by a central server.
        </p>
        <p>
          Also in traffic flow prediction, most literature describes
processes on central servers. There are two major ways
to model traffic: using a simulation (Raney &amp; Nagel, 2006)
or applying an imputation model, trained on previous
sensor measurements. Models are required for the estimation
of traffic flow at locations not being observed at all. Such
imputation is not the focus of our study, but the
prediction of traffic flow at sensor locations. We point the
interested reader to methods of simulation (e.g. cellular
automaton (Raney &amp; Nagel, 2006)) and model-based imputation
(e.g. (Liebig et al., 2012)). Most learning-based traffic flow
prediction methods analyse time series, where a popular
model is based on auto-regressive integrated moving
average (ARIMA) (Ahmed et al., 1979). Recently, an
application of a Gaussian Markov Model was proposed in
(Schnitzler et al., 2014), and more advanced graphical models,
namely Spatio-Temporal-Random-Fields (STRFs), were
applied to traffic modeling in (Piatkowski et al., 2013).
Distributed approaches comprise an approach that applies
kNN and Gaussian Process Regressi
          <xref ref-type="bibr" rid="ref2">on (Chen et al., 2014</xref>
          ),
on-line distributed prediction of traffic flow in a large-scale
road network (Wang et al., 2014), distributed traffic
modeling in a MapReduce framework (Chen et al., 2013),
Mapreduce parallel multivariate regressi
          <xref ref-type="bibr" rid="ref2">on (Dai et al., 2014</xref>
          ) and
MPI (Message Passing Forum, 1994) based high
performance computati
          <xref ref-type="bibr" rid="ref2">on based on SVM (Yang et al., 2014</xref>
          ). Few
distributed approaches combine sketches of neighbouring
sensors to get probabilistic estimates of the number of
vehicles co-occurring at different locations. Instead of counting
and re-identifying individual vehicles, we use aggregated
quantities.
        </p>
        <p>
          The task of learning from aggregated label information was
first introduced in (Ku¨ck &amp; de Freitas, 2005).
Theoretical bounds have only recently been proven in (Yu et al.,
2014). (Musicant et al., 2007) propose variants of
existing algorithms. The SVM optimization problem has been
adapted to the setting (Ru¨ping, 2010; Yu et al., 2013).
Mean Map (Quadrianto et al., 2009) estimates the mean
operator solving a system of linear equati
          <xref ref-type="bibr" rid="ref2">ons, while
(Patrini et al., 2014</xref>
          ) extend it with a manifold
regularization, outperforming both SVMs and Mean Map on
standard datasets. A modified Kernel k-Means algorithm (Chen
et al., 2009) minimizes the distance to the given label
proportions by matrix factorization. Recent work learns
Bayesian network (Hernndez-Gonzlez et al., 2013) and
generative (Fan et al., 2014) classifiers. The LLP algorithm
proposed in (Stolpe &amp; Morik, 2011) first determines
clusters and then tries to label them. LLP only has linear
running time, while its prediction performance competes with
the approaches in (Quadrianto et al., 2009; Ru¨ping, 2010)
and(Chen et al., 2009).
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>3. Distributed Learning of Spatio-Temporal</title>
    </sec>
    <sec id="sec-5">
      <title>Local Models</title>
      <p>Given are m distributed sensor nodes P1, . . . , Pm. Each
sensor node Pi delivers an infinite series of real-valued
measurements . . . , vt(−i)1, vt , vt+1, . . . for different time
(i) (i)
points . . . , t − 1, t, t + 1, . . ..</p>
      <p>Time spans between two
measurements are equidistant, given a constant sample rate.
Let t denote the current time of measurement, while t − a
and t+a are time points a steps in the past and future. Each
sensor node also has a spatial location.</p>
      <p>Many traffic flow management tasks require the prediction
of traffic flow categories, that are achieved by a
discretization of raw values into distinct intervals (e.g. risk level
assignment or decision for emergency traffic signals).The
task, given the current time point t, is therefore to predict a
label y from a set Y = {Y1, . . . , Yl} of distinct categories
at some arbitrary node Pi at future time point t + r, based
on the current and previous (raw) sensor readings at all or
a subset of nodes P1, . . . , Pm.</p>
      <p>We assume that for learning, measurements and labels
are somehow recorded (see below) over a fixed-length
time period.</p>
      <sec id="sec-5-1">
        <title>For the supervised training of prediction</title>
        <p>models, each node Pi thus provides a sequence Vi =
hv1(i), . . . , vn(i)i of measurements, vj
Li = hy1(i), . . . , yn(i)i of labels yj
(i) ∈ Y .</p>
        <p>(i) ∈ R, and a sequence
DISTRIBUTED LEARNING OF LOCAL MODELS
Instead of centralizing all data, we propose that each Pi
records and stores its own measurements and labels. For
predicting future traffic flow categories at node Pi, we
restrict learning to Pi itself and c topological neighboring
nodes around Pi. For instance, to learn and predict the
future type of traffic flow at some street junction, considered
are only measurements and labels recorded at the junction
itself and at c junctions closest to it.</p>
        <p>Before training, each Pi preprocesses measurements Vi as
follows.</p>
        <p>A window of size p is slided over the series
Vi with step size 1, storing all thereby created windows
x(i) = {vt(−i)p+1, . . . , vt(i)}, t = p, . . . , n as rows in a
t
dataset Di. Let N (i) = {n(1i), . . . , n(ci)} be the set of
indices for the c neighboring nodes around Pi. Based on
the datasets Di, Dn(i) , . . . , Dn(ci) and labels Li, we want
1
to learn a local function (model) f (i) that, given windows
n(i)
xt(i), xt 1 , . . . , xtn(ci) of sensor readings from node Pi and
its neighbors, predicts the label yt(+i)r at node Pi with
horizon r correctly.</p>
        <p>n(i)
Interpreting windows xt(i), xt 1 , . . . , xtn(ci)
single observation x that should be classified, the data is
vertically partitioned, since each neighboring node of Pi
only stores partial information about x, i.e. a subset of
feaas features of a
tures.</p>
        <p>An obvious choice for the training of f (i) at Pi is to ask
for the recorded measurements at each neighboring node,
concatenate their columns at Pi and join the labels stored
at Pi to the new dataset. The approach is more scalable
than centralizing all data, since the number c of neighbors
is fixed, avoiding the bottleneck problem of limited
bandwidth. However, each node still needs to transmit all
measurements to each of its neighbors, consuming at least as
much energy per node as sending all data to a single server.
Therefore, we propose to send only label information from
node Pi to its neighbors and to train models f0
and fn(i()i) , . . . , fn(i(c)i) at its neighbors. As model f (i) at node
1
Pi, we propose a majority vote over predictions from itself
and its neighboring nodes. All models are local, since they
(i) at node Pi
only consider measurements and labels of a fixed number
of close topological neighboring nodes around Pi.
Moreover, the approach works fully in-network without a central
coordinator, since each node only communicates with its
neighboring peer nodes. As learners at each node, one may
consider supervised learners, like kNN, Decision Trees or
SVMs. Considering the limited computational resources of
sensor nodes, however, our evaluation in Sect. 5 is solely
based on kNN.</p>
        <p>Since the number of bits to encode all labels is often less
than an encoding of all measurements, communication is
saved by sending labels from Pi instead of measurements
to Pi. However, supervised learning still requires
individual labels for all observations. The question is if
communication can be reduced even further, by sending fewer labels
or aggregated label information to each neighboring node.
Semi-supervised (Chapelle et al., 2006) and active
learning (Balcan et al., 2010) show that training on fewer labels
may achieve a similar performance as training on all labels.
However, such methods do not preserve the privacy of the
data, since they need individual labels of observations (see
Sect. 4). Instead, we propose to send only aggregated
label information, i.e. label counts, to neighboring nodes for
learning.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>4. Aggregation of Label Information</title>
      <p>Before sending label information to each of its
neighboring nodes, Pi divides its time-related sequence Li of labels
into consecutive batches C1(i), . . . , C(i) of a fixed size b
h
(see Fig. ??). It respects the prediction horizon r, such that
each C(i) consists of labels from time point t + (j − 1)b + r
j
to t + jb + r and align correctly with time points of
observations (i.e. windows of measurements) at other nodes.
Let n be from here on the size of datasets Di, i.e. the
number of windows stored. Then, h is ⌈n/bs⌉. For each
batch j, labels y ∈ Y are aggregated by counting how
often they occur, and stored in a h × l matrix of label counts
Q(i) = (qj(id)), where qj(id) = |{y ∈ Cj |y = Yd}| .
n(i)</p>
      <p>n(i)
Let Pn(ei) be a neighboring node receiving label counts from
Pi. Pn(ei) transforms Q(i) into a label proportion matrix
Π(i) = (πj(id)) = qj(id)/b, i.e. the counts of labels are divided
by batch size b. Since every node knows b and r, Pn(ei) can
partition its own windows x1 e , . . . , xnn(ei) of measurements
n(i)
into batches B1 e , . . . , Bhe . Since the sender respects r,
the time spans used for aggregating the labels align
correctly with the windows of measurements stored at Pn(ei) .
The learning task at node Pn(ei) now consists of learning a
model fn(i(e)i) , only based on its batches of (unlabeled)
measurements and the label information from node Pi, stored
in the label proportion matrix Π(i), such that the expected
prediction error over individual observations is minimized.</p>
      <sec id="sec-6-1">
        <title>This task is also known as learning from label proportions.</title>
        <p>Several methods have been developed to solve the task (see
Sect. 2). Considering the limited computational resources
of sensor nodes, the LLP algorithm (Stolpe &amp; Morik, 2011)
looked most promising for our evaluation in Sect. 5, since
LLP has a linear running time and its centroid model a
small memory footprint.</p>
        <p>Moreover, it can handle
multiclass classification problems as they arise in traffic
monitoring. However, we found that it still needs to be improved
for scalability issues and performance. The next section
describes LLP shortly, while Sect. 4 introduces a new local
search method.</p>
        <sec id="sec-6-1-1">
          <title>THE LLP ALGORITHM</title>
          <p>LLP learns from label proportions by first clustering all
observations and then assigning labels to each cluster. The
task of cluster analysis consists of partitioning a set of
observations into a set C of k disjunct groups (clusters)
C1, . . . , Ck, such that the similarity of observations in each
cluster is minimized. LLP relies on the idea that
observations having the same class also share similar features, i.e.
that clusters somehow correspond to classes. LLP allows
for several clusters per class and assumes that the majority
of elements of a cluster belongs to the same class. Once
given a clustering the only remaining problem is to assign
correct labels to each cluster.
defined as μ(x) = argminCk∈C ||x − ck||2 .</p>
          <p>More formally, let μ : X → C be a mapping that assigns
an arbitrary observation x ∈ X to a cluster C ∈ C. For
centroids c1, . . . , ck found with k-Means, μ(x) would be
n(i)
Further, let ℓ : C → Y be a mapping which assigns a label
λ ∈ Y to each cluster C ∈ C. For ease of notation, let f
denote model fn(i(e)i) to be learned at node Pn(ei) , Bi denote the
batch B e and Π denote matrix Π(i). f is the composition
i
of mappings ℓ and μ, i.e. f = ℓ ◦ μ.</p>
          <p>With prediction model f , entries γjd of a model-based
proportion matrix Γf = (γjd) can be calculated as
1
|Bj | x∈Bj
γjd =</p>
          <p>X I(f (x), Yd), I =
1 : f (x) = Yd
0 : f (x) 6= Yd
.</p>
          <p>The LLP algorithm now minimizes the mean squared error
MSE(Π, Γf ) =</p>
          <p>X(πjd − γjd)2 ,
1
h</p>
          <p>X
hl j=1 d=1
l
(1)
(2)
between the given label proportion matrix Π and the
modelbased proportion matrix Γf by trying different label
mappings ℓ.</p>
          <p>A LOCAL SEARCH STRATEGY WITH MULTISTARTS
The LLP algorithm as introduced in (Stolpe &amp; Morik,
2011) can work with different cluster algorithms and
labeling strategies. LLP with an exhaustive labeling strategy,
called LLPexh in the following, tries all possible labelings
of the clusters.</p>
        </sec>
        <sec id="sec-6-1-2">
          <title>We found it too time-consuming for the</title>
          <p>evaluations done in Sect. 5. The greedy strategy proposed
in (Stolpe &amp; Morik, 2011) didn’t achieve sufficient
accuracies for traffic prediction. Hence, a better search strategy is
demanded.</p>
          <p>We propose a local search that is started multiple times with
different random combinations of labels. LLP with this
search strategy will be called LLPlsm in the following. The
local search greedily improves on the current labeling of
clusters by trying all possible labels at each component of
a labeling vector λ. Fitness measures how well the
modelbased label proportion matrix Γf , as calculated from the
current labeling, matches the given label proportions. If
the fitness improves, the search starts from the first
component of the labeling vector λ, again. Otherwise, it resets
the label at the current position kpos to the label of the best
(local) solution found so far. Returned is the best labeling
found over all starts of the different greedy searches.
In each iteration, the greedy search runs until no further
improvement is possible. Moreover, at each step of the
algorithm, the fitness either improves or is staying the same
(which is a stopping criterion). Therefore, each search finds
a local minimum. Since the number of searches is finite, the
returned labeling vector is also locally minimal. In
comparison to LLPexh, it cannot be guaranteed that a globally
optimal solution is found. However, with regard to the
prediction results presented in Sect. 5, we found that a local
search performed sufficient enough, despite a much lower
running time.</p>
          <p>LLP as introduced in (Stolpe &amp; Morik, 2011) combines the
MSE with two other error measures. However, we found
that the use of these additional measures decreases the
accuracy in the traffic monitoring scenario. Hence, all
experiments in Sect. 5 are based on the MSE, only. Similarly, we
abstain from the evolutionary feature weighting presented
in (Stolpe &amp; Morik, 2011), since it would heavily increase
the algorithm’s running time.</p>
        </sec>
        <sec id="sec-6-1-3">
          <title>MPI IMPLEMENTATION</title>
          <p>We explicitly focus on the implementation with the
Message Passing Interface (MPI) (Message Passing Forum,
1994) as message passing in the top super computers in the
top500 list1 base on the MPI standard. MPI implements
the single program multiple data paradigm (Darema, 2001)
whereas every node of a distributed system executes the
same program but uses different data. To coordinate this
architecture a MPI program consists of 1 master node and
multiple slave nodes that perform computations, the results
are collected at the master. The art of MPI programming is
to divide the problem into multiple tasks that are transferred
to the slaves and processed thereby. Usually, the number of
tasks exceeds the number of slave nodes and the
distribution of the tasks has to be organized by the master.
The work in (Nupairoj &amp; Ni, 1994) analyses the
performance of such MPI systems and reveals that
communication is major bottleneck in MPI programs. A succeeding
publication (Piernas et al., 1997) provides empirical
estimates for computation of communication costs depending
on message lengths. Based on this publications two major
conclusions ban be made: (1) The shorter the messages the
lower the communication cost, and(2) broadcast messages
1http://www.top500.org/lists/2014/11/, last accessed May, 1st
should be avoided. Our algorithm respects both findings
and therefore seems suitable for an MPI implementation.
For ease of development we decided for the Cran-R
package Rmpi (Yu, 2002) which provides basic MPI
functionalities in Cran-R, thus matrix operators can be applied to the
data. Our program comprises the following generic steps:
1. Load Rmpi, and spawn slaves
2. Definition of the functions for the master
3. Definition of necessary functions for the LLP
algorithm at the slave</p>
        </sec>
        <sec id="sec-6-1-4">
          <title>4. Initialization of the data</title>
          <p>5. Send required data and functions to the slaves</p>
        </sec>
        <sec id="sec-6-1-5">
          <title>6. Tell slaves to execute their function</title>
          <p>7. Communicate with the slaves to perform computation</p>
        </sec>
        <sec id="sec-6-1-6">
          <title>8. Collect the results</title>
        </sec>
        <sec id="sec-6-1-7">
          <title>9. Close slaves and quit</title>
          <p>For the learning from label proportions, our
implementation presumes a shared network file system and initially
processes the data at the master such that the sliding
windows of the measurements are stored as Robjects on the
file system. Every task gets its pointer to the
corresponding slice of data and the label proportions of neighbouring
nodes. The LLP algorithm is executed in every task and the
trained models (cluster centers and their labels) are again
stored physically for later re-use. This also allows the
deployment of the parallel learned models in embedded
devices or the future application of the label proportion
models in high performance computation settings. Next
subsection analyses the communication cost of LLP in
comparison to kNN algorithm.</p>
        </sec>
        <sec id="sec-6-1-8">
          <title>ANALYSIS OF COMMUNICATION COSTS</title>
          <p>Each node Pi transmits a matrix Q to each of its
neighboring nodes, consisting of counts for each label Yd ∈ Y and
batch. Such counts may be assumed to be integers. The
maximum value of each integer is b, which means we need
to reserve at most ⌈log2 b⌉ bits for each label. The
number of batches, given n observations, is ⌈n/b⌉. The total
number of bits zAGG for encoding matrix Q is therefore
b
zAGG = l n m ⌈log2 b⌉|Y | .</p>
          <p>In comparison, the number of bits zALL required to encode
all labels of n observations, for |Y | different labels, is at
most
(3)
(4)
zALL = n⌈log2 |Y |⌉ .
The total costs are then either zAGG or zALL, multiplied by
the number of nodes m. Here we assume that label
information is broadcast to each neighboring node, which is not
unrealistic for sensors in topologically close regions. All
payloads reported in Sect. 5 base on this assumption.</p>
        </sec>
        <sec id="sec-6-1-9">
          <title>ANALYSIS OF PRIVACY</title>
          <p>The vulnerable data are the original sensor readings. These
traffic flow measurements bare the risk of re-identification
of individual vehicles. For example in a dense sensor
network with sparse observations of vehicles, their occurrence
may be tracked throughout the network. As mobility often
is a regular behaviour and contains patterns this risk is even
higher. In this section we show that our LLPlsm-based
algorithm transforms the data such that re-identification risk
is at most 1/s.</p>
          <p>In our distributed setting, adversaries of a particular sensor
node are malicious sensors that could use received
measurements of neighboring sensors for deduction of
individual mobility traces. The following attack model is
possible: The adversary analyses differences among
neighboring sensor readings and deduces individual movement. If
the difference among two neighboring sensor readings is
zero and both traffic flow counts are w, it is (depending
on network topology) likely that w vehicles moved
between the two sensors. In case of three neighboring sensors
Pa, Pb, Pc their measurements va, vb, vc can be combined
as follows: If va − vb = w = vc it may be deduced that
on the way from Pa to Pb w vehicles turned to Pc, in case
va − vb = −w = −vc w vehicles originated from the
location Pa.</p>
          <p>With our new LLPlsm-based approach we process
discretized traffic flow values and just communicate counts of
these value ranges. We denote the minimal (nonzero)
interval width by s. Thus, measurements may not be
distinguished up to a granularity of s vehicles and w is bounded
by s, w</p>
          <p>≥ s. In turn, the risk of re-identification with
the hereby described attack model is at most 1/s. Our
approach therefore provides s-anonymity by design. The
aggregation of label information reduces the remaining risk
for disclosure of neighboring labels at a malicious sensor
node.</p>
          <p>
            The solely transmission of label counts prevents
doubtless reconstructi
            <xref ref-type="bibr" rid="ref2">on of the labels (Yu et al., 2014</xref>
            ).
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>5. Experiments</title>
      <p>We perform tests of the method on data of the city of
Dublin. The Sydney Coordinated Adaptive Traffic
System (SCATS) provides information on vehicular traffic at
over 750 fixed sensor locations as spatio-temporal time
series (McCann, 2014). The data we use2 is a snapshot from
2Data is publicly available at http://dublinked.ie .
01/01/2013 till 14/05/2013, consisting of tuples (t, u, w),
where u is the location of the observation and consists of
an index for the junction, the arm and the lane number at
which the sensor is located at. The metric w contains the
aggregated vehicle count at sensor location since last
measurement. The time stamp t denotes the recording time.
95 Accuracy of kNN vs. LLP-lsm with dif erent aggregations
Payload of kNN vs. LLP with dif erent aggregations
90
65 kNN LLP-25 LLP-50 LLP-75 LLP-100
0 kNN LLP-25 LLP-50 LLP-75 LLP-100
model
model
15, 50 different random starting points, 500 iterations at
maximum) and label the clusters with LLPlsm (with 150
starts of the local greedy search) at each node for different
batch sizes b = 25, 50, 75 and 100. The accuracy of each
method is assessed by a 10-fold cross validation, i.e. all
models are trained and evaluated for different hold-out sets
10 times. In total 296 × 7 × 10 = 20, 720 models for kNN
need to be evaluated and 296 × 7 × 10 × 4 = 82, 880
models trained and evaluated for LLPlsm. The evaluation has
been done offline in parallel on different machines (about
36 CPU cores).</p>
      <p>Figure 1 shows the trade-off between accuracy and
payload sent for kNN and LLPlsm trained on differently sized
batches of aggregated labels. Besides the average accuracy
over all 10-fold cross-validations at each node, the bars in
Fig. 1 (left) also depict the standard deviation of accuracy
over all nodes.</p>
      <p>In general, LLPlsm performs slightly worse than kNN.
Nevertheless, there are still many junctions for which the traffic
flow is predicted quite well with LLPlsm. Some locations
have bad performance with both methods, a comparison to
the map reveals that these are locations of parking areas
e.g. inner-city parking houses and recreational areas where
many vehicles stay for a long period of time.</p>
    </sec>
    <sec id="sec-8">
      <title>6. Conclusions</title>
      <p>The task of scalable traffic flow prediction involves a
tradeoff between the accuracy of models and the amount of
communication between networked nodes. Especially in high
performance computation and embedded devices
commu</p>
      <sec id="sec-8-1">
        <title>Chen, S., Liu, B., Qian, M., and Zhang, C.</title>
      </sec>
      <sec id="sec-8-2">
        <title>Kernel k</title>
        <p>nication is costly.</p>
        <p>In this paper we presented a novel approach for local
models that trades-off communication costs to prediction
accuracy which is suitable for in-network deployment and
cluster computations.</p>
        <p>Future work will focus on examining more sophisticated
aggregation strategies for labels. We will study how to
include dynamic distributed traffic flow prediction in
stateof-the-art (multi-modal) route planning methods proposed
by (Bast et al., 2014).</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>Acknowledgements</title>
      <p>This research has received funding from the European
Union’s Seventh Framework Programme under grant
agreement number FP7-318225, INSIGHT. Additionally,
this work has been supported by Deutsche
Forschungsgemeinschaft (DFG) within the Collaborative Research
Center SFB 876, project B3. We thank Jan Czogalla for data
preprocessing.
mental. Analysis of Freeway Traffic Time Series Data</p>
      <sec id="sec-9-1">
        <title>Using Box and Jenkins Techniques. 1979.</title>
        <p>Balcan, M.-F., Hanneke, S., and Vaughan, J. W. The true
sample complexity of active learning. Machine
Learning, 80(2–3):111–139, 2010.</p>
        <p>Bast, Hannah, Delling, Daniel, Goldberg, Andrew,
Mu¨llerHannemann, Matthias, Pajor, Thomas, Sanders, Peter,
Wagner, Dorothea, and Werneck, Renato. Route
planning in transportation networks. Technical Report
MSRTR-2014-4, January 2014.</p>
        <p>Bellet, A., Liang, Y., Garakani, A. B., Balcan, M.-F., and
Sha, F. Distributed Frank-Wolfe algorithm: A unified
framework for communication-efficient sparse learning.</p>
        <p>CoRR, abs/1404.2644, 2014.</p>
        <p>Brefeld, U., Ga¨rtner, T., Scheffer, T., and Wrobel, S.
Efficient co-regularised least squares regression. In Proc.
of the 23rd Int. Conf. on Machine Learning (ICML), pp.
137–144, New York, NY, USA, 2006. ACM.</p>
        <p>Chapelle, O., Scho¨lkopf, B., and Zien, A. Semi-Supervised</p>
        <sec id="sec-9-1-1">
          <title>Learning. MIT Press, Cambridge, MA, 2006. Chen, Cheng, Liu, Zhong, Lin, Wei-Hua, Li, Shuangshuang, and</title>
        </sec>
        <sec id="sec-9-1-2">
          <title>Wang, Kai. Distributed modeling in a mapreduce framework for data-driven traffic flow fore</title>
          <p>casting. Intelligent Transportation Systems, IEEE
Transactions on, 14(1):22–33, 2013.</p>
          <p>Means based framework for aggregate outputs
classification. In Proc. of the Int. Conf. on Data Mining
Workshops (ICDMW), pp. 356–361, 2009.</p>
          <p>Chen, Xing-Yu, Pao, Hsing-Kuo, and Lee, Yuh-Jye.
Efficient traffic speed forecasting based on massive
heterogenous historical data. In Big Data (Big Data), 2014</p>
        </sec>
      </sec>
      <sec id="sec-9-2">
        <title>IEEE International Conference on, pp. 10–17, Oct 2014.</title>
        <p>
          Dai, Liang, Qin, Wen, Xu, Hongke, Chen, Ting, and Qian,
Chao. Urban traffic flow prediction: A mapreduce based
parallel multivariate linear regression approach. In
Intelligent Transportati
          <xref ref-type="bibr" rid="ref2">on Systems (ITSC), 2014</xref>
          IEEE 17th
        </p>
      </sec>
      <sec id="sec-9-3">
        <title>International Conference on, pp. 2823–2827, Oct 2014.</title>
        <p>Darema, Frederica. The spmd model: Past, present and
future. In Recent Advances in Parallel Virtual Machine
and Message Passing Interface, pp. 1–1. Springer, 2001.
Das, K., Bhaduri, K., and Votava, P. Distributed anomaly
detection using 1-class SVM for vertically partitioned
data. Stat. Anal. Data Min., 4(4):393–406, 2011.
Fan, K., Zhang, H., Yan, S., Wang, L., Zhang, W., and
Feng, J. Learning a generative classifier from label
proportions. Neurocomput., 139:47–55, 9 2014.</p>
        <p>Hernndez-Gonzlez, J., Inza, I., and Lozano, J. A.
Learning bayesian network classifiers from label proportions.</p>
        <p>Pattern Recognition, 46(12):3425–3440, 2013.</p>
        <p>Ku¨ck, H. and de Freitas, N. Learning to classify individuals
based on group statistics. In Proc. of the 21th UAI, pp.</p>
        <p>Lee, S., Stolpe, M., and Morik, K. Separable approximate
optimization of support vector machines for distributed
sensing. In Machine Learning and Knowledge
Discovery in Databases, volume 7524 of LNCS, pp. 387–402,</p>
        <sec id="sec-9-3-1">
          <title>Berlin, Heidelberg, 2012. Springer-Verlag.</title>
          <p>Liebig, Thomas, Xu, Zhao, May, Michael, and Wrobel,
Stefan. Pedestrian quantity estimation with trajectory
patterns. In Machine Learning and Knowledge
Discovery in Databases, pp. 629–643. Springer Berlin
Heidelberg, 2012.</p>
          <p>Liebig, Thomas, Piatkowski, Nico, Bockermann, Christian,
and Morik, Katharina. Predictive trip planning - smart
routing in smart cities. In Proceedings of the Workshops
of the EDBT/ICDT 2014 Joint Conference (EDBT/ICDT
pp. 331–338. CEUR-WS.org, 2014.
McCann, Barry. A review of scats operation and
deploySchnitzler, Franc¸ois, Liebig, Thomas, Mannor, Shie, and
2014.
1994.
2007.
ment in dublin. In Proceedings of the 19th JCT
Traffic Signal Symposium &amp; Exhibition. JCT Consulting Ltd,
Message Passing Forum. Mpi: A message-passing
interface standard. Technical report, Knoxville, TN, USA,
Musicant, D. R., Christensen, J. M., and Olson, J. F.
Supervised learning by training on aggregate outputs. In</p>
        </sec>
      </sec>
      <sec id="sec-9-4">
        <title>7th Int. Conf. on Data Mining (ICDM), pp. 252–261, 10</title>
        <p>Niu, Xiaoguang, Zhu, Ying, Cao, Qingqing, Zhang,
Xining, Xie, Wei, and Zheng, Kun.</p>
        <p>An
online-trafficprediction based route finding mechanism for smart city.</p>
      </sec>
      <sec id="sec-9-5">
        <title>International Journal of Distributed Sensor Networks,</title>
        <p>501:970256, 2015.</p>
        <p>Nupairoj, Natawut and Ni, Lionel M. Performance
evaluation of some mpi implementations on workstation
clusters. In Scalable Parallel Libraries Conference, 1994.,
Proceedings of the 1994, pp. 98–105. IEEE, 1994.
Patrini, G., Nock, R., Caetano, T., and Rivera, P. (almost)
no label no cry.</p>
      </sec>
      <sec id="sec-9-6">
        <title>In Advances in Neural Information</title>
        <p>Processing Systems 27, pp. 190–198. Curran Associates,
Inc., 2014.</p>
        <p>Piatkowski, Nico, Lee, Sangkyun, and Morik, Katharina.</p>
        <p>Spatio-temporal random fields: compressible
representation and distributed estimation. Machine Learning, 93
(1):115–139, 2013. ISSN 0885-6125.</p>
        <p>Piernas, Juan, Flores, A, and Garc´ıa, Jose´ M. Analyzing the
performance of mpi in a cluster of workstations based
on fast ethernet. In Recent advances in Parallel
Virtual Machine and Message Passing Interface, pp. 17–24.</p>
        <p>Quadrianto, N., Smola, A. J., Caetano, T. S., and Le,
Q. V. Estimating labels from label proportions. J. Mach.</p>
        <p>Raney, B. and Nagel, K. An improved framework for
largescale multi-agent simulations of travel behavior.
Towards better performing European Transportation
Sys</p>
        <p>Ru¨ping, S. SVM classifier estimation from group
probabilities. In Proc. of the 27th Int. Conf. on Machine Learning
(ICML), pp. 911–918, 2010.</p>
        <sec id="sec-9-6-1">
          <title>Morik, Katharina.</title>
        </sec>
        <sec id="sec-9-6-2">
          <title>Combining a gauss-markov model and gaussian process for traffic prediction in dublin city center.</title>
        </sec>
      </sec>
      <sec id="sec-9-7">
        <title>In Proceedings of the</title>
      </sec>
      <sec id="sec-9-8">
        <title>Workshops of</title>
        <p>the EDBT/ICDT 2014 Joint Conference (EDBT/ICDT
pp. 373–374. CEUR-WS.org, 2014.
detection in vertically partitioned data by distributed
core vector machines.</p>
      </sec>
      <sec id="sec-9-9">
        <title>In European Conf. on Ma</title>
        <p>chine Learning and Knowledge Discovery in Databases
(ECML/PKDD), pp. 321–336. Springer, 2013.</p>
        <sec id="sec-9-9-1">
          <title>Stolpe, Marco and Morik, Katharina.</title>
        </sec>
        <sec id="sec-9-9-2">
          <title>Learning from label proportions by optimizing cluster model selection.</title>
          <p>ence on Machine Learning and Knowledge Discovery in</p>
        </sec>
      </sec>
      <sec id="sec-9-10">
        <title>Databases - Volume Part III, ECML PKDD’11, pp. 349–</title>
        <sec id="sec-9-10-1">
          <title>Traffic flow prediction model for large-scale road network based on cloud computing.</title>
          <p>2014.
S. ∝SVM for learning with label proportions. In Proc.
504–512, 2013.</p>
          <p>Yu,</p>
          <p>Hao.
ing in r.</p>
        </sec>
        <sec id="sec-9-10-2">
          <title>Rmpi:</title>
        </sec>
        <sec id="sec-9-10-3">
          <title>Parallel</title>
          <p>statistical
comput2002.</p>
          <p>URL</p>
        </sec>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Ahmed</surname>
            ,
            <given-names>M.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cook</surname>
            ,
            <given-names>A.R.</given-names>
          </string-name>
          ,
          <source>of Oklahoma</source>
          . School of Civil Engineering, University, and Science,
          <source>EnvironIn Proceedings of the 2011 European Confer364</source>
          , Berlin, Heidelberg,
          <year>2011</year>
          . Springer-Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <article-title>On-line distributed prediction of traffic flow in a largescale road network</article-title>
          .
          <source>Simulation Modelling Practice and Mathematical</source>
          Problems in Engineering,
          <year>2014</year>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>F. X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jebara</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <surname>Chang</surname>
          </string-name>
          , Shih-Fu.
          <article-title>On learning with label proportions</article-title>
          .
          <source>CoRR, abs/1402</source>
          .5902,
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>F. X.</given-names>
          </string-name>
          <string-name>
            <surname>Yu</surname>
            , Liu,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jebara</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <source>and Chang, of the 30th Int. Conf. on Machine Learning (ICML)</source>
          , pp.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <article-title>Rnews 2002-2</article-title>
          .pdf.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>