=Paper=
{{Paper
|id=Vol-3028/D1-02-ESAAMM_2021_paper_4
|storemode=property
|title=Context Aware Software Stacks for Mobility
|pdfUrl=https://ceur-ws.org/Vol-3028/D1-02-ESAAMM_2021_paper_4.pdf
|volume=Vol-3028
|authors=Naci M. Dai,Deniz Memis,Burak Saglam
}}
==Context Aware Software Stacks for Mobility==
https://ceur-ws.org/Vol-3028/D1-02-ESAAMM_2021_paper_4.pdf
Context Aware Software Stacks for Mobility
Composive.ai Overview
Naci M. Dai Deniz Memis Burak Saglam
Eteration Eteration Eteration
Istanbul, Turkey Istanbul, Turkey Istanbul, Turkey
naci.dai@eteration.com deniz.memis@eteration.com burak.saglam@eteration.com
Abstract—Humans are very good at adapting to and driving in There are still many challenges to this learning process as
very different conditions and environments. Complex AI based huge learning data leads to many difficult to explain biases
software such as Autonomous vehicles (AV) are biased by the and edge cases where the algorithms come up with unexpected
training data and they do not do well in different contexts.
Environment perception and context awareness can be used to results. Deep neural networks learn from the data to cleanly
adapt and deploy complex networks of AI systems that are separate the dataspace for the intrinsic dimensionality (nicely
expected to perform safely on many different environments and demonstrated by Colah’s Neural Networks, Manifolds and
edge cases. Typical Autonomous Drive (AD) software stacks Topology [1]), and require careful engineering and modelling
provide localization, object detection and tracking using an to do this efficiently. These systems have very little or no
array of sensor data. Additional sources of information such as
V2X connectivity, passenger behavior, person-to-device mapping, contextual capability, or ability to abstract knowledge or
urban context, degradation of traffic etc. complements the sensor reason about the drive [2]. DARPA recently announced the
data that can be used to describe a context. This context is used Next AI challenge, and is focusing its investments on a third
to predict and adapt the system of software and applications that wave of AI that brings forth machines that understand and
are deployed to the vehicle. reason in context [3] (See Fig.1.).
Context Aware Software Stack for Mobility (CASSM) de-
The Contextual Adaptation means the systems construct
scribes a model based approach for definition of a software stack
for complex systems such as those found in autonomous vehicles. explanatory models for classes of real world phenomena and
Machine Learning is used to detect a context and adapt the the explanations and the context can be used to adapt both
software stack, which can then be used to identify and deploy the learning models and in our case the overall AD software
services to autonomous vehicles. CASSM will be an open source stack.
in-vehicle delivery of context-driven services and applications.
This paper describes an architecture to define context-aware
personalized & dynamically provisioned capabilities, applications
& services to autonomous vehicles.
Index Terms—Autonomous Vehicles, Context Awareness,
Adaptive Software Stacks, Machine Learning
I. I NTRODUCTION
Experienced human drivers are good at adapting to different
driving conditions and environments. They can instinctively
transfer their earlier learning to the current driving context.
The advent of big data and availability of computing allowed
statistical deep learning methods to power innovation in many
applications including AD. AD software are complex systems Fig. 1. DARPA - The Next Wave of AI (Figure Adapted from John
of various interconnected AI and deterministic modules. These Launchbury)
software do very well in the context that they are designed
and trained for, but perform poorly in other environments. Humans are very successful at understanding the context
Starting with Darpa challenges, statistical learning methods such as parking, urban driving, or driving fast on a highway,
have been used in implementing very successful AD software. abstracting experiences and transferring ideas and reacting
The quality of these solutions depend on their statistical appropriately. However, for AD software the design, capabili-
models and the big data that is used to train them. These ties, sensors and actions taken for these contexts can differ
systems are very good at learning, perceiving and adapting significantly. Even urban driving in a well organized city
to the environment. These cars can successfully, and safely versus a city with narrow streets (i.e. Istanbul) will require
drive on highways and in the cities, interacting with the training with completely different datasets, probably using
environment, the traffic, other cars and pedestrians. different models.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
We can define the context as the set of all the inputs from the C. Context Awareness
sensors, and additional inputs from the environment, people’s Numerous context-aware systems have been presented in
behavior, even if they are not used in the AD stack. The action the literature, they albeit differ in the way context related
of driving takes place and all these inputs interact with the data is acquired and utilized; the main commonalities of these
system and change its behavior. It involves the driver, the systems’ structure could be listed as below:
vehicle physics, its parts, brakes, steering, mechanical com-
• Acquiring information.
ponents, the weather condition, the destination etc. Context-
• Manipulating the information (filtering, compressing,
awareness means that a service, although given the same
transforming) to contextually represent the system.
request parameters, is perceived differently with respect to a
• Interpreting the context.
given context [4]. In autonomous driving context-awareness
• Distributing information based on the contextual interpre-
means, (1) Semantics and modelling the context for which
tation.
AD software is trained and designed for and (2) Using the
information about the context to adapt how AD perceives, In the AV domain, these systems have been utilized in a
predicts, and acts. variety of applications such as ”route planning and learn-
ing [7]–[9], real time perception of traffic congestion [10],
II. C ONTEXTUAL AI AND C ONTEXT AWARENESS FOR collision avoidance [11] and finding vacant car parks [12]”.
AUTONOMOUS V EHICLES Considering these use cases that provide a service to the user
A. Contextual AI and above mentioned system characteristics there are two key
Contextual AI is focused on providing a service or infor- aspects relevant to context-awareness: First is the knowledge
mation to the user that is relevant to the acquired (explicit acquired about the context of the service to be provided and
or implicit) context information [5]. This approach allows the second is to adjust the service to be contextually aware
contextually-aware systems to provide tailor-made services considering the information within the context.
and solutions under varying circumstances. There are mainly The first absolute benefit of context-awareness for AV, as
four pillars to Contextual AI, based on these the system must is true for all autonomous systems, is adaptation [13]. Adap-
be [5]: tation implies modifying a service that will, given the same
• Intelligible: Ability to explain what it knows, how it
parameters, yield a relevant distinguished output depending
knows and what it is doing. on the context and the objective. This may include different
• Adaptive: Ability to meet user’s needs and expectations
realizations such as refining or tracking other information,
under varying environmental circumstances. invoking or inhibiting other services. The other and relatively
• Customizable: Ability to be controlled fully by the user.
more sophisticated benefit compared to other autonomous
• Context-aware: Ability to perceive at a human level.
systems is proactivity, currently provided reactive services
work upon the invocation of the user (a pull type service), on
Even for the most successful AD software, its capabilities
the other hand proactive services could deliver the information
are still bounded by its compatibility to the current context
without any explicit action required by the user. Furthermore,
and the constraints it was trained for, making such systems
these proactive services could keep delivering based on the
heavily dependent on human intervention when it is faced with
predicted outcome of future context [13].
individual cases where the inferences and plan fail.
B. Explainable AI III. T HE M ODEL D RIVEN A DAPTIVE S OFTWARE S TACK
Like all other safety and sensitive applications of AI, Simon described means-end analysis with the notion of
Explainability of AD software is strongly related to context- adaptation, which is finding the difference between an existing
awareness. AD software travelled successfully for billions of state and a desired state and then finding the correlating
kilometers but when they fail, the reasons are not easily inter- process that will erase the difference [14].
pretable by a human. Interpreting and explaining is important Our work focuses on software systems that are amenable
to gain trust into the prediction, and to identify potential data to model-driven methods. It is possible to dynamically adapt
selection biases or artefacts. We need to know when an AD these models to better address a given context. A large
system is operating within the limits of its training. These percentage of AD software is based on the Robot Operating
limits identify contexts that the AD is designed to safely and System (ROS). ROS has the concept of computational nodes
reliably operate, and prevent dangerous inferences and outputs. that can be connected with the data flowing between them.
For example, if the perception system cannot classify the intent Many AD implementations involve sensor data, and higher
of a cyclist, being able to explain why the input data led level objects that are produced by sensor-fusion, prediction,
to an incorrect calculation will help in the adaptation of the planning and control tasks that are implemented with ROS.
underlying learning models. The explanations could be related This is typically done with a code-first, development heavy
to the structure or models of the system, of single decisions or practice.
cases, or generalize to the complete overall behavior. Samek et. RobMoSys attempts to address this difficulty by providing
al [6]. provide an extensive review of methods and applications standards for composable models and Software for Robotics
of some of the most recent advances in AI explanations. Systems. We are exploring RobMoSys model-driven approach
to define models for the proven ROS based open-source typical software stack like the ones used by Autoware (See
AD frameworks such as Autoware [15], Baidu Apollo [16]. Fig.4.).
Refactoring existing systems as model-based software will
provide fast and effective feedback for the functionality and
also will prove the platform independence. The metamodel-
driven approach, that composes the building blocks (or a
combination of them) into various systems in a purposeful
way, the context detection models and algorithms (ROS nodes)
will be in use. According to context, these nodes will be
used to create a context-level metamodel and will enable other
nodes to be deployed and to be used.
The contextual adaptation of an AD software stack would
change the active stack to better address the constraints and
trade-offs of the context. Each stack model is mapped to a
context that reflects the conditions of its training data. It is Fig. 4. Autoware AD Software Stack.
conceivable that one can define an “uber” model that would
satisfy most contexts and there will be no need for adaptation, In their work, they improve inferences by using the notions
but this is probably not achievable with today’s understanding. of adapting the composer model and policy preferences. Al-
Instead, we propose a pragmatic context-based adaptation ternatively, MROS [18] leverages the RobMoSys [19] model-
approach that will identify a context based on the inputs of the based approach at runtime, to provide a solution for ROS2
system and dynamically change the software stack (a model- systems, based on architectural self-adaptation driven by on-
based computation graph) to satisfy the requirements in the tology reasoning on the architecture models.
new context (See Fig.2. & Fig.3.). Of course this approach is not limited to AD. The ap-
plications in the mobility space exist in many different do-
mains. Connected cars and smart cities make use of cloud
based services and innovative human computer interactions.
The same context-based adaptation techniques can be applied
to model-based software applications in these domains. For
example, the service orchestrations based on BPMN-like flows
or component-based user interfaces that are amenable to view-
flow like structures would also benefit from this solution.
Composive.ai, will deliver our components and tools for
building adaptive software stacks as open-source software (See
Fig. 2. Metamodel extraction from existing systems. Fig.5.).
Fig. 5. Composiv.ai Adaptive Software Stacks.
Fig. 3. Adaptation using metamodel-driven feedback.
Similar approaches produced promising results for other C ONCLUSION
problems. Odena et.al [17] use reinforcement learning to State of the art AD software makes extensive use of statis-
change model behavior at test-time. They use a composer tical learning methods with proven results. These applications
model, which consists of a set of modules and a controller are typically limited by the context, size and quality of the
network. The composer network is a graph of deterministic and datasets that were used to train them. Contextual adaptivity
stochastic nodes made of neural networks that are organized and reasoning is the natural next step for AI methods to get
into ‘metalayers’. The composer network is very similar to the closer to human like capabilities. In our project, Composive -
Context Aware Software Stacks for Mobility, is a solution for
adapting model-driven complex autonomous systems (such as
AD). Composive will solve the adaptation problem by being
able to define a context and detecting the variations within the
defined context from a repository of pre-existing models using
a metamodel-driven approach, subsequently it will adapt and
deploy other context-driven services that are compatible with
the future contextual model.
IV. ACKNOWLEDGMENTS
This work is being partially funded by The Scientific and
Technological Research Council of Turkey, TEYDEB 1509
grant.
R EFERENCES
[1] http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
[2] Launcbury, John, A DARPA Perspective on Artificial Intelligence, 2017
https://youtu.be/-O01G3tSYpU
[3] AI Next Campaign, https://www.darpa.mil/work-with-us/ai-next-
campaign
[4] Pichler, Mario et al. ”Context-awareness and artificial intelligence”.
ÖGAI Journal 23. (2004): 4-.
[5] Contextual AI: The Next Frontier of Artificial
Intelligence,https://blog.adobe.com/en/2019/04/09/contextual-ai-the-
next-frontier-of-artificial-intelligence.htmlgs.ytw2bv
[6] W Samek, G Montavon, S Lapuschkin, C Anders, KR Müller, Explain-
ing Deep Neural Networks and Beyond: A Review of Methods and
Applications Proceedings of the IEEE, 109(3):247-278, 2021
[7] M. Younes, A. Boukerche, and G. Rom’an-Alonso, “An intelligent path
recommendation protocol (ICOD) for VANETs,” Comput. Netw., vol.
64, pp. 225–242, May 2014.
[8] M. B. Younes and A. Boukerche, “A performance evaluation of a con-
textaware path recommendation protocol for vehicular ad-hoc networks,”
in Proc. IEEE Global Telecommun. Conf. (GLOBECOM), Dec. 2013,
[9] M. Masikos, K. Demestichas, E. Adamopoulou, and M. Theologou,
“Energy-efficient routing based on vehicular consumption predictions of
a mesoscopic learning model,” Appl. Soft Comput., vol. 28, pp. 114–124,
Mar. 2015.
[10] J. Liu et al., “High-efficiency urban-traffic management in context-aware
computing and 5G communication,” IEEE Commun. Mag., vol. 55, no.
1, pp. 34–40, Jan. 2017.
[11] B. Kihei, J. A. Copeland, and Y. Chang, “Automotive Doppler sensing:
The Doppler profile with machine learning in vehicle-to-vehicle net-
works for road safety,” in Proc. IEEE Workshop Signal Process. Adv.
Wireless Commun. (SPAWC), Jul. 2017, pp. 1–5.
[12] A. Alhammad, F. Siewe, and A. Al-Bayatti, “An infostation-based
context-aware on-street parking system,” in Proc. Int. Conf. Comput.
Syst. Ind. Inform. (ICCSII), Dec. 2012, pp. 1–6.
[13] Pichler, Mario Bodenhofer, Ulrich Schwinger, W.. (2014). Context-
awareness and artificial intelligence. ÖGAI Journal. 23. 4-.
[14] Sciences of the Artificial, Herbert Simon, 1996, MIT Press
[15] Autoware Foundation - https://www.autoware.org/
[16] Apollo Auto - https://apollo.auto/
[17] Augustus Odena, Dieterich Lawson, Christopher Olah - Changing Model
Behavior at Test-Time using Reinforcement Learning, Workshop track
- ICLR 2017 https://arxiv.org/pdf/1702.07780.pdf
[18] https://robmosys.eu/mros/
[19] https://robmosys.eu Composable Models and Software for Robotics
Systems