=Paper= {{Paper |id=Vol-2699/paper09 |storemode=property |title=An Adaptive Semantic Stream Reasoning Framework for Deep Neural Networks |pdfUrl=https://ceur-ws.org/Vol-2699/paper09.pdf |volume=Vol-2699 |authors=Danh Le-Phuoc,Thomas Eiter |dblpUrl=https://dblp.org/rec/conf/cikm/PhuocE20 }} ==An Adaptive Semantic Stream Reasoning Framework for Deep Neural Networks== https://ceur-ws.org/Vol-2699/paper09.pdf

An Adaptive Semantic Stream Reasoning Framework for
Deep Neural Networks
Danh Le-Phuoca , Thomas Eiterb
a Technical University Berlin
b Technical University Vienna

Abstract
Driven by deep neural networks (DNN), the recent development of computer vision makes visual sensors such as stereo
cameras and Lidars ubiquitous in autonomous cars, robotics and traffic monitoring. However, due to operational constraints,
a processing pipeline like object tracking has to hard-wire an engineered set of DNN models to a fixed processing logic. To
overcome this, we propose a novel semantic reasoning approach that uses stream reasoning programs for in-cooperating
DNN models with commonsense and domain knowledge using Answer Set Programming (ASP). This approach enables us to
realize a reasoning framework that can adaptively reconfigure the reasoning plan in each execution step of incoming stream
data.

Keywords
Semantic Reasoning, Neural-symoblic, Stream Reasoning, Stream Processing

1. Motivation ple, the recent report of Uber’s accident in Arizona [4]
says "...The ADS detected the pedestrian 5.6 seconds be-
The recent development of computer vision (CV) driven fore impact. Although the ADS continued to track the
by deep neural networks (DNN) makes visual sensors pedestrian until the crash, it never accurately classified
such as stereo cameras and Lidars ubiquitous in au- her as a pedestrian or predicted her path. By the time the
tonomous cars, robotics and traffic monitoring. In par- ADS determined that a collision was imminent (1 second
ticular, many DNN models for object detection [1] and before impact), the situation exceeded the response spec-
tracking [2] are available. However making reliably ifications of the ADS braking system...". This accident
working in a real-life online processing pipeline such could have not happened if the ADS could reconfig-
as an Automated Driving System (ADS) or a traffic ure the object tracking pipeline on the fly, e.g chang-
surveillance query engine is still very challenging. For ing DNN models or using alternative sensor sources to
example, [2] reports that the most accurate DNN-driven improve the accuracy on detection and tracking.
multi-object tracking (MOT) pipelines can process only This motivates us to propose an approach of com-
4-5 frames/second. To make such a system work on- bining stream reasoning with probabilistic inference
line e.g. for ADS, where processing delay must be less to continuously configure such processing pipelines
than 100ms [3], one has to hard-wire a fixed sets of based in semantic information representing common-
DNN models with some sacrifices on accuracy and ro- sense and domain knowledge. The use of semantic in-
bustness as the design constraints of an ADS limit how formation together with DNNs has proved to be useful
much hardware can be put into a system [3]. For in- and led to better accuracy in image understanding [5]
stance, an additional 400 W power consumption trans- and in object tracking [6]. Similar to ours, these ap-
lates to a 3.23% reduction in miles per gallon for a 2017 proaches use declarative approaches to represent the
Audi A4 sedan or similarly, the additional power con- processing pipelines of visual data. However, none of
sumption will reduce the total driving range of electric them have considered how to deal with the aforemen-
vehicles. tioned operational constraints in the context of stream
Such a design-time trade-off often leads to unpre- processing. Our approach represents such constraints
dictable fatal errors in a real deployment. For exam- in an extension of Answer Set Programming (ASP).
This extension is proposed by leveraging LARS formu-
Proceedings of the CIKM 2020 Workshops, October 19-20, Galway, las [7] for expressing stream reasoning programmes to
Ireland. incorporate uncertainty of probabilistic inference op-
email: danh.lephuoc@tu-berlin.de (D. Le-Phuoc);
thomas.eiter@tuwien.ac.at (T. Eiter)
erations under weighted rules similar to LP𝑀𝐿𝑁 [8],
orcid: 0000-0003-2480-9261 (D. Le-Phuoc); 0000-0001-6003-6345 called semantic reasoning rules. As a result, we will
(T. Eiter) be able to dynamically express a visual sensor fusion
© 2020 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0). pipeline, e.g. MOT over multiple cameras, by seman-
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)
tic reasoning rules to fuse probabilistic inference op- b 2
b 4
b 6

erations with ASP-based solving processes. Moreover,
the expressive power of our approach also enables us b 1
b 3
b 5

to express operational constraints together with opti- det(yl,b ,car), trk(39,b ),
…
1 det(yl,b ,car), trk(39,b ),
2
…
3 det(yl,b ,car), trk(41,b ),
4
…
5 6

misation goals as a probabilistic planning programme 1 2 3 t
similar to 𝑝𝐵𝐶 + [9] so that our reasoning framework
can reconfigure the reasoning plan adaptively in each Figure 1: A Semantic Visual Stream Snapshot
execution step of incoming stream data.
reads are missing), based on commonsense principles.
2. Formalization of Semantic We thus use the LARS framework [7] to represent a
reasoning programme over our semantic stream data
Stream Reasoning with DNN which can evaluated using an ASP solver.
Models The LARS framework provides formulas with Boolean
connectives and temporal operators @𝜏 𝛼, □𝛼, and ◊𝛼
Semantically, a multi-sensor fusion data pipeline will to evaluate a formula 𝛼 at a time point 𝜏 (resp. ev-
consume the data that is observed by a Sensor as a ery, some time point) in the current stream 𝑆; win-
stream of observations (represented as an Observation) dow operators ⊞ 𝛼 take for evaluating 𝛼 a data snap-
𝑤

following standardized W3C/OGC Semantic Sensor Net- shot (substream) 𝑆 from 𝑆 by applying a function 𝑤 on
′

work Ontology (SSN) [10] . For instance, the example 𝑆. For example, the formula ⊞ ◊𝑑𝑒𝑡(𝑦𝑙, 𝐵, 𝑐𝑎𝑟) states
+5

in Figure 1 shows 3 image frames are observed by a that at evaluation time 𝜏 , 𝑑𝑒𝑡(𝑦𝑙, 𝐵, 𝑐𝑎𝑟) holds at some
traffic camera. These observations will then be fed into time point 𝜏 in the window [𝜏 − 5, .., 𝜏 ] selected by
′

a probabilistic inference process such as a DNN model 𝑤 = +5; in our representation, 𝑑𝑒𝑡(𝑦𝑙, 𝐵, 𝑐𝑎𝑟) is the
or a CV algorithm (represented as a Procedure) to pro- matching condition for "a car was detected in bound-
vide derived stream elements which then are repre- ing box 𝐵 by the YOLO detector". The formula @𝜏 𝛼
senting Sampling instances. In this example, we have is aligned with a fluent 𝐹 resp. an event 𝐸 in Event
𝑑𝑒𝑡(𝑦𝑙, 𝑏1 , 𝑐𝑎𝑟) representing for the output bounding Calculus (EC) [12], such that @𝜏 𝐹 ≡ ℎ𝑜𝑙𝑑𝑠𝐴𝑡(𝐹 , 𝜏 ) and
box 𝑏1 from the YOLO detector 𝑦𝑙∈𝐷𝑇 where 𝐷𝑇 (short @𝜏 𝐸 ≡ ℎ𝑎𝑝𝑝𝑒𝑛𝑠(𝐸, 𝜏 ). This will help us to employ
for Detector) represents for the set of detectors sup- ASP-based EC axioms for common sense reasoning rules
ported. Similarly, 𝑡𝑟𝑘(39, 𝑏2 ) represents for tracking as proposed in [6].
bounding box 𝑏2 generated by a tracking algorithm (a To deal with uncertainty, we extend LARS with weighted
Tracker) which associates 𝑏2 with the tracklet 39 via rules under LP𝑀𝐿𝑁 semantics as in [8]. In LP𝑀𝐿𝑁 ,
the popular object tracking algorithm SORT [11]. facts generated from feature extractors (DNN models
The symbol FeatureOfInterest (FoI) is used to repre- or OpenCV algorithms) as well as abduction rules on
sent the domain of physical objects which are subjects top can be annotated with uncertainty information given
for the sensor observations, e.g. tracking objects and by a weight, which allows reasoning under certain lev-
field of views (FoV) of the camera. The relationship be- els of uncertainty.
tween a Result generated by probabilistic inference al- For our concerns, a semantic reasoning program Π is
gorithms (e.g. YOLO detection model or Kalman filter a set of weighted rules 𝑟 of the form
algorithm) to such object is represented by the predi-
cate isSampleOf (denoted as iSO). As such algorithms 𝜔∶𝛼 ←𝛽 (1)
have output with uncertainty, we will use an abduc-
where 𝛼, 𝛽 are LARS formulas and 𝜔 ∈ ℝ ∪ {𝑥} is the
tion reasoning process to search for explainable evi-
weight of the rule. If 𝜔 = 𝑥, then 𝑟 is a hard rule, oth-
dences for iSO via rules driven commonsense and do-
erwise a soft rule; by Πℍ and Π𝕊 we denote the sets
main knowledge similar to [6].
of hard and soft rules of Π, respectively. The seman-
To formalise the reasoning process with such a se-
tics of Π is given by the answer streams [7] 𝑆 of the
mantic representation of stream data, we need a tem-
LARS program Π𝑆 obtained from Π by dropping the
poral model that allows us to reason about the prop-
weights and each 𝑟 where 𝑆 violates 𝛼 ← 𝛽; each
erties and features of objects from streams of sensor
such 𝑆 gets a probability 𝑃𝑟Π (𝑆) assigned calculated
observations. This model must account for the laws of
from the weights of the rules retained for Π𝑆 ; for more
the physical world movement and in particular be able
information, we refer to [8]. In Section 3 we will ad-
to fill gaps of incomplete information (e.g., if we do
dress how to translate restricted programs Π into ASP
not see objects appearing in observations, or camera
programs to be fed into an ASP solver, and how the

Reasonser
solve
weights 𝜔 can be learned from training data. genSnapshot

Stream Processor
To demonstrate how to build a semantic reasoning

ASP Solver
program, we will emulate DeepSORT tracking algo-

KB
rithm [13] via soft rules that can search for supporting lars2asp

DNN models
evidences to re-identify objects associated with track-

Planner
genPlanPf
lets created by Kalman filter above, by using visual ap- solve

pearance associations. DeepSORT extends SORT with
a DNN that is trained to discriminate targeted objects
(e.g. pedestrians or vehicles) on a labelled re-identificationFigure 2: Stream Reasoning Framework
dataset.
Hence, we will search for pairs of bounding boxes of 3. Dynamic Reasoning
two similar tracklets w.r.t. visual appearance. Due to
a large search space of possible matches, we will limit Framework
the search space by filtering the candidates based on
their temporal and spatial properties. Therefore, we To the realize our reasoning approach in Secion 2, we
use rules with windows to reason about two discon- proposes a dynamic reasoning framework illustrated
nected tracklets that have two bounding boxes matched in Figure 2. The key components Reasoner and Planner
within a time window of 𝛿 𝑀 time points which are of the framework are built on top an ASP Solver and
aligned with the DeepSORT’s gallery of associated ap- a Stream Processor which are pluggable and generic
pearance descriptors for each track. Based on this gallery modules. The control logic of the framework is gov-
of previous tracked boxes, the cosine distance is com- erned by Algorithm 1.
puted to compare appearance information that are par- While any ASP Solver supporting weak constraints
ticularly useful to recover identities after long term oc- can be used in our framework, existing stream proces-
clusions, when motion is less discriminative. Hence, sors such as relational or graph stream processing en-
for merging two adjacent tracklets that have visual ap- gines need to be extended with some prerequisite fea-
pearance matches, we use the parametrized soft rule tures to connect with the rest of the framework. For
(15) below. The pair of parameters (𝛿 , 𝑣𝑀 ) has to be
𝑀 𝑗 instance, in our under-development prototype, we ex-
specified for each reasoning step via the probabilistic tend CQELS [14] to enable DNN inference on GPUs as
planning component of our dynamic reasoning frame- built-in functions for CQELS-QL, the graph-based con-
work in Section 3; 𝑣𝑀 is one of the available visual
𝑗 tinuous query language of CQELS. Via CQELS-QL, the
matching models that represent the association met- auxiliary predicates such as 𝑐𝑜𝑙, 𝑛𝑒𝑥𝑡 and 𝑑𝑖𝑠𝑡 in Sec-
rics to discriminate comparing bounding boxes. tion 2 are expressed as continuous queries in order to
delegate the processing to the stream processor. This
𝜔1𝑗 ∶ 𝑖𝑆𝑂(𝐵1 , 𝑂) ← @𝜏 𝑡𝑟𝑘(𝑇1 , 𝐵1 ), 𝑐𝑜𝑙(𝐵1 , 𝑇2 ), 𝑡𝑟𝑘𝑙𝑒𝑡(𝑇2 , 𝑂), mechanism also helps us to avoid grounding overhead
𝑀 in continuous solving via materialised views similar to
@𝜏𝑒 𝑒𝑛𝑑𝑠(𝑇2 ), @𝜏𝑒 ⊞+𝛿 ◊𝑡𝑟𝑘(𝑇2 , 𝐵2 ),
the over-grounding approach in [15]. In particular, we
𝑗
𝜏 < 𝜏𝑒 + 3, 𝑣𝑀𝑎𝑡𝑐ℎ(𝑣𝑀 , 𝐵1 , 𝐵2 ) (2)
leverage continuous multiway-joins with windows of
Similarly, we can define rules to trigger the object CQELS to delegate the processing of LARS formulas
matching search based on visual appearances of two that do not occur in rule heads.
tracklets from two adjacent cameras. We use Moreover, the visual stream data with different for-
@𝜏 𝑑𝑖𝑠𝑡(𝐵1 , 𝐶, 𝑑 𝑡 ) to specify the time difference 𝑑 𝑡 from mats (e.g. RGB videos and Lidar PointCloud) together
the candidate camera 𝐶 at time point 𝜏 to start the with the knowledge base (KB) have to be normalized
search for the matches via the auxiliary predicate 𝑑𝑖𝑠𝑡. to the data model supported by the stream processor.
Also, 𝐶 is filtered by the auxiliary predicate 𝑛𝑒𝑥𝑡 stat- For example, ontologies and metadata and extracted
ing 𝐶 is adjacent to the camera where 𝐵1 was gener- symbols as outputs of DNN inference processes are
ated. 𝑙𝑒𝑓 𝑡𝐹 𝑜𝑉 (𝑂, 𝐶) represents for "object 𝑂 left the represented as temporal graphs of CQELS. With these
FoV of camera 𝐶". features, the stream processor is able to generate data
snapshots and planning profiles in ASP readable for-
𝜔2𝑗 ∶ 𝑖𝑆𝑂(𝐵1 , 𝑂) ← @𝜏 𝑡𝑟𝑘(𝑇1 , 𝐵1 ), 𝑛𝑒𝑥𝑡(𝐵1 , 𝐶), (3) mat via two methods 𝑔𝑒𝑛𝑃 𝑙𝑎𝑛𝑃𝑓 and 𝑔𝑒𝑛𝑆𝑛𝑎𝑝𝑠ℎ𝑜𝑡 re-
spectively.
𝑀
@𝜏 𝑑𝑖𝑠𝑡(𝐵1 , 𝐶, 𝑑 𝑡 ), @𝜏 +𝑑 𝑡 ⊞+𝛿 ◊𝑙𝑒𝑓 𝑡𝐹 𝑜𝑉 (𝑂, 𝐶),
𝑡𝑟𝑘𝑙𝑒𝑡(𝑇2 , 𝑂), 𝑡𝑟𝑘(𝑇2 , 𝐵2 ), 𝑣𝑀𝑎𝑡𝑐ℎ(𝑣𝑀 𝑗 , 𝐵1 , 𝐵2 ) The Planner calls the method 𝑔𝑒𝑛𝑃 𝑙𝑎𝑛𝑃𝑓 to prepare
the input for the first reasoning step of each time point
Algorithm 1 Semantic_Reasoning(S, 𝑡) LARS formulas to an ASP program using the method
Input: Semantic Stream 𝑆, new observation 𝕆, time 𝑙𝑎𝑟𝑠2𝑎𝑠𝑝, whose optimal models (answers sets) 𝐼 cor-
𝑝

point 𝑡 respond to the solutions of (5).
Output: Optimal answer set 𝐼 ∗
̂ argmax 𝑃𝑟Π̂𝑝 (Π𝐺 |𝑆, 𝜏 , Π𝑝 ) (5)
1: 𝑖 ← 0, Π𝕊 ← ∅ 𝐼 𝑝 ∶Π̂𝑝 (𝜏 )⊩𝐼 𝑝
2: for {𝜔 ∶ 𝛼 ← 𝛽} ∈ Π𝕊 do
3: 𝑖 ←𝑖+1 With a chosen plan embedded in some 𝐼 𝑝 , the Reasoner
4: Π̂𝕊 ← Π̂𝕊 ∪ {𝑢𝑛𝑠𝑎𝑡(𝑖) ← 𝛽, 𝑛𝑜𝑡 𝛼} calls the method 𝑔𝑒𝑛𝑆𝑛𝑎𝑝𝑠ℎ𝑜𝑡 to generate the input
5: Π̂𝕊 ← Π̂𝕊 ∪ {𝛼 ← 𝛽, 𝑛𝑜𝑡 𝑢𝑛𝑠𝑎𝑡(𝑖)} data of the reasoning program for the next step in line
(11) to carry out the second reasoning step from line
6: Π̂𝕊 ← Π̂𝕊 ∪ {∶∼ 𝑢𝑛𝑠𝑎𝑡(𝑖) [𝜔@0]}
(12) to (14) to generate the output of the whole pipeline
7: end for
̂ as the optimal model 𝐼 ∗ . To specify the weights of the
8: Π𝑝 ← 𝑔𝑒𝑛𝑃 𝑙𝑎𝑛𝑃𝑓 (𝑆, 𝕆, Π𝕊 , 𝑡 − 1)
̂ soft rules, we use the weight learning approach of [17]
9: Π̂𝑝 ← 𝑙𝑎𝑟𝑠2𝑎𝑠𝑝(Π𝕊 ∪Πℍ ∪Π𝑝 , 𝑡) which fits the weights using the training data via gra-
10: 𝐼 𝑝 ← Solve(Π̂𝑝 ) dient ascent. Training is done offline but uses Algo-
11: Π𝐷 ← 𝑔𝑒𝑛𝑆𝑛𝑎𝑝𝑠ℎ𝑜𝑡(𝑆, 𝐼 𝑝 , 𝑡) rithm 1 to compute an optimal stable model in each
12: Π̂ ← 𝑙𝑎𝑟𝑠2𝑎𝑠𝑝(Π̂𝕊 ∪Πℍ ∪Π𝐷 , 𝑡) step of updating weights in the gradient method.
13: 𝐼 ∗ ← Solve(Π
̂)
14: return 𝐼 ∗
4. Conclusion
𝜏 which finds the optimal reasoning plan to achieve a This position paper presented a novel semantic rea-
certain goal Π𝐺 under an operational constraint Π𝑝 , soning approach that enables probabilistic planning
following a probabilistic planning approach from [9] to adaptively optimize the sensor fusion pipeline un-
in lines (8) to (10) of Algorithm 1. The reasoning prob- der operational constraints expressed in ASP. The ap-
lem formalised in following is to find a configuration proach is realised with a dynamic reasoning mecha-
(𝑑𝑡 𝑖 , 𝑣𝑀 𝑗 , 𝛿 𝑀 ) to result the highest probability of our nism that can integrate the uncertainy of DNN infer-
tracking goal. For example, we can specify the goal of ence with semantic information, e.g common sense and
being able to track the objects that were tracked in the domain knowledge in conjunction with runtime infor-
previous time point 𝜏 − 1 as mation as inputs for operational constraints.
We are currently implementing an open sourced pro-
Π𝐺 = ⋀𝑂∈@𝜏 −1 𝑡𝑟𝑘𝑙𝑒𝑡(𝑂,_) @𝜏 𝑡𝑟𝑘𝑙𝑒𝑡(𝑂, _)
totype of the proposed reasoning framework in Java
Similarly, an operational constraint Π𝑝 can be expressed to exploit the code bases of CQELS and Ticker. We
ASP rules. For instance, the example rule below rep- use the Java native interface to wrap C/C++ libaries of
resents the constraint to limit executable plans Clingo 5.4.0 as the ASP Solver and NVidia CUDA 10.2
𝑝𝑙𝑎𝑛(𝐷, 𝑉 , 𝛿) at time point 𝜏 based on the estimations as the DNN Inference engine. The solving and infer-
the execution time of a candidate detection model 𝐷 ence tasks are coordinated in an asynchronous multi-
and visual a matching model 𝑉 together a candidate threading fashion to exploit the massive parallel capa-
window parameter 𝛿 𝑀 . The auxiliary predicates 𝑒𝑠𝑡 bilities of CPUs and GPUs.
and 𝑛𝑂𝑏𝑗 provide the time estimation for correspond-
ing DNN operations and the number of objects tracked
in camera 𝐶. Acknowledgments
This work was funded in part by the German Ministry
∶∼ @𝜏 𝑒𝑠𝑡(𝐷, 𝜏𝐷 ), @𝜏 𝑒𝑠𝑡(𝑉 , 𝜏𝑉 ), @𝜏 𝑝𝑙𝑎𝑛(𝐷, 𝑉 , 𝛿), (4)
@𝜏 −1 𝑛𝑂𝑏𝑗(𝐶, 𝑁 ), 𝜏𝐷 + 𝑁 ∗ 𝛿 𝑀 ∗ 𝜏𝑉 > 𝑚𝑎𝑥𝑇 𝑖𝑚𝑒 for Education and Research as BIFOLD - Berlin Insti-
tute for the Foundations of Learning and Data (refs
From Π , lines (8) to (10) carry out solving the reason- 01IS18025A and 01IS18037A) and supported by the Aus-
𝑝
ing problem formalised by formula (4). To generate the trian Federal Ministry of Transport, Innovation and
LARS program from the soft rules Π𝕊 , the algorithm Technology (BMVIT) under the program ICT of the
rewrites Π𝕊 into LARS formulas with weak constraints Future (FFG-PNr.: 861263, project DynaCon).
as Π̂𝕊 in lines (1) to (7) by extending the similar algo-
rithm for LP𝑀𝐿𝑁 in [8]. Then, we use the incremental
ASP encoding algorithm of Ticker [16] for rewriting
References [13] N. Wojke, A. Bewley, D. Paulus, Simple online
and realtime tracking with a deep association
[1] L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, metric, in: ICIP, 2017.
X. Liu, M. Pietikäinen, Deep learning for generic [14] D. Le-Phuoc, M. Dao-Tran, J. X. Parreira,
object detection: A survey, IJCV (2019). M. Hauswirth, A native and adaptive approach
[2] G. Ciaparrone, F. L. Sánchez, S. Tabik, L. Troiano, for unified processing of linked streams and
R. Tagliaferri, F. Herrera, Deep learning in video linked data, in: ISWC, 2011, pp. 370–388.
multi-object tracking: A survey, Neurocom- [15] F. Calimeri, G. Ianni, F. Pacenza, S. Perri, J. Zan-
puting (2019). URL: http://www.sciencedirect. gari, Incremental answer set programming with
com/science/article/pii/S0925231219315966. overgrounding, TPLP 19 (2019).
doi:https://doi.org/10.1016/j.neucom. [16] H. Beck, T. Eiter, C. Folie, Ticker:
2019.11.023. A system for incremental asp-based
[3] S.-C. Lin, Y. Zhang, C.-H. Hsu, M. Skach, M. E. stream reasoning, TPLP (2017). URL:
Haque, L. Tang, J. Mars, The architectural impli- https://doi.org/10.1017/S1471068417000370.
cations of autonomous driving: Constraints and doi:10.1017/S1471068417000370.
acceleration, in: ASPLOS ’18, 2018. [17] J. Lee, Y. Wang, Weight learning in a probabilistic
[4] NTSB, Collision between vehicle controlled extension of answer set programs, in: KR, 2018,
by developmental automated driving sys- pp. 22–31.
tem and pedestrian in Tempe, Arizona,
https://www.ntsb.gov/news/events/Documents/
2019-HWY18MH010-BMG-abstract.pdf, 2019.
Accessed: 2020-01-15.
[5] S. Aditya, Y. Yang, C. Baral, Integrating knowl-
edge and reasoning in image understanding, in:
IJCAI, 2019. URL: https://doi.org/10.24963/ijcai.
2019/873. doi:10.24963/ijcai.2019/873.
[6] J. Suchan, M. Bhatt, S. Varadarajan, Out of
sight but not out of mind: An answer set pro-
gramming based online abduction framework
for visual sensemaking in autonomous driv-
ing, in: IJCAI-19, 2019. URL: https://doi.org/
10.24963/ijcai.2019/260. doi:10.24963/ijcai.
2019/260.
[7] H. Beck, M. Dao-Tran, T. Eiter, LARS: A
logic-based framework for analytic reasoning
over streams, Artif. Intell. 261 (2018) 16–70.
URL: https://doi.org/10.1016/j.artint.2018.04.003.
doi:10.1016/j.artint.2018.04.003.
[8] J. Lee, Z. Yang, LPMLN, Weak Constraints, and
P-log, in: AAAI, 2017.
[9] J. Lee, Y. Wang, A probabilistic extension of ac-
tion language BC+ , TPLP 18 (2018) 607–622. URL:
https://doi.org/10.1017/S1471068418000303.
doi:10.1017/S1471068418000303.
[10] K. Janowicz, A. Haller, S. J. D. Cox, D. L. Phuoc,
M. Lefrançois, SOSA: A lightweight ontology for
sensors, observations, samples, and actuators, J.
Web Semant. 56 (2019) 1–10.
[11] A. Bewley, Z. Ge, L. Ott, F. Ramos, B. Upcroft,
Simple online and realtime tracking, in: ICIP,
2016, pp. 3464–3468.
[12] E. T. Mueller, Commonsense Reasoning: An
Event Calculus Based Approach, 2 ed., 2015.