CSK-SNIFFER: Commonsense Knowledge for Snifing Object Detection Errors

Abstract

This paper showcases the demonstration of a system called CSK-SNIFFER to automatically predict failures of an object detection model on images in big data sets from a target domain by identifying errors based on commonsense knowledge. CSK-SNIFFER can be an assistant to a human (as snifer dogs are assistants to police searching for problems at airports). To cut through the clutter after deployment, this “snifer” identifies where a given model is probably wrong. Alerted thus, users can visually explore within our demo, the model's explanation based on spatial correlations that make no sense. In other words, it is impossible for a human without the help of a snifer to flag false positives in such large data sets without knowing ground truth (unknown earlier since it is found after deployment). CSK-SNIFFER spans human-AI collaboration. The AI role is harnessed via embedding commonsense knowledge in the system; while an important human part is played by domain experts providing labeled data for training (besides human commonsense deployed by AI). Another highly significant aspect is that the human-in-the-loop can improve the AI system by the feedback it receives from visualizing object detection errors, while the AI provides actual assistance to the human in object detection. CSK-SNIFFER exemplifies visualization in big data analytics through spatial commonsense and a visually rich demo with numerous complex images from target domains. This paper provides excerpts of the CSK-SNIFFER system demo with a synopsis of its approach and experiments.

1. Introduction

Bbox2 car Trained object detection model M Human-AI collaboration, the realm of humans and AI bbox1 bbox3 H systems working together, typically achieves better per- collaborate formance than either one working alone [ 1 ]. Big data visualization and analytics can be used to foster interac- bPbrex1dicbtiboxn2 bbx3 tion [ 2 ]. Such areas receive attention, e.g. NEIL (Never Ending Image Learner) [ 3 ], active learning approaches CSK Snif er C Update [ 4 ], human-in-the-loop learning [ 5 ] etc. To that end, iImnpaugte x flag spatially we demonstrate a system “CSK-SNIFFER” exemplifying implausible bhyu mviasnu-aAlizIicnogllpaobtoernattiiaolnervrioaresninhalanrcginegcoombjpelcetxddeatteactsieotns, object pairs Inference rrkeenltoeriwveavlenedtogsbepjaetciatl ㎅ harnessing spatial commonsense. This system “snifs” errors in object detection using spatial collocation anoma- Figure 1: CSK-SNIFFER and the human-in-the-loop: A car lies, assisting humans analogous to snifer dogs aiding was detected in the image which was flagged bad by CSKpolice at airports. The process, (Figure 1), is as follows, SNIFFER based on its spatial knowledge w.r.t KB which the with the CSK-SNIFFER system (), human-in-the-loop human-in-the-loop can update after visualizing errors (), and inference model ( ) for object detection.

System interacts with human and provides object detection output visualizing potential errors, over model sion and recall, based on the spatial KB. Hence, the two , by deploying commonsense knowledge through a directions in this learning loop are as follows. spatial knowledge base (). The is derived by capturing spatial commonsense, especially as collocation • to : Feedback-based interactive learning anomalies. Then sees the visualized errors (output by • to : Assistance in object detection ) and can thereby enhance by increasing its precivisualized bbx1 bbx2 bbx3 Thus, the human and the AI work together with the goal of enhancing object detection in big data. Further, the inference model can potentially improve, as an added benefit of this adversarial learning via human-AI collaboration. The obtained information can be used to supply more examples to on the misclassified categories to make it more robust. If certain labels are inappropriate consistently across examples, it is a valuable insight. As data sets get bigger in volume and variety, such automation is even more significant in assisting object detection errors.

The use case in our work focuses on the smart mo

bility domain [ 6 ]. It entails autonomous vehicles, selfoperating trafic signals, energy-saving street lights dimming / brightening as per pedestrian usage etc. In such

AI systems, it is crucial to detect objects accurately, especially due to issues such as safety. CSK-SNIFFER plays an important role here, generating large adversarial training data sets by snifing object detection errors.

2. The CSK-SNIFFER Approach

People crossing city streets on pedestrian crossings Vehicles coming to a full hault at red signals Vehicles stopping or slowing down at stop signs Street lights dimming when occupants are few Street lights brightening when occupants are many Buses running on trafic-optimal routes Service dogs helping blind people People charging phones at WiFi stations People reading useful information at roadside kiosks People parking bikes at share-ride spots Vehicles flashing turning lights for L/R turns Bikes riding on bike routes only Trafic cops making hand signals in regular operations Vehicles driving beneath an overpass Dogs on a leash walking with their owners People jogging on sidewalks People entering and leaving trains when doors open People using prams for kids in buses Trees existing on sidewalks Ropeways carrying passengers to tourist spots Bikers wearing smart watches Maglev trains running between airports and cities Grass existing on freeway sides and city streets Solar panels existing on roofs of buildings People using smartphones for talking anytime anywhere Canal lights dimming when occupants are few Canal lights brightening with many occupants People wheeling shopping carts in grocery stores

We summarize the CSK-SNIFFER approach as per its

design and execution [ 7 ]. In this approach. we represent and construct a , along with the function () that generates a triple, i.e. < , ˆ , > from the predicted bounding boxes of objects and such that is a binary vector over relations ().

Gather using action-vocab( ): Table 1 presents some examples from action-vocab( ) where =smart mobility domain. These entries represent mostly typical and some unique scenes in this domain. This list was manually compiled by a domain Table 1 expert. Images are compiled using Web queries ∈ ∼ 10% examples from action-vocab( ) where =smart action-vocab( ) (on an image search engine), and an mobility domain. These are used as queries to compile the object detector predicts bounding boxes over ∈ . input to the object detector, and then CSK-SNIFFER can flag

KB construction: While in principle, we can directly images in where the detector failed to predict the correct use existing s, these have errors as elaborated in bounding boxes. some works [ 8 ]. CSK-SNIFFER isolates the efect of these errors by instead manually creating a at a very low cost. The is defined over a set of objects and rela- available at https://tinyurl.com/kb-for-csksnifer tions (). The relation set () comprises Function f(bbox): Similar to the triples in the , 5 relations (isAbove, isBelow, isInside, isNear, we define a function () to construct triples < overlapsWith). We are inspired by other works in the , ˆ , > using the predicted bounding boxes of imliterature such as [ 9 ] in picking these relations, and be- age . The () input consists of predicted boundcause our initial analysis proposed their suitability for ing boxes on an image, and the output is a list of triples bounding box relative relations. An entry in the in the format: < , ˆ , >, for every pair of objects comprises of a pair of objects , ∈ and a binary , ∈ the objects detected in the image. For every such vector denoting , ’s and their spatial relations pair, () compares the coordinates of the bounding over (). These spatial relations are manually an- boxes of and (this is a known process e.g. [ 9 ]). We notated by a domain expert, according to general likeli- illustrate this for the isInside relation. Let coordinates hood e.g., it is more likely that a dog is observed near a of a bounding box be 1, 1, 2, 2, then .1 denotes human, and much less likely that it is observed near a 1 coordinate of . If .2 ≤ .1 and .2 > .1 whale. The most popular objects on MSCOCO training and .1 < .2 then is inside . Similarly, data make up ; in our experiments = 10, and this other relations in () are built, compiling which leads to 2 entries in that need to be annotated with provides ˆ . For anomaly detection, we compare vectors . It is remarkable that our experiments demonstrate ˆ and for overlapping object-pairs , , detected that even with = 10, the allows CSK-SNIFFER to in the image and present in the . achieve good performance. We can infer that selecting Based on this discussion, the following algorithm a popular subset helps, even if it is small. An entry in summarizes the execution of CSK-SNIFFER. the KB is denoted as < , , > The is publicly

Algorithm 1: CSK-SNIFFER Approach

Input: Object detector trained on source domain

Manually compiled action-vocab( ) in target domain

Images compiled using Web queries ∈ action-vocab( ) 1. Define () comprising 5 relations: isAbove, isBelow, isInside, isNear, overlapsWith 2. Define commonsense , each entry < , , > where is a binary vector over () 3. Generate triples < , ˆ , > from predicted bounding boxes of ∈ using function (). 4. For each image ∈ , compare ˆ and , from bounding box triples < , ˆ , > and triples < , , > 5. For each , if ˆ ̸= then flag : , add to ′

Output: Subset ′ where failed

3. Excerpts from System Demo

We have built a live demo to depict the working of CSK

SNIFFER. This demo illustrates the functioning of CSK

SNIFFER to enhance its actual comprehension and aug- Table 2 ment its usage. In addition, this demo paper presents Distribution of spatial relations in . Spatial relations are the principles behind the human-in-the-loop functioning of the form: < , ^ , > of CSK-SNIFFER for snifing object detection errors in large, complex data sets, thereby being added contributions over our earlier work [ 7 ]. While this human-in-the- predicted, the demo moves to the home page. This conloop functioning is explained in the introduction with tains details on the output files generated. The “Images” an illustration and theoretical justification, its detailed option displays downloaded images as shown in Figure empirical validations with respect to interactive up- 2 herewith. dates constitute ongoing work, based on CSK-SNIFFER Output files generated by CSK-SNIFFER are illustrated being actively deployed in real-world settings. In fact, as follows. Table 2 shows the first output file “Collocathis demo paves the way for such interactive updates tions Map” with triples predicted by CSK-SNIFFER in via augmenting the usage of CSK-SNIFFER in suitable images with their respective counts. The final output applications to provide the human-in-the-loop feedback ifle “Error Set” Table 3 contains names of images with for the addition of such interactive updates. some odd visual collocations. It also indicates the triple

We present some screenshots illustrating the demo. that actually got predicted versus the expectation from Many more can be provided in a live setting. The user the model. These files help fathom the functioning of enters any search query related to smart mobility [ 6 ]. CSK-SNIFFER.

Images are downloaded from Google Images based on this query. Object detection is then performed on the images using YOLO [ 10 ] to start predicting triples in the image using the () function. Once the triples are person, overlapsWith, person car, is_near, car person, overlapsWith, car car, overlapsWith, person car, overlapsWith, car person, is_near, backpack backpack, is_near, person car, is_near, backpack backpack, is_near, car trafic light, is_near, trafic light each other, such that the AI (CSK-SNIFFER) provides Table 3 a visual demo of the object detection errors snifed by Canonical examples of errors flagged by CSK-SNIFFER If in- spatial CSK, thus generating large adversarial data sets ferred spatial relations over model-generated bounding boxes to assist object detection, while the human can use this are not consistent with expected spatial relations between ob- feedback to enhance the performance of CSK-SNIFFER, jects, then predicted bounding boxes are flagged as erroneous. thereby playing its role in the learning loop. 4.2. Error Analysis

4. Experimental Evaluation

We now present the precision and recall shortcomings.

We present examples from our experiments, showing the Recall issues: Actually bad, flagged good: Figure 5 correct and wrong predictions made by CSK-SNIFFER, depicts examples of these types of images. The reason for along with the error analysis. Here, “bad” refers to images CSK-SNIFFER predicting these images as good instead containing object detection errors while “good” refers to of bad is that the objects wrongly detected in the image correctly identified images with no such errors. are not present in our , hence it does not check for their locations. Thus, they are not found in any of the 4.1. Appropriate Identifications triples predicted, they are skipped so that they do not make their way to the error set.

Actually bad, flagged bad: Figure 3 illustrates examples Precision issues: Actually good, flagged bad: Our in this category. Experimental evaluation shows that our investigation of the source of these errors (see Figure model is good at identifying odd bounding boxes. 6), concluded that the relations are authored with a

Actually good, flagged good: Figure 4 portrays ex- 3D space in perspective, while the images only contain amples of this type. Experimental evaluation shows that 2 information. Therefore, relations such as above and CSK-SNIFFER is able to distinguish good predictions. below may be confused with farther and nearer. For

Other benefits: Interestingly, while analyzing mis- example, if a car is detected in the background, () takes of CSK-SNIFFER , we find that ∼ 10% of the refer- function make an incorrect interpretation as car is ence data on which is trained (MSCOCO, expected to above person. The will flag this as unlikely and be a high quality), contains wrong bounding boxes. This hence an erroneous detection, leading to a possibly good provides insights into potentially improving MSCOCO, prediction flagged as an error. constituting an added benefit of this work. Addressing 2D vs. 3D errors: We calculate the area

On the whole, the human and the AI collaborate with covered by a bounding box, such that if the area is less than an empirically estimated threshold that object is considered to be detected in the background and therefore does not predict the triple, e.g. car above person in that image. This helps to increase accuracy to ∼ 80%.

5. Conclusions and Roadmap

This paper synopsizes the demo (with approach and experiments) of a system “CSK-SNIFFER” that “snifs” object detection errors in a big data on an unseen target domain using spatial commonsense, with high accuracy at no additional annotation cost. Based on human-AI collaboration, the AI angle entails spatial CSK imbibed in the system deployed via visual analytics to assist humans, while an important human role comes from the domain expert perspective in image tagging and task identification for training the system (in addition to the obvious human contribution of commonsense knowledge in the system). More significantly, the human and the AI make contributions to the learning loop by feedback-based interactive learning, and assistance in object detection respectively. It is promising to note that our approach based on simplicity can automatically discover errors in data of significant volume and variety, and be potentially useful in this learning setting. We demonstrate that with high quality, we can generate large complex adversarial datasets on target domains such as smart mobility.

Future work includes harnessing existing, poten

tially noisy and incomplete commonsense in CSK

SNIFFER Another direction is to study whether auto

matic adversarial datasets compiled with assistance from

CSK-SNIFFER help train better models on novel target domains. Our work presents interesting facets from big data visualization and analytics along with human-AI collaboration.

6. Acknowledgments

A. Varde has NSF grants 2018575 (MRI: Acquisition of

a High-Performance GPU Cluster for Research & Education); 2117308 (MRI: Acquisition of a Multimodal

Collaborative Robot System (MCROS) to Support Cross

Disciplinary Human-Centered Research & Education).

She is a visiting researcher at Max Planck Institute for

Informatics, Germany.

References

[1] D. Wang , E. Churchill , P. Maes , X. Fan , B. Shneiderman , Y. Shi , Q. Wang , From human-human collaboration to human-ai collaboration: Designing ai systems that can work together with people , in: CHI , 2020 , pp. 1 - 6 .

[2] F. Zuo , J. Wang , J. Gao , K. Ozbay , X. J. Ban , Y. Shen , H. Yang , S. Iyer , An interactive data visualization and analytics tool to evaluate mobility and sociability trends during covid-19 , arXiv: 2006 . 14882 ( 2020 ).

[3] X. Chen , A. Shrivastava , A. Gupta , Neil: Extracting visual knowledge from web data , in: ICCV, 2013 , pp. 1409 - 1416 .

[4] K. Konyushkova , R. Sznitman , P. Fua , Learning active learning from data , Advances in Neural Information Processing Systems 30 ( 2017 ).

[5] D. Xin , L. Ma, J. Liu, S. Macke , S. Song , A. Parameswaran , Accelerating human-in-the-loop machine learning: Challenges and opportunities , in: ACM SIGMOD (DEEM workshop) , 2018 , pp. 1 - 4 .

[6] A. Orlowski , P. Romanowska , Smart cities concept: Smart mobility indicator, Cybernetics and Systems (Taylor & Francis) 50 ( 2019 ) 118 - 131 .

[7] A. Garg , N. Tandon , A. S. Varde , I am guessing you can't recognize this: Generating adversarial images for object detection using spatial commonsense , in: AAAI , 2020 , pp. 13789 - 13790 .

[8] N. Tandon , A. S. Varde , G. de Melo, Commonsense knowledge in machine intelligence , ACM SIGMOD Record 46 ( 2017 ) 49 - 52 .

[9] M. Yatskar , V. Ordonez , A. Farhadi , Stating the obvious: Extracting visual common sense , NAACL ( 2016 ).

[10] J. Redmon , A. Farhadi , Yolo9000: Better, faster, stronger, CVPR ( 2016 ).