Motivation

Multimodality in embodied interaction is an inherent aspect of human activity, be it in social, professional, or everyday mundane contexts. Next-generation human-centred AI technologies, operating in such contextualised everyday settings, will require an inherent foundational capacity to "make sense" of -e.g., perceive, understand, explain, anticipate-everyday, naturalistic interactional multimodality. This would be essential towards successfully achieving technology mediated ("human-in-the-loop" ) collaborative assistance, as well as ensuring compliance with emerging human-centred ethical and legal requirements, performance benchmarks, and inclusive usability expectations. It is therefore crucial that the foundational building blocks of such next-generation systems be semantically aligned with the descriptive, analytical, and explanatory characteristics and complexity of human task conceptualisation, performance benchmarks, and usability expectations. Against this backdrop, we define artificial visual intelligence [1] as: spectrum of high-level human-centred sensemaking capabilities. These capabilities encompass operational functions such as: Our central focus is on the development of general, domain-independent methods that may be seamlessly integrated as part of hybrid computational cognitive system, or even within computational cognitive models / cognitive architectures [2]. We also contextualise and demonstrate in the backdrop of applications in autonomous driving, cognitive robotics, visuoauditory media design, and cognitive psychology (e.g. [3,4,5,6], [7,8] ). Through applied casestudies, we provide a systematic model and general methodology showcasing the integration of diverse, multi-faceted AI methods pertaining Knowledge Representation and Reasoning, Computer Vision, Machine Learning, and Visual

Perception towards realising practical, human-centred, computational visual intelligence.

Neurosymbolic Visual Commonsense: Integrated Reasoning and Learning about Space, Motion, and Inter(A)ction

In the present status quo, our research in (computational) neurosymbolic visual commonsense categorically addresses three key questions:

I. What kind of (relational) abstraction mechanisms are needed to computationally "make-sense" of embodied multimodal interaction ?

II. How can (and why should) abstraction mechanisms (such as in I) be founded on behaviourally established cognitive human-factors emanating from naturalistic empirical observation in real-world applied contexts?

III. How to articulate behaviourally established abstraction mechanisms, preferences (etc) as formal declarative models suited for computational modelling aimed at operational"sensemaking" (encompassing capabilities such as abduction, relational learning, counterfactual inference) ?

Present work is particularly aimed at developing general methods for the semantic interpretation of (multimodal) dynamic visuospatial imagery with an emphasis on the ability to neurosymbolically perform abstraction, reasoning, and learning with cognitively rooted structured characterisations of commonsense knowledge pertaining to space and motion. Here, we specifically emphasise:

• General foundational commonsense abstractions of space, time, and motion needed for representation mediated (grounded) reasoning and learning with dynamic visuospatial stimuli (e.g., emanating from multimodal human behavioural signals in modalities such as RGB(D), video, audio, eye-tracking and possibly even bio signals [9]);

• Deep (visuospatial) semantics, entailing systematically formalised declarative (neurosymbolic) reasoning and learning with aspects pertaining to space, space-time, motion, actions & events, spatiolinguistic conceptual knowledge. Here, it is of the essence that an expressive ontology consisting of, for instance, space, time, space-time motion primitives as first-class 'neurosymbolic' objects is accessible within the (declarative) programming paradigm under consideration; and

• Explainable models of computational visuospatial commonsense based on a systematic integration of symbolic/relational methods on the one hand, and neural techniques aimed at low level quantitative (e.g., visual) data processing on the other; At a higher level of abstraction, deep (visualspatial) semantics (or deep semantics for short) entails inherent support for tackling a range of challenges concerning epistemological and phenomenological aspects relevant to dynamic spatial systems [10] where integrated reasoning about action and change [11,12] is involved:

• interpolation and projection of missing information, e.g., what could be hypothesised about missing information (e.g., moments of occlusion [13]); how can this hypothesis support planning an immediate next step?

• object identity maintenance at a semantic level, e.g., in the presence of occlusions, missing and noisy quantitative data, error in detection and tracking

• ability to make default assumptions, e.g., pertaining to persistence objects and/or object attributes Formal semantics and computational models of deep semantics manifest themselves as neurosymbolic spatio-temporal extensions of established declarative AI frameworks such as Constraint Logic Programming (CLP) [20], Inductive Logic Programming (ILP) [21], and Answer Set Programming (ASP) [22]. The more foundational aspects pertaining declarative spatial reasoning (built on top of CLP, ILP, ASP) independent of its relationship to cognitive vision research may be consulted in [23], [16,24], [18].

Discussion

The vision that drives our scientific methodology is:

» To shape the nature and character of (machine-based) artificial visual intelligence with respect to human-centred cognitive considerations, demonstrating an exemplar for developing, applying, and disseminating such methods in socio-technologically relevant application areas where:

(a) embodied (multimodal) human interaction is inherent;

(b) human-in-the-loop collaborative work is of the essence; and

(c) normative ethico-legal compliance based on regulatory requirement and humanfactors driven inclusive or universal design criteria is to be ensured.

Towards realising this vision, we adopt an interdisciplinary approach -at the confluence of Cognition, AI, Interaction, and Design-which we deem necessary to better appreciate the complexity and spectrum of varied human-centred challenges for the design and (usable) implementation of (explainable) artificial visual intelligence solutions in diverse human-system interaction contexts.

One of the key technical driving forces in our work is that of "representation mediated multimodal sensemaking".

In essence, we consider (neurosymbolic) representation mediated grounding as being significant in semiotic construction, e.g., enabling high-level meaning-making. This view stems from the long-established value of "grounding" in Artificial Intelligence and related disciplines [25]. Our research advances the theoretical, methodological, and applied understanding of "grounded representation" mediated multimodal sensemaking of embodied human interaction at the interface of spatial language, spatial logic, and spatial cognition. In our view, the significance of this form of (neurosymbolic) grounding must now be reiterated, re-asserted even, in view of recent advances in neural machine learning and the well-recognised "explainability" and "interpretability" requirements from the viewpoint of human-centred AI [26,27,28]. We believe that research in knowledge representation and reasoning (KR) has, since its inception, concerned itself with the "hard" problem of semantics, emphasising explainability, formal verification and diagnosis, elaboration tolerance amongst other things. Research in KR, and more broadly in symbolic AI and semantics, and their role and contribution towards large-scale hybrid "human-in-theloop" intelligence is of even greater significance now than ever before given the tremendous synergistic opportunities afforded by the widely demonstrated power of deep learning driven techniques in computer vision (and beyond). The onus now, we posit, is on KR research to drive itself towards developing methods that can seamlessly integrate (and be "usable") with other kinds of AI methods, be data-centric neural learning techniques, or otherwise.

In this invited position statement, we have attempted to summarise our mindset and ongoing work in the CoDesign Lab towards:

» Establishing a human-centric foundation and roadmap for the development of neurosymbolically grounded inference about embodied multimodal interaction as identifiable in a range of real-world application contexts.

This summary is not meant to be a comprehensive literature review; this may be obtained through the cited works. For key technical details and to obtain a summary of open directions, we direct interested readers to select publications as follows: a compact starting point may be obtained via the comprehensive summary in [1], or through the shorter/focussed components in [15,5,4,13,3]. Longer summaries in the form of (recent) doctoral dissertations are available in [29] and [30,31].