In the era of the Internet of Things and Big Data, data scientists are required to extract valuable knowledge from the given data. This challenging task is not straightforward. Data scientists rst analyze, cure and pre-process data. Then, they apply Arti cial Intelligence (AI) techniques to automatically extract knowledge from data. However, nowadays the focus is set on knowledge representation and how to enhance the human-machine interaction. Non-expert users, i.e., users without a strong background on AI, require a new generation of explainable AI systems. They are expected to naturally interact with humans, thus providing comprehensible explanations of decisions automatically made. In this paper, we sketch how certain computational intelligence techniques, namely interpretable fuzzy systems, are ready to play a key role in the development of explainable AI systems. Interpretable fuzzy systems have already successfully contributed to build explainable AI systems for cognitive cities.
The quest of \comprehensibility" and \explanation" in the eld of
Computational Intelligence is rooted in the more general paradigm of Computing With
Words (CWW) stated by Zadeh more than two decades ago [
CWW operates at the semantic level, i.e., inference is carried out by considering the semantic de nition of words (and their connectives) in terms of fuzzy sets and operators. In fact, CWW relies on the assumption that fuzziness is intrinsic in the semantics of most linguistic terms. Moreover, as a distinctive feature, it deals naturally with fuzziness at the inference process and produces results represented linguistically.
Copyright © 2018 for this paper by its authors. Copying permitted for private and academic purposes.
Fuzzy (rule-based) systems use CWW to represent linguistic knowledge and
carry out approximate reasoning under imprecision and uncertainty, which makes
their use appealing in many application domains. Interpretable fuzzy systems
(IFS) are fuzzy systems whose behavior is human-centric, i.e., they can be
easily understood, trusted on or accounted for human beings. IFS are especially
appreciated in domains with advanced human-computer collaboration such as
Medicine [
IFS combined with NL generation techniques are a recent development [
\Even though current AI systems o er many bene ts in many
applications, their e ectiveness is limited by a lack of explanation ability
when interacting with humans."
Notice that powerful AI techniques like Deep Learning [
Black-box models are not suitable for Cognitive Cities. A Cognitive City [
In this paper, we will brie y describe the concept of Interpretability within CWW (Sec. 2), then we outline a novel framework to build explainable AI systems for Cognitive Cities, also enumerating some examples of applications where this framework has been applied (Sec. 3). Finally, Section 4 draws main conclusions and sketch future work. 2
Human perceptions are de ned by intrinsic and extrinsic attributes regarding
human senses (sight, smell, taste, touch and hearing). In addition, human
pleasantness depends on what people experience (Perceptions), but also on what
people expect (Cognitions) which is in uenced by common and personal background
(e.g., context or mood). In most cases, human beings use NL for describing their
perceptions and for construing their experience [
The use of FST and CWW for designing comprehensible intelligent systems ts with the intuitive idea that many concepts represented in NL have a perception-based nature. An agent can perceive objects of the reality and associate them to concepts which can be verbalized by means of linguistic labels (see Fig. 1). On the one hand, perception is a cognitive act of transforming sensory data into mental representations. On the other hand, linguistic labels designate concepts which refer to the objects of the world. Fuzzy logic (in the wide sense) moves from the assumption that objects belong to a reality that is characterized by continuous features, which are perceived by sensors able to observe and process such continuity. In consequence, the concepts re ect the continuity of the reality and, therefore, accommodate the properties of graduality (because objects' features vary without sharp boundaries) and granularity (because each concept refers to a multitude of objects). Graduality and granularity are the key components of fuzzy sets which are the basic building blocks of several theories and methodologies, including CWW.
Thus, FST seems very suitable to formalize perception-based intelligent
system which are interpretable to human beings. However, the use of FST to design
intelligent systems is not enough to ensure interpretability. More than 30 years
ago, Michalski introduced his \Comprehensibility Postulate" (CP) [
\The results of computer induction should be symbolic descriptions of given entities, semantically and structurally similar to those a human expert might produce observing the same entities. Components of these descriptions should be comprehensible as single chunks of information, directly interpretable in natural language, and should relate quantitative and qualitative concepts in an integrated fashion."
Notice that the CP was formulated without explicitly considering FST;
nevertheless, it highlights several key-points that are worth observing. Firstly, the
human-centrality of the results of a computer induction process, which should
be described symbolically as a necessary condition to communicate information.
However, such symbols are not \empty", but they should represent chunks of
information, namely, information granules, that are groups of data tied together
by semantic relationships (e.g., proximity or similarity) [
Moreover, such symbols should be directly interpretable in NL. However, this does not boil down to a simple selection of symbols within a corpus, i.e., a vocabulary made of NL terms for a speci c domain. In fact, the CP requires the interpretation of symbols to be in NL. This is a requirement on the semantics of symbols and relations between them, i.e., on the information granules they denote. More precisely, on one hand information granules (resulting from computing processes) should conform with concepts a human can conceive; on the other hand, NL terms convey an implicit semantics (which depends also on the context), that is shared among all human beings speaking the same language. Therefore, a symbol coming from NL can be used to denote an information granule only if the implicit semantics of the symbol highly matches with the explicit semantics of the information granule.
This relation between both implicit and explicit semantics is called \semantic
co-intension". It is inspired by the concept of model co-intension of Zadeh [
A model is co-intensive if its proximity is high." We use the Zadeh's notion of co-intension to relate the semantics de ned by information granules with the underlying semantics held by the used symbols: the results of an inductive process are interpretable if the explicit semantics embedded in the model is co-intensive with the implicit semantics inferred by the user while interpreting the model. In this sense, we think FST and CWW can naturally contribute to explainable AI. 3
The concept of \Cognitive City" was rst introduced by Novak [
We have already empirically shown the bene ts of combining IFS with NL
generation techniques to make the interaction between humans and fuzzy
systems more natural [
The framework consists of three main stages: 1. Data acquisition is made through both humans and machines. Raw data are pre-processed in accordance with the communicative goal, the user model, the knowledge source, and the discourse history. 2. Data analysis and interpretation is carried out by IFS which are carefully designed o -line. Notice that domain knowledge is embedded into linguistic rules which relate semantically co-intensive concepts taken from a corpus speci c for the target application. They are in the core of the so-called Granular Linguistic Model of Phenomena (GLMP) which consists of a hierarchical network of perception mappings and computational perceptions de ned by IFS. Moreover, the relevance calculation is the basis to generate customized messages. Only the most relevant pieces of information will be added to the nal report. 3. The extracted knowledge (expressed in the form of linguistic pieces of information coming out of the so-called content determination stage) is presented to humans in NL (after document planning and careful computational linguistic realization which includes lexicalization, referring expression generation and aggregation).
We have already applied this framework (1) to provide citizens in Gijon
(Spain) [
Let's introduce brie y how the framework sketched in Fig. 2 was implemented for one of the pilots in the NatConsumers project. Input data elements were as follows: { Communicative goal. The main factors in uencing residential energy consumption in European countries were identi ed in a previous study. Here, the goal was the automatic generation of linguistic advice for saving energy at home, remarking the main factors to consider and the main actions to carry out. Three kind of energy consumptions are considered: General, speci c and standby consumptions. { User model. We identi ed the target consumers who will receive the customized messages. Moreover, we classify consumers regarding both attitudinal and physical taxonomies which were previously de ned by other partners in the project. { Knowledge source. Information related to the energy consumption for each household involved in the project. { Discourse history. The sequence of messages already generated for each speci c consumer.
Fig. 3 shows a schematic excerpt of the IFS/GLMP described in [
In addition, Fig. 3 includes some illustrative examples of sentences which were generated as instantiations of the related templates, after running the fuzzy inference process, for a speci c case. Moreover, Fig. 4 depicts the nal report generated for the same illustrative case. Notice that the report combines both NL texts and graphs with the aim of maximizing interpretability. 3 H2020 Coordination and Support Action (CSA) [http://www.natconsumers.eu]
Taking pro t of our previous background as designers of interpretable fuzzy systems, we have gone a step forward in the generation of explainable AI systems. They are ready to convey citizens with valuable knowledge (represented by NL texts) which is automatically extracted from data. Nevertheless, we are aware this is only a rst step and a lot of work remains to do. For example, coste ectiveness, scalability or interface with other AI systems will turn up as key issues to address with the aim of applying our proposal universally in a cognitive city. In the short term, we plan to extend our framework with interactive dialog systems (endowed with argumentation theory approaches) which are likely to enhance the human-machine interaction capability of our explainable AI systems.
Results on applied research can be pro tably used in theoretical research on interpretability. Roughly speaking, the results achieved on studying interpretability of fuzzy systems are mostly based on common-sense arguments and heuristic approaches [24]. This recently gave rise to some critical positions on the state-of-art [25], which stimulates new directions of investigation. From a theoretical viewpoint, research on comprehensibility can be cast into the eld of communications between granular worlds, where agents are endowed with a granular representation of knowledge and want to communicate some information. In this abstract setting, several levels of representation of information can be established, from numerical (representing signals) to symbolic (representing terms), possibly passing through intermediate, hybrid levels. In this sense, Granular Computing could be intended as the cognitive bridge from the numerical reality to the symbolic world, where abstract relations emerge from speci c dependencies. Namely, in analyzing the main theme of communication between granular worlds, the main properties of semantic communication could be investigated by using the tools o ered by Granular Computing. Examples of these properties are imprecision, uncertainty, fuzziness or vagueness. All these properties are intimately connected with NL, and, traditionally, systematically removed in view of communicating information to machines and between machines. A lot of work has already been done in literature, by the de nition of theories like Possibility Theory [26], Precisiated NL [27] or the theory of Z-numbers [28]. Yet, there is large room for further research, especially if aimed at giving computational solutions, i.e., solutions that can be translated into computer programs.
This work was supported by TIN2014-56633-C3-3-R (ABS4SOWproject) from the Spanish \Ministerio de Econom a y Competitividad". Financial support from the Xunta de Galicia (Centro singular de investigacion de Galicia accreditation 2016-2019) and the European Union (European Regional Development Fund ERDF), is gratefully acknowledged. 23. Conde-Clemente, P., Alonso, J.M., Trivino, G.: rLDCP: R package for text generation from data. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Naples, Italy (2017) 1{6 http://dx.doi.org/10.1109/FUZZIEEE.2017.8015487. 24. Alonso, J.M., Castiello, C., Mencar, C.: Interpretability of Fuzzy Systems: Current Research Trends and Prospects. In Kacprzyk, J., Pedrycz, W., eds.: Springer Handbook of Computational Intelligence. Springer Berlin / Heidelberg (2015) 219{237 25. Hullermeier, E.: Does machine learning need fuzzy logic? Fuzzy Sets and Systems 281 (2015) 292{299 26. Dubois, D., Prade, H.: Possibility theory and its applications: Where do we stand? In Kacprzyk, J., Pedrycz, W., eds.: Springer Handbook of Computational Intelligence. Springer Berlin / Heidelberg (2015) 31{60 27. Zadeh, L.A.: Toward human level machine intelligence - Is it achievable? the need for a paradigm shift. IEEE Computational Intelligence Magazine 3(3) (2008) 11{22 28. Zadeh, L.A.: A Note on Z-numbers. Information Sciences 181(14) (2011) 2923{ 2932