=Paper=
{{Paper
|id=Vol-2071/paper1
|storemode=property
|title=Building Cognitive Cities with Explainable Artificial Intelligent Systems
|pdfUrl=https://ceur-ws.org/Vol-2071/CExAIIA_2017_paper_1.pdf
|volume=Vol-2071
|authors=Jose M. Alonso,Corrado Mencar
|dblpUrl=https://dblp.org/rec/conf/aiia/AlonsoM17
}}
==Building Cognitive Cities with Explainable Artificial Intelligent Systems==
https://ceur-ws.org/Vol-2071/CExAIIA_2017_paper_1.pdf
Building Cognitive Cities with Explainable
Artificial Intelligent Systems
Jose M. Alonso1 and Corrado Mencar2
1
Centro Singular de Investigación en Tecnoloxı́as da Información (CiTIUS),
Universidade de Santiago de Compostela, Santiago de Compostela, Spain
josemaria.alonso.moral@usc.es
2
Department of Informatics, University of Bari “Aldo Moro”, Bari, Italy,
corrado.mencar@uniba.it
Abstract. In the era of the Internet of Things and Big Data, data sci-
entists are required to extract valuable knowledge from the given data.
This challenging task is not straightforward. Data scientists first ana-
lyze, cure and pre-process data. Then, they apply Artificial Intelligence
(AI) techniques to automatically extract knowledge from data. How-
ever, nowadays the focus is set on knowledge representation and how
to enhance the human-machine interaction. Non-expert users, i.e., users
without a strong background on AI, require a new generation of explain-
able AI systems. They are expected to naturally interact with humans,
thus providing comprehensible explanations of decisions automatically
made. In this paper, we sketch how certain computational intelligence
techniques, namely interpretable fuzzy systems, are ready to play a key
role in the development of explainable AI systems. Interpretable fuzzy
systems have already successfully contributed to build explainable AI
systems for cognitive cities.
Keywords: Explainable Computational Intelligence, Interpretable Fuzzy
Systems, Natural Language Generation, Cognitive Cities
1 Introduction
The quest of “comprehensibility” and “explanation” in the field of Computa-
tional Intelligence is rooted in the more general paradigm of Computing With
Words (CWW) stated by Zadeh more than two decades ago [1]. CWW comes
into play when there is the need of representing and manipulating knowledge
expressed in Natural Language (NL). CWW is closely related to the Compu-
tational Theory of Perceptions (CTP) [2] and is based on Fuzzy Set Theory
(FST) [3].
CWW operates at the semantic level, i.e., inference is carried out by con-
sidering the semantic definition of words (and their connectives) in terms of
fuzzy sets and operators. In fact, CWW relies on the assumption that fuzziness
is intrinsic in the semantics of most linguistic terms. Moreover, as a distinctive
feature, it deals naturally with fuzziness at the inference process and produces
results represented linguistically.
Copyright © 2018 for this paper by its authors. Copying permitted for private and academic purposes.
Fuzzy (rule-based) systems use CWW to represent linguistic knowledge and
carry out approximate reasoning under imprecision and uncertainty, which makes
their use appealing in many application domains. Interpretable fuzzy systems
(IFS) are fuzzy systems whose behavior is human-centric, i.e., they can be eas-
ily understood, trusted on or accounted for human beings. IFS are especially
appreciated in domains with advanced human-computer collaboration such as
Medicine [4]. However, it is noteworthy that building IFS is a matter of care-
ful design [5]. In a nutshell, interpretability is not granted by the adoption of
FST, which represents a necessary yet not a sufficient requirement for CWW
and human-centric computing [6]. Accordingly, fuzzy designers must look for a
good interpretability-accuracy trade-off when designing fuzzy systems for spe-
cific applications.
IFS combined with NL generation techniques are a recent development [7],
well suited to address one of the last challenges stated by the USA Defense
Advanced Research Projects Agency (DARPA) [8]:
“Even though current AI systems offer many benefits in many ap-
plications, their effectiveness is limited by a lack of explanation ability
when interacting with humans.”
Notice that powerful AI techniques like Deep Learning [9] perform very well
when dealing with low-level perceptions and pattern recognition tasks but the
resulting models act as black boxes, i.e. their main drawback is their lack of
explanation ability what makes them hard to be trusted by humans.
Black-box models are not suitable for Cognitive Cities. A Cognitive City [10]
evolves from a smart city by introducing collaborative intelligence between the
city and its citizens who are willing to produce and receive information (thus
requiring mutual communication of information and knowledge between humans
and machines). Accordingly, Cognitive Cities should be populated with explain-
able AI systems able to share with humans their view of the world.
In this paper, we will briefly describe the concept of Interpretability within
CWW (Sec. 2), then we outline a novel framework to build explainable AI sys-
tems for Cognitive Cities, also enumerating some examples of applications where
this framework has been applied (Sec. 3). Finally, Section 4 draws main conclu-
sions and sketch future work.
2 Interpretability as Semantic Co-intension
Human perceptions are defined by intrinsic and extrinsic attributes regarding
human senses (sight, smell, taste, touch and hearing). In addition, human pleas-
antness depends on what people experience (Perceptions), but also on what peo-
ple expect (Cognitions) which is influenced by common and personal background
(e.g., context or mood). In most cases, human beings use NL for describing their
perceptions and for construing their experience [11].
The use of FST and CWW for designing comprehensible intelligent sys-
tems fits with the intuitive idea that many concepts represented in NL have a
Fig. 1. Flow diagram of a perception-based system.
perception-based nature. An agent can perceive objects of the reality and asso-
ciate them to concepts which can be verbalized by means of linguistic labels (see
Fig. 1). On the one hand, perception is a cognitive act of transforming sensory
data into mental representations. On the other hand, linguistic labels designate
concepts which refer to the objects of the world. Fuzzy logic (in the wide sense)
moves from the assumption that objects belong to a reality that is character-
ized by continuous features, which are perceived by sensors able to observe and
process such continuity. In consequence, the concepts reflect the continuity of
the reality and, therefore, accommodate the properties of graduality (because
objects’ features vary without sharp boundaries) and granularity (because each
concept refers to a multitude of objects). Graduality and granularity are the key
components of fuzzy sets which are the basic building blocks of several theories
and methodologies, including CWW.
Thus, FST seems very suitable to formalize perception-based intelligent sys-
tem which are interpretable to human beings. However, the use of FST to design
intelligent systems is not enough to ensure interpretability. More than 30 years
ago, Michalski introduced his “Comprehensibility Postulate” (CP) [12], which
can be conveniently used as a starting point for discussing comprehensibility (or
interpretability) in knowledge-based systems:
“The results of computer induction should be symbolic descriptions
of given entities, semantically and structurally similar to those a human
expert might produce observing the same entities. Components of these
descriptions should be comprehensible as single chunks of information,
directly interpretable in natural language, and should relate quantitative
and qualitative concepts in an integrated fashion.”
Notice that the CP was formulated without explicitly considering FST; nev-
ertheless, it highlights several key-points that are worth observing. Firstly, the
human-centrality of the results of a computer induction process, which should
be described symbolically as a necessary condition to communicate information.
However, such symbols are not “empty”, but they should represent chunks of
information, namely, information granules, that are groups of data tied together
by semantic relationships (e.g., proximity or similarity) [13, 14].
Moreover, such symbols should be directly interpretable in NL. However, this
does not boil down to a simple selection of symbols within a corpus, i.e., a vo-
cabulary made of NL terms for a specific domain. In fact, the CP requires the
interpretation of symbols to be in NL. This is a requirement on the semantics
of symbols and relations between them, i.e., on the information granules they
denote. More precisely, on one hand information granules (resulting from com-
puting processes) should conform with concepts a human can conceive; on the
other hand, NL terms convey an implicit semantics (which depends also on the
context), that is shared among all human beings speaking the same language.
Therefore, a symbol coming from NL can be used to denote an information gran-
ule only if the implicit semantics of the symbol highly matches with the explicit
semantics of the information granule.
This relation between both implicit and explicit semantics is called “semantic
co-intension”. It is inspired by the concept of model co-intension of Zadeh [15]:
“In the context of modeling, co-intension is a measure of proximity
of the input/output relations of the object of modeling and the model.
A model is co-intensive if its proximity is high.”
We use the Zadeh’s notion of co-intension to relate the semantics defined by
information granules with the underlying semantics held by the used symbols:
the results of an inductive process are interpretable if the explicit semantics
embedded in the model is co-intensive with the implicit semantics inferred by
the user while interpreting the model. In this sense, we think FST and CWW
can naturally contribute to explainable AI.
3 Explainable AI Systems for Cognitive Cities
The concept of “Cognitive City” was first introduced by Novak [16] and it em-
phasizes the role of learning, memory creation and experience retrieval as central
processes for coping with current challenges of efficiency, sustainability and re-
silience [17]. In a Cognitive City, an intelligent and distributed collaboration
takes place between the city and its citizens who act as “sensors” as well as
“recipients”. Thus, it arises a novel form of intelligence, which may be called
“collaborative intelligence”, where people and machines collaborate to solve com-
plex problems [18]. Of course, effective human-machine communication requires
mutual understanding. To achieve this goal, a Cognitive City must be popu-
lated with explainable AI systems able to share with humans their semantic
co-intensive view of the world.
We have already empirically shown the benefits of combining IFS with NL
generation techniques to make the interaction between humans and fuzzy sys-
tems more natural [7]. In addition, we described the role of interpretable fuzzy
systems in designing Cognitive Cities in [19]. In short, we created a new frame-
work (see Fig. 2) where we combined (1) the so-called Linguistic Description
of Complex Phenomena (LDCP) [20], which implements and enhances Zadeh’s
CTP, and (2) the NL generation pipeline proposed by Reiter and Dale [21].
Fig. 2. A Framework for building explainable AI systems in Cognitive Cities.
The framework consists of three main stages:
1. Data acquisition is made through both humans and machines. Raw data are
pre-processed in accordance with the communicative goal, the user model,
the knowledge source, and the discourse history.
2. Data analysis and interpretation is carried out by IFS which are carefully
designed off-line. Notice that domain knowledge is embedded into linguis-
tic rules which relate semantically co-intensive concepts taken from a corpus
specific for the target application. They are in the core of the so-called Gran-
ular Linguistic Model of Phenomena (GLMP) which consists of a hierarchical
network of perception mappings and computational perceptions defined by
IFS. Moreover, the relevance calculation is the basis to generate customized
messages. Only the most relevant pieces of information will be added to the
final report.
3. The extracted knowledge (expressed in the form of linguistic pieces of infor-
mation coming out of the so-called content determination stage) is presented
to humans in NL (after document planning and careful computational lin-
guistic realization which includes lexicalization, referring expression genera-
tion and aggregation).
We have already applied this framework (1) to provide citizens in Gijón
(Spain) [19] with information related to the public bus transport; and (2) to pro-
vide citizens of European cities with details about their energy consumption [22].
Notice that the last work was carried out in the context of the NatConsumers
Horizon2020 EU project3 . In addition, the interested reader is referred to the
R software package called rLDCP [23] which provides a first open-source imple-
mentation of this framework.
Let’s introduce briefly how the framework sketched in Fig. 2 was implemented
for one of the pilots in the NatConsumers project. Input data elements were as
follows:
– Communicative goal. The main factors influencing residential energy con-
sumption in European countries were identified in a previous study. Here,
the goal was the automatic generation of linguistic advice for saving en-
ergy at home, remarking the main factors to consider and the main actions
to carry out. Three kind of energy consumptions are considered: General,
specific and standby consumptions.
– User model. We identified the target consumers who will receive the cus-
tomized messages. Moreover, we classify consumers regarding both attitudi-
nal and physical taxonomies which were previously defined by other partners
in the project.
– Knowledge source. Information related to the energy consumption for each
household involved in the project.
– Discourse history. The sequence of messages already generated for each spe-
cific consumer.
Fig. 3 shows a schematic excerpt of the IFS/GLMP described in [22]. On the
one hand, the perception mappings (PM) at the lowest level of the hierarchy
(e.g., 1P MCD ) are implemented by strong fuzzy partitions with linguistic terms
established in accordance with the corpus given for this application domain.
Thus, we preserve interpretability at partition level. Then, PMs in upper levels
(2P M ) are implemented by fuzzy IF-THEN rule sets, also paying attention to
interpretability constraints. For example, 2P MRD includes rules such as “IF
Average Cluster Consumption is High AND Hourly Household Consumption is
High THEN The Household Consumption is Similar to Other Households in the
Same Cluster”.
In addition, Fig. 3 includes some illustrative examples of sentences which
were generated as instantiations of the related templates, after running the fuzzy
inference process, for a specific case. Moreover, Fig. 4 depicts the final report
generated for the same illustrative case. Notice that the report combines both
NL texts and graphs with the aim of maximizing interpretability.
3
H2020 Coordination and Support Action (CSA) [http://www.natconsumers.eu]
Fig. 3. Example of IFS/GLMP for NatConsumers project.
Fig. 4. Illustrative example of customized report for one of the anonymous households
involved in the NatConsumers project.
4 Conclusions and Prospects
Taking profit of our previous background as designers of interpretable fuzzy
systems, we have gone a step forward in the generation of explainable AI systems.
They are ready to convey citizens with valuable knowledge (represented by NL
texts) which is automatically extracted from data. Nevertheless, we are aware
this is only a first step and a lot of work remains to do. For example, cost-
effectiveness, scalability or interface with other AI systems will turn up as key
issues to address with the aim of applying our proposal universally in a cognitive
city. In the short term, we plan to extend our framework with interactive dialog
systems (endowed with argumentation theory approaches) which are likely to
enhance the human-machine interaction capability of our explainable AI systems.
Results on applied research can be profitably used in theoretical research
on interpretability. Roughly speaking, the results achieved on studying inter-
pretability of fuzzy systems are mostly based on common-sense arguments and
heuristic approaches [24]. This recently gave rise to some critical positions on
the state-of-art [25], which stimulates new directions of investigation. From a
theoretical viewpoint, research on comprehensibility can be cast into the field
of communications between granular worlds, where agents are endowed with a
granular representation of knowledge and want to communicate some informa-
tion. In this abstract setting, several levels of representation of information can
be established, from numerical (representing signals) to symbolic (representing
terms), possibly passing through intermediate, hybrid levels. In this sense, Gran-
ular Computing could be intended as the cognitive bridge from the numerical
reality to the symbolic world, where abstract relations emerge from specific de-
pendencies. Namely, in analyzing the main theme of communication between
granular worlds, the main properties of semantic communication could be inves-
tigated by using the tools offered by Granular Computing. Examples of these
properties are imprecision, uncertainty, fuzziness or vagueness. All these proper-
ties are intimately connected with NL, and, traditionally, systematically removed
in view of communicating information to machines and between machines. A lot
of work has already been done in literature, by the definition of theories like
Possibility Theory [26], Precisiated NL [27] or the theory of Z-numbers [28]. Yet,
there is large room for further research, especially if aimed at giving computa-
tional solutions, i.e., solutions that can be translated into computer programs.
Acknowledgements
This work was supported by TIN2014-56633-C3-3-R (ABS4SOWproject) from
the Spanish “Ministerio de Economı́a y Competitividad”. Financial support from
the Xunta de Galicia (Centro singular de investigación de Galicia accreditation
2016-2019) and the European Union (European Regional Development Fund -
ERDF), is gratefully acknowledged.
References
1. Zadeh, L.A.: From computing with numbers to computing with words - From
manipulation of measurements to manipulation of perceptions. IEEE Transactions
on Circuits and Systems I: Fundamental Theory and Applications 46(1) (1999)
105–119
2. Zadeh, L.A.: A new direction in AI: toward a computational theory of perceptions.
Artificial Intelligence Magazine 22(1) (2001) 73–84
3. Zadeh, L.A.: Fuzzy sets. Information and Control 8(3) (1965) 338–353
4. Alonso, J.M., Castiello, C., Lucarelli, M., Mencar, C.: Modeling interpretable
fuzzy rule-based classifiers for medical decision support. In: Medical Applications
of Intelligent Data Analysis: Research Avancements. IGI GLOBAL (2012) 255–272
5. Trillas, E., Eciolaza, L.: Fuzzy Logic: An Introductory Course for Engineering
Students. Springer (2015)
6. Bargiela, A., Pedrycz, W.: Human-Centric Information Processing Through Gran-
ular Modelling. Studies in Computational Intelligence. Springer, Berlin, Heidelberg
(2009)
7. Alonso, J.M., Ramos-Soto, A., Reiter, E., van Deemter, K.: An exploratory study
on the benefits of using natural language for explaining fuzzy rule-based systems.
In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Naples, Italy
(2017) 1–6 http://dx.doi.org/10.1109/FUZZ-IEEE.2017.8015489.
8. Gunning, D.: Explainable Artificial Intelligence (XAI). Technical report, Defense
Advanced Research Projects Agency (DARPA), Arlington, USA (2016) DARPA-
BAA-16-53.
9. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)
http://www.deeplearningbook.org.
10. Finger, M., Portmann, E.: What Are Cognitive Cities? In Portmann, E., Finger,
M., eds.: Towards Cognitive Cities. Volume 63 of Studies in Systems, Decision and
Control. Springer International Publishing (2016) 1–11
11. Halliday, M.A.K., Matthiessen, M.I.M.: Construing Experience through Meaning:
A Language-based Approach to Cognition. Continuum (1999)
12. Michalski, R.S.: A theory and methodology of inductive learning. Artificial Intel-
ligence 20(2) (1983) 111–161
13. Zadeh, L.A.: Toward a theory of fuzzy information granulation and its centrality
in human reasoning and fuzzy logic. Fuzzy Sets and Systems 90(2) (1997) 111–127
14. Bargiela, A., Pedrycz, W.: Granular computing: an introduction. Kluwer Academic
Publishers (2003)
15. Zadeh, L.A.: Is there a need for fuzzy logic? Information Sciences 178(13) (2008)
2751–2779
16. Novak, M.: Cognitive Cities. In: Intelligent Environments. Elsevier (1997) 386–420
17. Mostashari, A., Arnold, F., Mansouri, M., Finger, M.: Cognitive cities and intelli-
gent urban governance. Network Industries Quarterly 13(3) (2011) 4–7
18. Epstein, S.L.: Wanted: Collaborative intelligence. Artificial Intelligence 221 (4
2015) 36–45
19. Alonso, J.M., Castiello, C., Mencar, C.: The role of interpretable fuzzy systems
in designing cognitive cities. In Portmann, E., Seising, R., Tabachi, M., eds.:
Designing Cognitive Cities: Linking Citizens to Computational Intelligence to Make
Efficient, Sustainable and Resilient Cities a Reality. Studies in Systems, Decision
and Control. Springer Verlag (2017) 1–21
20. Trivino, G., Sugeno, M.: Towards linguistic descriptions of phenomena. Interna-
tional Journal of Approximate Reasoning 54 (2013) 22–34
21. Reiter, E., Dale, R.: Building natural language generation systems. Cambridge
University Press (2000)
22. Conde-Clemente, P., Alonso, J.M., Trivino, G.: Towards automatic generation
of linguistic advice for saving energy at home. Soft Computing (2016) 1–15
http://dx.doi.org/10.1007/s00500-016-2430-5.
23. Conde-Clemente, P., Alonso, J.M., Trivino, G.: rLDCP: R package for text gener-
ation from data. In: Proceedings of the IEEE International Conference on Fuzzy
Systems (FUZZ-IEEE), Naples, Italy (2017) 1–6 http://dx.doi.org/10.1109/FUZZ-
IEEE.2017.8015487.
24. Alonso, J.M., Castiello, C., Mencar, C.: Interpretability of Fuzzy Systems: Current
Research Trends and Prospects. In Kacprzyk, J., Pedrycz, W., eds.: Springer
Handbook of Computational Intelligence. Springer Berlin / Heidelberg (2015)
219–237
25. Hüllermeier, E.: Does machine learning need fuzzy logic? Fuzzy Sets and Systems
281 (2015) 292–299
26. Dubois, D., Prade, H.: Possibility theory and its applications: Where do we stand?
In Kacprzyk, J., Pedrycz, W., eds.: Springer Handbook of Computational Intelli-
gence. Springer Berlin / Heidelberg (2015) 31–60
27. Zadeh, L.A.: Toward human level machine intelligence - Is it achievable? the need
for a paradigm shift. IEEE Computational Intelligence Magazine 3(3) (2008) 11–22
28. Zadeh, L.A.: A Note on Z-numbers. Information Sciences 181(14) (2011) 2923–
2932