=Paper=
{{Paper
|id=Vol-2350/paper1
|storemode=property
|title=Knowledge Engineering and Machine Learning for Design and Use in Cyber-Physical Environments
|pdfUrl=https://ceur-ws.org/Vol-2350/paper1.pdf
|volume=Vol-2350
|authors=Michael Walch
|dblpUrl=https://dblp.org/rec/conf/aaaiss/Walch19
}}
==Knowledge Engineering and Machine Learning for Design and Use in Cyber-Physical Environments==
https://ceur-ws.org/Vol-2350/paper1.pdf
Knowledge Engineering and Machine Learning
for Design and Use in Cyber-Physical Environments
Michael Walch
University of Vienna
Universitätsring 1
1010 Vienna, Austria
Abstract at design-time. The resulting semi-formal artifacts also have
a potential to be connected to CPS at run-time. To achieve
A required task for developing cyber-physical systems (CPS) this potential, a specialization of the conceptual modeling
with people and business aspects in the loop is to capture
human knowledge & design in an explicit manner. Knowl-
approach is necessary. Correspondingly, the s*IoT concep-
edge engineering can be applied to tackle this task. Thereby, tual modeling approach has been proposed to bring together
the idea is to utilize human knowledge & design in an au- in an intelligent manner (1) conceptual models that decom-
tomated manner throughout the life-cycle of CPS. In partic- pose human knowledge & design and (2) operation envi-
ular, one challenge is to connect conceptual models and op- ronments that further abstract from intricate capabilities of
eration environments. The former focuses on capturing and CPS (Walch and Karagiannis 2019). The result thereof are
decomposing human knowledge & design about people, busi- ”smart” models that can be understood by humans and CPS.
nesses, and CPS using semi-formal concepts that can be exe-
Connectivity between conceptual models and operation
cuted through procedures for sequential semantics, while the
latter focuses on continuous-time models and CPS that op- environments can be realized by different means. One option
erate in the physical world at run-time. By connecting con- is to develop conceptual models and operation environments
ceptual models and operation environments in an intelligent by hand, which implies that different stakeholders have to
manner, the s*IoT conceptual modeling approach is able to invest a great amount of effort. Connecting these manually
align two levels of iterpretability: one for people concerned developed artifacts is possible by developing application-
with feasible, desirable, and viable designs and one for ef- specific interfaces, which again requires human effort for
ficient, automated, and reliable use of CPSs. Therby, s*IoT each interface. Another option is to employ the semantic
supersedes the approach of developing application-specific web stack to automate the connection of conceptual models
interfaces between conceptual models and operation environ- and operation environments. The semantic web stack pro-
ments. Rather, s*IoT employs the semantic web stack to re-
vides benefits for topics that require diversity, synthesis, and
duce the human effort for developing application-specific in-
terfaces. While this is a promising approach, the question is definiteness (Janowicz et al. 2014). As a consequence, tech-
if the integration of machine-learning approaches offers ad- nologies from the semantic web stack are adopted in the cur-
ditional benefits for s*IoT, as machine-learning approaches rent version of the s*IoT conceptual modeling approach. In
can presumably further eliminate human effort associated detail, ontologies and reasoning are employed by the s*IoT
with technologies from the semantic web stack. This paper modeling method and tool. This enables automation by fur-
presents an arguable opinion about the issue. ther decomposing conceptual models into elements with for-
mal semantics that are matched to the formal semantics ab-
stracting capabilities of operation environments. While em-
Introduction ploying the semantic web stack allows for the elimination of
While most cyber-physical systems (CPS) are intended to a large portion of manual work, some aspects still have to
enhance the capabilities of people and businesses, this is a be largely developed by hand in a labor-intensive and error-
problem because it is difficult for CPS to know people’s and prone process that has become a key bottleneck (Doan et
businesses’ requirements (Sowe et al. 2016). Making human al. 2004). Therefore, a third potential option is to employ
knowledge & design accessible can help CPS to make intel- machine learning. Thereby, the focus is on opportunities for
ligent decisions and achieve their goals which are ultimately advanced automation.
the goals of people and businesses. Conceptual modeling is The methodology of this paper is to present an arguable
an approach to make human knowledge & design explicit opinion about the combination of knowledge engineering
in a semi-formal manner that can be understood by humans and machine learning. In particular, a potential update of
Copyright held by the author(s). In A. Martin, K. Hinkelmann, A. the s*IoT conceptual modeling approach is explored by ex-
Gerber, D. Lenat, F. van Harmelen, P. Clark (Eds.), Proceedings of amining the opportunities of machine learning. Therefore,
the AAAI 2019 Spring Symposium on Combining Machine Learn- three cases are discussed on the topic of automating the
ing with Knowledge Engineering (AAAI-MAKE 2019). Stanford connection between conceptual models and operation envi-
University, Palo Alto, California, USA, March 25-27, 2019. ronments. The goal is to describe a direction along which
Language
Abstraction
Model of
Concepts Modeling
f
no indirect Method
os itio ign model of
s Engineering
co mp & De
De ge
led Models using
n ow Concepts Modeling
K
Conceptual
Semantic
Technologies Models
CPS Operation
Environment
Execution f
Environment no
a c tio i es
str abilit
Realizer Role Ab a p
C "Smart"
CPS Models
Run-Time
Environment
Domain
Abstraction
Figure 1: Topic for which the applicability of machine learning is analyzed.
future research can progress. This direction is framed by plicit knowledge & design that is decomposed by human-
the conceptual framework of specializing the design sci- oriented and machine-oriented representations. These rep-
ence paradigm with a model-based approach. Additionally, a resentations are conceptual models. Conceptual models are
meta-level view is applied that considers the resulting mod- semi-formal in the sense that they can be processed by ICT
els as systems under study. Research questions for this pa- systems but also contain semantics that require human in-
per are to map the opportunities of machine learning for terpretation. To build conceptual models, modeling methods
connecting conceptual models and operation environments, and tools are required. A linguistic, procedural, and algorith-
to structure information from concrete cases in which ma- mic abstraction of conceptual models, their modeling meth-
chine learning is needed, and to conclude future research di- ods, and modeling tools is provided by metamodels (Kara-
rections. To answer the research questions, the method of giannis and Kühn 2002). Together, metamodels and models
conceptual analysis is applied. An analysis of the results support the engineering of knowledge & design in an ag-
in terms of strengths, weaknesses, opportunities and threats ile cycle (Karagiannis 2015). Thereby, engineering can be
(SWOT) is conducted for validation purposes. viewed as a task of assembling representational components
Following the introduction, the paper is structured in five rather than axiom-writing (Clark et al. 2001). As a conse-
sections. First, foundations and related work are summarized quence, an engineer is not always necessary when knowl-
on conceptual modeling, CPSs, and their connection. Af- edge & design of subject matter experts is made explicit,
terwards, the s*IoT conceptual modeling approach is intro- as the latter can directly interact with conceptual models
duced briefly, also in terms of how it benefits from employ- using representations familiar or intuitive to them. Cyber-
ing the semantic web stack. Based on these two sections, an physical systems are feedback systems involving cyber and
update is suggested on how the s*IoT modeling method and physical components, which enables innovative applications
tool can be combined with machine learning. The results are for, e.g., Industry 4.0, Society 5.0, and Smart Cities. The
critically reflected in a discussion section before the conclu- difference to traditional ICT systems is that there is no
sion. clear separation, but rather an intersection of physical pro-
cesses and software (Shi et al. 2011). However, modeling
Foundations and Related Work is required to enable different multidisciplinary teams to
Figure 1 shows the topic addressed by the s*IoT concep- work together on the problem of designing and using CPS.
tual modeling approach. This topic is analyzed in this paper As a consequence, CPSs create new challenges for model-
regarding the applicability of machine learning. Therefore, ing not covered by traditional modeling methods for ICT
foundations and related work are briefly discussed. systems (Derler, Lee, and Sangiovanni-Vincentelli 2011;
The topic under scrutiny can be structured in three main Sharma et al. 2014). That is because traditional ICT sys-
parts: conceptual modeling, CPS, and the connection be- tems rely on models that encoding knowledge & design
tween the two. Conceptual modeling can be employed in a through sequential steps, while CPSs are deeply rooted in
distilling cycle to make human knowledge & design explicit the physical world, which requires continuous-time models
(Karagiannis, Buchmann, and Walch 2017). The result is ex- that are working with, e.g., solvers that numerically approx-
imate the solutions to differential equations. Connectivity Design-Time Aspects of Connection Run-Time
between conceptual models and CPS requires an integra-
Descriptions
Decomposition
tion of design-time and run-time aspects. Therefore, con-
of
Behaviour
Abstraction
ceptual models can be extended by operational semantics Human Realization of
Requirements
Knowledge Cyber-Physical
and Design Systems
(Lehmann et al. 2010) on the one end of the connection. On
for
Function Structure
the other, CPS can be understood as a run-time environment
for executable models. The run-time environment can be en-
Line of
capsulated by an execution environment that provides inter- Alignment
faces on the same level of abstraction as executable mod-
els. Together, run-time environment and execution environ- Figure 2: Aspects of connecting conceptual models and op-
ment make up the operation environment of executable mod- eration environments.
els. However, in reality the connection between executable
models and execution environments is a complex issue, as effort that does not scale. To alleviate this issue, the s*IoT
there is no fixed point of alignment (Walch and Karagiannis modeling method and tool integrates technologies from the
2019). semantic web stack.
After this short introduction to the foundations of the topic
Figure 2 shows aspects of connecting conceptual mod-
under scrutiny, related work for the connection between con-
els and operation environments in the space between hu-
ceptual models and operation environments is discussed. Re-
man knowledge & design and CPS capabilities. Require-
garding the execution of conceptual models, there is a ben-
ments can be derived from the former while descriptions can
efit for conceptual models that are cognitively adequate for
be derived for the latter. Requirements and descriptions can
humans and processable by machines, as such models could,
be modelled in terms of function, structure, and behaviour.
e.g., enable communication and collaboration, support deci-
Structural aspects refer to components and their relation-
sion makers through analysis and simulation, and automate
ships, functional aspects to the hierarchy of abstract roles
enterprise operations through model execution (Hinkelmann
and concrete realizations (i.e., goals and measurable effects),
et al. 2018). To harness these benefits, formal semantics of
and behavioural aspects to the performance over time. Be-
conceptual models are essential (Hinkelmann et al. 2016).
tween all these aspects, gaps may exist with regards to com-
Examples of conceptual models that are extended by formal
putational paradigms, granularity of detail, and language of
operational semantics are model types like UML which are
presentations. In s*IoT, connecting these aspect in models
extended by fUML (Dévai et al. 2015), SysML which re-
is supported by technologies from the semantic web stack.
quires dedicated execution environments (Wolny 2017), and
The resulting benefit is that the point of alignment between
BPMN which can be put to use by workflow engines (De Gi-
requirements and descriptions is not fixed for specific ap-
acomo et al. 2017). However, only few types of models can
plications, but rather it allows for added flexibility, intelli-
be executed (Thalheim 2018), which is a problem due to ag-
gence, and automation when connecting different kinds of
ile and fast changing modeling requirements and especially
conceptual models and operation environments. This is pos-
considering that CPS could be employed to operational-
sible because the semantic web stack provides technologies
ize models. Regarding the abstraction of CPS in conceptual
that elevate the connection from application-specific inter-
models, the PRINTEPS project is a recent example (Morita
faces to semantic mappings between the involved elements.
et al. 2018). PRINTEPS commits to the robot operating sys-
An example for a concrete application case is to model hu-
tem (ROS) as an abstraction of the run-time environment
man knowledge & design about, e.g., an Industry 4.0 pro-
that different robots offer. This execution environment is re-
duction process, to annotate the resulting conceptual model
flected in domain-specific conceptual models that are ex-
with formal semantics, and to discover suitable services of
tended with operational semantics for model execution. Fur-
CPS for model execution.
ther model abstraction allows for conceptual models that are
intuitive for domain experts. In PRINTEPS, some of the ab-
straction and decomposition mechanisms that relate differ- s*IoT and Machine Learning
ent conceptual models and ROS are automated. However, To improve the s*IoT conceptual modeling approach, the
one problem is that the commitment to ROS is not applicable benefits of machine learning are examined with regards
to all kinds of CPS, especially as CPS architectures change to the issue of connecting conceptual models and opera-
from hierarchical to service-oriented (Foehr et al. 2017; tion environments. Therefore, three cases are presented. In
Gruettner, Richter, and Basten 2017). these three cases, the current version of the s*IoT model-
ing method and tool are applied. As this implies the use
The s*IoT Conceptual Modeling Approach of technologies from the semantic web stack, the results
The s*IoT conceptual modeling approach has been pro- are ”smart” models. Additionally, ”smart” models are also
posed due to new requirements that emerged from AMME extended by employing machine learning on a proof-of-
and changing architectures of CPS (Walch and Karagian- concept basis in the three presented cases.
nis 2019). In particular, problems have been identified when Case One - Recognizing the Structure of Cyber-
conceptual models are put to use, as the manual align- Physical Environments: In this case, the s*IoT modeling
ment of conceptual models and operation environments by method and tool are applied to model a mock-up coffee mak-
application-specific interfaces requires human development ing process and to execute that process in a cyber-physical
environment that contains a robotic arm and coffee ingredi- Discussion
ents. To enable model execution, the structure of the cyber- A conclusive SWOT analysis is an effective approach for
physical environment is abstracted to the modeling layer. rationalization. Therefore, a SWOT analysis is conducted
This is done manually by humans who created an ontology to validate the opinions formed in this paper about the po-
that extends the model of the mock-up coffee making pro- tentials of machine learning for the s*IoT conceptual mod-
cess. The ontology contains information about objects in the elling approach. Furthermore, the SWOT analysis general-
cyber-physical environment like the coffee ingredients and izes from the three presented cases.
the robotic arm, e.g., their x, y, and z positions. By com-
The strengths of machine learning for s*IoT are: (1) Hu-
bining all these elements in ”smart” models, the execution
man effort associated with technologies from the semantic
of the process becomes possible. Currently, the options that
web stack can be reduced. This allows for greater flexibil-
machine learning provides to this case are being evaluated.
ity when connecting conceptual models and operation envi-
In particular, image recognition was used to update the on-
ronments. (2) New application scenarios become possible as
tology of objects based on real-time data. As a consequence,
modeling methods and tools evolve. (3) The quality of con-
it is feasible that no manual intervention would be necessary
ceptual models and CPS is increased as machine learning en-
in case the amount, position, or size of coffee ingredients
ables a tighter connection between the two. The weaknesses
changes, if machine learning approaches were to be inte-
of machine learning for s*IoT are: (1) Additional complex-
grated in the s*IoT modeling method and tool.
ity is introduced as the workload of human stakeholders gets
Case Two - Reasoning Function from Structure: In automated. New sources of error and a lack of tractability
this case, the s*IoT modeling method and tool are applied to are a problem for modeling method engineers and model-
model the function and structure of CPS. By using technolo- ers. (2) Machine learning requires human effort to select ma-
gies from the semantic web stack, it is possible to reason the chine learning paradigms, prepare training data, and super-
function of CPS from their structure. This requires knowl- vise learning algorithms. (3) The applicability of machine
edge engineers and domain experts to define the relation be- learning is related to the availability of training data. This
tween function and structure, e.g., a robotic vehicle that can is somewhat contradictory to conceptual modeling which is
drive and steer has - among other things - two independent often used to capture innovative and creative ideas. The op-
motors, wheels, and motor controllers. Currently, it is evalu- portunities of machine learning for s*IoT are: (1) Collab-
ated how this kind of reasoning can be supported by machine oration is facilitated among the machine learning commu-
learning. Previously, the structure of a CPS had to be mod- nity, the conceptual modeling community, and the CPS com-
eled by hand, as well as the relation between function and munity. This creates new chances for research, application,
structure. Existing models of that kind were used to orga- and education. (2) The dissemination of the s*IoT modeling
nize training sets for machine learning. Based on these train- method and tool can be accelerated by embracing the current
ing sets and machine learning technologies, it was possible trend of machine learning. (3) By automating human effort,
to identify the structure of CPS from images and to clas- human resources become available. These human resources
sify CPS by their function. A thorough comparison of ben- can be used for creative and innovative tasks. The threats
efits and drawbacks between the currently employed tech- of machine learning for s*IoT are: (1) Machine learning is a
nologies from the semantic web stack and machine learning complex topic and human resources are sparse. Furthermore,
should be able to provide further insights. projects that involve machine learning are often difficult to
plan due to the lack of previous results. (2) It is possible that
Case Three - Modeler Assistant based on CPS Be- the trend of machine learning changes as it did before. The
haviour: In this case, the goal is to reduce the time and danger is to focus on soon to be outdated aspects of machine
cognitive effort modelers spent, by providing intelligent as- learning. (3) A social and ethical perspective has to be con-
sistants to modelers. These assistants should actively clas- sidered when tasks of humans are automated. Furthermore,
sify the modeler’s activities, predict future tasks, and proac- all kinds of risks have to be considered when humans are
tively perform those tasks automatically (Panton et al. 2006). replaced by automation.
One example for this is case-based reasoning, where knowl-
edge of previously experienced cases is used to propose so-
lutions to changing requirements (Martin and Hinkelmann Conclusion
2018). Currently, s*IoT offers no intelligent assistants for Knowledge engineering is necessary in the life-cycle of
modelers. Therefore, machine learning can be explored to CPS, as human knowledge & design is essential for CPS
fill this gap. The concept is that, as processes are being put with people and businesses in the loop. In the life-cycle
to use by CPS, the feedback data from CPS behaviour can of CPS, conceptual models are knowledge engineering ar-
be collected. This feedback can be used in machine learn- tifacts that have to be connected to operation environments.
ing to classify good and bad patterns of processes. Based Connecting conceptual models and operation environments
on this classification, it should be possible to predict how is elevated by s*IoT from an application-specific develop-
newly modeled processes will behave. This prediction could ment effort towards a systematic approach that makes use
be made available to the modelers of processes during their of technologies from the semantic web stack. While this
modeling task. After reviewing the necessary machine learn- is a promising endeavor, this paper is exploring advanced
ing technologies, it is feasible that progress can be made to- options for elevating the connection of conceptual models
wards developing a prototype for this case. and operation environments even further. In particular, the
reemerging trend of machine learning is evaluated regarding Janowicz, K.; Van Harmelen, F.; Hendler, J. A.; and Hitzler,
benefits it could provide for s*IoT. P. 2014. Why the data train needs semantic rails. AI Maga-
Three cases are presented in which machine learning sup- zine.
ports connecting conceptual models and operation environ- Karagiannis, D., and Kühn, H. 2002. Metamodelling plat-
ments. In the first, the recognized structure of a cyber- forms. In Bauknecht, K.; Tjoa, A. M.; and Quirchmayr, G.,
physical environment is made available for conceptual mod- eds., E-Commerce and Web Technologies, 182–182. Berlin,
els. In the second, functional capabilities of CPS are clas- Heidelberg: Springer Berlin Heidelberg.
sified based on the structure of CPS components. In the
Karagiannis, D.; Buchmann, R.; and Walch, M. 2017. How
third, the behaviour of processes is predicted during mod-
can diagrammatic conceptual modelling support knowledge
eling based on their previous execution by CPS. Preliminary
management? In 25th European Cenference on Information
results from the three cases are promising. The next step is
Systems, ECIS 2017, Proceedings of the 25th European Con-
to integrate the machine learning technologies used in those
ference on Information Systems (ECIS), 1568–1583.
three cases as part of the s*IoT modeling method and tool,
which will allow modelers unfamiliar with the technologies Karagiannis, D. 2015. Agile modeling method engineer-
to make use of them. Furthermore, this allows other model- ing. In Proceedings of the 19th Panhellenic Conference on
ing method engineers to integrate them into their modeling Informatics, PCI ’15, 5–10. New York, NY, USA: ACM.
methods as well. Lehmann, G.; Blumendorf, M.; Trollmann, F.; and Al-
bayrak, S. 2010. Meta-modeling runtime models. In In-
References ternational Conference on Model Driven Engineering Lan-
Clark, P.; Thompson, J.; Barker, K.; Porter, B.; Chaudhri, guages and Systems, 209–223. Springer.
V.; Rodriguez, A.; Thomere, J.; Mishra, S.; Gil, Y.; Hayes, Martin, A., and Hinkelmann, K. 2018. Case-Based Reason-
P.; et al. 2001. Knowledge entry as the graphical assem- ing for Process Experience. Cham: Springer International
bly of components. In Proceedings of the 1st international Publishing. 47–63.
conference on Knowledge capture, 22–29. ACM. Morita, T.; Kashiwagi, N.; Yorozu, A.; Walch, M.; Suzuki,
De Giacomo, G.; Oriol, X.; Estanol, M.; and Teniente, E. H.; Karagiannis, D.; and Yamaguchi, T. 2018. Practice of
2017. Linking data and bpmn processes to achieve exe- multi-robot teahouse based on PRINTEPS and evaluation of
cutable models. In International Conference on Advanced service quality. In 2018 IEEE 42nd Annual Computer Soft-
Information Systems Engineering, 612–628. Springer. ware and Applications Conference (COMPSAC), 147–152.
Derler, P.; Lee, E. A.; and Sangiovanni-Vincentelli, A. L. IEEE.
2011. Addressing modeling challenges in cyber-physical Panton, K.; Matuszek, C.; Lenat, D.; Schneider, D.; Wit-
systems. Technical report, California Univ. Berkeley Dept. brock, M.; Siegel, N.; and Shepard, B. 2006. Common sense
of Electrical Engineering and Computer Science. reasoning–from cyc to intelligent assistant. In Ambient In-
Dévai, G.; Karácsony, M.; Németh, B.; Kitlei, R.; and telligence in Everyday Life. Springer. 1–31.
Kozsik, T. 2015. UML model execution via code genera- Sharma, A. B.; Ivančić, F.; Niculescu-Mizil, A.; Chen, H.;
tion. In EXE@ MoDELS, 9–15. and Jiang, G. 2014. Modeling and analytics for cyber-
Doan, A.; Madhavan, J.; Domingos, P.; and Halevy, A. 2004. physical systems in the age of big data. ACM SIGMETRICS
Ontology matching: A machine learning approach. In Hand- Performance Evaluation Review 41(4):74–77.
book on ontologies. Springer. 385–403. Shi, J.; Wan, J.; Yan, H.; and Suo, H. 2011. A survey of
Foehr, M.; Vollmar, J.; Calà, A.; Leitão, P.; Karnouskos, cyber-physical systems. In Wireless Communications and
S.; and Colombo, A. W. 2017. Engineering of next gen- Signal Processing (WCSP), 2011 International Conference
eration cyber-physical automation system architectures. In on, 1–6. IEEE.
Multi-Disciplinary Engineering for Cyber-Physical Produc- Sowe, S. K.; Simmon, E.; Zettsu, K.; de Vaulx, F.; and Bo-
tion Systems. Springer. 185–206. janova, I. 2016. Cyber-physical-human systems: Putting
Gruettner, A.; Richter, J.; and Basten, D. 2017. Explaining people in the loop. IT professional 18(1):10–13.
the role of service-oriented architecture for cyber-physical Thalheim, B. 2018. Conceptual model notions–a matter
systems by establishing logical links. In Information Sys- of controversy: Conceptual modelling and its lacunas. En-
tems (ECIS), 25th European Conference on. terprise Modelling and Information Systems Architectures
Hinkelmann, K.; Gerber, A.; Karagiannis, D.; Thoenssen, 13:9–27.
B.; van der Merwe, A.; and Woitsch, R. 2016. A new Walch, M., and Karagiannis, D. 2019. How to connect de-
paradigm for the continuous alignment of business and IT: sign thinking and cyber-physical systems: the s*IoT concep-
Combining enterprise architecture modelling and enterprise tual modelling approach. In Proceedings of the 52nd Hawaii
ontology. Computers in Industry 79:77 – 86. Special Is- International Conference on System Sciences.
sue on Future Perspectives On Next Generation Enterprise Wolny, S. 2017. A runtime model for SysML. Doctoral
Information Systems. College Cyber-Physical Production Systems 43.
Hinkelmann, K.; Laurenzi, E.; Martin, A.; and Thönssen, B.
2018. Ontology-Based Metamodeling. Cham: Springer In-
ternational Publishing. 177–194.