=Paper=
{{Paper
|id=Vol-3214/WS1Paper3
|storemode=property
|title=Predicting the Unpredictable through Realism in Interoperable Digital Twins
|pdfUrl=https://ceur-ws.org/Vol-3214/WS1Paper3.pdf
|volume=Vol-3214
|authors=Ryan Williamson,Hazel Carlin,Steven Hayward,Paul Goodall,Katherine van Lopik,Bob Young,Andrew West
|dblpUrl=https://dblp.org/rec/conf/iesa/WilliamsonCHGLY22
}}
==Predicting the Unpredictable through Realism in Interoperable Digital Twins==
https://ceur-ws.org/Vol-3214/WS1Paper3.pdf
Predicting the Unpredictable through Realism in Interoperable
Digital Twins
Ryan Williamson1, Hazel Carlin1, Steven Hayward1, Paul Goodall1, Katherine van Lopik1,
Bob Young1, and Andrew West1
1
Loughborough University, Ashby Road, Loughborough, LE11 3TU, UK
Abstract
Current Digital Twin solutions provide valuable support to production engineers and product
designers in terms of instantaneous feedback on machine performance and simulations of
potential product revisions. However, the information provided by these solutions is focused
on well understood scenarios from real world situations. Current twins cannot deal with
problems that are less well understood. More effective analysis and modelling is required to
provide additional insight so that unexpected issues can be resolved prior to their occurrence.
This workshop paper considers the solution requirements to meet future manufacturing needs
from a perspective of more realistic modelling of manufacturing systems. This requires the
identification of the range of variables that need to be sensed from the physical device, the
complexity of the models that need to be maintained, the range of analyses that need to be
available, the depth and range of knowledge that needs to be applied to these analyses, the
interaction methods with human stakeholders, and the interoperability requirements that need
to be met across multiple sets of digital twins.
Keywords 1
Digital Twin, embedded intelligence, interoperability, manufacturing
1. Introduction
Digital Twins (DTs) can support manufacturing decision making in several areas including: supply
chain and production optimisation to mitigate the challenges of complex and rapidly changing factors,
both internal and external; assessing the manufacturability of new products at the design stage;
enabling smart maintenance approaches to predict when maintenance is required rather than purely
relying upon reactive and preventative methods. Whilst there is a great deal of hype surrounding the
prospect of realistic digital twins, the current reality is that DTs are often developed for bespoke
applications and only mirror aspects of reality, in well-defined and understood scenarios within the
domain of the DT developer.
There are multiple different definitions for the concept of a digital twin [1]. However, there is a
growing consensus that a DT must include more than just data but include sufficient analyses to
predict the behaviour of the physical twin that can thereby enable responsive decisions to be taken to
enhance the physical twin’s performance [2]. The response time necessary will depend on the rate at
which the physical twin is degenerating. Considering this view of a digital twin as a behaviour
predictor leads to several points that need consideration. These include (i) do we know the sorts of
changes in behaviour of the physical system that we want to predict? (ii) do we understand the
potential causes of these changes? (iii) do we have the sensor capability to collect the data that is
Proceedings of the Workshop of I-ESA’22, March 23–24, 2022, Valencia, Spain
EMAIL: r.williamson@lboro.ac.uk (R. Williamson); h.m.carlin@lboro.ac.uk (H. Carlin); s.hayward@lboro.ac.uk (S. Hayward);
P.A.Goodall@lboro.ac.uk (P. Goodall); k.van-lopik@lboro.ac.uk (K. van Lopik); R.I.Young@lboro.ac.uk (B. Young);
a.a.west@lboro.ac.uk (A. West)
ORCID: 0000-0003-4286-3130 (H. Carlin); 0000-0003-1583-1782 (S. Hayward); 0000-0002-3086-8076 (P. Goodall); X 0000-0003-3620-
8375 (K. van Lopik)
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
necessary to predict these changes? (iv) do we have sufficiently accurate models of the physical
system, and any other related systems, to be able to provide realistic data analysis? (v) do we have the
knowledge of the physical system and its environment to be able provide realistic analysis? (vi) Do
we have an appropriate digital environment that can support the breadth, complexity and
responsiveness required to support and manage the data, knowledge, analysis and interaction
requirements of a behaviour prediction system?
The above questions all relate to whether a realistic result can be achieved from the digital twin.
As we progress towards answering these questions and developing a greater understanding of
behaviour prediction systems this raises a further question: can we start to predict behaviour that has
previously been unpredictable? The solution requires a greater level of knowledge and knowledge
sharing, connectedness, interoperability, adaptability and trust in the digital twins than is currently
available. In the next section we discuss the issues involved in moving towards the development of
such solutions.
2. Issues to be resolved to offer realistic digital twins
Figure 1 illustrates a digital twin, both in relation to its physical twin or real system, and in the
context of its operating environment which will involve multiple related twins. This illustrates our
proposal that improvements towards predicting currently ‘unpredictable’ events should be based on
progressive enhancements to the digital twin, based on new understanding of the real system in the
context of its environment. This new understanding will come through the evaluation of DTs though
their ongoing operation. Six issues for developing more realistic twins are identified and discussed
within this paper; (i) assessing sensor requirements in context, (ii) model enhancement requirements,
(iii) analyses required to support real world understanding, (iv) knowledge support requirements, (v)
understanding the needs of human stakeholders and (vi) meeting interoperability requirements.
Figure 1: A cyclic approach to predicting the unpredictable through realistic digital twins
2.1. Assessing sensor requirements in context
Suitable sensor selection and placement is critical to enable an accurate and reliable digital twin
particularly for model calibration. However, the ideal solution may include viable given real-world
constraints. To determine what sensors are required it is important to work backwards from what the
analysis is aiming to decide and determine what physical models are required to calculate the output.
The physical models will then drive what sensors are required and the preferred point in the system at
which the sensing is required [2].
The first instance should always focus on leveraging available sensors and determining what
useful data can be extracted from analysis of resources readily available before modifications are
made. If it is determined that more information is required, sensor selection must be conducted
according to several constraints, namely (i) type – what physical phenomena needs to be measured to
support the analytics for digital-twin development, (ii) cost – weighing up an acceptable sensor
accuracy and sensor cost, (iii) spatial – compromising between sensor quantity, size, location and
mounting options to extract the most useful data whilst not impacting equipment functionality and (iv)
accessibility – how is the sensor accessed (is a wired connection required) and what data rate is
necessary (often driven by the analytics).
One aspect often overlooked, of particular importance for wireless sensors, is the security of
devices being introduced to a system. The importance of device security increases when a digital twin
is used to inform or even automate operations as any corruption or falsification of data injected to the
system can have dramatic impacts on the system and those reliant on the system, necessitating
safeguarding and verification of the genuine physical and digital twins and their sensor data.
2.2. Model enhancement requirements
By their very nature, DT models are a simplification of the real world. In creating the model, the
DT creator has appraised the complete system and decided on which items are of interest to the DT
user. Where a system has fully predictable failures, then the choice of items to model is relatively
obvious. However, for complex systems, unexpected types of failure may start to occur and
understanding the causes of these problems is fundamental to building realism into digital twins to
increase the chance of discovering unpredicted events [3].
To analyse this complexity further, the following three aspects should be considered to create
ultra-realistic DTs capable of detecting unexpected events. Firstly, more items may need to be
modelled. The mindset of the DT creator should change from modelling what has failed in the past to
modelling all items capable of failing. Secondly, each physical item should be assessed as to whether
including its details would help to predict unexpected failures. For example, if a bolted connection
exists with an adjacent item, should this be modelled in detail in case loss of tension causes a failure?
Should a measured thickness be input into the model instead of nominal CAD data? Should the DT
model be designed to degrade as the real system ages? Thirdly, the level of physics needed to detect
all failures should be considered, e. g. just simple on-off activation or kinematics or dynamics. The
types of analyses that are required dictate what information is needed by the DT.
However, the complexity of the DT model cannot be increased without limit – a cost/benefit
balance needs to be made. Unnecessary complexity and fidelity lead to long run-times as well as
increased effort in building and maintaining the model. The DT creator needs to decide when the DT
model is complex enough and this can be decided only by appropriate validation tests on the model.
Provided all possible failures, not just the predictable ones, can be detected by the DT then it is
sufficiently complex.
2.3. Analyses required to support real world understanding
Typical condition-based monitoring of manufacturing systems relies on the use of machine
learning analytical techniques with historic big data, which have been refined over recent years for a
variety commonplace well understood examples [4]. While this research area is producing
increasingly more promising results, the logic behind its application will no longer be applicable in
scenarios where rare or unpredictable events occur, as there will be no foundation of historic data
available from which to identify this behaviour. Naturally this then shifts the analysis of data acquired
from digital twins more towards anomaly detection, to establish events where data recorded from the
physical model does not align with its simulated counterpart.
This shift in analytical technique then raises two significant questions; (i) how far can blind
anomalous warnings be developed into meaningful and therefore useful information for human users
and (ii) the question of trust in digital twins and the level of autonomy that should be given to them
regarding their decisions in unanticipated scenarios? To support automated or semi-automated
understanding of these anomalies with further insight, a decision support tool and database may be
required to make assumptions based on the type and scale of these events in the context of the
equipment under observation, presenting the most likely of these to a human user for further inquiry
or a final decision. The requirements, structure and limitations of such tools are all subjects in need of
further understanding to support automated decision making. At such time that human intervention is
required in this decision-making process, the digital twin’s capacity to simulate various machine
states and operating conditions will be key for remote identification of the root cause of failure. In its
application to unforeseen events however it is possible the digital model will not have been suitably
validated in this context and this therefore raises doubt in the confidence of the conclusions and
decisions made by or with these digital twins. This becomes increasingly important when moving
towards autonomous operation. This concern may further inform what granularity is deemed
appropriate for the creation of digital twins to then encompass the ability to simulate events which are
both expected and unforeseen.
2.4. Knowledge support requirements
To create DTs capable of predicting unexpected failures, an increased level of knowledge is
required. Knowledge is needed not just for areas of the equipment that historically have a tendency to
fail, but for all areas that may impinge on the effective operation of the equipment. This knowledge
can be broadly grouped into three areas: physical system-based, software-based and strategic-based
knowledge [5]. Knowledge on the physical system includes where problems could potentially occur;
optimum sensor locations; details such as geometry, material, and physical conditions between the
sensor locations; and what patterns of behaviour initiate a failure. Particularly important for predicting
unexpected failures is the knowledge of actual system conditions, such as how it has aged, material
defects, dimensional irregularities, wear and dirt build-up. Knowledge of the DT software includes
which software is most appropriate for geometry, simulation and data processing; the capability of
different physics engines to replicate the motion of items; the fidelity of meshes to allow contact to be
modelled correctly; and the extent that variables can be modified. Strategic-based knowledge includes
production aspects such as scheduling, operation, maintenance, and interaction between machines;
and business knowledge such as demand for products and cost aspects. Obtaining detailed production
knowledge allows the DT to be tailored to the specific application. For example, details of scheduling
and maintenance routines would prevent the machine learning algorithm flagging up a sudden
stoppage, when it is in fact scheduled down-time for a certain machine. Business knowledge again
gives context to system behaviour so that training data is comprehensive and accurate.
Detailed knowledge would need to be elicited from operators, equipment designers, material
experts and simulation experts. Since the complexity of the knowledge is high with multiple
interrelationships and potential semantic issues, it would ideally be modelled using ontologies. This
allows concepts and relationships to be stored in a logical, machine interpretable, structure [6]. The
provision of effective ontologies to support manufacturing knowledge interactions is an ongoing area
of research.
2.5. Understanding the needs of human stakeholders
The DT is part of an organisations digitalisation strategy that requires preparation for digitalisation
through the above steps considering sensor suitability, knowledge availability and quantification as
well as current system function. Requirement engineering is therefore a crucial part of DT design to
establish user needs and goals. An initial step in the preparation for digitalisation is through a system
function and knowledge stock take. Stakeholders must review their current system and establish goals
for the DT, identify if performance data are currently monitored (and how) and if the system is
understood from an ‘as-is’ rather than ‘as-intended’ perspective. The activities, interfacing and
interaction of humans in the system should be well understood and modelled appropriately. These
models can be used to ensure the DT provides the right data in the right timeframe and context to
support user goals. To facilitate system evaluation in future, requirements may be quantified to
develop performance indicators for continued improvement. Goal oriented requirements engineering
may have potential for this purpose through the representation of goals as numerical values.
Once the system preparation is complete, the design of the twin should align with the nature of the
system and the goals of the user. A technology focussed DT may misalign with a real-world system
due to the negation of the influence of human activity in the system. Human activity recognition
(HAR) using wearable sensors and computer vision have been researched to develop digital models of
human-system interaction to interpret human behaviour. In addition to the sensor constraints
mentioned in the section above there are concerns regarding privacy and dignity, ergonomics, comfort
and the practicalities of wearable sensor [7]. An emerging research topic related to the use of HAR is
the Human Digital Twin (HDT). The HDT requires a cross disciplinary research effort from
technological, psychological and sociological domains. Some of the challenges in HDT concern
content creation, data ownership and twin interoperability within existing digital twins. Additionally,
there are significant challenges in identifying, monitoring, and managing human activities relating to
dynamic states of knowledge, decision and actions, alongside psychosocial factors and through
system interaction.
Lastly, humans need to accept and trust any system changes that affect their working practices. To
improve trust and acceptance (resulting in improved performance of the system) human centred
design (HCD) approaches should be used. This requires inclusion of stakeholders at all levels of the
business (operational, tactical, and strategic) and communication of how changes to the system will
improve work. A combination of well-established HCD methods as well as emerging frameworks
could be explored to support this analysis. A further challenge is the reliance on human factors and
ergonomics expertise, which may not be available in smaller enterprises. There is little guidance
available on rectifying specific issues that occur when digitalisation strategies negatively affect
human performance.
2.6. Meeting interoperability requirements
The interoperability of digital twins in manufacturing becomes significant from two aspects. The
first concern is that any implementation of an ideal theoretical digital twin would be a significant
focal point within manufacturing processes communications, connecting to many other systems
seeking both input and output including databases, analytical tools, machine control logic and various
human user interfaces. Secondly, the digital twin itself is likely to be composed of various interacting
subsystems responsible for maintaining the digital twin processes. Of these, primarily there exists the
separation of the real and cyber aspects of the model, which are required to work together seamlessly
to ensure a realistic visualisation of the live system. The real-world side of this twin relies on physical
sensor networks and data processing tools to relay information to its cyber counterpart, whereas the
cyber side depends on simulation tools to recreate these states and inform control of the physical
apparatus. Additionally, the digital model may also be composed of multiple simulations depending
on the complexity of the system under question, where each of these could be constructed
independently using the software most suited to that task. These internal factors then compound the
importance of effective interoperability surrounding digital twins.
In other fields, widespread adoption of a standard ontology has improved interoperability between
heterogenous data sources substantially, in manufacturing however there still exists a need for the
development and the widespread industrial implementation of a set of standardised ontologies to
support interoperability [8]. To further support the creation of digital twins the creation of an
additional DT reference ontology is required to inform developers of the required scope of
instrumentation and simulation that is required for suitable representation of the target problem. Used
in conjunction these systems will inform manufacturers as to the necessary components of their digital
twins and how these systems will need to communicate both internally and externally.
3. Discussion and conclusions
This paper discusses the issues towards providing more effective digital twin solutions that are
currently available and proposes a framework that encompasses the elements of (i) effective sensor
availability and selection, (ii) model complexity requirements and capability, (iii) data analytic set
requirements and provision, (iv) knowledge support understanding and provision, (v) interoperability
requirements within and between interacting digital twins and (vi) provision of responsive human-
computer interactions. The ability to populate such a framework with effective solutions to this range
of key interacting elements will offer manufacturing businesses the potential to predict and resolve
manufacturing problems that are currently perceived as unpredictable.
4. Acknowledgements
This work is supported by the UK’s Engineering and Physical Science Research Council (EPSRC)
from funding for the project “Embedded Integrated Intelligent Systems for Manufacturing” [grant
reference EP/P027482/1]. In addition, funding from the Engineering and Physical Science Research
Council Centre for Doctoral Training in Embedded Intelligence (grant no. EP/L014998/1) is also
acknowledged.
5. References
[1] B. R. Barricelli, E. Casiraghi, D. Fogli, A Survey on Digital Twin: Definitions, Characteristics,
Applications, and Design Implications, IEEE Access 7 (2019) 167653-167671. doi:
10.1109/ACCESS.2019.2953499.
[2] W. Kritzinger, M. Karner, G. Traar, J. Henjes, W. Sihn, Digital Twin in manufacturing: A
categorical literature review and classification, IFAC-PapersOnLine 51 (2018) 1016–1022. doi:
10.1016/j.ifacol.2018.08.474
[3] M. Hausmann, Y. Koch, E. Kirchner, E. Managing the uncertainty in data-acquisition by in situ
measurements: A review and evaluation of sensing machine element-approaches in the context of
digital twins, International Journal of Product Lifecycle Management 13 (2021) 48–65. doi:
10.1504/IJPLM.2021.115700.
[4] M. Liu, S. Fang, H. Dong, C. Xu, Review of digital twin about concepts, technologies, and
industrial applications, Journal of Manufacturing Systems 58 (2021), 346-361.
[5] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. P. Francisco, J. P. Basto, S. G. S. Alcala, A
systematic literature review of machine learning methods applied to predictive maintenance,
Computers and Industrial Engineering 137 (2019) 106024. doi: 10.1016/j.cie.2019.106024.
[6] M. Grieves, Digital twin: Manufacturing excellence through virtual factory replication, 2014.
URL:
https://www.researchgate.net/publication/275211047_Digital_Twin_Manufacturing_Excellence_
through_Virtual_Factory_Replication.
[7] International Standards Organisation, Industrial automation systems and integration – Formal
semantic models for the configuration of global production networks. ISO 20534:2018.
[8] B. Hartmann, Human Worker Activity Recognition in Industrial Environments, 2011. URL:
http://books.google.com/books?hl=en&lr=&id=UBOJyJ6XMpgC&oi=fnd&
pg=PA1&dq=Human+Worker+Activity+Recognition+in+Industrial+Environments&ot
s=crbc-Ya4iy&sig=e7-EXLPAzxaqTwla3izWLsMDY4I
[9] M. Karray, N. Otte, R. Rai, F. Ameri, B. Kulvatunyou, B. Smith, D. Kiritsis, C. Will, C., R.
Arista, The Industrial Ontologies Foundry (IOF) perspectives, Industrial Ontology Foundry
(IOF) - achieving data interoperability Workshop, in: International Conference on
Interoperability for Enterprise Systems and Applications, Tarbes, France, 2021. URL:
https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=925879.