=Paper=
{{Paper
|id=None
|storemode=property
|title=Linking Process Models and Operating Data for Exploration and Visualization
|pdfUrl=https://ceur-ws.org/Vol-886/paper_3.pdf
|volume=Vol-886
}}
==Linking Process Models and Operating Data for Exploration and Visualization==
https://ceur-ws.org/Vol-886/paper_3.pdf
Linking Process Models and Operating
Data for Exploration and Visualization
Ken WENZEL a,1 , Marcel TISZTL a
a Fraunhofer IWU, Department for Production Management,
Germany
Abstract. Data integration between different life cycle stages of a production sys-
tem is a crucial requirement for the realization of continuous computer support
what is often referred to as the Digital Factory. This paper presents an approach
that leverages planning data within the operation phase of a production system by
applying Semantic Web technologies and linked data concepts.
Keywords. production system, life cycle, process planning, semantic web, sensor
networks
Introduction
When considering the design and operation of a production system, one can observe that
there exists an information gap between those life cycle phases [1]. This usually results
in a complete redesign of models for observation and control of such systems instead of
reusing structural and functional knowledge that was already created during the design
phase. We believe that this is due to the wide range of tools and complex data formats
making it hard to transfer knowledge from one phase to another. Our approach trans-
forms existing process models of production systems into a linked data representation
and combines them with ontologies for Semantic Sensor Networks and units of mea-
surement to enable the linkage with collected operating data. The development of our
concepts is based on a usage scenario from the automotive industry where media con-
sumption data for a body-in-white assembly line is collected and should be linked to
processes, products and resources for the purpose of visualization and analysis.
Section 1 outlines the related ontologies and applications used in our approach.
Section 2 introduces a simple ontology for the description of manufacturing processes.
Based on this ontology an approach for exploration and visualization of operating data
is introduced by Section 3. Finally, a summary and outlook is given by Section 5.
1. Overview on related ontologies and software systems
Figure 1 gives an overview on ontologies which were created for our approach to rep-
resent and integrate process models with operating data. These encompass the manu-
1 Corresponding Author: Ken Wenzel, Fraunhofer-Institute for Machine Tools and Forming Technology
IWU, Department for Production Management, Reichenhainer Str. 88, 09126 Chemnitz, Germany; E-mail:
ken.wenzel@iwu.fraunhofer.de.
1
Dolce+DnS
SSN Time QUDT
UltraLite
Manufacturing
Processes
Process Designer MET MET
Models Data Visualization
Process Operating
Models Data
Process Operating Data
Designer Explorer
owl:imports read/write uses
Figure 1. Ontologies and software systems used by our approach.
facturing process ontology (Section 2) for the description of process models, the MET
data ontology for the representation of measurement data (Section 3) as well as the
MET visualization ontology for the mapping of measured properties to suitable graphical
charts (Section 4). The latter two are combined within our Measurements and Evaluation
Toolkit (MET) – a collection of vocabularies and software tools to acquire, manage and
visualize measurement data. The existing vocabulary that was reused for the creation of
our ontologies is shown at the top of Figure 1.
Additionally, a domain-specific ontology was developed as an extension of the man-
ufacturing process ontology to (partially) represent Tecnomatix Process Designer models
as linked data. These process models are automatically imported from their proprietary
XML serialization and transformed into RDF data.
Finally, the presented approach is implemented within the Operating Data Explorer
using the Knowledge Modeling and Management Architecture (KOMMA) [2].
2. A lightweight ontology for manufacturing processes
Diverse engineering disciplines tried to adopt ontologies for describing their respective
domains. For example, the research projects COGents and IMPROVE [3] demonstrated
that ontologies are a well-suited means to describe models of chemical process systems.
Besides these, there are also some efforts towards creating base ontologies for the man-
ufacturing domain. MASON [5] defines an upper ontology for manufacturing and gives
2
DUL:Method DUL:PhysicalArtifact DUL:Object DUL:Diagnosis
subclass subclass subclass subclass
hasInput
must be Artifact State
hasPossibleState
hasOutput must be
must be uses
must be
requires
Process must be Resource
Figure 2. Manufacturing process ontology.
examples of its usage within the fields of cost estimation and production control. The
ADACOR ontology [4] contains concepts to describe manufacturing systems for the pur-
pose of scheduling and control. Both ontologies capture some common but also some
distinct parts of the manufacturing domain.
Following the principle of minimal ontological commitment as proposed by Gruber
[6], we argue that separation of concerns into even more lightweight ontologies simpli-
fies their reuse and accelerates their adoption. Hence, we have created the simple on-
tology depicted by Figure 2 for the description of manufacturing operations with their
related products and resources. This ontology is also aligned to DOLCE+DnS UltraLite
(DUL)2 , an enhanced version of the Descriptive Ontology for Linguistic and Cognitive
Engineering (DOLCE) [7].
The ontology is loosely based on the core idea of the Structured Analysis and De-
sign Technique (SADT) [8] that is able to describe systems as a hierarchy of functions.
In our case those functions are manufacturing Processes which may have one or more
Artifacts as inputs and outputs and may require one or more Resources for operation.
The ontology introduces also the concept State, allowing a more specific description of
resources, especially in regard to operating data.
The proposed ontology does intentionally not provide any concepts for the descrip-
tion of raw materials, tools, workers and other application specific entities. Also deeper
classification of artifacts or processes as done within MASON is omitted.
The alignment to DUL allows easy extension of the ontology for various use
cases. Process is considered a subclass of DUL:Method to express that this concept
should be used to describe manufacturing operations in the sense of process planning.
DUL:PhysicalArtifact is chosen as a superclass of Artifact since inputs and outputs of
manufacturing operations are usually physical objects with an intended function. Our
model unifies SADT’s notion of control and mechanisms into one Resource concept that
is a subclass of DUL:Object. Resources can be further described by using States repre-
senting a DUL:Diagnosis of these systems.
2 http://ontologydesignpatterns.org/wiki/Ontology:DOLCE+DnS UltraLite
3
mfg:Artifact
mfg:Resource
mfg:requires mfg:hasInput innerPart
robot
ssn:FeatureOfInterest mfg:hasInput mfg:Artifact
mfg:uses
mfg:requires mfg:Process frame
spotWelding
mfg:Resource
spotWeldingGun mfg:hasOutput mfg:Artifact
ssn:hasProperty weldedPart
ssn:Property
qudt:MechanicsQuantityKind ssn:featureOfInterest
ssn:onPlatform quantity:Power
ssn:observes ssn:observedProperty
ssn:observationResultTime time:Instant
2012-03-27 16:08:19.163
ssn:SensingDevice ssn:madeObservation ssn:Observation
powerMeter powerObservation1
time:DateTimeInterval
ssn:observationSamplingTime 10 ms
ssn:isProducedBy ssn:observationResult
qudt:PowerUnit
met:hasUnit unit:KiloWatt
ssn:hasValue ssn:ObservationValue
ssn:SensorOutput
met:QuantityObservationValue
output1
value1 xsd:float
met:hasQuantityValue 243
Figure 3. Example of an electric power observation.
3. Representation of operating data
The W3C Semantic Sensor Networks Incubator Group (SSN-XG)3 has built an exten-
sive ontology for describing sensor networks and related observations. For this reason,
their work encompasses an in-depth review of existing sensor and observation ontologies
where selected concepts were used to create the Semantic Sensor Network (SSN) ontol-
ogy. This ontology intentionally excludes descriptions of domain concepts like time or
location information to enable its applicability for multiple use cases and domains.
We are mostly using concepts of the SSN ontology that were developed for the use
case of data discovery and linking. These concepts allow to describe Observations made
by a SensingDevice regarding some Properties of a certain FeatureOfInterest.
In the case of operating data the FeatureOfInterest is usually either the process itself
or the involved resources and product artifacts. When combining the SSN ontology with
the ontology for manufacturing processes introduced in Section 2 the feature of interest
of an observation may be an instance from one of the classes Process, Resource, State
or Artifact. More complex use cases can be covered by using instances of DUL:Event as
feature of interest.
Figure 3 denotes an example of an electric power observation expressed by applying
the SSN ontology in combination with the manufacturing process ontology. The example
covers a body-in-white assembly process with spot welding where the actual power con-
sumed by the spot welding gun is observed. For the representation of observable proper-
ties and their associated units the QUDT4 (Quantities, Units, Dimensions and Data Types
3 http://www.w3.org/2005/Incubator/ssn/
4 http://qudt.org/
4
in OWL and XML) ontology is used. An alternate ontology with comparable coverage is
the Measurement Units Ontology5 (MUO). Both ontologies provide a formal framework
for defining measurable properties (quantities in QUDT, qualities in MUO) with corre-
sponding base units and their derived forms. Additionally, both ontologies provide a set
of predefined properties and unit systems like SI or CGS. MUO’s data is extracted from
UCUM6 , the Unified Code for Units of Measure, while QUDT uses its own definitions.
Time points and intervals are expressed by using the Time Ontology in OWL7
(OWL-Time).
For representing observation result values a special concept QuantityObservation-
Value as subclass of ssn:ObservationValue is introduced by our MET data ontology. It
allows to describe measured quantities (met:hasQuantityValue) and their associated units
(met:hasUnit).
4. Exploration and visualization
The semantic linking of operating data, aligned with the SSN, to the manufacturing pro-
cess ontology, like we have shown above, can support an analyst in exploring and exam-
ining the collected data, for example, to identify specific resource usage patterns. There-
fore, we want to demonstrate, how visualization and navigation on the described model
can be achieved. The starting point is the process model of the production system that
can be explored within the process, resource and artifact dimensions. Each element from
each dimension may have a hierarchical and topological structure.
Moreover, we want to support the visualization of a Process, a Resource or an Arti-
fact by generating a chart of its properties and related observations. Figure 4 exemplifies
this compound visualization for a selected Resource.
4.1. Generic diagrams for observed properties
Data visualization of FeatureOfInterests is a complex matter, because their observed
properties, that should be visualized, may differ in some aspects. For example, some
properties like power consumption and temperature have numeric values for individual
points in time and are suitably visualized by XY charts. Other properties like the current
system state or the modified artifacts represent some kind of state that is valid over a
time interval (e.g. the duration of a manufacturing operation) and can be suitably visu-
alized by gantt or bar charts. To support a variety of visualizations, the MET-VIZ (Mea-
surements and Evaluation Toolkit - Visualization) ontology introduces concepts to for-
mally describe the associations between visualizable properties and suitable charts. An
example is depicted in Figure 5.
The ontology defines the classes Chart and ChartDescription. Each chart supported
by our visualization framework is represented as an instance of Chart and observed prop-
erties are associated to these charts by using ChartDescriptions.
5 http://idi.fundacionctic.org/muo/muo-vocab.html
6 http://unitsofmeasure.org/
7 http://www.w3.org/TR/owl-time/
5
quantity:Power
quantity:
Thermodynamic
Temperature
mfg:Process idle welding handling handling
mfg:State idle busy
mfg:Artifact none weldedPart weldedPart frame
10:49:00 10:49:20 10:49:40 10:50:00 10:50:20 10:50:40 10:51:00
Figure 4. Visualization of observed property values for a given Resource.
4.2. Visualization framework based on data-driven documents
Like Figure 4 denotes, each observed property of a given FeatureOfInterest is visualized
by a specific chart. These charts are combined into a composite chart with a timeline,
where the user can select the global time window for all contained charts. To support
this functionality, we have developed a visualization framework that implements versa-
tile charts based on data-driven documents (d3) [10] and their corresponding Javascript
implementation D3.js8 . D3 allows to use powerful declarative transformations between
data and SVG representations. This eases the creation of generic charts and highly sim-
plifies the development of domain-specific visualizations for various use cases. D3’s
transformation capabilities are leveraged to define basic generic diagrams which can be
composed with each other to create compound visualizations of multiple properties.
These compositions are supported by two design rationales of our framework. The
first one is that the chart’s data access is controlled by a dedicated data provider interface.
8 http://mbostock.github.com/d3/
ssn:FeatureOfInterest ssn:Property met-viz:ChartDescription met-viz:Chart
instance instance instance instance
my:process1 quantity:Power my:powerChartDesc met-viz:xyChart
ssn:hasProperty met-viz:hasChartDescription met-viz:withChart
Figure 5. Associate properties with suitable charts.
6
If a chart is part of a composite then the composite assigns a specific data provider to
this chart. This enables the composite to simply pass through data requests and responses
from and to its children or intercept the communication to implement data conversion or
caching (e.g. within a time window).
The second building block is the interconnection of multiple chart axes. Thus a
composite can define the axis of a contained chart as parent of an axis of another child.
If a parent axis is changed in its dimensions or selection then it propagates these changes
to the respective child axes.
4.3. Composite visualization of observed properties
For the creation of a composite visualization, the database is queried for all observed
properties of a given FeatureOfInterest. This is exemplified by the following SPARQL9
query.
PREFIX ssn:
PREFIX met-viz:
SELECT DISTINCT ?property ?chart WHERE {
[ a ssn:FeatureOfInterest; ssn:hasProperty ?property ] .
?property met-viz:hasChartDescription ?description .
?description met-viz:withChart ?chart
}
Needless to say, the user can narrow down the search results to constrain the visu-
alization to certain properties. For each of these selected properties an instance of the
corresponding chart is added to the composite.
A timeline within the top-level composite enables the user to select a specific time
window whose corresponding data should be visualized. Based on this selection all charts
representing observed properties are updated to show the correct values within the se-
lected time interval. Our visualization framework supports this use case by intercon-
necting the charts and their axes like described above. After selecting the time window,
each chart individually retrieves its data by using a data provider that itself queries the
database.
The following SPARQL query retrieves measurement data for a given time window
and an observed property. This query selects all observations for a given Property of a
FeatureOfInterest and retrieves the associated data values by filtering the results using
the selected time window with start time and end time points.
PREFIX dul:
PREFIX time:
PREFIX ssn:
SELECT ?value ?rtime WHERE {
?o a ssn:Observation;
ssn:observedProperty ?property; ssn:featureOfInterest ?foi;
9 http://www.w3.org/TR/rdf-sparql-query/
7
ssn:observationResultTime [
time:inXSDDateTime ?rtime
];
ssn:observationResult [
ssn:hasValue [ dul:hasRegionDataValue ?value ]
] .
FILTER (?rtime > ?startTime && ?rtime < ?endTime)
}
4.4. Navigation using the manufacturing process ontology
We have shown how observed properties of a particular FeatureOfInterest (a Process, a
Resource or an Artifact of the manufacturing process ontology) can be visualized. How-
ever, it would be helpful to have some kind of visualization about their relations to each
other and also to directly use these relations to switch between different FeatureOfInter-
ests. Hence, additionally to the visualization of the properties of a FeatureOfInterest the
following information can be displayed:
• If the visualized FeatureOfInterest is a Resource then we can visualize all the
Processes that were active on that Resource in a gantt chart.
• If the visualized FeatureOfInterest is a Process then we can visualize the states of
its required Resources in a gantt chart and set a reference to its input and output
Artifacts.
• If the visualized FeatureOfInterest is an Artifact then we can visualize all Pro-
cesses that caused changes and their corresponding Resources in a gantt chart.
This can be done by querying the process model (expressed using our manufacturing
process ontology) for this information using SPARQL and by visualizing the results
accordingly. The following query exemplifies the retrieval of active processes belonging
to a resource. It assumes that the current state of processes (e.g. active or inactive) is also
observed. For example, an observed property ex:ProcessState of a process may have the
values ex:ActiveState or ex:InactiveState where the latter may be instances of mfg:State.
Now the query retrieves all processes with their corresponding start times and durations
that require the given Resource and were active at some point in time.
Please note that it would also be possible to model a resource’s current active process
as one of its observed properties (ssn:observedProperty).
PREFIX dul:
PREFIX time:
PREFIX ssn:
PREFIX mfg:
PREFIX ex:
SELECT ?process ?startTime ?duration WHERE {
?process mfg:requires ?resource .
?o a ssn:Observation;
ssn:featureOfInterest ?process;
8
ssn:observedProperty ex:ProcessState;
ssn:observationResultTime [ time:inXSDDateTime ?startTime ];
ssn:observationSamplingTime ?duration;
ssn:observationResult [
ssn:hasValue [ dul:hasRegionDataValue ex:ActiveState ]
] .
}
5. Conclusions and future work
We have introduced a lightweight ontology for describing manufacturing processes. Due
to the alignment with DOLCE+DnS UltraLite an easy extension and combination with
other vocabularies is ensured. Moreover, an approach was presented that combines our
ontology with the SSN vocabulary to represent operating data of production systems and
link it to elements of a process model. Based on these results we have shown the creation
of mashups with charts for multiple observed properties by leveraging ontologies and
advanced visualization technologies. This is supported by linking to an existing process
model that improves the navigation and filtering of operating data.
Our future work will investigate how computation rules for indicators can be ex-
pressed by embedding mathematical formulas into RDF data sets [11]. This technology
will allow the automatic computation of aggregated performance information based on
the linked representation of process models and operating data.
Moreover, while currently data has only been collected from a running production
system, the next steps may investigate how ontologies can help to integrate rules for
manufacturing control into process models for using them within the operating phase.
For example, we are currently working on augmentation of process and infrastructure
models with additional knowledge and rules for energy-oriented control of production
systems. The resulting solution should be able to operate a production system along with
its infrastructure in an energy-efficient way.
Acknowledgements
The work presented in this paper is co-funded within the Cluster of Excellence Energy-
Efficient Product and Process Innovation in Production Engineering (eniPROD R ) by the
European Union (European Regional Development Fund) and the Free State of Saxony.
References
[1] B. Denkena, M. Shpitalni, P. Kowalski, G. Molcho, and Y. Zipori, Knowledge management in process
planning, CIRP Annals - Manufacturing Technology, vol. 56, no. 1, 2007, pp. 175–180.
[2] K. Wenzel, Ontology-Driven Application Architectures with KOMMA, 7th International Workshop on
Semantic Web Enabled Software Engineering, Bonn, 2011. http://iswc2011.semanticweb.org/
fileadmin/iswc/Papers/Workshops/SWESE/6.pdf
[3] J. Morbach, A. Wiesner, and W. Marquardt, OntoCAPE – A (re)usable ontology for computer-aided
process engineering, Computers & Chemical Engineering, vol. 33, no. 10, 2009, pp. 1546–1556.
9
[4] S. Borgo and P. Leito, Foundations for a Core Ontology of Manufacturing, in: Ontologies, vol. 14, R.
Sharman, R. Kishore, and R. Ramesh, Eds. Springer US, 2007, pp. 751–775.
[5] S. Lemaignan, A. Siadat, J.-Y. Dantan, and A. Semenenko, MASON: A Proposal For An Ontology Of
Manufacturing Domain, in: IEEE Workshop on Distributed Intelligent Systems: Collective Intelligence
and Its Applications, 2006. DIS 2006, 2006, pp. 195–200.
[6] T. Gruber, Toward Principles for the Design of Ontologies Used for Knowledge Sharing, International
Journal Human-Computer Studies Vol. 43, Issues 5–6, Novemer 1995, pp. 907–928.
[7] A. Gangemi, N. Guarino, C. Masolo, A. Oltramari, and L. Schneider, Sweetening Ontologies with
DOLCE, in: Proceedings of the 13th International Conference on Knowledge Engineering and Knowl-
edge Management, Ontologies and the Semantic Web, London, 2002, pp. 166–181.
[8] D. Marca and C. L. McGowan, SADT : structured analysis and design technique, McGraw-Hill, New
York 1988.
[9] J. R. Hobbs and F. Pan, An Ontology of Time for the Semantic Web, ACM Transactions on Asian Lan-
guage Information Processing, vol. 3, 2004, pp. 66–85.
[10] M. Bostock, V. Ogievetsky, and J. Heer, D3: Data-Driven Documents, in: IEEE Trans. Visualization &
Comp. Graphics (Proc. InfoVis), 2011.
[11] K. Wenzel and H. Reinhardt, Mathematical Computations for Linked Data Applications with OpenMath,
24th OpenMath Workshop, Bremen, 2012. http://www.informatik.uni-bremen.de/cicm2012/
papers/OM2012/paper_1.pdf
10