To increase the utility of semantic industrial information models we propose a methodology to incorporate extracted operator knowledge, which we assume to be present in the form of rules, in knowledge graphs. To this end, we present multiple modelling patterns that can be combined depending on the required complexity. Aiming to combine information models with learning systems we contemplate desired behaviours of embeddings from a predictive quality perspective and provide a suited embedding methodology. This methodology is evaluated on a real world dataset of a fused deposition modelling process.
On the one hand, standardised semantic information models (IMs) and standards for their hosting, such as the Industry 4.0 asset administration shell, are gaining traction in the industrial internet of things where they can be used to facilitate interoperability and data interchange between different companies, production plants, lines or machines [1,2,3]. On the other, knowledge graphs (KGs) are a popular data structure to integrate knowledge of multiple heterogeneous sources [4,5,6]. Combined with approaches that allow reasoning over knowledge graphs, e. g. for link prediction to facilitate knowledge graph completion, they are a logical choice for semantic industrial information models [7,8,9]. However, the knowledge typically represented in these industrial information models is of mostly factual and conceptual nature, reaching the level of "knowledge of principles and generalizations" of Krathwohl's taxonomy [10]. It concerns equipment [7,9], material [7], process segments [7], parts [5], products [5,7], events [9], underlying measurements [11] and geospatial information [12] as well as relations interconnecting these entities. Based on this information, use cases addressed range from anomaly detection [8] over process monitoring [13,8] to locating parts and equipment in plants [12] and risk management [13]. We are not aware of the use of industrial IMs containing extracted expert knowledge, which belongs to the more advanced conceptual, i. e. "knowledge of models, theories, structure" [10], and procedural categories of knowledge. This knowledge is of heuristics-like nature and usually obtained through multiple years of expertise. A more detailed explanation of expert knowledge and procedural knowledge for the manufacturing scenario investigated in this paper is given in Section 3.
Since expert knoweldge is playing a crucial role in many industrial day to day processesfrom the design of components to reparametrisation processes necessary to deal with quality defects during production-and only available to a limited number of people, representing it in a standardised way would be of great interest for the industry. Therefore, we propose that including procedural knowledge would enhance the applicability of IMs in several ways:
1. a semantic integration between the what of given machinery and the how to operate it 2. suitability for predictive quality use cases, e. g. by utilizing the resulting knowledge graphs, which would contain quantified knowledge, to increase the performance of learning systems. This could lead to an increase in the ability to generalize and cope with coarse data-both frequent challenges in industrial contexts 3. a standardised representation of procedural knowledge which would (1) enable the creation of digital process twins that could be supplied alongside machinery for operating and training purposes, thereby reducing the impact of changes in a production line, (2) provide a standardised way to combine it with varying kinds of knowledge from different sources, e. g. physical limits of machinery provided by process engineering and (3) enable the fusion of knowledge extracted by traditional [14] as well as data-based [15] methods
Based on an overview of related work (see Section 2) we explore how extracted tacit operator knowledge available as rules can be incorporated in KGs serving as IMs by modelling patterns (Section 3). Section 4 tries to answer the question whether these KGs are able to be embedded in a form that benefits predictive quality use cases. Section 5 provides an outlook describing the next steps towards realizing our vision of knowledge graphs containing extracted expert knowledge as actionable rules while Section 6 concludes this paper.
Characteristics of established industrial information models and the expected benefits of incorporating rules in these models are described in Section 1. While rules have, to the best of our knowledge, not been directly represented in knowledge graphs used as industrial information models, they have been frequently used in different semantic contexts.
Representations of rules in graphs have been addressed by Chein and Mugnier [16]. They explored how bi-coloured graphs can be used to encode condition and conclusion of rules on both a general level as well as, optionally, for specific entities encoded via attributes. However, their approach does not offer a way to provide quantifications to either conditions or conclusions. Also, established embedding methods are not directly applicable to bi-coloured graphs, which limits the usability of their approach for e. g. knowledge infused learning.
Assisted Embeddings More often than being directly represented in graphs, logical rules are used as auxiliary information for knowledge graph embeddings [17,18]. Here, rules are either provided by algorithm designers or domain experts to capture common sense knowledge or automatically mined from knowledge graphs [18]. Zhang et al. [19] create relations based on rules that load to an increase in embedding performance. Ringsquandl et al. [20] conclude that the performance of KG completion can be increased by utilizing embeddings of events, i. e. time-series data [9] during KG embedding. However, in contrast to the events or logical rules employed in these approaches, rules founded on extracted operator knowledge frequently contain quantifications of conditions or quantifications which makes them more complex and unsuited to the described approaches.
Rule Representations in Embeddings It has been shown that "existential rules can be exactly represented using convex regions of knowledge graph embeddings" [21]. While a methodology that provides exact representation of general rules in embeddings would provide greatly helpful in evaluating the suitability of different methodological choices of rule representation in knowledge graphs we cannot rely on Gutierrez's methodology since the rules containing the experts' extracted knowledge are more complex than the existential rules considered. Furthermore, representing rules in knowledge graphs as opposed to embeddings, provides a significant benefit for information models as the representation is more direct and can be independently accessed.
Establishing Embedding Quality Link prediction and entity classification are the standard scenarios to evaluate embedding methods [22,23,24,25,19,26,27,28,29]. However, doubt has been cast both on biases in the used datasets [30,31,32] as well as on the more general capability of KG embeddings to capture semantics [33]. Therefore, behavioural testing of embedding methodologies is gaining attention [34]. Since we are aiming at using embeddings not only for link prediction but to improve the performance of learning systems, an evaluation of the embeddings' encoding of the required semantic information is necessary in our case.
Based on a manufacturing scenario, this section will introduce modelling patterns for different representations of expert knowledge along with an overview of their properties.
In manufacturing use cases, a high proportion of expert knowledge is tacit operator knowledge which pertains to parametrisation of machinery. As such, it contains knowledge about both conceptual relationships between process parameters and quality characterisitcs as well as procedural behaviours that lead to the achievement of goals, i. e. how and in what order to adapt process parameters to mitigate occurring quality defects and achieve a perfect parametrisation.
In this paper we focus on the aspect of how parameters are adjusted. This tacit operator knowledge can be extracted by various methodologies [e.g. 14,15] leading to rules at different levels of abstraction. Also, information concerning the same problem might be available from alternative sources such as process engineering documents or handbooks which could be combined or contrasted with knowledge operators gained in practice. As such, adapting and building on definitions for operator knowledge presented in [35], we inspect several modelling patterns for representing the underlying knowledge at different abstraction levels that can be combined at will. In the following, we will align our terminology with manufacturing scenarios to increase the readability of examples. From these modelling patterns, a fitting degree of abstraction can be chosen to either reflect the kind of knowledge that is available or of the information models' specific domain. This is relevant since we expect that the higher complexity, i. e. through hierarchies, provides challenges for embedding approaches. Avoiding unneeded complexity in the representation is therefore likely to achieve better results. As such, we recommend choosing the highest abstraction level, that is able to encode all present information. Note that the representations of higher abstraction levels can be easily converted to representations of lower abstraction levels. Therefore including operator knowledge of a different source which utilises a different abstraction level is still possible.
A rule at the highest level of abstraction, i. e. an unquantified rule, could be verbalised as If quality characteristic q is unsatisfactory then adjust process parameter p. It can be viewed as an implies relation between the condition quality characteristic q and the conclusion parameter p. This corresponds to the triple notation of (head, relation, tail) common in knowledge graphs. Adapting the definitions of [35] we define parameters 𝑝 ∈ 𝑃 and quality characteristics 𝑞 ∈ 𝑄. This yields the triple 𝑟 𝜂 = (𝑞, ⟨implies⟩, 𝑝). Whereas in [35] an index for the process iteration was included, we omit it here for the sake of readability as it is not relevant for the contents of this paper. A graphical representation of 𝑟 𝜂 is shown in Figure 1a. A modelling alternative would be a is a relation between 𝑞 and the semantic meaning of quality characteristics. However, this semantic information is already encoded by the directed relation. Making it explicit would only increase syntactic complexity and hierarchy.
Rules with quantified conclusions, i. e. parameters, 𝑟 𝜌 ^can be verbalised as If quality characteristic q is unsatisfactory then adjust process parameter p by 𝜈 with 𝜈 ∈ R. Therefore, 𝑟 𝜌 ^= (𝜂 𝑞,𝑝 , 𝜈) = (𝑞, ⟨implies⟩, 𝑝, 𝜈). Generally, we want to keep the representation as succinct as possible. Therefore, the representation of unquantified rules is extended to use 𝜈 as a weight of the implies relation (cf. Figure 1b). Here, the quantified parameter 𝜌 ∈ P, where P = {(𝜈, ⟨quantifies⟩, 𝑝) | 𝑝 ∈ 𝑃 and 𝜈 ∈ R} is implicitly modelled. We denote the rules of quantified conclusions as 𝑟 𝜌 ^since several observations are aggregated into the quantification.
In addition to quantified parameters that serve as actionable recommendations for operators, the conditions, i. e. quality characteristics, can also be quantified to arrive at more descriptive rules.
With quantified conditions, it is possible to represent more advanced concepts, e. g. for higher defects in a specific quality characteristic, parameters need to be adjusted more substantially. While in theory quantified quality characteristics could be used without quantified parameters, it is not beneficial in practice since the conclusion of the rule would remain the same. As such, we consider quantified parameters as a prerequisite of quantified quality characteristics.
Rules with aggregated quantified quality characteristics and parameters 𝑟 o ^,𝜌 ^can therefore be verbalised as If quality characteristic q is within 𝜇, then adjust process parameter p by 𝜈, where 𝑟 o ^,𝜌 ^= ((𝜇, ⟨quantifies⟩, 𝑞), ⟨implies⟩, (𝜈, ⟨quantifies⟩, 𝑝))
Sometimes a specific parametrisation is only relevant if multiple conditions align. These rules, 𝑟 𝑞 𝑛 , could be verbalised as If quality characteristic x and quality characteristic z are unsatisfactory, then adjust process parameter p. For this modelling pattern, we omitted quantifications for the sake of brevity. 𝑟 𝑞 𝑛 could be trivially encoded by having multiple separate rules for each quality characteristic influencing the same process parameter in the IM, e. g. 𝑟 𝜂𝑞,𝑝 and 𝑟 𝜂𝑠,𝑝 , with 𝑞, 𝑠 ∈ 𝑄 and 𝑝 ∈ 𝑃 . However, in this case it would not be clear whether the rules are related according to a logical AND, OR, or a different operator altogether. As such, we propose the introduction of a relator vertex representing the logical operator required by the 𝑟 𝑞 𝑛 in question. For the example of an AND-relator shown in Figure 3 be expanded to the following hierarchical triple:
𝑟 𝑞 𝑛 = {((𝑞, ⟨combined by⟩, 𝑙), ⟨implies⟩, 𝑝), ((𝑠, ⟨combined by⟩, 𝑙), ⟨implies⟩, 𝑝)} AND In theory, there is no limit to the number of relations being combined by a relator, the definitions here are based on the example in Figure 3.
The case of one condition influencing several conclusions can be unambiguously represented by adding a separate rule for each of the conclusions. Therefore, this case does not require a special relator vertex.
Noy et al. make the point that "it is critical not to lose the linkage between the relationships stored in the graph and where those relationships come from" [5]. While they refer to the discovery process, we assume that capturing semantics of operator knowledge in IMs could be aided by including process knowledge, especially since Ringsquandl et al. [20] have achieved promising results in knowledge graph completion by considering event embeddings. In manufacturing, orders Ω, which can be viewed as compositions of process iterations 𝐼, are produced for certain amounts of time. We propose to explicitly model this process data as (𝑖, ⟨belongs to⟩, 𝜔), where 𝑖 ∈ 𝐼 and 𝑜 ∈ 𝑂. The process iterations can be connected with the resulting quantified parameters, (𝜌 𝑖 , ⟨chosen in⟩, 𝑖), where 𝜌 ∈ P, quantified quality characteristics (𝑎 𝑖 , ⟨is exhibited after⟩, 𝑖), and quantified influences (𝑏 𝑖−1 , ⟨influences⟩, 𝑖), where 𝑎, 𝑏 ∈ O and (𝑏 𝑖−1 , ⟨is exhibited after⟩, 𝑖 − 1). P and O are defined in the modelling patterns for quantified parameters and quantified conditions, respectively. We note that process iteration 𝑖 is preceded by 𝑖 − 1 in order 𝜔, i. e. the quality characteristic 𝑏 𝑖−1 is exhibited after the conclusion of the preceding process iteration.
Connecting the process data with the modelling patterns described above is beneficial, especially in the case of data-based knowledge extraction, since it allows explanation by example. The definition is analogous for parameters 𝑝. Using this, we can denote a relationship between process data and aggregate as (𝑎 𝑖 , ⟨contributes to⟩, 𝑎 ^), where 𝑎 ^is an aggregation of quantified quality characteristic expressed in a specific rule. Both, 𝑎 𝑖 and 𝑎 ^, are expressed values of a quality characteristic and as such are equivalent in regards to their hierarchical level. The only difference is that the value of 𝑎 𝑖 is sampled from the real world process whereas 𝑎 ^is calculated based on a set of observations. If all process data is included in the IM, a supports relation could also be defined following the quality metric used in rule mining [36,37]. However, since data-based rules rely on aggregates, a direct comparison between 𝑞 ^𝑗 and 𝑞 𝑖 , as well as 𝑝 ^𝑗 and 𝑝 𝑖 would fail and need to be relaxed by an interval in which they are considered equal. Also, since the rules are split between parameters and quality characteristics there would have to be two separate relations.
Here, we will give an overview of the properties shown by different modelling patterns. Firstly, with exceeding expressiveness of and information contained in the representation its complexity is increasing. This increase of abstraction can be seen in the increase in hierarchy hierarchy for each additional quantification or multiple conditions. Secondly, if process data is included in the IM it is likely to lead to a strong imbalance, since process data is much more readily available than extracted operator knowledge. Both aspects highlight the challenges embedding methodologies for IMs with operator knowledge have to address. As most of the relations described in Section 3 are not symmetric it would be easy to generate inverse relations, e. g. implied by for implies, which could be beneficial in knowledge graph completion settings.
In this section we aim to give a first indication whether it is possible to embed knowledge graphs containing extracted operator knowledge available as rules. As such, we present a first step towards a methodology for constructing such an embedding and provide a preliminary evaluation. To this end, we utilize the dataset presented in [35] of a fused deposition modelling (FDM) process. We choose to model the knowledge with the pattern of quantified conclusions, since the dataset does not provide the data for more complex patterns, i. e. quantified conditions. Figure 4: Information Model for process data, i. e. parametrisation processes and their iterations, as well as a rule. The rule's condition and conclusion is quantified relying on aggregations of quantified process parameters and quality characteristics that result from the iteration processes.
The accompanying code is available on github1 .
To embed operator knowledge that is intended to assist learning systems instead of knowledge graph completion, subgraphs containing the knowledge that is particularly relevant for the given input should be embedded. In our case, the input is a defective quality characteristic, e. g. stringing, a common problem in FDM, that can be alleviated through a fitting parametrisation by the operator or learning system. Our embedding methodology (cf. Figure 5) is loosely based on the methodology outlined in Kursuncu et al. [38] that is an approach towards embeddings for learning systems addressing classification in an natural language processing (NLP) setting, rather than KG completion or predictive quality scenarios.
We follow a sum-based approach [39] that aggregates individual node embeddings, that are particularly relevant to the input quality characteristic. To identify the fitting subgraph 𝒮, the input is mapped to the respective node in the knowledge graph. Then, this node is propagated by one step for all outgoing edges to arrive at the parameters adjusted to alleviate this quality defect. Based on this, the propagated nodes, embedded by TransH [29], 𝑧 are aggregated by 𝑧 𝒮 = ∑︀ 𝑣 𝑖 ,𝑣 𝑗 ∈𝒮 𝑧 𝑗 ⊗ 𝐷(𝑧 𝑖 , 𝑧 𝑗 ) to form the subgraph embedding 𝑧 𝑆 . Here, 𝑣 𝑖 , 𝑣 𝑗 are the pairs of head and tail nodes resulting from the graph propagation and 𝐷(𝑧 𝑖 , 𝑧 𝑗 ) is the euclidean distance between the node embeddings of 𝑣 𝑖 and 𝑣 𝑗 . Dependent on which semantic information should be represented in the subgraph embedding, it must be decided which node embeddings to aggregate in 𝑧 𝒮 . If the head node does not hold semantic information, we suggest ignoring the head node in the subgraph embedding. As this is the case in our scenario, we only aggregated the node embeddings for the parameters. If a modelling pattern for a different abstraction level is used, e. g. quantified conditions, the propagation step has to be increased to deal with the introduced indirections.
Metrics commonly used to evaluate embeddings in knowledge graph completion settings, e. g. mean reciprocal rank and hits@k, are unsuited to establish the quality of embeddings of operator knowledge in our scenario since the required ground truth is not present. Instead, we propose that the fundamental behaviour of rule embeddings in predictive quality scenarios should be that the parameters adjusted for similar quality characteristics in the (sub)graph, should be as equal as possible to those in the embedding space. Therefore, we define a metric in analogy to hits@k, matches@k, based on the amount of overlap between the K closest quality characteristics in embedding and graph space. To be able to establish the amount of overlap, a set of the 𝐾 closest quality characteristics is prepared by ordering them descendingly according to their similarity-amount of overlapping parameters adjusted (higher is better) and euclidean distance (lower is better) for graph and embedding space, respectively. Then, the number of matches, #matches, between the respective sets is calculated for each quality characteristic. By conducting the comparison on a set, we ensure that small differences in similarity are not unduly exaggerated in the overall metric since the order is not important to determine a match.
For matches@k, 𝐾 has to be chosen according to the respective dataset. It decreases in expressiveness with increasing size since the unordered nature of the comparison leads to number of matches equalling the number of quality characteristics |𝑄| if 𝐾 = |𝑄|, which would equal an overlap of 100 %. Therefore, inspecting the actual similarities between quality characteristics in the graph space is necessary to determine the 𝐾 which is representative for real world similarity. This can either be done by relying on domain knowledge or by determining the point at which the similarities are abruptly decreasing. In the following experiment we use 𝐾 = 3 since we established experimentally that for greater 𝐾 similarities between the quality characteristics are rapidly increasing.
To determine whether it is possible to embed knowledge graphs containing explicit operator knowledge we conduct an experiment on the dataset described in [35]. After preprocessing and removing categorical parameters, it contains ratings of 13 quality characteristics and a total of 46 parameters that are adjusted to optimise the quality characteristics. The distribution of quality characteristics is skewed with more operator knowledge being present for those that occur more often.
Applying the methodology and metric described above we receive a mean #matches of 2.85 ± 0.38 (95.00 % ± 12.67 %) for 𝐾 = 3 over all quality characteristics for 46 dimensional embeddings. While this indicates a relatively high overlap, we investigate its distribution for the individual quality characteristics combined with their occurrence in Figure 6. In Figure 6a we can see that that the performance seems to generally increase with increasing occurrence of quality characteristics in the dataset. However, there seems to be a second cluster of well performing quality characteristics with relatively low occurrence that yields good results. Inspecting Figure 6b confirms this notion. Since we assume more operator knowledge to be present in the graph for quality characteristics with higher occurrence the fact that higher occurring quality characteristics lead to better results seems to underpin the conclusion that extracted operator knowledge in embeddings can be represented by embeddings. However, a significant portion of quality characteristics with low occurrence also leads to good results.
While the presented preliminary evaluation strengthens our hypothesis, a more thorough evaluation is needed to arrive at a firm conclusion. This includes a comparison of the proposed embedding method on the different modelling patterns. Also, the influence of increasing hierarchies, due to increasing complexity, on embedding methods will be investigated. In this context, evaluating more complex embedding methods which have been shown to deal well with hierarchies between concepts such as RotH [23] would be interesting. Moreover, an evaluation on multiple datasets would allow greater confidence in regards to the transferability of the described concepts. However, we are not aware of any suitable public datasets in the industrial domain at this time.
Furthermore, the behaviour of the embedding methods for varying levels of noise in the data should be investigated, since complex information models are rarely error free. Additionally, uncertainties of operators could be encoded using soft rules.
In the patterns modelling operator knowledge presented in this work we strongly relied on relations to represent properties of vertices and relations. These properties could also be represented as attributes of vertices. While this would reduce the involved hierarchies it imposes other complexities for embedding methodologies. As such the integration of attribute-based embeddings [40,41] could be beneficial.
In addition, the applicability of common KG completion approaches on KGs containing operator knowledge could be researched to infer relations or nodes that have not been encountered in reality, thereby increasing the information content of the representation.
Lastly, by an integration with learning systems, as outlined by Kursuncu et al. [38] for NLP, we could directly measure the impact of the knowledge contained in IMs on the predictive power of learning systems.
In this paper we presented several modelling patterns for including extracted operator knowledge into industrial information models, represented as knowledge graphs. These modelling patterns can be conceived as architectural patterns and can be combined and applied depending on the required complexity that should be expressed. Furthermore, we presented an embedding methodology to represent this knowledge as a vector that could be used to combine learning systems with operator knowledge. We established a metric suited to evaluate the embedding's capability to capture semantic relations between conditions, i. e. quality characteristics, based on their resulting conclusions, i. e. parametrisations. In a preliminary evaluation, we have shown that the chosen information model and the proposed embedding methodology are able to express and capture semantic relationships between conditions that lead to similar conclusions if they occurred in a sufficient quantity.
This work is funded by the Bavarian Ministry of Economic Affairs, Energy and Technology in the scope of the ADELeS project.