Literal-Aware Knowledge Graph Embedding for Welding Quality Monitoring

Abstract

Recently there has been a series of studies in knowledge graph embedding (KGE), which attempts to learn the embeddings of the entities and relations as numerical vectors and mathematical mappings via machine learning (ML). However, there has been limited research that applies KGE for industrial problems in manufacturing. This paper investigates whether and to what extent KGE can be used for an important problem, that is quality monitoring for welding in manufacturing industry. It is an important process accounting for production of millions of cars annually. The work is in line with our research of data-driven solutions that intends to replace the traditional costly quality monitoring. The paper tackles two challenging questions simultaneously: how large the welding spot diameter is; and to which car body the welded spot belongs to. The problem setting is dificult for traditional ML because there exist a high number of car bodies that should be assigned as class labels. We formulate the problem as link prediction, and experimented popular KGE methods with literals on real industry data, with consideration of literals. This paper accompanies the full paper in in-use track and provides additional discussion on problem formulation, literal handling strategies, and included information in industrial KG construction.

1. Introduction

Background and Challenge. Research in knowledge graphs and their industrial applications has attracted increasing attention [ 1 ]. Recently there has been a series of studies in knowledge graph embedding (KGE), but there has been limited research that applies KGE for industrial problems in manufacturing. This paper investigates whether and to what extent KGE can be used for an important problem, that is quality monitoring for welding in manufacturing industry. We discuss automated welding, which plays a critical role in the automotive industry for manufacturing high-quality car bodies, with millions of cars produced annually. The welding process generates a vast amount of data, considering the number of welding machines in car production lines and the thousands of spots on each carbody (Fig. 1a). This large amount of data increases the demand of data-driven solutions, which aim to reduce and eventually replace a Current, voltage, Resitance,

Time, Power etc.

Moving direction ! spQoutedsiatimonet1e:r?

regression: predict real number discretisation entity creation Question 2: Which

carbody part? classification: predict carbody part label diameDtei<ra0ma.e1stecrlasses linpkrepdreicdtictatiioln: rrooddwwff:llst:::yCOsuplbabejseCscltaPsrsoOpfertaby

D0i.a1m-0e.t2er ? Spot103 Diameter D0i.a2.m-..0e.t3er SSSppphoooettta111d000333 prrrrdddefffd:::itttcyyyapppteeee DDDiiiaaammmeeettteeetrrrai000l...123---000...234??? s000c...o653r524e lparbeedli,ctriaonnk,=1 caarsboednytitpieaCsrtasr body Caprabrto3dy bc part1 conventional destructive, yet extremely expensive and ineficient, methods. Addressing this challenges, two core questions need to be answer here as shown in Fig. 1b and Fig. 1c. First, spot diameter (Q1) is the key quality indicator for evaluating welding quality. Since the diameter must be neither too large nor too small. Second, the carbody part of the spot is important because it is essential to know the percentage of good spots for each car body, but the carbody-spot correspondence information does not exist in a large amount of historical data silos from past welding systems. Currently there are research applying ML and Digital Twin in industry [ 2 ], but limitedly solve this challenge especially the two core questions.

Our Approach. We investigate KGE for answering the two questions in the automotive industry with production data, and compare with representative classic ML. We first construct KGs from tabular data with special handling on literals; then, we conduct experiments and compare mainstream KGE methods such as TransE, RotatE, AttH with multilayer perceptron (MLP), and compare with variant KGE applied in [ 3 ]. This poster paper accompanies our full paper in ISWC 2023 [ 4 ]. This paper provides additional discussions on problem formulation, literal handling, and industrial KG construction, which were not present in the full paper.

2. Approach

Welding KG construction. A welding KG is constructed from tabular data collected from a German factory. We have used welding-related information, such as time of welding processes, welding machines, welding programs, and welding parameters (e.g., voltage, current, resistance). The constructions are conducted on welding spots and the car body and diameters. We transform the values of the welding data table into entities and the relationships between these entities as edges in the KG. Fig.2b shows the construction of literal entities, which are entities generated from numeric values. The literals are handled as described in the later part.

Problem formulation. Fig. 1b and Fig. 1c shows in more details the two research questions of the quality monitoring in the use case and our approach to reformulate them: given the information of the welding spot, we want to predict the carbody of the this welding spot and the diameter of the welding spot. As shown in the Fig. 1, both of the problems are reformulated. The Spot diameter prediction was a regression problem based on the welding data to predict the real values for the diameters size. Due to resolution when measuring the spot diameter, hasVoltagemean

Diameter 0.2-0.3

?

Diameter ...

Spot103

belongsTo ?

Car body part2

a b diameter as classes

Diameter <0.1

Diameter

0.1-0.2 time literal entities conntoecKtG

I_mean <0.1

Machine1 U_mean <0.1

hasCurrentmean

producedBy operatedBy owl:Class rdf:type owl:ObjectProperty rdfs:subClassOf hasProgram hasCarbodypart hasComponhaesnCto1mponent2 h a so C m p o n e nt 3

Program1

Component1

Component2 Component3 current (I) stage2 aggregation stage3 stage1 spot_id I_mean I_1_mean I_2_mean I_3_mean Spot103 0.72 0.63 0.80 0.60 Spot104 0.57 0.52 0.70 0.49 discretisation for each feature

I_mean <0.1

I_mean 0.3-0.4

...

I_mean 0.1-0.2

I_mean 0.4-0.5

...

I_mean 0.2-0.3

I_mean 0.5-0.6

... we discretise the diameters into diferent diameter classes as classification problem. We then constructed the entities based on the diameter classes and carbody class. The links between welding spots and the diameter classes or the carbody are predicted.

Literal handling. We did the following steps for literal embeddings inspired by [ 5 ]. The numeric literals of the knowledge graphs are embedded following aggregation, discretisation of features, entity creation, and linking. As shown in the Fig. 2, in the aggregation step, the sensor measured values are aggregated into the mean values of the three stages and the overall mean values in real numbers. Then in the discretisation step, we discretise the real values into diferent ranges. And then we create entities based on the discretised ranges and link them.

3. Evaluation and Discussion

We discuss the experiments and provide additional discussion on the problem formulation, the literal handling and discretisation techniques, and the industrial KG construction. Discussion on problem formulation. We discuss three promising ways: (1) classic ML with MLP; (2) classic KGE with link Table 1: Model performance comparison on answering Q1 and prediction; (3) binary triple classifi- Q2. Bold: best results. Underlined: second best. cation inspired by [ 3 ].

MLP TransE RotatE AttH MLP. In manufacturing, quality Acc(Hits@1) 0.39 0.42 0.25 0.31 monitoring aims at predicting diam- Q1 MRR - 0.65 0.49 0.57 eters and carbody. These questions nrmse 0.05 0.06 0.08 0.06 are formulated as classification prob- Acc(Hits@1) 0.61 0.64 0.52 0.53 lems, where carbodies and diame- Q2 MRR - 0.77 0.69 0.70 ters are formulated as the predicted Hits@Groupby3 - 0.85 0.81 0.79 classes. This formulation is verified by the MLP model. This model is proven to be most eficient and have best performance only in nrmse in Q1, but provides inadequate performance for Q2, see Tab. 1.

KGE. Since there can be over hundreds and thousands of carbodies, Q2 is not very suitable as a classic ML problem. We adapt the problem and reformulate it as link prediction (Fig. 2), where the correct link between a welding spot and the correct carbody should have the highest score among all the other carbodies. Similar principles also hold for diameter predictions. This formulation is verified by the KGE model. We use metrics, including Acc, MRR, and Hits@Groupby3. We introduce a new metric Hits@Groupby3, because no KGE model deliver satisfactory Acc. We thus adopt the adaptation to relax the metric, to test the accuracy based on the group of 3 carbodies. The results, see Tab. 1, show TransE delivers the best results for both Q1 and Q2 on Acc, MRR, and Hits@Groupby3.

KGE-MLP. The third possible problem formulation is inspired by [ 3 ], where the link is formulated as binary classification with the output score between 0 and 1, where a value closer to 1 indicates the link exists. The score is compared with all other potential diameters or carbodies, and the predicted link the highest score is selected as the prediction. This formulation is verified by the KGE-MLP model. This model proved to be not good on Acc, MRR, and other metrics compared with the MLP and KGE models in our welding quality monitoring use case, but is State-of-the-Art (SotA) model on other use case [ 3 ]. So we still choose this model in our paper. Discussion on literal landling. In Table 2: Best KGE model compared with KGE-MLP models. SotA research, there exist mainly The KGE-MLP models are marked with*. Bold: best. two ways to handle literals: discretisation (KGA) [ 5 ] and literal as em- Metric TransE TransE* DistMult* HolE* bedding vector (LiteralE) [ 6 ]. Ac- Acc(Hits@1) 0.42 0.17 0.22 0.21 cording to the experiments in [ 5 ], Q1 MRR 0.65 0.45 0.48 0.48 the discretisation with bins meth- nrmse 0.06 0.11 0.09 0.10 ods yield SotA results with large im- Hits@1 0.64 0.34 0.34 0.37 provements on the traditional KGE. Q2 Hits@MGRroRupBy3 00..8757 00..4485 00..5426 00..5421 We also consider [ 6 ] not suitable, because it requires fixed embedding size for fixed number of literals, while in real application the number of literals can vary a lot. This paper chooses the discretisation with bins as in [ 5 ] method to encode the literal information. Other discretisation methods are compared and discussed in the next part.

Discussion on discretisation strategy. There are diferent discretisation approaches discussed in the paper, including the single setting without overlapping, overlapping and hierarchical settings. There are also two diferent bins creation methods based on frequencey or the fixed value. According to the experiment results the single bin with fixed value is simple and the performance diferences between diferent discretisation strateties are insignificant, so we choose this method on the welding quality monitoring.

Discussion on included information in industrial KG construction. In the KG construction step, we also notice it is important to consider the impact of the tabular columns on the KGE performance. The number of columns in production data are very large (over 200), but only few information are important based on domain knowledge, and most of them are meta-settings, or overlapping information that do not contribute to the KGE performance. This is a common problem for industrial KGs, especially for industries such as manufacturing, mining, chemistry, where large amount of information is collected, but only few are essential for the quality monitoring.. Thus, we need to choose the least amount of columns that represent the most important information for welding. This selection process is done by iteratively updating the features and evaluating the performance. Our observations are, the most important information are (1) those that are crucial to the graph structure of the KG, such as welding machine, welding program, the materials and thickness of carbody, that have impact define the graph structure; (2) sensor values (literals in KG).

4. Conclusion and Outlook

This poster paper presents an extended abstract of our full paper [ 4 ] and provides additional discussions. The research is under the under the umbrella of Neuro-Symbolic AI for Industry 4.0 at Bosch. We aim at enhancing manufacturing technology with both symbolic AI [ 7 ] (such as semantic technologies) for improving transparency [ 1 ], and ML for prediction power. We will further improve the performance of the KG embedding method and develop other complementary technologies, such as ontologies [ 8, 9 ], ontology-based data access, etc. Acknowledgements. The work was partially supported by EU projects Dome 4.0 (953163), OntoCommons (958371), DataCloud (101016835), Graph Massiviser (101093202), EnRichMyData (101093202), and SMARTEDGE (101092908) and the Norwegian Research Council funded projects (237898, 308817).

References

[1] Z. Zheng , B. Zhou , D. Zhou , A. Soylu , E. Kharlamov, Executable knowledge graph for transparent machine learning in welding monitoring at bosch , in: CIKM, 2022 , pp. 5102 - 5103 .

[2] Z. Huang , M. Fey , C. Liu , E. Beysel , X. Xu , C. Brecher , Hybrid learning-based digital twin for manufacturing process: Modeling framework and implementation , Robotics and Computer-Integrated Manufacturing 82 ( 2023 ) 102545 .

[3] E. B. Myklebust , E. Jimenez-Ruiz , J. Chen , R. Wolf , K. E. Tollefsen , Knowledge graph embedding for ecotoxicological efect prediction , in: ISWC , 2019 .

[4] Z. Tan , B. Zhou , Z. Zheng , O. Savkovic , Z. Huang , I. G. Gonzalez , A. Soylu , E. Kharlamov, Literal-aware KGE for welding quality monitoring , in: ISWC , 2023 .

[5] J. Wang , F. Ilievski , P. A. Szekely , K.-T. Yao, Augmenting knowledge graphs for better link prediction , in: IJCAI , 2022 .

[6] A. Kristiadi , M. A. Khan , D. Lukovnikov , J. Lehmann , A. Fischer , Incorporating literals into knowledge graph embeddings , in: ISWC , 2019 .

[7] D. Rincon-Yanez , M. H. Gad-Elrab , D. Stepanova , K. T. Tran , C. C. Xuan , B. Zhou , E. Karlamov, Addressing the scalability bottleneck of semantic technologies at bosch , ESWC Industry ( 2023 ).

[8] B. Zhou , Z. Zheng , D. Zhou , Z. Tan , O. Savković , H. Yang , Y. Zhang , E. Kharlamov, Knowledge graph-based semantic system for visual analytics in automatic manufacturing , ISWC , 2022 .

[9] Z. Zheng , B. Zhou , D. Zhou , A. Q. Khan , A. Soylu , E. Kharlamov, Towards a statistic ontology for data analysis in smart manufacturing , in: ISWC Posters , volume 3254 , 2022 .