Introduction

With the advent of the fourth industrial revolution, the manufacturing environments are evolving towards digital ecosystems. Propelled by the improvement in sensing and communication technologies, the Internet of Things (IoT) has taken root in traditional industries, and this has opened the world of Big Data, where the volume, velocity and variety of data is rapidly rising [1]. This has been no different for the case of Bosch, currently undergoing a transformation from a traditional manufacturing company to an IoT one [2].

One of the most critical manufacturing processes at Bosch is Resistance Spot Welding (RSW), primarily employed for joining car body components. This process involves passing a high-intensity electrical current through two electrodes and the metal sheets positioned between them, generating localized heat that fuses the sheets together at specific points, creating a weld spot [3]. Even if it is considered a conventional method, this is still a very complex yet critical procedure [4], as the failure of a single spot can halt an entire production line [3]. In the industry, quality assurance for RSW is typically achieved via offline destructive inspection methods on samples selected by predefined empirical rules, but due to the heavy class imbalance on the sets, detecting faulty spots is usually a very rare occurrence. Consequently, data-driven approaches have been proposed in recent years, with the objective of detecting defective samples more efficiently and reducing scrap and downtime [4]. Some of these approaches focus on image processing, and they are oriented towards assessing the quality of the spots in an online fashion with the help of cameras mounted on the welding devices Prediction of the Weld Qualities Using Surface Appearance Image in Resistance Spot Welding, Monitoring of Resistance Spot Welding Process). Others, mainly directed by the work of Chen et al., developed systems for weld quality estimation based on infrared thermography [5] [6]. However, implementing these methods is often challenging due to the need to acquire, install, and integrate specialized hardware, which can be costly and time-consuming. Therefore, the research focused on using intrinsic process variables for monitoring is especially valuable. From the early works of Cho et al. [7], many different data-driven methods have been developed, either defined as classification or prediction tasks, depending on whether the intention is to estimate the welding nugget diameter, or to identify defective welds. In this paper, developed under the Graph Massivizer project, we introduce the need for a hybrid approach for welding quality estimation, one that leverages on existing ontologies to act as an inductive bias on top of the available sensor data, with the objective of obtaining a more grounded and efficient algorithm.

Use case

Resistance Spot Welding (RSW) is widely used in the automotive industry, as over 90% of assembly work in a car body is completed by RSW [8]. These spots are critical for the structural integrity of the vehicle, and the amount of spot welds in a car are counted in the thousands [9]. Resistance Spot Welding is a process used to join two or more metal sheets together by applying pressure and passing an electric current through the materials. The process generates heat due to the resistance to the electrical current in the contact area between the metal sheets, causing the metals to melt and fuse together. The welding electrodes also provide the necessary force to hold the materials in place during welding [10]. This is a very complicated process involving electromagnetic, thermal, mechanical and metallurgical variables [8]. Even in very controlled environments, defects have been shown to be very difficult to detect, as the welds are hidden between two sheets of metal. As such, in resistance spot welding, quality assurance typically involves the use of offline destructive inspection techniques on selected samples. These methods involve directly measuring the mechanical strength, fatigue resistance, and failure modes of the weld joints. By physically testing these samples to the point of failure, critical information about the integrity and durability of the weld can be obtained. This approach, while effective in assessing joint quality, is limited by its destructive nature and the fact that it only provides insight into the specific samples tested, rather than the entire production run [4].

Most modern RSW operations are very efficient, and defective welds are a rare yet critical occurrence, as such, defining data-driven methods to target these defects is a priority for the automotive industry, and it has been a very rich subject of research. The main difficulties explored in the literature for the implementation of efficient quality prediction algorithms are the following: (i) limit on labeled data amount, (ii) limit on features, (iii) limit on coverage of relevant situations [11]. Increasing the amount of labeled data is a resource-intensive task, therefore, it is often best avoided. Regarding (ii), some authors have made use of additional sensors (infrared sensors [5], cameras...) for reliable quality prediction, however, this is often difficult to implement and over-complicates the process, which leads to more failures and downtime. Lastly, (iii) is the most pressing challenge, because as quality is good under normal conditions, the datasets are often times heavily imbalanced, and creating more anomalous labels requires expensive laboratory testing.

Therefore, it is specifically interesting to detect RSW defects using the highly available dynamic electrical features captured during the process, namely the current and the voltage, which are used to conform to the dynamic resistance. These methods treat the dynamic curves as time series and aim at identifying the underlying patterns that lead to anomalies in the welds. In [12], a random forest classifier was trained with features derived from this signal and achieved up to 98% accuracy. Similarly, [13] used a radial basis function neural network to predict nugget diameter and weld strength, achieving 98% accuracy by classifying features into different quality levels. In [14], a kernel extreme machine model optimized with a particle swarm algorithm also reached 98% accuracy. The performance of these models is very solid in their specific scenarios, and it leads to the incorrect idea that quality assurance in RSW is a solved topic. However, as mentioned by Stavropoulos et al. [4], RSW is a very heterogeneous process, and the variation of the elements that comprise the RSW ecosystem and process (machines, materials, thickness, position...) heavily affect the final quality of the joint.

Towards this direction, a need for a more holistic approach has been defined, one that takes into account a bigger portion of the welding ecosystem and is able to adapt to the different flavors that RSW can have. It is this approach that Tan et al. followed [15] by framing the quality prediction problem as one of link prediction in a literal-aware RSW welding knowledge graph. By integrating background knowledge on the prediction, the model is able to get a higher-level view of the operation and tackle the heterogeneity of RSW in a more holistic manner. Even if the performance of this approach is notable, in an attempt to frame it under the more traditional embedding methods, it deals with the time-series nature of the curves by aggregating the values of the dynamical electrical features, leading to a great loss of expressivity and decrease in performance, as Dickinson et al. stated in [16].

Proposed methodology

It has been shown that while data-driven methods lack the capacity to model the complexities of the RSW ecosystem, knowledge-graph-driven approaches lack the expressivity and granularity afforded by time-series data. Thus, motivated by the work of Jain et al. [17], the objective of Bosch and the consortium partners under the Graph Massivizer project is to merge both approaches and design the next-generation quality monitoring methods that go beyond the more traditional data-driven methods by combining knowledge and sensor measurements.

On the one hand, we refer to sensor measurements as the time series that are produced during the welding operation, which can have the form of currents, voltages, temperatures. . . These are variables that evolve during each weld and are fundamental to the estimation of the quality (e.g., if an abnormality has been detected in the current that flows through the cathode of the welding machine, the presence of an anomaly will be likely). On the other hand, we refer to knowledge that encompasses diverse and rich prior information about the process, derived from expert knowledge, such as the ontology for a specific RSW machine. This ontology is used as the backbone to create knowledge graphs for each specific operation, similar to the approach that Tan et al. followed in [15], but instead of pointing at the aggregated values of the dynamic features, it points at the entire representation of the time-series, thus maintaining the expressivity in the analysis. This ontology is then used as a structural inductive bias that inherently captures the nature of RSW. Subsequently, a Graph Neural Network (GNN) is trained for prediction, leveraging the ontology-based knowledge graph to enhance its learning capability [18]. This method is not only expected to improve the model's accuracy in predicting outcomes but also enable it to generalize better across different RSW operations by integrating domain-specific knowledge. Additionally, the use of GNNs allows for the incorporation of complex relationships and dependencies between various process parameters, providing a more holistic and detailed understanding of the RSW process dynamics. This approach demonstrates the potential to bridge the gap between data-driven methods and expert knowledge, resulting in a more robust and comprehensive modeling framework.

Drawbacks

Despite the high expectations for the project and the computational soundness of the method, we are currently encountering some challenges.

• A significant challenge lies in effectively representing time-series data within a graph-based framework. While time-series data provide rich, dynamic information about the welding process, converting this data into a graph structure that preserves its temporal properties and contextual relevance is complex. One possible solution is to use visibility graphs, which transform time-series data into networks where each data point (or "node") is connected to other nodes based on specific visibility criteria. Although visibility graphs can capture the intrinsic patterns and relationships within the time series, they may struggle with scalability when dealing with large datasets or high-dimensional data. Furthermore, it remains unclear how to best integrate these graph-based representations with existing domain knowledge in a cohesive manner that enhances the model's predictive capability.

• Combining knowledge graph data with time-series data in a meaningful way remains an open question. For example, a potential approach could involve appending embeddings of events generated by a BERT-like model [19] to embeddings derived from a feature tokenizer of the time-series data, as proposed by Jain et al. [17]. However, this strategy may not be effective in our use case due to the scarcity of events, which limits the potential information gain from such embeddings. • Using ontologies as structural inductive biases, while theoretically sound, raises questions about scalability. For instance, one could use a BERT-like model to create embeddings for materials, static conditions, and other domain-specific knowledge, and then train a Graph Neural Network (GNN) with this data. However, it is uncertain how well this approach would scale to larger datasets or more complex welding scenarios. Moreover, techniques like node2vec [20], which are often used for node embeddings in graphs, may not be suitable for capturing the nuanced, multi-relational nature of the knowledge graph in this context.

Conclusion

In conclusion, this paper highlights the need for a hybrid approach to quality assurance in Resistance Spot Welding (RSW) by combining knowledge graphs and time-series data to address the limitations of traditional data-driven and knowledge-graph-based methods. The proposed methodology, leveraging Graph Neural Networks (GNNs) and domain-specific ontologies, offers a more comprehensive and adaptive modeling framework that captures the complex dependencies and dynamics of the RSW process, leading to improved prediction accuracy and robustness. However, challenges remain in effectively integrating time-series data within a graph-based framework and ensuring scalability across diverse welding scenarios.