Advancements in automation and increasingly capable AI models are transforming industrial quality control and shift the human involvement toward higher-level tasks. This development brings both risks and opens up potentials, requiring a careful and conscious shaping of this change to ensure an optimal outcome. In this position paper, we approach this challenge from the perspective of human potential loss and reflect on our engagement with industry stakeholders from the automotive, aerospace, and coating domains. We outline key principles for both preserving human expertise and engagement in hybrid workflows and active building of new skills, proposing a conceptual model for interactive visual inspection and labeling. In addition, we discuss initial visualization interface considerations and outline research directions for enhancing human-AI collaboration in quality control.
Automation is widely used in industrial quality control (QC) to enhance consistency and eficiency
while reducing manual labour. Traditional automated inspection systems often rely on predefined rules
and threshold-based evaluations. Advancements in machine learning and deep learning now enable
the automation of more complex QC tasks [
CEUR Workshop
ISSN1613-0073
This shift reinforces the need to actively preserve, utilize and augment human expertise in quality
control processes [
In this work, we reflect on findings from a large-scale industry project with multiple partners, including technology providers and manufacturing companies from sectors such as automotive, aerospace, and coatings. By analyzing workflows and challenges, we investigate how human-AI collaboration in quality control can be strengthened. Specifically, we explore how interactive visual analytics can support human oversight, intervention, and knowledge transfer while advancing AI-driven automation.
We outline ten key principles to enhance human potential (Sec. 2), propose a conceptual model for interactive visual inspection and labeling (Sec. 3), and discuss initial visualization design considerations (Sec. 4). Finally, we outline research directions to improve hybrid human-AI quality control (Sec. 5).
Growing automation heightens the risk of human potential loss, a major concern for industry. While
automation enhances eficiency, it can lead to disengagement, knowledge erosion, and reduced
adaptability among workers [
Strengthening human potential is essential for maintaining expertise and decision-making quality,
but has also critical links to employee satisfaction, workforce stability, and long-term value creation [
Over two years, we collaborated with industry stakeholders, including technology providers and
manufacturers, and conducted a series of workshops and interviews on human-centric aspects of
zero-defect manufacturing [
P2. Transparent and Trustworthy Decision-Making. AI outputs should be interpretable to help
users build trust, improve oversight, and develop expertise [
Clearly defined roles and collaborative decision-making mechanisms ensure that human workers remain in control throughout the process.
P4. Reducing Ergonomic Strain. Automation and AI should be used to alleviate repetitive and
high-strain QC tasks, to allow human workers focus on higher-level decision-making.
P5. Situational Assistance. AI should provide context-aware recommendations and synthesize
knowledge to support workers in both routine and non-routine tasks [
P7. Knowledge Documentation and Sharing. Findings and interventions should be systematically recorded to support organizational learning and eficient problem resolution. Knowledge retention should not be outsourced to the AI but should remain accessible and transferable.
P8. Capturing Tacit Knowledge, which is often dificult to articulate and embedded in daily practice, should be systematically recorded across diferent levels of interactions. In addition to traditional documentation and training approaches, methods such as behavior-oriented data collection and gamification can help surface implicit expertise and support continuous learning.
P9. AI-Assisted Pattern Recognition. Revealing patterns and root causes of quality issues can help to anticipate failures and optimize production processes [19].
P10. AI Literacy and Oversight. AI should help workers understand its strengths and limitations, fostering overall AI literacy [20].
Building on these human potential principles, we propose a conceptual model that integrates human expertise into AI-driven QC. This model, illustrated in Fig. 1, structures human-AI collaboration into two interrelated processes (P3): continuous monitoring, which runs mostly automatically, and on-demand visual inspection and labeling, performed by users. Since our focus is on human interaction, we provide a high-level overview of the continuous monitoring before detailing the user’s role in the workflow.
Throughout the production monitoring process, data is collected automatically and continuously (P4). Depending on the setup—which varies by domain and product—samples are taken at fixed intervals, at specific stages, or based on other predefined conditions. These samples may include images, infrared scans, structured sensor data, or other modalities. A machine learning model processes each sample, predicts and assigns a label, which may take the form of a numerical score, a boolean value, or a categorical assessment. The results, along with timestamps and, ideally, confidence scores, are saved in a database. While automated quality control has existed for years, earlier systems often relied on rule-based checks and frequently required human involvement to interpret results. With advances in deep learning, these processes are now largely autonomous and allow the AI to perform more complex analyses and detect quality issues (P9). This three-step process of data collection, sample selection, and AI-based labeling may be repeated across diferent products and QC workflows.
User involvement in this process occurs through two mechanisms: alerts, which are system-triggered notifications, and manual fetching, where users actively request information. Interaction takes place through dedicated user interfaces with embedded visualizations (P1). These interfaces support various user tasks, which we outline here, while preliminary design considerations are discussed in Sec. 4.
Aggregated Summary. This interface enables users to assess the overall state of the AI-driven monitoring process, as schematically shown in Fig. 2. Users analyze temporal patterns, KPIs, and interpret explainability features (e.g., feature importance analysis; see P2). This supports high-level situational awareness (P5) and aligns with the typical overview-first, details-on-demand approach [ 21]. Users can quickly evaluate the number of inspections performed, detect anomalies, and identify areas that require further investigation. Alerts and recommendations guide users toward critical incidents (P6), while filtering options help them refine their focus.
Individual Results with Labels. By reviewing individual samples, users compare AI-generated
labels with their domain expertise to confirm or override model results, as illustrated in Fig. 3. This is
done for spot checks, regulatory compliance, trust calibration, model improvement, or learning from
AI insights, among other reasons (P1). Depending on the QC tasks, users assess simple visual cues
like color deviations or cracks [
Documented Cases. Users can retrieve, review, and contribute to past cases, including previous quality issues and per-sample interventions (P7 ). In addition, they can document broader corrective actions and best practices, such as those addressing recurring quality issues.
Recommendations. The AI system synthesizes computational results (e.g., predicted labels) with information from the knowledge base to generate actionable recommendations (P5, P6). These may include checklists, suggested machine or process adjustments, and other decision-support elements.
Users also play an active role in refining the automated monitoring process by applying insights from inspections, making corrective decisions, and changing broader aspects of the production process.
We highlight visualization and interaction techniques that may be adapted for the QC context. While the conceptual model outlines hybrid collaboration, its efective implementation relies on well-designed interface components. These approaches represent initial ideas that must be refined and complemented by additional visualizations and interface components to support all outlined human potential principles.
A core objective is to provide users with a high-level overview while enabling them to explore subsets and patterns before inspecting individual samples. Inspired by LineUp [25] and related techniques [29, 30], we propose a multi-attribute ranking, in combination with an annotated timeline, and embedded in a coordinated multiple-views system [31], as illustrated in Fig. 2. The multi-attribute ranking allows users to analyze heterogeneous attributes by ranking samples based on predicted labels, confidence scores, or other QC-relevant metrics. Each row can represent an individual sample or a group of samples. To further support pattern recognition and anomaly detection, word-scale visualizations can be embedded within each row [32]. However, a limitation of traditional ranking techniques is their lack of temporal context, which is essential for QC. Identifying temporal patterns [33], such as defects occurring at regular intervals or systematic issues, but also AI-judgment mistakes and factors relating to these, can provide critical insights and support AI oversight capabilities. Additionally, understanding when users override AI predictions helps in refining models and workflow processes. Thus, we suggest the integration of an annotated timeline that highlights user corrections, and system-detected anomalies, and other essential events. The timeline is interactively linked with the ranking and allows users to correlate data attributes with temporal trends. In large-scale time series data, where samples may be collected at high frequencies, temporal guidance mechanisms [34] become essential.
Feature importance and attention visualizations help to extract high-level insights from data. While displaying raw or aggregated data is essential for certain tasks, visualizations should also reveal key patterns to support deeper analysis. This becomes particularly relevant when working with large datasets and in settings where AI models perform a high degree of automation. Feature importance visualizations help users understand which variables most influence the model’s decision-making process. Methods such as SHAP explanations can highlight key features, but they must be presented in a way that is understandable for lay users [38, 26]. For image-based QC tasks, additional visualizations can indicate why a model made a certain prediction or, for instance, highlight regions of interest with AI-human mismatches. Examples include heatmaps [39], contour plots [40], or attention maps [27], which can reveal typical patterns the model associates with defects or quality issues.
Uncertainty visualization is essential for reliable and transparent decision-making, as well as for building trust in AI [41]. In addition to helping users learn from AI decisions and develop their skills, it also supports a better understanding of AI limitations, fostering AI literacy regarding its strengths and constraints in the QC context. To support this, uncertainty quantification and model explainability techniques [42] should be integrated at both the overview level and in per-sample inspections.
This directly connects to human-AI alignment. Explainable AI and uncertainty displays should also incorporate human feedback. For instance, when the AI lacks confidence, the system could present similar cases labeled by both AI and human inspectors to aid comparison. Mismatches between human and AI labels should be factored into future uncertainty assessments. Additional overviews of alignment patterns could highlight areas where AI consistently performs well or struggles.
The increasing automation of industrial QC through more capable models presents both opportunities and challenges. While AI can enhance eficiency and accuracy, human expertise remains critical for ensuring reliability, adaptability, resilience, and long-term knowledge transfer.
From technology push and societal needs to a concept. We approach this challenge from the perspective of human potential loss and have compiled a set of principles to address it. Building on these principles and insights from QC across various sectors, we developed a workflow abstraction to guide our thinking about human-AI collaboration and support discussions with stakeholders.
These represent an initial, opportunistic sample rather than a definitive framework. We welcome active dialogue and will continue to refine both the principles and conceptual model through iterative engagement with industry stakeholders.
From conceptual model to interface design. A major challenge lies in translating workflow abstractions into usable interaction paradigms and interactive visualization techniques [43]. We outlined initial considerations for overview and detail page designs to support human oversight, intervention, and knowledge transfer. However, further research is needed to refine these interface ideas, increase their level of fidelity and evaluate their efectiveness in real-world settings. Key research questions are: How can QC interfaces support the systematic capture and transfer of tacit knowledge (P8)? How can QC interfaces dynamically adjust based on user expertise, confidence levels, or task complexity (P5)? How should AI uncertainty be communicated (P2)? How can a system distill actionable recommendations from AI predictions and the knowledge base, and how should these be presented (P6, P7)?
From interface to counterpart. Integrating AI into QC workflows is not just about building models and interfaces but about fostering hybrid teams where humans and AI collaborate efectively ( P1) [44]. Besides the interface design, successful human-AI collaboration depends on cultural and organizational considerations that extend beyond HCI and visualization research (P3).
Quality issues often emerge later in the production cycle or post-production [ 45] which requires systems to integrate feedback from multiple stages. Research is needed to explore how AI-assisted QC can support this long-term traceability. In general, the transition to hybrid workflows requires a deeper understanding of the human role in QC [46]. Designing intelligent automation in a way that supports human engagement, trust, and skill development is essential to ensure that workers remain active participants rather than passive overseers, supporting the development of resilience both on an organizational and an employee-level.
In this position paper, we explored efective human-AI collaboration in industrial quality control. We emphasized the importance of preserving human potential and outlined a conceptual model to support this goal. We plan to develop prototypes and engage further with industry stakeholders to refine our approach and validate assumptions. Additionally, we aim to assess whether our proposed workflow and design ideas efectively support human-AI collaboration in real-world quality control settings.
While our focus has been on industrial QC, we believe many ideas are transferable to other hybrid environments where humans and AI systems must interact efectively. Therefore, this work shall also serve to encourage further research on human-centered AI collaboration across diferent domains.
This work is part of the ZERO3 project (FO999896399), funded by the Austrian Research Promotion Agency (FFG). The authors thank Profactor, i-RED Infrarot Systeme, and the other project partners for their valuable contributions.
The authors used GPT-4o for grammar and spelling checks. After using these services, the authors reviewed and edited the content and take full responsibility for the publication’s content. digital divide between “new” and “old” experts, in: Proc. Nordic Human-Computer Interaction Conference, 2022, pp. 1–6. [19] O. El Melhaoui, I. El Melhaoui, F. Jefali, S. Said, S. Elouaham, Integration of artificial intelligence algorithms for defect detection and shape recognition in mechanical quality control, Interactions 246 (2025) 1–14. [20] D. Long, B. Magerko, What is AI literacy? competencies and design considerations, in: Proc. CHI
Conf. on Human Factors in Computing Systems, 2020, pp. 1–16. [21] B. Shneiderman, The eyes have it: A task by data type taxonomy for information visualizations, in: The Craft of Information Visualization, Elsevier, 2003, pp. 364–371. [22] G. Cao, S. Ruan, Y. Peng, S. Huang, N. Kwok, Large-complex-surface defect detection by hybrid gradient threshold segmentation and image registration, IEEE Access 6 (2018) 36235–36246. [23] F. Frustaci, F. Spagnolo, S. Perri, G. Cocorullo, P. Corsonello, Robust and high-performance machine vision system for automatic quality inspection in assembly processes, Sensors 22 (2022) 2839. [24] J. Zhao, F. Chevalier, E. Pietriga, R. Balakrishnan, Exploratory analysis of time-series with chronolenses, IEEE Transactions on Visualization and Computer Graphics 17 (2011) 2422–2431. [25] S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, M. Streit, LineUp: Visual analysis of multi-attribute rankings, IEEE Transactions on Visualization and Computer Graphics 19 (2013) 2277–2286. [26] B. Moreira Cunha, S. Diniz Junqueira Barbosa, Evaluating the efectiveness of visual representations of shap values toward explainable artificial intelligence, in: Proc. Brazilian Symp. on Human Factors in Computing Systems, 2024, pp. 1–11. [27] T. Shimizu, F. Nagata, K. Arima, K. Miki, H. Kato, A. Otsuka, K. Watanabe, M. K. Habib, Enhancing defective region visualization in industrial products using grad-cam and random masking data augmentation, Artificial Life and Robotics 29 (2024) 62–69. [28] M. Zhuxi, Y. Li, M. Huang, Q. Huang, J. Cheng, S. Tang, A lightweight detector based on attention mechanism for aluminum strip surface defect detection, Computers in Industry 136 (2022) 103585. [29] S. Pajer, M. Streit, T. Torsney-Weir, F. Spechtenhauser, T. Möller, H. Piringer, Weightlifter: Visual weight space exploration for multi-criteria decision making, IEEE Transactions on Visualization and Computer Graphics 23 (2016) 611–620. [30] Q. Liu, Y. Ren, Z. Zhu, D. Li, X. Ma, Q. Li, Rankaxis: Towards a systematic combination of projection and ranking in multi-attribute data exploration, IEEE Transactions on Visualization and Computer Graphics 29 (2022) 701–711. [31] J. C. Roberts, State of the art: Coordinated & multiple views in exploratory visualization, in: Int.
Conf. on Coordinated and Multiple Views in Exploratory Visualization, 2007, pp. 61–71. [32] P. Gofin, W. Willett, J.-D. Fekete, P. Isenberg, Exploring the placement and design of word-scale visualizations, IEEE Transactions on Visualization and Computer Graphics 20 (2014) 2291–2300. [33] M. Brehmer, B. Lee, B. Bach, N. H. Riche, T. Munzner, Timelines revisited: A design space and considerations for expressive storytelling, IEEE Transactions on Visualization and Computer Graphics 23 (2016) 2151–2164. [34] Y. Shi, B. Chen, Y. Chen, Z. Jin, K. Xu, X. Jiao, T. Gao, N. Cao, Supporting guided exploratory visual analysis on time series data with reinforcement learning, IEEE Transactions on Visualization and Computer Graphics 30 (2023) 1172–1182. [35] R. Dabh, N. Kantesaria, P. Vaghasia, J. Hirpara, R. Bhoraniya, Casting product image data for quality inspection, 2020. URL: https://www.kaggle.com/ravirajsinh45/ real-life-industrial-dataset-of-casting-product/notebooks, last accessed: 2025-03-02. [36] X. Lv, F. Duan, J.-j. Jiang, X. Fu, L. Gan, Deep metallic surface defect detection: The new benchmark and detection network, Sensors 20 (2020) 1562. [37] M. Wieler, T. Hahn, Weakly supervised learning for industrial optical inspection, in: Symp. of the
German Association for Pattern Recognition (DAGM), volume 6, 2007, p. 11. [38] M. A. Meza Martínez, A. Mädche, Designing interactive explainable ai systems for lay users, in:
Int. Conf. on Information Systems, 2023, pp. 1–17. [39] S. Lysdahlgaard, Utilizing heat maps as explainable artificial intelligence for detecting abnormalities on wrist and elbow radiographs, Radiography 29 (2023) 1132–1138. [40] J. Zhang, Y. Wang, P. Molino, L. Li, D. S. Ebert, Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models, IEEE Transactions on Visualization and Computer Graphics 25 (2018) 364–373. [41] A. Chatzimparmpas, R. M. Martins, I. Jusufi, K. Kucher, F. Rossi, A. Kerren, The state of the art in enhancing trust in machine learning models with the use of visualizations, in: Computer Graphics Forum, volume 39, 2020, pp. 713–756. [42] U. Bhatt, J. Antorán, Y. Zhang, Q. V. Liao, P. Sattigeri, R. Fogliato, G. Melançon, R. Krishnan, J. Stanley, O. Tickoo, et al., Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty, in: Proc. AAAI/ACM Conf. on AI, Ethics, and Society, 2021, pp. 401–413. [43] T. Munzner, A nested model for visualization design and validation, IEEE Transactions on
Visualization and Computer Graphics 15 (2009) 921–928. [44] A. Arslan, C. Cooper, Z. Khan, I. Golgeci, I. Ali, Artificial intelligence and human workers interaction at team level: a conceptual assessment of the challenges and potential hrm strategies, Int. Journal of Manpower 43 (2022) 75–88. [45] Y. Ali, S. W. Shah, A. Arif, M. Tlija, M. R. Siddiqi, Intelligent framework design for quality control in industry 4.0, Applied Sciences 14 (2024) 7726. [46] T. Hoch, J. Martinez-Gil, M. Pichler, A. Silvina, B. Heinzl, B. Moser, D. Eleftheriou, H. D. EstradaLugo, M. C. Leva, Multi-stakeholder perspective on human-ai collaboration in industry 5.0, in: Artificial Intelligence in Manufacturing, Springer, 2023, pp. 407–421.