Knowledge graph construction and maintenance process: Design challenges for industrial maintenance support

Abstract

Knowledge graphs (KGs) structure knowledge to develop intelligent systems in several application domains. Industrial maintenance support requires knowledge and expertise on a variety of aspects of the factory, machinery, and components. However, the actual creation and maintenance process of KGs has remained unelaborated. We review the KG literature to integrate previous models into one process model also incorporating knowledge engineering principles within. The literature review and a subsequent case study together represent the problem and objectives definition phases of a design science project. The contributions include the integrated process model for KG creation and maintenance and the initially observed design challenges in the KG process operationalisation in a context of supporting industrial maintenance. knowledge base, knowledge graph, knowledge engineering, knowledge graph construction process Graph (KG) marked a turn in the usage of KGs in 2012, KGs have been implemented to organise KBs in a myriad of domains [2]. Conceptually, a KB thus consists of stored knowledge in a relevant domain of interest. A KG is an approach to model and implement a KB, in which a schema of classes of information entities and their relations can be presented as a graph, allowing also arbitrary interrelations among the entities [3].

1. Introduction

Knowledge bases (KBs) form a central part of knowledge engineering, which was established in the 1950’s as its own field in research on artificial intelligence (AI) [ CEUR diagnostics to cultivate their products and services [ 5 ]. KGs are often in the core of such systems. However, the actual creation (and maintenance) process of the KGs as such remains imprecise.

The study reviews KG research and creates an integrated process model of KG creation and maintenance from the literature. The model includes five main stages and 14 tasks. The practical motivation for our research resides in an industry-academia collaboration project Oxilate 1 that develops an intelligent assistant to provide knowledge support for maintenance personnel of a complex cyber-physical system. Creation and maintenance of a KG was chosen as the overall approach to organising and maintaining knowledge to be utilised by the assistant. A case study utilises the literature-based model to describe the first design challenges that will guide the future research and development eforts of the intelligent assistant at hand, while informing also other similar eforts. As such, the study comprises the problem and objectives definition phases of a wider design science research (DSR) [ 6 ] project.

The paper is organised as follows. The next section describes the research methodology. Section 3 summarises the previous literature, resulting in an integrated model for knowledge graph construction process (KGCP). Section 4 describes the case illustrating an operationalisation of the KGCP and the emerging design challenges for a solution. Section 5 discusses the contributions. Finally, section 6 concludes with our future research objectives.

2. Methodology

KBs are needed to store expert knowledge together with relevant data and documentation available and reuse it for digital assistants. Although such cases on industrial maintenance have been reported in the literature and they use KG for organising knowledge, none of them explain explicitly the process how KG is designed, created, and utilised. Therefore, we need to investigate how the process is described in other fields. This paper covers the problem and objectives definition steps of the DSR approach [ 6 ], utilising both literature and a problem with unfolding design challenges emerging in the project-related case study, simultaneously.

The literature review focused on process elements of KG creation. Because KB has longer history as a related concept, we used both, KB and KG, as search terms. The literature was drawn from main search engines in 10/2021 (Table 1). Quotation was used for whole terms to eliminate irrelevant papers from construction field. Google search was conducted in two parts due to restrictions in the number of terms per search. Altogether more than 400 papers were found and after duplicate removal we had 331 studies to screen. The main question throughout the literature selection process was: ”Does the paper describe a process for creating a KB/KG?”, regardless of the application domain. After screening and full text reviews we had 43 studies dating from 1997 to 2021. For the summary, we further excluded 6 papers, because they referenced the process from one of the other studies already in the review.

To provide design research input from our industrial case, one of the coauthors is constantly developing the actual KG for the company, providing professional and practical interaction to crystallise the design objectives. The academia-industry interaction took place through ongoing 1Oxilate= Operational eXcellence by Integrating Learned information into AcTionable Expertise https://itea4.org/ project/oxilate.html biweekly workshops in May-November 2021, complemented with 3 other expert interviews and e-mail correspondence in the beginning of the project for problem definition.

3. Literature review

This section summarises the literature on KG construction processes in and beyond the industrial maintenance field. Among the early works on knowledge engineering (KE), Turban et al. [ 1 ] describe KE as an iterative process with the stages of knowledge acquisition, knowledge representation, knowledge base, knowledge validation, inferencing, and explanation and justification. (Figure 1). The knowledge acquisition includes a task of data acquisition from diverse sources, e.g. documents and experts, so it can be adapted to modern data intensive environment. Knowledge representation describes tasks for modelling knowledge for computer processing, and knowledge validation involves tasks of confirming data integrity and solving factual disputes among data. Explanations given by intelligent systems were important for the expert systems, therefore the explanation task here indicate the reasons given by the system for suggested actions.

For many years, the descriptions of the KB followed similar lines of reasoning with the KE process. In 2017, Pan et al. [ 7 ] suggested an iterative process for knowledge management and KG creation for businesses. Their naming convention of the process stages difers from the previous, being almost unused elsewhere. Nevertheless, they describe a high-level process of KG creation with three main stages: construction, storage, and consumption, and elaborate the KG life cycle. The construction stage includes ontology development, data lifting, data annotation and quality assurance. The consumption includes understanding and exploitation as sub tasks. A major diference is that they include KG consumption as one of the stages, identifying that the way we plan to utilise the KG, in turn, afects the other stages.

Also in 2017, another three-stage process for KG creation is introduced with the stages of data and information acquisition, knowledge fusion and knowledge processing [ 3 ]. The knowledge fusion concept is adopted from data fusion to indicate identification of true triples instead of accurate data. Yet another KGCP was proposed in 2020 [ 8 ]. The process includes the stages of knowledge creation, knowledge hosting, knowledge curation and knowledge deployment. Here, the curation stage includes knowledge enrichment as a well-detailed subtask. However, the enrichment methods are essentially the same as in the earlier inferencing methods.

After gaining an overview of the general-level KG process models, we reviewed the rest of the process descriptions (Table 2). Because the naming conventions and structuring of the processes vary, we compiled these tasks under the most used stage names to present them in a comparable format. Consequently, we recognised 14 tasks under 5 overarching stages: knowledge acquisition, knowledge fusion, knowledge processing, knowledge storage and knowledge utilisation. These stages can be found in many of the sources, especially in the traditional KE principles [ 1 ] with updates from modern data science [ 9 ]. Many of the tasks consist of sub tasks, as reported, e.g., in [ 10, 11 ].

Knowledge acquisition includes all the tasks that are required to discover knowledge or structure data to acquire knowledge. As shown in the table, we make a distinction between data acquisition and knowledge extraction. Here, data acquisition is simply the process of acquiring raw data from the sources relevant to the field and task at hand and preprocessing it to enable further use. It is important to make this distinction, because the raw or preprocessed data are not yet in the form to ofer knowledge for the users. Knowledge extraction is the task where the patterns in the data are discovered, most often through entity, relation, and attribute extraction [ 12 ].

Knowledge fusion and knowledge processing are not yet set in the literature, instead they may be described interchangeably between diferent studies. In this paper, knowledge fusion refers to the process of knowledge validation and integration with sub tasks such as entity co-reference or entity disambiguation. The knowledge processing on the other hand, refers to the model creation, quality evaluation in the sense of fact-checking, and knowledge inferencing [ 13 ].

Knowledge storage includes all the tasks that are directly connected to the stored knowledge, i.e. knowledge representation as it focuses on modelling the knowledge for computer processing, storage technologies, retrieval of the data for use and visualising it [ 1 ]. Knowledge utilisation includes the tasks that are conducted on the built KG and the ways knowledge is exploited for a use case. Source [ 14 ] 1997 [ 15 ] 2001 [ 16 ] 2005 [ 17 ] 2009 [ 18 ] 2009 [ 19 ] 2012 [ 20 ] 2012 [ 21 ] 2014 [ 22 ] 2014 [ 23 ] 2015 [ 3 ] 2017 [ 7 ] 2017 [24] 2017 [25] 2018 [ 12 ] 2018 [26] 2019 [27] 2019 [28] 2019 [29] 2019 [30] 2019 [31] 2019 [ 13 ] 2019 [32] 2019 [33] 2020 [34] 2020 [35] 2020 [ 8 ] 2020 [36] 2020 [37] 2020 [ 38 ] 2020 [ 39 ] 2020 [ 40 ] 2020 [ 41 ] 2021 [ 42 ] 2021 [ 43 ] 2021 [ 44 ] 2021 [ 45 ] 2021 Number of refs x x x x x 5 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 29

KE x x x x x x x x x x x x x x x x x x x x x x x x x x 26 x x x x x x x x 8 x x x x x x x x x x x x x x x x x

Altogether 14 tasks need to be implemented for KG creation and usage. Most of the articles do not include all the 14 while a few only discussed two or three of them. Depending on the chosen approach, top-down or bottom-up, the KG creation process starts either by ontology creation or data acquisition. The top-down approach refers to the KG creation beginning from the conceptual model or schema. The bottom-up approach, on the contrary, begins from acquiring the data and analysing it to form a model [ 7 ]. Most often the final stage of the construction process is the knowledge storage, although some models include the tasks for KB usage and maintenance. Only one study [36] identified all the tasks, except ontology selection, as it followed the bottom-up approach. However, the suggested process was a waterfall model, where only reasoning can change the KG once finished. Furthermore, each task was described only briefly without the focus on the entire process.

We separated ontology selection from ontology creation/update because these tasks difer. Most of the articles use the bottom-up approach, creating an ontology from data. Fewer articles discuss the possibility of selecting an ontology of-the-rack. Although it is possible to select a predefined ontology, it needs often to be adapted to the use case [ 26, 42 ]. Consequently, the ontologies may be updated in the knowledge processing stage.

The suggested processes from 1997 to 2021 were, to an extent, similar: knowledge is acquired (35), fused (20), processed further (27) and stored (33) in a KB (the numbers in this paragraph correspond to the number of papers describing the stage in question). Knowledge utilisation, however, is ignored in most papers as a stage in KGCP, even though many describe a process for an application-specific KG. The least discussed tasks were ontology selection (5), cleaning (8), updates (9) and application (4) (Table 2). In terms of completing the tasks, there are already helpful reviews about many of the sub tasks [ 10, 11 ], hence we do not detail them here but instead focus on the higher-level view of the process. Furthermore, this review supports the idea that we can create one process representation that incorporates the previous KGCPs, which is also shown in figure 2.

The sources treat the top-down and the bottom-up approaches for knowledge graph creation as diferent processes, e.g. [ 12 ], but it would be clearer for a developer if they could be described within one process model. Our process model starts from knowledge acquisition stage regardless of the approach chosen. For the top-down approach, either an ontology is chosen of-the-rack or only data structures needed for ontology building are extracted, after which knowledge fusion, storage and processing stages are conducted. Next, knowledge acquisition is done again, this time for data instances to create the KG following the structure created. For the bottomup approach, the process starts in knowledge acquisition by data acquisition and knowledge extraction and continues from there to knowledge fusion and further.

Knowledge and the related structures change over time. The equipment in industry change, the applications change, the processes develop, and people come and go. To ofer timely knowledge for work processes in such an environment, the KG construction and maintenance must be treated as a continuous process. [27] mentions that the current, ever improving environment requires spiral evolution of KGs, which is illustrated in our iterative process.

4. Design Challenges in a Case of Industrial Maintenance

This case involves a company that ofers maintenance services to the machines they manufacture. The machines are sold globally, and the maintained machines are often spread in various locations, while the maintenance experts (possessing most comprehensive knowledge) work near the headquarters in Finland. Industrial maintenance (IM) relies heavily on human efort aided by technology. When a local maintenance engineer meets trouble, they contact a maintenance expert. The experts are often overloaded with requests, which increases maintenance time. To solve this, the company develops a digital assistant for the onsite maintenance engineers e.g. The assistant utilisation should ease the workload of the maintenance experts and allow more time for challenging cases. The assistant will also reduce the need for travel by the experts, thus reducing environmental impact and costs.

The digital assistant allows maintenance engineers to identify causes for the problem and ofers relevant documentation. We have identified the need to better understand the expert knowledge to support the engineers in their knowledge-based tasks as one of the design objectives in earlier publication [ 46 ]. The design challenges related to KG construction are identified considering the KG process stages above.

Firstly, we need to decide whether to build ontology bottom-up or select an ontology topdown. As a reference ontology is a domain specific vocabulary of common entities, it can be used to create application ontologies and then KGs [ 47 ]. We evaluated two IM ontologies for the use case. [ 48 ] develops an ontology for industry 4.0 based on extensive literature review. However, this ontology presents only high-level concepts for IM and was not applicable in this case. [ 47 ] develops a reference ontology designed directly for IM. This included all the required concepts, being openly available, it was thus chosen.

While the ontology gave the structure and data needs, data was gathered from various sources, and entities, attributes and relations were matched from documents to the ontology structure. A same structure for the same type of document was essential to ease the automatic detection of nodes and edges. However, many of the documents are completed by humans, leading to various contents and styles in individual documents. The iterative nature of KGCP became visible when new data is added to the system and knowledge is inferred based on available data. There are several types of data that are variably changeable, e.g. metadata is more static, whereas runtime data has higher velocity. There are also several types of documents with diferent attributes on update times and other contents to draw suggestions upon. Furthermore, the content format and structure may vary as well between data sources.

In addition to having efect on knowledge acquisition, the diverse sources afect the integration task, i.e. duplicate information is identified and removed. For instance, when data is integrated and suggestions given, the first mock-ups gave links to documents beyond the user’s access rights. Because the company trades internationally, workers often write the documents in local languages, which complicates finding similar cases and connecting same entities. Computer translation tackled a part of this problem, and it works well enough between the main languages (Finnish and English). Further development is needed for other languages.

As content structures vary and relevant information for solving a problem often involves multiple information sources, problem solution often requires human thinking. For example, understanding and solving a single problem can require analysis of tickets, updates, warnings, manuals etc. For a digital assistant to work, text analysis with natural language processing (NLP) techniques could be used to understand similar or most common cases. The sameness of an issue has multiple levels, i.e. it is possible to compare only the recurring issue of the same equipment instance, or the same type of equipment in the same factory, or the same type of equipment across all factories. This poses challenges for inferencing, as the algorithms must be carefully designed to address the needs.

Machine learning methods for knowledge processing must be studied further. One learning method might involve the communications that workers have with the digital assistant. The reliability of the information must also be evaluated, e.g. the information can contain measuring mistakes or incorrect information.

While developing the KG, GraphQL and Neo4j were found useful. As a proof-of-concept, a web server application is used to test and trial the system. The initial application requirements will be reviewed and revised with a group of users. Further research and development are needed to determine a verified solution for the application.

While the KG is used, users can give feedback on the usability of the suggestions, which has efect on the stored knowledge and how it is represented later. It could also be possible to gather usage data and rank knowledge usefulness based on it. For instance, how quickly an issue is solved or if it is solved without human expert involvement would be indications for how well the assistant performs. These aspects are the key to enable the digital assistant to learn, without which we cannot talk of an intelligent assistant.

To summarise, the early steps of our design research collaboration indicate the following design challenges in practice considering our process model. The challenges draw attention toward evolvability and adaptability of KGs for best utilisation of knowledge. • Iterative nature of the process sets challenges and requires compromises across all of the stages. • Knowledge acquisition – Various content sources and formats set challenges for the data acquisition methods, as diferent formats must be declared separately for automatic acquisition and knowledge extraction. • Knowledge fusion – Integration challenges include duplicate removal and resolving contradicting information, especially after AI-based learning solutions will be introduced to the system. • Knowledge storage • Knowledge processing • Knowledge utilisation – Usability of knowledge is an important attribute when considering visualisation techniques and varies as per use case. – NLP techniques can be used for inferencing new knowledge, the state-of-the-art is vast and must be investigated. – Relevance of knowledge should be maintained when suggestions are given to users. – Feedback methods should also be developed so that any dificulties in utilising knowledge can be resolved in further iteration as an integral element for continuously evolving KG that adapts to the changing environment.

5. Discussion

The contribution suggested in our research is twofold. Firstly, our integrative summary of the literature on KGCP presents a general, iterative model of 5 stages and 14 tasks to guide KG initiatives. Secondly, our initial steps of identifying the problem and design challenges for KG operationalisation in our case of IM support demonstrates the technological and development challenges in this context.

Table 2 illustrates how the previous literature, while discussing individual elements of KGCP, has focused on details of varying stages and tasks. While the traditional concerns of knowledge acquisition and knowledge storage were the most recognised elements throughout the literature, several of the other necessary tasks from the viewpoint of building KG solutions were ignored in the previous works on IM and beyond. Hence, our work contributes by outlining an integrated process model. Our model recognises the roots of the KE tradition that identified from early on the iterative nature of KE work [ 1 ]. In comparison to the hitherto most complete model that recognised all tasks (a case of failure prediction of elevators) [36] in a waterfall model, our study identifies the iterative nature of KG creation and maintenance, and the case introduces design challenges from another industrial domain.

The practical implications include design challenges identified in a case of IM. The case already confirms the need for continuous change to update the KG, for the evolving visions for utilising intelligent systems in the field. That is, KGCP should be designed from the beginning as an iterative, continuous process, acknowledging the need to learn from the continuous interactions of the workers utilising knowledge. Contradictory to the hitherto reported experiences from the IM, e.g. [ 4, 49 ], which have so far followed the bottom-up approach to ontology creation, our case represents and will continue with the top-down ontology selection when proceeding to further prototyping and evaluation stages of the ongoing design research process.

As our research so far represents the first steps of design research, problem and challenges definition, in industry-academia collaboration, the natural step for further research is to follow the implementation in the style of action design research [ 50 ] until a functioning prototype can be tried out with the experts of the target organisation. This work takes place under an industry led European ITEA project Oxilate, targeted to end in February 2023. We expect that our contemporary design challenges will develop towards more mature design principles, while there is an opportunity for new design objectives and principles to emerge on the way. The literature-oriented KG process model will be also demonstrated and initially verified in relation to other ongoing industrial use cases and domains for KG creation and utilisation within the same research project.

6. Conclusion

The study was to determine a KG construction process, and we have summarised and integrated an iterative model of KGCP. The aim was to enhance the development of digital assistants in industrial maintenance work. Therefore, we have outlined the related design challenges observed during initial stages of a co-operative, industry-academia design science project. The model identifies 5 main stages and 14 tasks necessary for obtaining a functioning and continually maintainable KG for the purpose. The model can be used to guide design and development eforts of similar systems beyond the context at hand, while the specified design challenges provide practical implications in the field of industrial maintenance. The future work involves continuing with action-oriented collaborative design research with the reported case, resulting in unfolding design objectives and principles with reported experiences of the solutions. Information Technology, and Biomedical Engineering (ICICI-BME), 2015, p. 196–201. doi:1 0 . 1 1 0 9 / I C I C I - B M E . 2 0 1 5 . 7 4 0 1 3 6 2 . [24] M. Niknam, S. Karshenas, A shared ontology approach to semantic representation of bim data, Automation in Construction 80 (2017) 22–36. doi:1 0 . 1 0 1 6 / j . a u t c o n . 2 0 1 7 . 0 3 . 0 1 3 . [25] J. L. Martinez-Rodriguez, I. Lopez-Arevalo, A. B. Rios-Alvarado, Openie-based approach for knowledge graph construction from text, Expert Systems with Applications 113 (2018) 339–355. [26] M. A. Islam, Leveraging knowledge graph for UK environmental legislation related to building construction and maintenance, Ph.D. thesis, Lappeenranta University of technology, 2019. [27] Q. Li, B. Wen, D. Hao, The construction of knowledge graph based on securities information, in: 2019 IEEE International Conference on Computer Science and Educational Informatization (CSEI), IEEE, 2019, p. 181–186. [28] R. Miao, X. Zhang, H. Yan, C. Chen, A dynamic financial knowledge graph based on reinforcement learning and transfer learning, in: 2019 IEEE International Conference on Big Data (Big Data), 2019, p. 5370–5378. doi:1 0 . 1 1 0 9 / B i g D a t a 4 7 0 9 0 . 2 0 1 9 . 9 0 0 5 6 9 1 . [29] L. Qin, Z. Hao, L. Zhao, Food safety knowledge graph and question answering system, in: Proceedings of the 2019 7th International Conference on Information Technology: IoT and Smart City, 2019, p. 559–564. [30] N. Wang, E. Haihong, M. Song, Y. Wang, Construction method of domain knowledge graph based on big data-driven, in: 2019 5th International Conference on Information Management (ICIM), 2019, p. 165–172. doi:1 0 . 1 1 0 9 / I N F O M A N . 2 0 1 9 . 8 7 1 4 6 6 4 . [31] T. Wang, L. Duan, C. He, G. Deng, R. Qin, Y. Zhang, Atom: Construction of anti-tumor biomaterial knowledge graph by biomedicine literature, in: 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE, 2019, p. 1256–1258. [32] W. Zeng, H. Liu, Y. Feng, Construction of scenic spot knowledge graph based on ontology, in: 2019 18th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES), 2019, p. 120–123. doi:1 0 . 1 1 0 9 / D C A B E S 4 8 4 1 1 . 2 0 1 9 . 0 0 0 3 7 . [33] E. N. Alemu, J. Huang, Healthaid: Extracting domain targeted high precision procedural knowledge from on-line communities, Information Processing & Management 57 (2020) 102299. [34] Z. Chen, Y. Zhao, The technology of military knowledge graph construction based on multiple open data sources, in: 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE), 2020, p. 1993–1997. doi:1 0 . 1 1 0 9 / I C M C C E 5 1 7 6 7 . 2 0 2 0 . 0 0 4 3 6 . [35] P. Constantopoulos, V. Pertsas, From publications to knowledge graphs, in: G. Flouris, D. Laurent, D. Plexousakis, N. Spyratos, Y. Tanaka (Eds.), Information Search, Integration, and Personalization, Communications in Computer and Information Science, Springer International Publishing, 2020, p. 18–33. doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 0 3 0 - 4 4 9 0 0 - 1 _ 2 . [36] J. Hou, R. Qiu, J. Xue, C. Wang, X.-Q. Jiang, Failure prediction of elevator running system based on knowledge graph, in: Proceedings of the 3rd International Conference on Data Science and Information Technology, 2020, p. 53–58. [37] F. Li, W. Xie, X. Wang, Z. Fan, Research on optimization of knowledge graph construction

References

[1] E. Turban , J. E. Aronson , T.-P. Liang , R. V. McCarthy , Decision Support Systems and Intelligent Systems , 7th edition ed., Prentice Hall , 2004 .

[2] J. Chicaiza , P. Valdiviezo-Diaz , A comprehensive survey of knowledge graph-based recommender systems: Technologies, development, and contributions , Information 12 ( 2021 ) 232 . doi:1 0 . 3 3 9 0 / i n f o 1 2 0 6 0 2 3 2 .

[3] A. Ktob , Z. Li , The arabic knowledge graph: Opportunities and challenges , in: 2017 IEEE 11th International Conference on Semantic Computing (ICSC) , 2017 , p. 48 - 52 . doi: 1 0 . 1 1 0 9 / I C S C . 2 0 1 7 . 2 2 .

[4] G. Fenza , M. Gallo , V. Loia , D. Marino , F. Orciuoli , A cognitive approach based on the actionable knowledge graph for supporting maintenance operations , in: 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS) , 2020 , p. 1 - 7 . doi:1 0 . 1 1 0 9 / E A I S 4 8 0 2 8 . 2 0 2 0 . 9 1 2 2 7 5 9 .

[5] H. Hossayni , I. Khan, M. Aazam , A. Taleghani-Isfahani , N. Crespi , Semkore: Improving machine maintenance in industrial iot with semantic knowledge graphs , Applied Sciences 10 ( 2020 ) 6325 . doi:1 0 . 3 3 9 0 / a p p 1 0 1 8 6 3 2 5 .

[6] K. Pefers , T. Tuunanen , M. A. Rothenberger , S. Chatterjee , A design science research methodology for information systems research , Journal of Management Information Systems 24 ( 2007 ) 45 - 77 . doi:1 0 . 2 7 5 3 / M I S 0 7 4 2 - 1 2 2 2 2 4 0 3 0 2 .

[7] J. Z. Pan , G. Vetere, J. M. Gomez-Perez , H. Wu , Exploiting Linked Data and Knowledge Graphs in Large Organisations , Springer, 2017 . Google-Books-ID: qZoNDgAAQBAJ.

[8] D. Fensel , U. Şimşek , K. Angele , E. Huaman , E. Kärle , O. Panasiuk , I. Toma , J. Umbrich , A. Wahler , Knowledge Graphs: Methodology, Tools and Selected Use Cases, Springer International Publishing, 2020 .

[9] X. Zhao , Y. Jia , A. Li , R. Jiang , Y. Song , Multi-source knowledge fusion: a survey , World Wide Web 23 ( 2020 ) 2567 - 2592 . doi:1 0 . 1 0 0 7 / s 1 1 2 8 0 - 0 2 0 - 0 0 8 1 1 - 0 .

[10] J. Yan , C. Wang , W. Cheng, M. Gao , A. Zhou , A retrospective of knowledge graphs , FRONTIERS OF COMPUTER SCIENCE 12 ( 2018 ) 55 - 74 . doi:1 0 . 1 0 0 7 / s 1 1 7 0 4 - 0 1 6 - 5 2 2 8 - 9 .

[11] A. Hogan , E. Blomqvist, M. Cochez , C. D'Amato , G. Melo , C. Gutierrez , S. Kirrane , J. Gayo , R. Navigli , S. Neumaier , A.-C. Ngomo , A. Polleres , S. Rashid , A. Rula , L. Schmelzeisen , J. Sequeda , S. Staab , A. Zimmermann , Knowledge graphs , ACM Computing Surveys 54 ( 2021 ). doi:1 0 . 1 1 4 5 / 3 4 4 7 7 7 2 .

[12] Z. Zhao , S. - K. Han, I.-M. So , Architecture of knowledge graph construction techniques , International Journal of Pure and Applied Mathematics 118 ( 2018 ) 1869 - 1883 .

[13] H. Yun , Y. He , L. Lin , Z. Pan , X. Zhang, Construction research and application of poverty alleviation knowledge graph , in: International Conference on Web Information Systems and Applications , Springer, 2019 , p. 430 - 442 .

[14] G. P. Ias , V. Moustakis , G. Charissis, Interactive knowledge based construction and maintenance , Applied Artificial Intelligence 11 ( 1997 ) 697 - 717 .

[15] M. Matthee , H. Viktor , Data mining for organizational knowledge management: Aiding decision, sense and policy making , in: European Conference on Knowledge Management (ECKM) , 2001 , p. 353 - 366 .

[16] P. Parpola , Inference in the sookat object-oriented knowledge acquisition tool, Knowledge and information systems 8 ( 2005 ) 310 - 329 .

[17] Y. Chi , C. Chen , Project teaming: Knowledge-intensive design for composing team members , EXPERT SYSTEMS WITH APPLICATIONS 36 ( 2009 ) 9479 - 9487 . doi:1 0 . 1 0 1 6 / j . e s w a . 2 0 0 8 . 1 2 . 0 1 5 .

[18] H. L. Zhang , C. Van der Velden , X. Yu , T. Jones , I. Fieldhouse, C. Bil , A knowledge-based system approach for feature recognition in aerospace components design processes , in: Proc. of 2009 AutoCRC Conference , 2009 .

[19] H. Ghadimi , D. Bird , A knowledge base for the world's energy rich regions , Regional Research Institute Working Papers 8 ( 2012 ). URL: https://researchrepository.wvu.edu/rri_ pubs/8.

[20] J. H. Beevi , N. Deivasigamani , A new approach to the design of knowledge base using xcls clustering , in: International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME- 2012 ), 2012 , p. 14 - 19 . doi:1 0 . 1 1 0 9 / I C P R I M E . 2 0 1 2 . 6 2 0 8 2 8 0 .

[21] P. Bellini , P. Nesi , N. Rauch , Knowledge base construction process for smart-city services , in: 2014 19th International Conference on Engineering of Complex Computer Systems , 2014 , p. 186 - 189 . doi:1 0 . 1 1 0 9 / I C E C C S . 2 0 1 4 . 3 3 .

[22] S. Shin , J.-H. Um , S.-P. Choi , H. Jung , S. Xu , L. Zhu , K-Base: Platform to Build the Knowledge Base for an Intelligent Service , Springer, 2014 , p. 273 - 277 .

[23] J. Lam , Y. A. Noor , E. Supriyanto, Ontology driven knowledge base for high risk pregnancy management , in: 2015 4th International Conference on Instrumentation, Communications , lfow chart, in: 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC) , volume 9 , 2020 , p. 1386 - 1390 . doi:1 0 . 1 1 0 9 / I T A I C 4 9 8 6 2 . 2 0 2 0 . 9 3 3 8 9 0 0 .

[38] Y. Li , R. Li , Research on construction method of knowledge graph-based on mobile phone quality detection , in: 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC) , IEEE, 2020 , p. 695 - 699 .

[39] J. Wang , Knowledge graph analysis of internal control field in colleges , Tehnički vjesnik 27 ( 2020 ) 67 - 72 . doi:1 0 . 1 7 5 5 9 / T V - 2 0 1 9 1 0 0 4 0 9 2 6 5 9 .

[40] J. Zhao , X. Yang , Q. Qiao , L. Chen, Personalized learning design of ideology and politics of distance education courses based on big data , in: 2020 IEEE International Conference on Progress in Informatics and Computing (PIC) , 2020 , p. 135 - 139 . doi: 1 0 . 1 1 0 9 / P I C 5 0 2 7 7 .

[41] A. Dsouza , N. Tempelmeier , R. Yu , S. Gottschalk , E. Demidova, Worldkg: A world-scale geographic knowledge graph , arXiv:2109 .10036 [cs] ( 2021 ). doi:1 0 . 1 1 4 5 / 3 4 5 9 6 3 7 . 3 4 8 2 0 2 3 .

[42] E. Filtz , S. Kirrane , A. Polleres , The linked legal data landscape: linking legal data across diferent countries , Artificial Intelligence and Law ( 2021 ) 1 - 55 . doi: 1 0 . 1 0 0 7 / s 1 0 5 0 6 - 0 2 1 - 0 9 2 8 2 - 8 .

[43] Y. Jiang , C. Chen , X. Liu , Assembly process knowledge graph for digital twin , in: 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE) , IEEE, 2021 , p. 758 - 763 .

[44] Y. Ma , D. Hong , F. Dan , X. Yang , X. Li , Research on the construction method of knowledge graph for power grid education resources , in: 2021 IEEE 3rd International Conference on Computer Science and Educational Informatization (CSEI) , IEEE, 2021 , p. 99 - 103 .

[45] S. Tiwari , D. Gaurav , A. Srivastava , C. Rai , K. Abhishek , A preliminary study of knowledge graphs and their construction , Lecture Notes in Networks and Systems 164 ( 2021 ) 11 - 20 . doi:1 0 . 1 0 0 7 / 9 7 8 - 9 8 1 - 1 5 - 9 7 7 4 - 9 _ 2 .

[46] H. Bomström , E. Annanperä, M. Kelanti , Y. Xu , S.-M. Mäkelä , M. Immonen , P. Siirtola , A. Teern , K. Liukkunen , T. Päivärinta, Digital twins about humans-design objectives from three projects , Journal of Computing and Information Science in Engineering 22 ( 2022 ) 050907 .

[47] M. H. Karray , F. Ameri , M. Hodkiewicz , T. Louge, Romain: Towards a bfo compliant reference ontology for industrial maintenance , Applied Ontology 14 ( 2019 ) 155 - 177 . doi:1 0 . 3 2 3 3 / A O - 1 9 0 2 0 8 .

[48] M. Yahya , J. G. Breslin , M. I. Ali , Semantic web and knowledge graphs for industry 4 .0, Applied Sciences 11 ( 2021 ). URL: https://www.mdpi.com/2076-3417/11/11/5110. doi: 1 0 .

3 3 9 0 / a p p 1 1 1 1 5 1 1 0 .

[49] B. Vogel-Heuser , F. Ocker , I. Weiß , R. Mieth , F. Mann , Potential for combining semantics and data analysis in the context of digital twins , Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 379 ( 2021 ) 20200368 . doi:1 0 . 1 0 9 8 / r s t a . 2 0 2 0 . 0 3 6 8 .

[50] M. K. Sein , O. Henfridsson , S. Purao , M. Rossi , R. Lindgren , Action design research, MIS Quarterly 35 ( 2011 ) 37 - 56 . doi:1 0 . 2 3 0 7 / 2 3 0 4 3 4 8 8 .