Model, Information Technology Management, Design Science Research
1. Introduction
Artificial Intelligence (AI) is increasingly evolving from a pure research domain to a driver for
innovative companies. Almost all industries are already intensively engaged in the application of AI
solutions and this emerging technology is widely seen as a key game changer [39]. Despite its more
than 50-year history, AI research also seems to be at its peak today [
31
]. New publications, AI
architectures and programming frameworks are proposed daily and each field is trying to implement AI
solutions for its own specific industry [39]. AI is increasingly benefiting from the growing data volumes
enabled by other emerging trends such as cloud computing and many experts promise a bright future
for AI [
9, 21
]. Due to the current attention given to this field, governments, private organizations and
individuals are allocating more and more resources to support research opportunities for AI [39].
The field of AI encompasses many technical methods such as Machine Learning (ML) [
6, 27
],
Natural Language Processing [
19
], Computer Vision [
13
], Knowledge Representation [
26
] or Robotics
[36]. While these technical methods and implementations of AI have been intensively researched in
areas such as computer science or cognitive science since the 1950s [
24, 31, 37
], a more business and
strategy-oriented view of managing AI technologies for entire organizations is still relatively new [
4
].
In addition, the practical application of AI in organizations still lags behind the research advances in AI
algorithm development. The key challenge for the future use of AI in businesses is to develop suitable
application scenarios, to convert these into prototypes and to develop operationally usable applications
2022 Copyright for this paper by its authors.
that can then be operated productively and continuously evolved [
7
]. Only when applications are used
productively AI can be applied either to reduce costs through automation or to generate additional
revenue through new business models or better interaction with customers. Therefore, the adoption and
diffusion of AI in organizations appears to be heavily influenced by more impact factors than the pure
technical engineering of AI-based information systems. For example, the proper integration of AI into
business processes requires large, high-quality and balanced data sets [
14
], new explanatory approaches
[
29
] or compliance with ethical principles in the autonomous operation of AI systems [
20
]. However,
no overarching framework exists in the literature that addresses all of these specific AI management
challenges. Therefore, the aim of our research is to identify the dimensions, which determine the
adoption and diffusion of AI innovations and its management in organizations, provide new
methodological approaches to support the application of AI by analyzing the previously identified AI
management dimensions and finally develop an AI management framework. Thereby, the entire
research process follows the following overarching Research Question (RQ):
RQ: How can a management framework be designed that fosters the adoption and diffusion of
Artificial Intelligence in organizations?
To answer this research question, a multi-step, design science-oriented research approach is followed
(cf. section 3). The research procedure consists of four main iterations, each answering a different
subquestion with regard to the overarching research question. Additionally to these four main iterations, a
parallel research stream investigates new approaches and methods on how AI-based information
systems can be managed in each of the individual dimensions of the AI management framework.
2. Managing Artificial Intelligence with Maturity Models
The use of Information Technology (IT) must be systematically managed and coordinated to optimally
support corporate goals and enable IT to create added value for the organization. This systematic
management task is known as IT management [
30
]. However, IT innovations based on different
subareas of AI pose new challenges for IT management and in some cases require new approaches [
5, 7,
8
]. For these new challenges, the term AI management has been introduced [
7, 8
]. AI describes the
ability of artificially constructed systems to act intelligently or mimic cognitive and intelligent behavior
as well as the science and engineering involved [
15
]. Furthermore, AI is understood as a technology
cluster closely linked to the concept of “Narrow AI” introduced by Kurzweil [
22
] and which has an
increasing impact on today’s businesses and industries [39]. AI management subsumes systems, models
and methods for the systematic steering of AI in organizations [
5
]. It recognizes the potential of AI for
business challenges, develops productive solutions, operates them and continuously evolves them [
7
].
AI management is not only focused on the development of new algorithms, software or hardware, but
it ensures that AI can be productively used in organizations and that a real value contribution is
achieved. Managing AI requires a deep understanding of the respective algorithms and the specifics of
AI methods require new processes, new structures and new competences to ensure the professional
handling of AI [
8
]. Thereby, AI management is a component of IT management [
7
].
In IT management, Maturity Models (MMs) are a widely used tool for managing and coordinating
various relevant aspects like organizational competences and technologies [
3
]. They are used as a
strategic component for continuously comparing and defining a path or a roadmap in different
evolutionary states [
2
] and as benchmarks to assess and improve organizational capabilities [34]. MMs
were originally proposed in the 1970s by Nolan and Gibson [
28
] and mostly consist of a sequence of
maturity levels for a class of objects. Since then, MMs have defined an own research area and many
different MMs have been developed for different application domains such as software development,
quality management and data analytics [
25, 33
]. The most prominent example is the Capability Maturity
Model (CMM), which is used to assess the quality of the organization’s software process and to
determine actions for its improvement [
10
]. An initial overview of 237 different MMs in 20 application
domains including an initial classification scheme can be found in the paper of Wendler [38]. As digital
transformation has accelerated in recent years, the number of MMs has also increased significantly and
the use of MMs seems to be a suitable tool for the successful integration of key technologies into the
processes of a digitalized enterprise [
12
]. This includes the use of MMs for AI management. While
initially several generic AI MMs were developed at a high level of abstraction [
1, 17, 23, 32
],
industryspecific AI MMs have been increasingly developed in recent years to ensure better applicability for
companies [
11, 15, 35
]. Therefore, an abstract view of management capabilities analogous to generic
AI MMs can serve as an initial guide, while an industry-specific description of management capabilities
and industry-specific AI MMs address the concrete application of AI management in business practice.
Subsequently, our AI management framework should also include these two buildings blocks (generic
metamodel and industry-specific application model).
3. Research Approach
The goal of this research is to design an AI management framework based on the idea of the AI maturity
of organizations and to provide initial methodological approaches on how to advance in the different
dimensions of the management framework. Therefore, this work follows an overall design science
research approach with several iterations (cf. Figure 1). Following the first phases of the methodological
considerations of Becker et al. [
3
], existing AI MMs will be analyzed with a systematic literature review
in the beginning of the research process. Depending on the results of this analysis, either the dimensions
of an existing AI MM will suffice as a first orientation or a new AI MM is designed and evaluated using
qualitative research methods (e.g. expert interviews and focus group discussion) in an application
domain of choice. The resulting artifact will be an industry-specific AI MM that can be directly applied
by practitioners such as IT managers or consultants. In the next iteration, the dimensions of the
analyzed, mostly industry-specific MMs are abstracted independently of their application domain and
an initial metamodel for the AI management framework is designed. The dimensions of the metamodel
are then evaluated with a survey using a quantitative-qualitative questionnaire with information systems
and AI experts of different companies. In this step, the resulting artifact will be a metamodel that serves
researchers and practitioners as an initial guide to AI management. After a successful evaluation of the
metamodel, a web-based information system for the AI management framework with the underlying
AI MM will be developed. In the first step, it will be conceptualized using an User Interface (UI)
prototype. This initial UI prototype is evaluated again with a cognitive walkthrough methodology. The
resulting artifact will be a conceptual UI model that help scientist and MM developers in the application
of their specific AI MM. Finally, the UI prototype will be developed into a complete web-based
information system. By disseminating the self-assessment part for an initial AI maturity analysis of the
web-based AI management framework via an openly accessible web interface, the framework will be
evaluated and (hopefully) used by many researchers and practitioners. The final artifact is a web-based
AI management and maturity assessment tool that can be used by practitioners such as IT managers or
consultants to manage AI technologies in the target company.
During this central, design-science oriented and multi-method research process, small-scale studies for
each dimension will be conducted to provide scientific and practical examples of how AI can be
managed in different dimensions of the AI management framework. So far, the first iteration with the
development of the Auditing Artificial Intelligence Maturity Model (A-AIMM) [
15
] and the four
smallscale studies [
14, 16, 20, 29
] have already been conducted and published. The other three iterations
(metamodel, UI prototype and web-based tool) are currently in progress and are expected to be
published in the next two years. In the following, the results of the already completed research [
14–16,
20, 29
] as well as some preliminary results of the research in progress are briefly described.
4. Preliminary Ideas and Results
4.1. Artificial Intelligence Management Dimensions
The first steps of this work have already been accomplished. In 2021, one of the first industry-specific
AI MMs was developed following the design science-oriented development procedure of Becker et al.
[
15
]. The systematic literature review at the beginning of the A-AIMM development process revealed
that some scientific attempts already met the requirements for a well-developed MM for AI
management of Becker et al. [
3
] and therefore these models were combined into a new one covering all
relevant aspects found so far. Subsequently, the new and combined AI MM was extended by
interviewing experts in the field of auditing, revealing pressing ethical and regulatory issues related to
the adoption and diffusion of AI in organizations. A detailed description of the dimensions that guide
the further development of the AI management framework can be found in Table 1 whereas a condensed
version of the AI MM with a total of five AI maturity levels is illustrated in Figure 2.
With the A-AIMM, an initial design of different levels and control questions was also developed to
assess the state an organization is in regarding its management of AI. For the implementation of the AI
management framework as a web-based information system, these levels and control questions are
currently being further developed with an additional scoring system.
4.2.
Additional Studies for the Management Dimensions
In addition to the central research approach described in section 3, we have already conducted
smallscale methodological and evaluative studies on each dimension during our research to provide scientific
and practical examples of how AI-based information systems can be managed in the different
dimensions of the AI management framework (cf. Table 2):
One example on how organizations can make progress in the Technologies dimension of AI
management is by combining AI with other promising technological approaches. This was shown in a
study that examined how combining AI with the Progressive Web App technology can realize
economic, environmental as well as social benefits and create additional value [
16
].
Another study as part of our research process provides an idea on how to manage data for ML-based
financial fraud detection, which suffers greatly from dataset imbalance [
14
]. By using Generative
Adversarial Networks to create synthetic data samples for minority classes, datasets can be balanced,
which ultimately enables better training on datasets for fraud detection. This study has demonstrated
the importance of the Data dimension to AI management and provided an example of how practitioners
can handle datasets with similar problems for the use of AI-based information systems.
A third study found that different roles have different requirements for the application of AI-based
information systems in organizations [
29
]. Moreover, it is necessary to give personal explanations to
the different roles involved in the application of AI in order for AI to be accepted and used in an
organization. This study highlights the importance of the People & Competences as well as the Ethics
& Regulations dimension for AI management and provides a role model that can help practitioners to
manage their various team members in the application of AI-based information systems.
Finally, a fourth study revealed that even state of the art AI-based face recognition algorithms suffer
from discrimination against ethnic groups [
20
]. Based on this result, a roadmap for designing
nondiscriminatory ML-Services was derived that will be further developed in future research. This study
has shown that especially AI-based information systems, which are not explicitly programmed but
implicitly developed through a training and test process, need new AI management policies regarding
the Ethics & Regulations dimension.
5. Theoretical and Practical Contributions
As mentioned in section 1 and 2, the management of AI-based information systems is a pressing issue
to enable today’s organizations to adopt and leverage AI on a large scale. In the past, research mostly
focused on purely technical implementation of AI-based information systems. But in recent years, the
AI research community has become more and more aware that several socio-technological factors are
important for the use of AI in today’s organizations. Therefore, a fairly new research area has emerged
in recent years that focuses on the adoption and diffusion as well as the management of AI.
With our research we contribute directly to this current research field and extend the body of
knowledge in the field information systems engineering [
4
]. We provide the first AI management
framework that ties directly to the extensively researched field of MMs. Our AI management framework
is developed on a solid scientific foundation by using a multi-method, design science-oriented research
approach. The different dimensions can guide other AI management researchers to develop methods
that promote the adoption and diffusion of AI in organizations. These methods (including our AI
management framework) should then be applied in the organizational practice to bridge the gap between
scientific AI research and practical AI application.
Practitioners benefit from our research through the application of the AI management framework. It can
serve organizations as an internal benchmarking tool and as an enabler to determine their current status
quo with regard to the adoption and diffusion of AI technologies. With the planned web-based
information system (cf. section 3), the AI management framework can be easily integrated into the daily
operations of any organization. Through a continuous improvement process (cf. Figure 3), each
organization can iteratively become an AI-enabled organization by receiving recommendations for
further use of AI based on its current capabilities. Finally, our AI management framework forms the
basis for certified AI management systems that are currently being developed by the International
Organization for Standardization (ISO) [
18
]. By applying our AI management framework,
organizations could more easily achieve a potential AI management systems certification.
6. Conclusion and Outlook
In this paper, we present a multi-step, design science-oriented research approach to develop a
management framework that fosters the adoption and diffusion of AI in organizations. We clearly
formulate the research question (cf. section 1), identify a significant problem in the field of research
(cf. section 1 and section 2) and outline the current status of the problem domain and related solutions
(cf. section 2). Our research methodology consists of four main iterations with an additional parallel
research stream to investigate new approaches and methods on how AI-based information systems can
be managed in each of the individual dimensions of the AI management framework (cf. section 3).
Following this research methodology, we present preliminary ideas and the results achieved so far (cf.
section 4). Finally, we outline the theoretical and practical contributions of our work to the problem
domain and highlight their uniqueness (cf. section 5). In the future, we will follow up the presented
research methodology and conduct the next steps to develop the AI management framework. We will
design an industry-independent AI management metamodel and operationalize it by developing a
webbased information system with a systematic scoring system. The final integrated management
framework, together with all its components (industry-specific maturity model, generic metamodel,
web-based tool and methodical guidelines), will enable the continuous improvement of AI adoption
and guide companies on their way in becoming AI-enabled organizations.
7. Acknowledgements
I want to thank Prof. Dr. Oliver Thomas (oliver.thomas@uni-osnabrueck.de) from the Information
Management and Information Systems group of Osnabrück University and the Smart Enterprise
Engineering group of the German Research Center of Artificial Intelligence for the supervision of this
thesis.
8. References
[34] A. Schumacher, S. Erol, W. Sihn, A Maturity Model for Assessing Industry 4.0 Readiness and
Maturity of Manufacturing Enterprises, Procedia CIRP. 52, (2016) 161–166.
[35] J. Seger, N. Miailhe, S. Mueller, The AIGO: A Framework for Planning, Developing, and
Deploying Artificial Intelligence in Intergovernmental Organizations, Future Society, Cambridge,
2019.
[36] S. Thrun, Robotic Mapping: A Survey, in: G. Lakemeyer, B. Nebel (Eds.), Exploring Artificial
Intelligence in the New Millennium, Morgan Kaufmann, San Francisco, 2003, pp. 1–35.
[37] A. M. Turing, Computing Machinery and Intelligence. Mind. 49 (1950) 433–460.
[38] R. Wendler, The maturity of maturity model research: A systematic mapping study. Inf. Softw.
Technol. 54 (12) (2012) 1317–1339.
[39] D. Zhang, N. Maslej, E. Brynjolfsson, J. Etchemendy, T. Lyons, J. Manyika, H. Ngo, J. C. Niebles,
M. Sellitto, E. Sakhaee, Y. Shoham, J. Clark, R. Perrault, The AI Index 2022 Annual Report, AI
Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, 2022.