=Paper= {{Paper |id=Vol-3855/facete1 |storemode=property |title=Generating Business Capability Maps using GenAI: A Case Study |pdfUrl=https://ceur-ws.org/Vol-3855/facete1.pdf |volume=Vol-3855 |authors=Jürgen Jung,Pierre Wienke |dblpUrl=https://dblp.org/rec/conf/ifip8-1/JungW24 }} ==Generating Business Capability Maps using GenAI: A Case Study== https://ceur-ws.org/Vol-3855/facete1.pdf
                         Generating Business Capability Maps using GenAI: A Case
                         Study
                         Jürgen Jung1,*,† , Pierre Wienke2,†
                         1
                             Frankfurt University of Applied Sciences, Nibelungenplatz 1, 60318 Frankfurt, Germany
                         2
                             FINIUS GmbH, Kölner Straße 3, 65760 Eschborn, Germany


                                        Abstract
                                        The concept of business capability is well accepted and a powerful tool for enterprise architecture and business
                                        management. However, creating a business capability map for a specific company is still a time-consuming task
                                        for experienced consultants as well as a challenging endeavour for junior consultants due to the abstract notion of
                                        business capabilities. Systems like ChatGPT are currently showing surprisingly accurate results when generating
                                        text and there are first research approaches investigating their potential for enterprise modelling. The paper at
                                        hands describes the prototypical implementation and evaluation of a generator for business capability maps. The
                                        project is based on a pre-trained language model and uses prompt engineering for deriving a prompt that can be
                                        used for first draft business capability maps. Experienced consultants may then focus on improving the result
                                        from GenAI instead of creating a new map from scratch for each customer.

                                        Keywords
                                        business capability, business capability map, generative AI, consulting




                         1. Introduction
                         Business Capability (BC) is a widely used abstraction for describing the activities of an organisation (i.e.
                         functional capabilities [1]). They are a less detailed representation compared to business processes and,
                         therefore, require less time and effort for their documentation [2]. A Business Capability Map (BCM)
                         is a visual representation of the structure of BC’s of an organisation or parts of it. Such a BCM can
                         support business-IT alignment [3], strategic management [4] or are part of an entire discipline, Business
                         Architecture [5].
                            There are a few methods for BC mapping available (e.g. [6, 7]) but creating a BCM is still a time-
                         consuming and challenging task. A BCM needs to reflect the essence of an organisation but the
                         abstraction of a BC is often hard to grasp by business representatives. Being abstract is a benefit of
                         BCMs but it is also a disadvantage as they might not directly relate to common concepts in Enterprise
                         Modelling. A BC does not reflect a business process as it should fade out details associated with a
                         business process model [2]. Furthermore, a BC should be technology agnostic and not represent the
                         organisational chart of a company [8].
                            In recent years, Artificial Intelligence (AI) systems have made tremendous progress. Especially,
                         Generative AI (GenAI)[9] got some popularity because of services like ChatGPT. These services are not
                         only capable of generating text but current research also aims to evaluate their potential for creating
                         enterprise models from textual descriptions [10, 11, 12]. There are also first publications investigating
                         the generation of BCM [13].
                            Against this background, the objective of the research presented in the paper at hand is to develop
                         and evaluate a prototype for generating a BCM using GenAI based on text documents describing a
                         company. The prototype was subject to a student project that has been conducted together with a
                         consulting firm, specialised in consulting services around BCM and Enterprise Architecture.
                         Companion Proceedings of the 17th IFIP WG 8.1 Working Conference on the Practice of Enterprise Modeling Forum, M4S, FACETE,
                         AEM, Tools and Demos co-located with PoEM 2024, Stockholm, Sweden, December 3–5, 2024
                         *
                           Corresponding author.
                         †
                           These authors contributed equally.
                         $ jung.juergen@fb2.fra-uas.de (J. Jung); pierre.wienke@finius.de (P. Wienke)
                          0000-0002-8649-4827 (J. Jung)
                                       © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).


CEUR
                  ceur-ws.org
Workshop      ISSN 1613-0073
Proceedings
   An extension of the STAR (situation, task, action, result) model [14, 15] is used for documenting the
case study. The initial situation of the consulting firm–including the motivation for the joint research
project–is depicted in section 2. The definition of the objective and major requirements are presented
as task in section 3. Actions that have been done for implementing the prototype as well as design
alternatives are subject to section 4. The resulting prototype’s architecture and the prompts used for
generating the BCM are discussed in section 5. In addition to the STAR model, there is a separate
section 6 for reflecting the prototype and future developments.


2. Situation
The research project was initiated by a consultancy firm specialised on the finance industry. The firm
provides services around Business and Enterprise Architecture aiming to harmonize the application
landscape of their clients. Key stakeholders of the firm are active members of the so called Business
Architecture Guild which provides method support and reference models for business architecture—
including industry-specific BCM [16]. Customer-specific BCM’s are created by senior consultants based
on their experience, industry reports and reference models, like for example HERM (Higher Education
Reference Model)[17] or the reference models provided by the Business Architecture Guild[16]. These
reference models provide common knowledge on a given industry, but still need to be adjusted with
respect to the specifics and terminology of a client.
   Even though, BCM are one of the core artefacts for their services, creating a BCM is still a challenging
and time-consuming task:

    • Developing a customer-specific BCM requires several iterations of alignment with the customer.
      A BCM needs to use the company-specific terminology and should reflect any specifics of its
      business model so that it can be understood and gets accepted by the client. [8]
    • Creating (and communicating) a BCM requires several years of industry experience. Due to their
      abstract notion, BCM are often beyond information systems students’ (or junior consultants’)
      experience of the world. [18]
    • A BCM has to meet several quality criteria in order to be used effectively, for example: disjoint
      (BC should not overlap), complete (the whole business model needs to be covered), minimal (it
      does not contain unnecessary BC) or stability (a BCM should not change over time) [19]

   Hence, creating a customer-specific BCM is quite expensive as it requires the involvement of a
senior consultant for several days–or even weeks or months. At the same time, there is no immediate
value-add as the BCM is not a goal in itself for the consultancy but a tool required for understanding
and documenting the business perspective of the client or used as a navigator for future changes. Some
senior consultants, therefore, regard it as cumbersome, especially as they tend to reuse bits and pieces
from previous work or reference maps. The process of creating a BCM is a chance for the client company
to understand its organisation and activities. For most consultants, there is only limited value from this
as their assignment is only for a limited amount of time.
   Consequently, the firm aims to investigate GenAI technologies with respect to their potential for
generating a customer-specific BCM. There are already a couple of papers available documenting
promising research results concerning the creation of models from text documents. Görgen et al.
evaluate the generation of business process models using ChatGPT (GPT4) by comparing against the
result modelled by a domain expert [10]. In a similar way, Sandkuhl et al. perform two experiments to
check whether ChatGPT can substitute a domain expert [11]. Sjilov et al. investigate the potential of
supporting modelling activities with Graph Neural Networks GNN [12].
   In order to test ChatGPT’s capabilities to generate a BCM in a similar way, the firm collaborated with
the Frankfurt University of Applied Sciences. A team of three students was tasked to implement a proof-
of-concept (PoC) for a BCM generator (BCMGen) as part of their post-graduate study in information
systems (German: Wirtschaftsinformatik) [18]. The student team had three months time and each
team member was supposed to work on the project for around 15 hours per week. None of the team
members had any experience in business capability mapping and only minor exposure to GenAI or
Natural Language Processing (NLP). They had access to scientific papers via the university library and
test data as well as reference maps was provided by the consultancy firm.


3. Task
The objective of the research project was defined by the consultancy firm. BCMGen–as a proof-of-
concept (PoC)–is supposed to generate a BCM based on text documents describing the client organisation.
There was no specific expectation on which GenAI to be used and the decision was left to the student
team. It was clarified in the beginning that the BCMGen result might still supposed to be edited by an
experienced consultant (i.e. it is not the final result for the client).
   The requirements were jointly documented by the consultancy firm as well as the student team and
can be summarised as follows:
   1. BCMGen provides a web-interface so that text documents can be uploaded for further processing
   2. The result is a hierarchical BCM consisting of two levels
   3. Each BC needs to have a concise description
   4. The BCM needs to have a format that can subsequently be edited by a consultant
   5. BC should be disjoint and the BCM needs to be complete with respect to the text documents
      (input)
   6. The grouping of level-2 BC to level-1 BC needs to be coherent based on business concepts and
      activities
   7. Customer-specific terminology, concepts and principles need to be reflected in the BCM. Conse-
      quently, an industry-specific or reference BCM will have to be adjusted by BCMGen accordingly.
   Text documents describing individual companies have been provided by the consultancy firm and,
therefore, a non-disclosure agreement was needed as they contain confidential information. Only a few
of the consultancy’s clients were willing to share such documents so that the team was also tasked to
find appropriate input on their own (e.g. documents from public sector or published business reports).
Those documents need to contain as much information as possible on what the company is doing.
   The PoC was developed iteratively as the consultancy firm was simultaneously learning from the
project. Intermediate results were presented at the end of each phase and relevant feedback was provided
by the consultancy firm. In essence, the firm was acting like a customer in an agile project.


4. Actions
After the project has been kicked-off, the student team first needed to prepare a project plan and define a
collaboration mode. Furthermore, they got familiar with the concepts of BC, BCM as well as GenAI. The
consultancy firm provided their own training video for consultants in capability mapping. With respect
to GenAI, the team members did their own research on similar projects and available technologies.

4.1. Prompt Engineering
During a first meeting for defining the scope and objectives of the project, the consultancy restricted
BCMGen to the public transport industry. This allowed for a focused development of a first prototype.
The students needed to collect text documents and reference capability maps for this respective industry.
The first version was then developed using ChatGPT and prompting methods. Such methods can be
used for pre-trained models and provide contextual information as part of the prompt (i.e. query to the
Large Language Model (LLM)) [20, 21].
   The basic idea of using prompt engineering for BCMGen is depicted in figure 1. The students needed
to decide for a pre-trained LLM, in this case GPT, provided via the OpenAI Application Programming
Interface (API). The students then needed to investigate how to define the prompt for the query to GPT.
Figure 1: Elements of Prompt Engineering (source: [20])


The prompt consists of documents provided by the user (e.g. documents on a specific public transport
company) together with instructions and context information (e.g. reference models or information on
capability maps). The team aimed at zero-shot prompting, hence, they focused on evaluating different
instructions and context information to improve the quality of the output.

4.2. Training of pre-trained model
A second option has been considered: training a pre-trained LLM with respect to capability mapping.
There are a couple of pre-trained LLM available (e.g. Llama) that can be be enhanced for individual
needs by using additional training data [22]. This might have lead to a better quality of BCMGen if such
an LLM is trained with a sufficient number of BCM. However, the idea was discarded for the following
reasons:
    • Time constrained: The planned project duration of three months was quite short and the team
      might have run into the risk of not meeting the deadline by adding an additional training phase
      half-way through the project.
    • Lack of training data: There is not sufficient training data (i.e. annotated BCM) publicly available,
      so that the team would have needed to collect and annotate existing reference maps.
    • Performance: There was not enough computing power available for training the LLM. Procuring
      a respective virtual server would have taken additional time.
    • Bias: Using existing reference BCM might have influenced the result so that BCMGen creates
      only maps similar to the reference maps.
    • Uncertainty: The impact of the additional training on the quality of BCMGen was hard to assess
      in advance.
   There are solutions or workarounds available for some of these concerns. However, the team decided
to focus on prompting rather that training their own model, given the time restriction.

4.3. Graphical BCM editor
The initially agreed output formats of BCMGen are comma-separated files (CSV), PDF (Portable Doc-
ument Format) and JSON (JavaScript Object Notation). CSV can be imported into a spreadsheet
application and then easily edited by consultants. Figure 2 shows an example output in Microsoft Excel.
This tool is installed on the consultants’ standard computers so that they can edit the business capability
map immediately. However, the visualisation does not explicitly show the decomposition hierarchy
of BC. Column B indicates the level and column E the parent capability. The level-1 capability Data
Management for example consists of the level-2 capabilities Data Coordination, Data Leveraging, Data
Flow Management, Data Quality Management and Data Integration Management. Column C indicates
whether the corresponding BC is corporate (tier 1), core (tier 2) or support (tier 3).
Figure 2: Example BCMGen output in MS Excel


   Some consultants raised an additional requirement for a graphical tree visualisation of the BCM.
There are frameworks for graphical editors available (for example GoJS)1 . Such frameworks allow
for visualising the map and even provide measures for immediately editing it. The requirement was
discarded due to time restrictions. However, as JSON is generated already, a visual tool based on GoJS
can be added later on.


5. Result
5.1. System Architecture
The architecture of the BCMGen PoC is shown in figure 3. The user interface is implemented using
the Python-based Flask2 framework for web applications. A consultant may upload several documents
describing the client company. In case of non-text documents, Tesseract is used for recognising and
extracting text from, for example, bitmaps or PDF. The text then needs to be separated into several
chunks due to size restrictions of the OpenAI API. For the first version using the OpenAI API, the chunk
size was restricted to 5,000 bytes. The context (i.e. the reference models) do not need to be split as the
file size is rather small.
   In the phase of AI analysis the prompt will be created together with the consultants input text from
the document processing stage and submitted to the OpenAI API. The prompt contains the instruction
for GPT together with reference BCM. A separate list with customer specific-terminology is added
so that rather abstract terms from the reference model can be replaced by a language familiar to the
customer. The resulting capability list is then the input for creating the BCM.
   The creation of the BCM is done in the AI generation phase. The hierarchical structure is created
and each capability will be extended with a concise description. The output is then stored as PDF for
human readers, CVS for further editing by consultants and JSON so that it can be processed by further
applications (e.g. a visualising tool based on GoJS). The CVS output is required as BCMGen was not
expected to create the final BCM.

5.2. Prompt
The prompt for generating the BCM consists of several parts. The first part is used for each text chunk
in order to extract capabilities from the text:
Take t h e r o l e a s an e x p e r t e n t e r p r i s e a r c h i t e c t .
You w i l l be p r o v i d e d a t e x t where you need t o i d e n t i f y r e l e v a n t
   business c a p a b i l i t i e s .
1
    https://gojs.net
2
    https://flask.palletsprojects.com/en/3.0.x/
Figure 3: System Architecture of BCMGen


Take t h e f o l l o w i n g c a p a b i l i t i e s a s a r e f e r e n c e :   {
   level_1_capabilities } { level_2_capabilities }
Make s u r e t h a t t h e c a p a b i l i t i e s a r e t r a n s l a t e d i n t o E n g l i s h
   Reference capabilities are provided as context in a list of capabilities ({level_1_capabilities}
and {level_2_capabilities}). These instructions are then followed by the text chunk.
   After all text chunks have been processed, each BC will be classified, whether it is a corporate,
core or a support capability as defined in [2]. Corporate capabilities refer to managing the company
or providing guidance to the organisation. They are often referred to as strategic capabilities. Core
capabilities represent the core business and, therefore, value-adding functions. Support capabilities
(sometimes referred to as enabling) provide support functions for the core capabilities. The classification
as corporate, core or support will be done for each of the resulting capabilities:
Now go t h r o u g h a l l t h e c h a t r e s p o n s e s a b o v e .
Note t h a t t i e r s mean t h e f o l l o w i n g : T i e r 1 = S t r a t e g i c C a p a b i l i t i e s ,
    Tier 2 = Operational C a p a b i l i t i e s , Tier 3 = Support C a p a b i l i t i e s
   The next step deals with creating the hierarchy of capabilities visualised in the BCM. The number
of levels has been restricted to two to keep the PoC compact. The following prompt creates level-2
capabilities for each level-1 capability:
Now go t h r o u g h a l l t h e r e l e v a n t c a p a b i l i t i e s and c r e a t e l e v e l 2
      c a p a b i l i t i e s f o r each c a p a b i l i t y
Here a r e t h e l e v e l 2 c a p a b i l i t i e s you s h o u l d t a k e a s a r e f e r e n c e : {
      level_2_capabilities }
P l e a s e be aware t h a t t h e t i e r l e v e l i s d e t e r m i n e d by t h e c a p a b i l i t y
      from which i t i s d e r i v e d .
Make s u r e t h a t t h e l e v e l 2 c a p a b i l i t i e s do n o t have t h e same name a s
        the l e v e l 1 c a p a b i l i t y
  Reference BC are provided as {level_2_capabilities} and certain conditions need to be ensured.
The tier of a level-2 capability has to be the same as the corresponding level-1 capability in order to
have a coherent BCM. Furthermore, GPT does not care for naming without a clear statement that a
sub-capability is not allowed to have the same name as its parent capability. After completing this step,
the JSON file can be created:
As an e x p e r t e n t e r p r i s e a r c h i t e c t , your t a s k i s t o a n a l y s e t h e
   l i s t e d business c a p a b i l i t i e s .
These c a p a b i l i t i e s s h o u l d be s t r u c t u r e d i n t o a JSON f o r m a t ,
   r e f l e c t i n g a h i e r a r c h i c a l c a p a b i l i t y map .

C r e a t e a JSON S t r u c t u r e : S t r u c t u r e t h e e x t r a c t e d c a p a b i l i t i e s i n t o a
       JSON f o r m a t a s shown below . E n s u r e t h e JSON s y n t a x i s c o r r e c t
      and t h e r e a r e no e r r o r s .
Example :
{{
" capabilities ": [
        {{
        " name " : " Brand Management " ,
        " level ": "1" ,
        " tier ": 1,
        " subCapabilities ": [
              {{
              " name " : " Brand D e f i n i t i o n " ,
              " level ": "2" ,
              " tier ": 1,
              }} ,
              {{
              " name " : " Brand P o r t f o l i o Management " ,
              " level ": "2" ,
              " tier ": 1,
              }}
        ]
        }}
        {{
        " name " : " Customer Management " ,
        " level ": "1" ,
        " tier ": 2,
        " subCapabilities ": [
              {{
              " name " : " Customer D e f i n i t i o n " ,
              " level ": "2" ,
              " tier ": 2,
              }} ,
              {{
              " name " : " Customer M a t c h i n g " ,
              " level ": "2" ,
              " tier ": 2,
              }}
        ]
        }}
]
}}
R e t u r n j u s t t h e JSON message
  The result is the complete BCM in JSON format, but the descriptions are still missing. These are
generated with the following instructions:
Be a h e l p f u l a s s i s t a n t
P l e a s e en h a n c e t h e JSON f i l e by i n t r o d u c i n g a new ’ d e s c r i p t i o n ’
      f i e l d f o r e a c h e l e m e n t , c o n t a i n i n g a b r i e f one − s e n t e n c e
      d e s c r i p t i o n of the c o r r e s p o n d i n g c a p a b i l i t y . { JSON_input }
Make s u r e t h a t t h e JSON o u t p u t doesn ’ t c o n t a i n any JSON s y n t a x e r r o r
  The term {JSON_input} is replaced by the previous JSON. The result is now ready to be saved as a
JSON file or exported as CVS or PDF file. The PDF version contains a graphical representation of the
BCM as a tree.
6. Reflection
BCMGen has been evaluated and developed further by two consultants for several months now. One of
the major changes was the shift from OpenAI to Claude Sonnet 3 provided via Amazon Web Services
(AWS) due to the availability of licences in the consultancy firm. The quality of the BCM was further
improved by using industry-specific reference models and domain-specific word mappings for client-
specific terminology.
   The results are quite promising. Both consultants emphasize that BCMGen helps them with creating
a client-specific BCM much faster than before (“a couple of hours rather than days”). The capability
naming is accurate and the descriptions usually match the purpose of the capabilities. There is also an
import of the BCM into the standard modelling tool (MagicDraw) of the consultancy available now.
   However, a BCM still has to be revised by a senior consultant–as already intended during the project
initiation. First of all, the quality of the input is decisive. BCMGen is only capable to recognise
capabilities mentioned in the text documents provided by the client. Also inconsistencies, which are
common in natural language texts that have been evolving over time, hamper the result’s quality. Beside
of this, BCMGen shows common phenomena of GenAI. In case of missing information, GenAI tends
to hallucinate by inventing non-existing capabilities for filling in the gaps. According to one of the
consultants, there seemed to be less hallucinations by Claude Sonnet 3.4 compared to OpenAI GPT.
Nevertheless, especially Claude needs to be reminded frequently that the output has to be in English
language4 .
   An example output from BCMGen (with Claude Sonnet) is shown in table 1 without any manual
modifications on the contents. The capabilities have been created based on the classic book The Art of
War by Sun Tzu. The structure of the table is slightly different in order to enhance readability. The
middle column shows the capabilities’ names and the right one their descriptions. The left column show
the corresponding level-1 capability in case of a level-2 capability. The field is empty for any level-1
capability. Dedicated columns indicating the hierarchy level and tier have been omitted.
   The military context is visible in the capabilities Intelligence Management or Strategic Planning. There
are no Finance capabilities as financial aspects are not covered by the book. Claude recognised the
military context and already deducted that the capabilities might be transferred to a business context.
When asked for an explanation for the result, Claude provided the following output:

          The business capabilities have been identified based on the military and espionage context of
          the text. These capabilities, while framed in a military context, have analogous applications
          in business strategy and operations.
             1. Intelligence Management is crucial for gathering and analysing information about
                competitors and market conditions.
             2. Strategic Planning involves assessing the competitive landscape and allocating re-
                sources effectively.
             3. Human Resource Management focuses on recruiting, training, and deploying person-
                nel with specialized skills.
             4. Risk Management is essential for maintaining operational and information security
                in a competitive business environment.
             5. Communication Management ensures secure and strategic dissemination of informa-
                tion.
             6. Financial Management covers compensation and funding for operations.
             7. Operational Management deals with executing strategies and planning infiltration
                into new markets.

  BCMGen will be further refined while being used in customer projects. Some of the requirements to
be considered in the future are as follows:
3
    https://www.anthropic.com/claude
4
    Statement from one of the consultants: “This is funny but annoying at the same time.”
Table 1
Business capabilities derived from The Art of War (Generated by AI tool Claude Sonnet)
superior capability          capability name               description
                             Intelligence Management       Overarching capability for managing intel-
                                                           ligence operations and information
Intelligence Management      Spy Network Management        Managing and coordinating a network of
                                                           intelligence sources to gather valuable in-
                                                           formation
Intelligence Management      Information Collection        Systematically gathering data and intelli-
                                                           gence from various sources
Intelligence Management      Information Verification      Validating and cross-checking collected in-
                                                           formation to ensure accuracy and reliabil-
                                                           ity
                             Strategic Planning            Overall capability for developing and im-
                                                           plementing organizational strategies
Strategic Planning           Resource Allocation           Efficiently distributing available resources
                                                           to maximize organizational effectiveness
Strategic Planning           Risk Assessment               Identifying and evaluating potential risks
                                                           to the organization and its operations
Strategic Planning           Decision Making               Making informed choices based on avail-
                                                           able information and strategic objectives
                             Human Resource Manage-        Managing the organization’s workforce
                             ment                          and talent pool
Human Resource Manage-       Talent Acquisition            Recruiting and hiring skilled individuals to
ment                                                       support organizational goals
Human Resource Manage-       Reward and Recognition        Implementing systems to motivate and re-
ment                                                       tain valuable employees
                             Security Management           Ensuring the protection of organizational
                                                           assets and information
Security Management          Information Security          Safeguarding sensitive data and commu-
                                                           nications from unauthorized access or
                                                           breaches
Security Management          Counterintelligence           Identifying and neutralizing threats from
                                                           hostile intelligence services
                             Knowledge Management          Effectively handling and utilizing organi-
                                                           zational knowledge and information
Knowledge Management         Information Analysis          Examining and interpreting collected data
                                                           to derive meaningful insights
Knowledge Management         Knowledge Dissemination       Sharing relevant information and insights
                                                           across the organization
                             Operational Efficiency        Optimizing organizational processes and
                                                           performance
Operational Efficiency       Cost Optimization             Identifying and implementing measures to
                                                           reduce operational costs
Operational Efficiency       Performance Measurement       Evaluating and tracking organizational
                                                           and individual performance against set
                                                           goals


• Improving the naming of BCs: Names do not always follow naming conventions for capabilities
  (e.g. “Counterintelligence” in table 1).
• Implementing a graphical editor: CVS output is preferred by the consultants as they can edit it in
  Excel immediately. However, a graphical representation is perceived as more catchy by clients.
• Training a pre-trained LLM: The current version uses a standard GenAI without any further
  training. There is a plan to investigate whether additional training of an existing model will
  improve the quality of the result.
7. Summary and Open Topics
Business Architecture Management takes time when set up to deliver value for a client organisation.
Currently, Business Capability Mapping is a technique that is performed manually while performing
interviews and by analysing documents. Errors might occur, capabilities can be missed or false positives
can be discovered. Therefore, a prototype for generating capability maps has been implemented and
evaluated as part of a research project presented in the paper at hand. Of course, problems still arise:
GenAI systems, especially ChatGPT, tend to hallucinate, are generally lazy and sometimes ignore
relevant aspects. Especially, determinism is yet to be achieved. Nevertheless, there is a positive impact
on our role as business architecture consultants. BCMGen saves time and increases quality and, therefore,
value for the client. Although yet only a PoC, we will use and extend BCMGen in our business.
   The student project presented in the paper at hand already revealed some peculiarities with generating
capability maps. There were only three iterations with the consultants as the project was scheduled
for three months. These feedback rounds where crucial as the students did not have experience with
capability mapping. Consultants needed to asses the result’s quality and more feedback rounds may
have improved the output. The quality depends on the GenAI system together with the prompt, the
context and the input provided. While the students tried to improve the prompt with the OpenAI API,
there was no further investigation with different contexts (i.e. reference models) or variations of the
input (e.g. kinds of documents or length). These will have to be evaluated in follow-up projects.


References
 [1] J. Brits, G. Botha, M. Herselmann, Conceptual framework for modeling business capabilities, in:
     Proceedings of the 2007 Informing Science and IT Education Joint Conference, Ljubljana, Slovenia,
     Juni 22–25, 2007, 2007, pp. 151–170.
 [2] J. Jung, B. Fraunholz, Masterclass Enterprise Architecture Management, Springer Nature, Cham,
     2021.
 [3] G. Bondel, A. Faber, F. Matthes, Reporting from the implementation of a business capability map
     as business-it alignment tool, in: 2018 IEEE 22nd International Enterprise Distributed Object
     Computing Workshop (EDOCW), 2018, pp. 125–134. doi:10.1109/EDOCW.2018.00027.
 [4] W. Keller, Using capability models for strategic alignment, in: D. Simon, C. Schmidt (Eds.), Business
     Architecture Management: Architecting the Business for Consistency and Alignment, Springer
     International Publishing, Cham, 2015, pp. 107–122.
 [5] W. Kuehn, Strategy to Reality: Making the Impossible Possible for Business Architects, Change
     Makers and Strategy Execution Leaders, Morgan James Publishing, New York, 2023.
 [6] M. Wißotzki, The capability management process: Finding your way into capability engineering,
     in: D. Simon, C. Schmidt (Eds.), Business Architecture Management: Architecting the Business for
     Consistency and Alignment, Springer International Publishing, Cham, 2015, pp. 77–105.
 [7] Van Riel, Jonas and Poels, Geert, A method for developing generic capability maps : a design
     science study in the professional sport industry, Business & Information Ssystems Engineering 65
     (2023) 403–424. URL: http://doi.org/10.1007/s12599-023-00793-z.
 [8] R. Burlton, Developing your capability architecture: It’s all about being able to get things done,
     2017. URL: https://www.brcommunity.com/articles.php?id=b927.
 [9] S. Feuerriegel, J. Hartmann, C. Janiesch, P. Zschech, Generative ai, Business & Information Systems
     Engineering 66 (2024) 111–126. URL: https://doi.org/10.1007/s12599-023-00834-7. doi:10.1007/
     s12599-023-00834-7.
[10] L. Görgen, E. Müller, M. Triller, B. Nast, K. Sandkuhl, Large language models in enterprise modeling:
     Case study and experiences, in: International Conference on Model-Driven Engineering and
     Software Development, 2024, pp. 74–85. URL: https://api.semanticscholar.org/CorpusID:267968062.
[11] K. Sandkuhl, B. Barn, S. Barat, Neural text generators in enterprise modeling: Can chatgpt
     be used as proxy domain expert?, in: A. da Silva, M. da Silva, J. Estima, C. Barry, M. Lang,
     H. Linger, C. Schneider (Eds.), Information Systems Development, Organizational Aspects and
     Societal Trends (ISD2023 Proceedings). Lisbon, Portugal: Instituto Superior Tecnico., 2023. URL:
     https://doi.org/10.62036/ISD.2023.44.
[12] N. Shilov, W. Othman, M. Fellmann, K. Sandkuhl, Machine learning for enterprise model-
     ing assistance: an investigation of the potential and proof of concept, Software and Systems
     Modeling 22 (2023) 619–646. URL: https://doi.org/10.1007/s10270-022-01077-y. doi:10.1007/
     s10270-022-01077-y.
[13] C. Drieschner, M. Sensoy, J. Weking, , H. Krcmar, Business capability mining: Opportunities and
     challenges, in: ECIS 2023 Research Papers, 388, 2023. URL: https://aisel.aisnet.org/ecis2023_rp/388.
[14] S. W. Handayani, Y. K. S. Pranoto, S. Puspitaningrum, M. N. A. B. Ghazali, Star technique in
     the abcd5e learning model to improve teacher assessment capability in early chilhood education,
     Educative: Jurnal Ilmiah Pendidikan 2 (2024) 80–86. URL: https://publikasi.abidan.org/index.php/
     educative/article/view/380. doi:10.70437/educative.v2i2.380.
[15] S. Cook, Coaching for High Performance: How to develop exceptional results through coaching,
     IT Governance Publishing, Ely (UK), 2009.
[16] Business Architecture Guild (Ed.), A Guide to the Business Architecture Body of Knowledge
     (BIZBOK Guide), 13th ed., Business Architecture Guild, 2024.
[17] G. Nauwerck, P. Maltusch, V. L. Strat, E. Suominen, Towards a sector specific higher education
     reference model: introducing herm, European Journal of Higher Education IT 1 (2022).
[18] J. Jung, P. Wienke, Teaching research for impact in post-graduate information systems education,
     in: Research for Impact in Digital Innovation: Proceedings of the 2nd RIDI Workshop, Utrecht
     (NL), April 11 & 12, 2024, 2024, pp. 10–13.
[19] W. Pfannenstiel, dpunkt.verlag, Heidelberg, 2023.
[20] P. Sahoo, A. K. Singh, S. Saha, V. Jain, S. Mondal, A. Chadha, A systematic survey
     of prompt engineering in large language models: Techniques and applications, CoRR
     abs/2402.07927 (2024). URL: https://doi.org/10.48550/arXiv.2402.07927. doi:10.48550/ARXIV.
     2402.07927. arXiv:2402.07927.
[21] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, G. Neubig, Pre-train, prompt, and predict: A systematic
     survey of prompting methods in natural language processing, Comput. Surveys 55 (2023). URL:
     https://doi.org/10.1145/3560815. doi:10.1145/3560815.
[22] J. Alammar, M. Grootendorst, Hands-On Large Language Models: Language Understanding and
     Generation, O’Reilly, 2024.