Business Capability (BC) is a widely used abstraction for describing the activities of an organisation (i.e. functional capabilities [1]). They are a less detailed representation compared to business processes and, therefore, require less time and effort for their documentation [2]. A Business Capability Map (BCM) is a visual representation of the structure of BC's of an organisation or parts of it. Such a BCM can support business-IT alignment [3], strategic management [4] or are part of an entire discipline, Business Architecture [5].
There are a few methods for BC mapping available (e.g. [6,7]) but creating a BCM is still a timeconsuming and challenging task. A BCM needs to reflect the essence of an organisation but the abstraction of a BC is often hard to grasp by business representatives. Being abstract is a benefit of BCMs but it is also a disadvantage as they might not directly relate to common concepts in Enterprise Modelling. A BC does not reflect a business process as it should fade out details associated with a business process model [2]. Furthermore, a BC should be technology agnostic and not represent the organisational chart of a company [8].
In recent years, Artificial Intelligence (AI) systems have made tremendous progress. Especially, Generative AI (GenAI) [9] got some popularity because of services like ChatGPT. These services are not only capable of generating text but current research also aims to evaluate their potential for creating enterprise models from textual descriptions [10,11,12]. There are also first publications investigating the generation of BCM [13].
Against this background, the objective of the research presented in the paper at hand is to develop and evaluate a prototype for generating a BCM using GenAI based on text documents describing a company. The prototype was subject to a student project that has been conducted together with a consulting firm, specialised in consulting services around BCM and Enterprise Architecture.
An extension of the STAR (situation, task, action, result) model [14,15] is used for documenting the case study. The initial situation of the consulting firm-including the motivation for the joint research project-is depicted in section 2. The definition of the objective and major requirements are presented as task in section 3. Actions that have been done for implementing the prototype as well as design alternatives are subject to section 4. The resulting prototype's architecture and the prompts used for generating the BCM are discussed in section 5. In addition to the STAR model, there is a separate section 6 for reflecting the prototype and future developments.
The research project was initiated by a consultancy firm specialised on the finance industry. The firm provides services around Business and Enterprise Architecture aiming to harmonize the application landscape of their clients. Key stakeholders of the firm are active members of the so called Business Architecture Guild which provides method support and reference models for business architectureincluding industry-specific BCM [16]. Customer-specific BCM's are created by senior consultants based on their experience, industry reports and reference models, like for example HERM (Higher Education Reference Model) [17] or the reference models provided by the Business Architecture Guild [16]. These reference models provide common knowledge on a given industry, but still need to be adjusted with respect to the specifics and terminology of a client.
Even though, BCM are one of the core artefacts for their services, creating a BCM is still a challenging and time-consuming task:
• Developing a customer-specific BCM requires several iterations of alignment with the customer.
A BCM needs to use the company-specific terminology and should reflect any specifics of its business model so that it can be understood and gets accepted by the client. [8] • Creating (and communicating) a BCM requires several years of industry experience. Due to their abstract notion, BCM are often beyond information systems students' (or junior consultants') experience of the world. [18] • A BCM has to meet several quality criteria in order to be used effectively, for example: disjoint (BC should not overlap), complete (the whole business model needs to be covered), minimal (it does not contain unnecessary BC) or stability (a BCM should not change over time) [19] Hence, creating a customer-specific BCM is quite expensive as it requires the involvement of a senior consultant for several days-or even weeks or months. At the same time, there is no immediate value-add as the BCM is not a goal in itself for the consultancy but a tool required for understanding and documenting the business perspective of the client or used as a navigator for future changes. Some senior consultants, therefore, regard it as cumbersome, especially as they tend to reuse bits and pieces from previous work or reference maps. The process of creating a BCM is a chance for the client company to understand its organisation and activities. For most consultants, there is only limited value from this as their assignment is only for a limited amount of time.
Consequently, the firm aims to investigate GenAI technologies with respect to their potential for generating a customer-specific BCM. There are already a couple of papers available documenting promising research results concerning the creation of models from text documents. Görgen et al. evaluate the generation of business process models using ChatGPT (GPT4) by comparing against the result modelled by a domain expert [10]. In a similar way, Sandkuhl et al. perform two experiments to check whether ChatGPT can substitute a domain expert [11]. Sjilov et al. investigate the potential of supporting modelling activities with Graph Neural Networks GNN [12].
In order to test ChatGPT's capabilities to generate a BCM in a similar way, the firm collaborated with the Frankfurt University of Applied Sciences. A team of three students was tasked to implement a proofof-concept (PoC) for a BCM generator (BCMGen) as part of their post-graduate study in information systems (German: Wirtschaftsinformatik) [18]. The student team had three months time and each team member was supposed to work on the project for around 15 hours per week. None of the team members had any experience in business capability mapping and only minor exposure to GenAI or Natural Language Processing (NLP). They had access to scientific papers via the university library and test data as well as reference maps was provided by the consultancy firm.
The objective of the research project was defined by the consultancy firm. BCMGen-as a proof-ofconcept (PoC)-is supposed to generate a BCM based on text documents describing the client organisation. There was no specific expectation on which GenAI to be used and the decision was left to the student team. It was clarified in the beginning that the BCMGen result might still supposed to be edited by an experienced consultant (i.e. it is not the final result for the client).
The requirements were jointly documented by the consultancy firm as well as the student team and can be summarised as follows:
1. BCMGen provides a web-interface so that text documents can be uploaded for further processing 2. The result is a hierarchical BCM consisting of two levels 3. Each BC needs to have a concise description 4. The BCM needs to have a format that can subsequently be edited by a consultant 5. BC should be disjoint and the BCM needs to be complete with respect to the text documents (input) 6. The grouping of level-2 BC to level-1 BC needs to be coherent based on business concepts and activities 7. Customer-specific terminology, concepts and principles need to be reflected in the BCM. Consequently, an industry-specific or reference BCM will have to be adjusted by BCMGen accordingly.
Text documents describing individual companies have been provided by the consultancy firm and, therefore, a non-disclosure agreement was needed as they contain confidential information. Only a few of the consultancy's clients were willing to share such documents so that the team was also tasked to find appropriate input on their own (e.g. documents from public sector or published business reports). Those documents need to contain as much information as possible on what the company is doing.
The PoC was developed iteratively as the consultancy firm was simultaneously learning from the project. Intermediate results were presented at the end of each phase and relevant feedback was provided by the consultancy firm. In essence, the firm was acting like a customer in an agile project.
After the project has been kicked-off, the student team first needed to prepare a project plan and define a collaboration mode. Furthermore, they got familiar with the concepts of BC, BCM as well as GenAI. The consultancy firm provided their own training video for consultants in capability mapping. With respect to GenAI, the team members did their own research on similar projects and available technologies.
During a first meeting for defining the scope and objectives of the project, the consultancy restricted BCMGen to the public transport industry. This allowed for a focused development of a first prototype. The students needed to collect text documents and reference capability maps for this respective industry. The first version was then developed using ChatGPT and prompting methods. Such methods can be used for pre-trained models and provide contextual information as part of the prompt (i.e. query to the Large Language Model (LLM)) [20,21].
The basic idea of using prompt engineering for BCMGen is depicted in figure 1. The students needed to decide for a pre-trained LLM, in this case GPT, provided via the OpenAI Application Programming Interface (API). The students then needed to investigate how to define the prompt for the query to GPT. The prompt consists of documents provided by the user (e.g. documents on a specific public transport company) together with instructions and context information (e.g. reference models or information on capability maps). The team aimed at zero-shot prompting, hence, they focused on evaluating different instructions and context information to improve the quality of the output.
A second option has been considered: training a pre-trained LLM with respect to capability mapping. There are a couple of pre-trained LLM available (e.g. Llama) that can be be enhanced for individual needs by using additional training data [22]. This might have lead to a better quality of BCMGen if such an LLM is trained with a sufficient number of BCM. However, the idea was discarded for the following reasons:
• Time constrained: The planned project duration of three months was quite short and the team might have run into the risk of not meeting the deadline by adding an additional training phase half-way through the project. • Lack of training data: There is not sufficient training data (i.e. annotated BCM) publicly available, so that the team would have needed to collect and annotate existing reference maps. • Performance: There was not enough computing power available for training the LLM. Procuring a respective virtual server would have taken additional time. • Bias: Using existing reference BCM might have influenced the result so that BCMGen creates only maps similar to the reference maps. • Uncertainty: The impact of the additional training on the quality of BCMGen was hard to assess in advance.
There are solutions or workarounds available for some of these concerns. However, the team decided to focus on prompting rather that training their own model, given the time restriction.
The initially agreed output formats of BCMGen are comma-separated files (CSV), PDF (Portable Document Format) and JSON (JavaScript Object Notation). CSV can be imported into a spreadsheet application and then easily edited by consultants. Figure 2 shows an example output in Microsoft Excel. This tool is installed on the consultants' standard computers so that they can edit the business capability map immediately. However, the visualisation does not explicitly show the decomposition hierarchy of BC. Column B indicates the level and column E the parent capability. The level-1 capability Data Management for example consists of the level-2 capabilities Data Coordination, Data Leveraging, Data Flow Management, Data Quality Management and Data Integration Management. Column C indicates whether the corresponding BC is corporate (tier 1), core (tier 2) or support (tier 3). Some consultants raised an additional requirement for a graphical tree visualisation of the BCM. There are frameworks for graphical editors available (for example GoJS) 1 . Such frameworks allow for visualising the map and even provide measures for immediately editing it. The requirement was discarded due to time restrictions. However, as JSON is generated already, a visual tool based on GoJS can be added later on.
The architecture of the BCMGen PoC is shown in figure 3. The user interface is implemented using the Python-based Flask2 framework for web applications. A consultant may upload several documents describing the client company. In case of non-text documents, Tesseract is used for recognising and extracting text from, for example, bitmaps or PDF. The text then needs to be separated into several chunks due to size restrictions of the OpenAI API. For the first version using the OpenAI API, the chunk size was restricted to 5,000 bytes. The context (i.e. the reference models) do not need to be split as the file size is rather small.
In the phase of AI analysis the prompt will be created together with the consultants input text from the document processing stage and submitted to the OpenAI API. The prompt contains the instruction for GPT together with reference BCM. A separate list with customer specific-terminology is added so that rather abstract terms from the reference model can be replaced by a language familiar to the customer. The resulting capability list is then the input for creating the BCM.
The creation of the BCM is done in the AI generation phase. The hierarchical structure is created and each capability will be extended with a concise description. The output is then stored as PDF for human readers, CVS for further editing by consultants and JSON so that it can be processed by further applications (e.g. a visualising tool based on GoJS). The CVS output is required as BCMGen was not expected to create the final BCM.
The prompt for generating the BCM consists of several parts. The first part is used for each text chunk in order to extract capabilities from the text: Take t h e r o l e a s an e x p e r t e n t e r p r i s e a r c h i t e c t . You w i l l be p r o v i d e d a t e x t where you need t o i d e n t i f y r e l e v a n t b u s i n e s s c a p a b i l i t i e s .
Reference capabilities are provided as context in a list of capabilities ({level_1_capabilities} and {level_2_capabilities}). These instructions are then followed by the text chunk.
After all text chunks have been processed, each BC will be classified, whether it is a corporate, core or a support capability as defined in [2]. Corporate capabilities refer to managing the company or providing guidance to the organisation. They are often referred to as strategic capabilities. Core capabilities represent the core business and, therefore, value-adding functions. Support capabilities (sometimes referred to as enabling) provide support functions for the core capabilities. The classification as corporate, core or support will be done for each of the resulting capabilities:
The next step deals with creating the hierarchy of capabilities visualised in the BCM. The number of levels has been restricted to two to keep the PoC compact. The following prompt creates level-2 capabilities for each level-1 capability: Now go t h r o u g h a l l t h e r e l e v a n t c a p a b i l i t i e s and c r e a t e l e v e l 2 c a p a b i l i t i e s f o r e a c h c a p a b i l i t y Here a r e t h e l e v e l 2 c a p a b i l i t i e s you s h o u l d t a k e a s a r e f e r e n c e : { l e v e l _ 2 _ c a p a b i l i t i e s } P l e a s e be aware t h a t t h e t i e r l e v e l i s d e t e r m i n e d by t h e c a p a b i l i t y from which i t i s d e r i v e d .
Reference BC are provided as {level_2_capabilities} and certain conditions need to be ensured. The tier of a level-2 capability has to be the same as the corresponding level-1 capability in order to have a coherent BCM. Furthermore, GPT does not care for naming without a clear statement that a sub-capability is not allowed to have the same name as its parent capability. After completing this step, the JSON file can be created:
As an e x p e r t e n t e r p r i s e a r c h i t e c t , your t a s k i s t o a n a l y s e t h e l i s t e d b u s i n e s s c a p a b i l i t i e s . These c a p a b i l i t i e s s h o u l d be s t r u c t u r e d i n t o a JSON f o r m a t , r e f l e c t i n g a h i e r a r c h i c a l c a p a b i l i t y map .
C r e a t e a JSON S t r u c t u r e : S t r u c t u r e t h e e x t r a c t e d c a p a b i l i t i e s i n t o a JSON f o r m a t a s shown below . E n s u r e t h e JSON s y n t a x i s c o r r e c t and t h e r e a r e no e r r o r s . Example : { { " c a p a b i l i t i e s " : [ { { " name " : " Brand Management " , " l e v e l " : " 1 " , " t i e r " : 1 , " s u b C a p a b i l i t i e s " : [ { { " name " : " Brand D e f i n i t i o n " , " l e v e l " : " 2 " , " t i e r " : 1 , } } , { { " name " : " Brand P o r t f o l i o Management " , " l e v e l " : " 2 " , " t i e r " : 1 , } } ] } } { { " name " : " Customer Management " , " l e v e l " : " 1 " , " t i e r " : 2 , " s u b C a p a b i l i t i e s " : [ { { " name " : " Customer D e f i n i t i o n " , " l e v e l " : " 2 " , " t i e r " : 2 , } } , { { " name " : " Customer M a t c h i n g " , " l e v e l " : " 2 " , " t i e r " :
The result is the complete BCM in JSON format, but the descriptions are still missing. These are generated with the following instructions: Be a h e l p f u l a s s i s t a n t P l e a s e en h a n c e t h e JSON f i l e by i n t r o d u c i n g a new ' d e s c r i p t i o n ' f i e l d f o r e a c h e l e m e n t , c o n t a i n i n g a b r i e f one − s e n t e n c e d e s c r i p t i o n o f t h e c o r r e s p o n d i n g c a p a b i l i t y . { J S O N _ i n p u t } Make s u r e t h a t t h e JSON o u t p u t doesn ' t c o n t a i n any JSON s y n t a x e r r o r
The term {JSON_input} is replaced by the previous JSON. The result is now ready to be saved as a JSON file or exported as CVS or PDF file. The PDF version contains a graphical representation of the BCM as a tree.
BCMGen has been evaluated and developed further by two consultants for several months now. One of the major changes was the shift from OpenAI to Claude Sonnet3 provided via Amazon Web Services (AWS) due to the availability of licences in the consultancy firm. The quality of the BCM was further improved by using industry-specific reference models and domain-specific word mappings for clientspecific terminology.
The results are quite promising. Both consultants emphasize that BCMGen helps them with creating a client-specific BCM much faster than before ("a couple of hours rather than days"). The capability naming is accurate and the descriptions usually match the purpose of the capabilities. There is also an import of the BCM into the standard modelling tool (MagicDraw) of the consultancy available now.
However, a BCM still has to be revised by a senior consultant-as already intended during the project initiation. First of all, the quality of the input is decisive. BCMGen is only capable to recognise capabilities mentioned in the text documents provided by the client. Also inconsistencies, which are common in natural language texts that have been evolving over time, hamper the result's quality. Beside of this, BCMGen shows common phenomena of GenAI. In case of missing information, GenAI tends to hallucinate by inventing non-existing capabilities for filling in the gaps. According to one of the consultants, there seemed to be less hallucinations by Claude Sonnet 3.4 compared to OpenAI GPT. Nevertheless, especially Claude needs to be reminded frequently that the output has to be in English language 4 .
An example output from BCMGen (with Claude Sonnet) is shown in table 1 without any manual modifications on the contents. The capabilities have been created based on the classic book The Art of War by Sun Tzu. The structure of the table is slightly different in order to enhance readability. The middle column shows the capabilities' names and the right one their descriptions. The left column show the corresponding level-1 capability in case of a level-2 capability. The field is empty for any level-1 capability. Dedicated columns indicating the hierarchy level and tier have been omitted.
The military context is visible in the capabilities Intelligence Management or Strategic Planning. There are no Finance capabilities as financial aspects are not covered by the book. Claude recognised the military context and already deducted that the capabilities might be transferred to a business context. When asked for an explanation for the result, Claude provided the following output:
The business capabilities have been identified based on the military and espionage context of the text. These capabilities, while framed in a military context, have analogous applications in business strategy and operations.
1. Intelligence Management is crucial for gathering and analysing information about competitors and market conditions. 2. Strategic Planning involves assessing the competitive landscape and allocating resources effectively. 3. Human Resource Management focuses on recruiting, training, and deploying personnel with specialized skills. 4. Risk Management is essential for maintaining operational and information security in a competitive business environment. 5. Communication Management ensures secure and strategic dissemination of information. 6. Financial Management covers compensation and funding for operations. 7. Operational Management deals with executing strategies and planning infiltration into new markets.
BCMGen will be further refined while being used in customer projects. Some of the requirements to be considered in the future are as follows: • Improving the naming of BCs: Names do not always follow naming conventions for capabilities (e.g. "Counterintelligence" in table 1). • Implementing a graphical editor: CVS output is preferred by the consultants as they can edit it in Excel immediately. However, a graphical representation is perceived as more catchy by clients. • Training a pre-trained LLM: The current version uses a standard GenAI without any further training. There is a plan to investigate whether additional training of an existing model will improve the quality of the result.
Business Architecture Management takes time when set up to deliver value for a client organisation. Currently, Business Capability Mapping is a technique that is performed manually while performing interviews and by analysing documents. Errors might occur, capabilities can be missed or false positives can be discovered. Therefore, a prototype for generating capability maps has been implemented and evaluated as part of a research project presented in the paper at hand. Of course, problems still arise: GenAI systems, especially ChatGPT, tend to hallucinate, are generally lazy and sometimes ignore relevant aspects. Especially, determinism is yet to be achieved. Nevertheless, there is a positive impact on our role as business architecture consultants. BCMGen saves time and increases quality and, therefore, value for the client. Although yet only a PoC, we will use and extend BCMGen in our business.
The student project presented in the paper at hand already revealed some peculiarities with generating capability maps. There were only three iterations with the consultants as the project was scheduled for three months. These feedback rounds where crucial as the students did not have experience with capability mapping. Consultants needed to asses the result's quality and more feedback rounds may have improved the output. The quality depends on the GenAI system together with the prompt, the context and the input provided. While the students tried to improve the prompt with the OpenAI API, there was no further investigation with different contexts (i.e. reference models) or variations of the input (e.g. kinds of documents or length). These will have to be evaluated in follow-up projects.