During a software project lifecycle, the estimation of cost, effort, and schedule are essential aspects since they are used to allocate resources, plan product releases, and negotiate payments. It has been demonstrated that the dominant variable to estimate these aspects is the software functional size. One of the methods to measure software is COSMIC (ISO/IEC 19761). The COSMIC organization has published guidelines to help the measurement process for some types of software applications like real-time software, SOA, business applications, etc. However, there is a lack of information on how to measure some new or trending technologies. One of these trending technologies is Robotic Process Automation, also known as RPA. Even though the RPA market has been growing rapidly, there has not been an approach of using formal metrics to measure this kind of software. This paper illustrates the process of measuring an RPA software using the COSMIC method.
Companies naturally are looking to be more efficient and save costs. This kind of
situation causes technological innovations to arise. Robotic Process Automation (RPA) is
one of these innovations which, in terms of return on investment, most case studies
show positive results. Šimek & Šperka [
According to van der Aalst, Bichler & Heinzl [
RPA can be cataloged as a type of software implementing a type of functionality,
maybe in a distinct way of traditional development. During the lifecycle of any software
development project, including RPA, the use of formal metrics to estimate cost, effort,
and schedule is relevant since these aspects are used to allocate resources, plan product
releases and negotiate payments. However, currently in the software development
activities, the lack of formal metrics and standards in the estimation of cost and schedule
is a constant in the industry. The dominant variable to estimate effort, cost, or time is
the software's size to be developed, as has been described by several studies [
The size of the software can be measured in many ways and at different points in a software project life cycle. Up to now, there are five standards of software functional size measurement methods (FSMM). The ISO/IEC 19761 COSMIC method is the only standard belonging to the second generation, including several domains of use, like MIS (Management Information Systems), real-time infrastructure, etc., and also solving most of the problems that come with the FSMM of the 1st generation.
The COSMIC method arises in the year 2000; until now, in the literature reviewed, there is no documentation about the use of the COSMIC method to measure or estimate RPA software, as could be reviewed in the knowledge base of the COSMIC site (www.cosmic-sizing.org).
Any approach that helps determine if the RPA software can be measured with COSMIC would be useful in order to generate more formal estimations and compare the productivity behavior of this type of project against the other types of software in distinct domains like MIS.
The article presents a case study that is organized as follows: Section 2 explains the characteristics of RPA and how the industry has grown. In this section, an overview of COSMIC (ISO/IEC 19761) is presented. Section 3 explains the RPA robot used in this paper, along with the Robot’s environment, the process that has been automated, and the application of the COSMIC method to measure the RPA functionality. Section 4 presents a summary of the study, validity threats and suggestions for future work. 2 2.1
RPA
Redesigning old information systems or designing new information systems is often
costly in an automation project. Companies have been trying to automate routine tasks
and business processes for a long time, often without proper Return on Investment
(ROI). This kind of problem caused RPA technology to arise [
RPA tools’ aim is to reduce monotonous tasks on employees so that workers have
more time for value-added tasks [
According to van der Aalst et al. [
Furthermore, the relationship between the RPA Robot and humans is so that when a
case turns out to be exceptional, the RPA Robot may hand over the case to a human.
Syed et al. [
Aguirre & Rodriguez [
There are a few papers, most of them presented in the form of a case study, that show
the productivity and cost-saving results of implementing RPA. For example, in the case
study presented by Aguirre & Rodriguez [
The positive results, along with the RPA trend, have caused the RPA market to grow.
Forrester's estimated growth is that the RPA market will reach US $2.9 billion by 2021
and US $12 billion by 2023. There are approximately 15 main vendors of RPA
solutions on the market [
Considering the feature where RPA creates the functionality required easily by
dragging, dropping, and linking icons, a natural source of cost savings could be the
productivity increase, because the productivity could be defined as "the ratio of the number of
products delivered by a process to the number of inputs" [
In the software engineering field, various ISO/IEC norms have been developed oriented to measurement. Many years verifying various approaches showed that the concept of software size measurement based on its functionality, is a key attribute to the user and the technical team.
The ISO/IEC 14143 norm [
“Functional size is defined as the size of the software derived by quantifying the
Functional User Requirements” [
“Functional User Requirements (FUR) stands for a sub-set of the User Requirements
describing what the software does, in terms of tasks and services” [
After about 40 years of improving various software FSM techniques, five of them
(out of over 20) have been acknowledged by the ISO/IEC as conforming the rules laid
down in the ISO/IEC 14143 norm and have taken on the form of the following
standards: ISO/IEC 20926 (IFPUG), ISO/IEC 20968 (MKII), ISO/IEC 24570 (NESMA),
ISO 29881 (FISMA) and ISO/IEC 19761 (COSMIC) [
COSMIC introduces its own homologated and standardized measure unit called Cosmic Function Point (CFP). 1 CFP represents the size of one data movement (Entry, Exit, Read, or Write). Therefore, functional size can be measured by counting the number of handled data movements (CFP). More information about the COSMIC method can be found in the COSMIC website (www.cosmic-sizing.org).
The official COSMIC website provides guidelines to apply the COSMIC method in
specific situations. Some of the situations covered by COSMIC are real-time software
[
This study is based on an RPA Robot that has been developed as a proof of concept (PoC). The Robot involves two companies, due to confidentiality issues, the name of the client company is not presented. The company Robot Digital TI (company A) was developing and providing RPA solutions to a second company (company B), a banking organization. Company A developed a project implementing RPA in the operations department of company B. Company A had communication with company B to understand the process that has been automated. When the process was well understood, a BPMN diagram that models the business process was created.
The input to develop this paper was the BPMN diagram (Figure 3), received from Company A; several interviews were developed with this organization’s employees to understand each of the tasks. We also saw a demonstration of how the robot performs these tasks. With all this we had enough information to identify FUR at a functional process level, considering the RPA as a software that will act as a functional user for the other applications involved.
Having all the information allowed us to perform the 3 phases of the cosmic method following the principles from the COSMIC Software Context Model and the COSMIC Generic Software Model. Firstly, during the measurement strategy phase, we identified the FUR and the functional users of the robot. The mapping phase allowed us to identify the functional process derived from the FUR, along with data groups and the data movements of the functional process. Finally, in the measurement phase, the cosmic unit of measurement was applied to obtain the functional size of the software. 3.2
The process named fixed-term deposit was chosen to be automated as a PoC because the activities that belong to this are repetitive, and the parts of the process are well structured. It is worth noticing that not all the variants of the fixed-term deposit nor all the activities of the process were automated. This Robot covers the simplest case of a fixed-term deposit.
The process diagram is presented in figure 3, showing 17 activities the operations department performed during the fixed-term deposit process. Out of these 17 tasks, 15 are automated by the Robot (marked with a green tick); the two remaining tasks (marked with a red cross) are not part of the PoC, so they are not taken into account for this case study. This process starts when, at a certain time of the day, the robot starts waiting for the business department to send a registration request.
The BPMN diagram presented in figure 3 shows the FTD business process of company B at a business level, that means, it does not have the granularity level to develop a measurement. Therefore, some subprocesses, like logging in to their systems, getting some specific data, downloading PDF files, etc., are missing in figure 3. The whole process, as it is done by the robot, is explained in section 3.3.
The RPA Robot has been developed using the “Automation Anywhere” tool and will be called FTD (acronym of “fixed-term deposit”). Three software pieces are essential to understand the Robot’s functionality: the CRM System, the banking system, and a server that runs on CITRIX. 3.3
According to the COSMIC website, the COSMIC Functional Sizing Methodology “is based on fundamental software principles, and is therefore suited to all types of software”. Therefore, it can be applicable to measure RPA robots. COSMIC is based on representing the functionality of the software, that is mapping the functionality to four types of data movements that could apply to any software (Entry,eXit,Write,Read).
The approach we have taken to measure RPA robots is that when the robot manipulates the user interface, as a human would, by sending or receiving data groups, these manipulations are classified as entries and exits. According to the COSMIC manual, a functional user is a sender and/or intended recipient of data of a piece of software. When the robot writes and reads from other software’s user interface, the RPA robot commits to the definition of functional user for the other software.
Fig. 3. BPMN diagram showing the fixed-term deposit process. The tasks that have been automated by the Robot are marked with a green tick. The tasks that have not been automated, therefore not taken into account in this case study, are marked with a red cross. The diagram has numbers in circles/ovals to identify the tasks automated by the robot, and it has letters in squares to identify decisions automated by the robot.
The robot reads and writes in the user interface, this could cause confusion when having, for example, an “Entry” type data movement and the verb “Read” on the subprocess description. To avoid this, it has been decided not to use the “write” and “read” verbs in the subprocess descriptions presented in Table 1, nor in the functional process description. Instead, when talking about the robot writing in the user interface, we use the verb “send”; when talking about the robot reading from the user interface, we use the verbs “request” and “receive”.
Below, Figure 4, presents the visualization of the RPA interacting with other
software as defined by COSMIC Manual [
Fig. 4. Comparison between RPA functional user and human functional user when executing the same functionality
The purpose of this measurement is to measure the FTD software using the Application Layer approach, that means “as a whole”, to enable the possibility to make a benchmark to compare the effort required to develop these applications against traditional web applications, and for estimation purposes. As presented in this paper, to show that the functionality provided by an RPA software can be measured using COSMIC (ISO/IEC 19761).
The scope of the measurement is the complete functionality of the FTD software, which only includes one functional process called “Fixed Term Deposit FUR”.
The COSMIC context diagram will be used in the strategy phase to help identify and
show the functional users of the software to be measured. The context diagram, defined
by the COSMIC method [
The FTD Robot has four functional users: • CRM: The Customer Relationship Management System that company B uses. • Banking System: The system that company B uses to do banking transactions and operations. • Citrix: A server running on CITRIX operative system, where company B uploads some of their archives. • Clock: The clock that notifies FTD to start working.
RPA does not store any transactional data; this leads to not having read or written data movements when measuring this kind of software.
In this case, the tasks that the robot should do were presented with a BPMN diagram (Figure 3), which shows the business view of how the Fixed Term Deposit process works, along with a detailed explanation of how humans do this process. The BPMN diagram, along with the explanation, allows the development team to have FUR at a functional process level of granularity, considering that RPA uses an “outside-in” approach. Functional Process – Fixed Term Deposit FUR
At a particular time, the Robot receives a clock tick to start the process. The Robot accesses the CITRIX server and then enters the CRM System. The Robot is requesting the CRM, every specific time, for a new client number to process. When there is a client number, the Robot receives it. The Robot requests the CRM for the client’s request document by sending the client number. The Robot receives the client’s request document (in PDF format) from the CRM.
The Robot obtains the data through the request document via OCR (Optical Character Recognition). Then, it verifies that the information supplied by the client is valid and saves the request document data in variables. It also sends the client’s request document to CITRIX to be saved.
After sending the document to CITRIX, the Robot requests the banking system the client’s information along with the account balance by sending the client number. The Robots receive the client’s information along with the account balance. The Robot validates if the balance (included in the account data) is enough for it to continue the process. If this validation is successful, the Robot sends the term and amount to the banking system, after this, it sends the subproduct (all of these contained in the client’s request document) to the banking system. It also sends the account data to CITRIX, so a new folder is created (the folder name has information from the account data).
Then it requests the client’s subaccounts by sending the client data to the banking system. It receives the client’s subaccounts from the banking system. The Robot selects and sends two subaccounts, the subaccount where the money generated by the investment will be liquidated, and the subaccount from where the funds for the investment will be taken.
After all this, the Robot requests the banking system for all the information saved by the system during this process (Client’s request information). It receives all the information that the banking system has saved. The Robot verifies that the saved information is correct. The Robot confirms the operation, once the information is verified.
Following the operation's confirmation, the Robot requests the operation voucher (in PDF format) to the banking system by sending the client’s information. The Robot receives the operation voucher (in PDF format). Then it requests the banking system for the account status by sending the client information. The Robot receives the account status from the banking system, then verifies that the fixed term deposit transactions are present in the account status. The Robot sends the operation voucher document to CITRIX to be saved. The Robot sends the operation voucher to the CRM to be saved as an attachment.
During all the process, the Robot sends CITRIX the instructions that it executes, so they are saved in a log.
Knowing the information presented in table 1, it is correct to say that the FTD functional size is 25 CFP.
Table 1 has 28 rows (1 row per subprocess) and 5 columns. The first column helps identify every subprocess. The second column shows the functional users related to each subprocess, from the RPA Robot’s viewpoint. The third column contains the subprocesses details. The fourth column shows the data groups presented in each subprocess. Finally, the fifth column shows the data movement types related to every subprocess.
ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14
tional
Clock CITRIX CRM CRM CRM CRM CRM
Banking System
ing System CITRIX 0 1 1 1 1 2 2 3 3 4 4 A 7,8 5
ument (in PDF format).
Receive the client’s request document (in PDF format).
Get the data from the request document via OCR. Verify that the information supplied by the client is valid and save the client’srequest data in variables. Send the client’s request document to CITRIX to be saved. Request the client’s information along with the account balance.
tials Citrix Credentials CRM Client
Number Client Number Client Solicitude n/a
along with the account balance.
ment
E n/a X X X E X E X X X X Banking System Banking System Banking System Banking System Banking System Banking System Banking System Banking System Banking System Banking System CITRIX CRM 11,12 9 9 9 10 11 14 14 13 13 15 15
where the money generated by the investment will be liquidated. Select and send the subaccount from where the funds for the investment will be taken.
Request the information that the banking system has saved during the process.
Receive the information that was captured and verify if it is correct. Confirm the operation. Request to download the operation voucher (in PDF format).
ify that the fixed term deposit transactions are present.
Send the operation voucher to CITRIX to be saved.
Send the operation voucher so it is saved as an attachment in CRM.
Send the instructions executed so that CITRIX can save it in a log.
E X X X E X E X X X 24 CFP v5.0
It is worth noticing that row 27 has “n/a” in the BPMN activity column, this is because, even though the robot does this task, it is not related to any activity presented in the BPMN Diagram.
The COSMIC method is applied to get the functional size value of FTD by following the rule that says Functional Size = E + X + W + R. The functional size of the FTD software is the sum of all the data movements from the Fixed Term Deposit FUR functional process. The FTD software receives 8 Entries, sends 18 exits and has no writing or reading. This gives FTD a functional size of 24 CFP v5.0. 4
The RPA market has been growing, and according to the market projections, it will continue growing. Thus, having a correct estimation for this kind of project will become more and more critical. Using COSMIC functional size to help estimate is a step forward to achieving this goal of having better estimations.
In this paper, an RPA robot from a banking operation was used to illustrate how the measurement of an RPA software can be made by using the COSMIC method, showing the appliance of the COSMIC method measurement process to this kind of software.
The use of the COSMIC method to measure the RPA software was successful. The FTD Robot has a functional size of 24 CFP v5.0. With the use of the COSMIC method, a productivity model for RPA projects can be generated, so that productivity can be compared with other kinds of software.
This paper obtains the FUR from a BPMN diagram generated by observing how a human does a certain task, which is the approach applied by the company that develops the RPA robot. The process should be well defined and monotonous, this leads to having certain facility to obtain the FUR at a functional process granularity. 4.1
This paper presents a case study where the FUR were presented by a BPMN diagram because that is how company A works; there could be other ways to obtain FUR, but these other approaches are out of the scope for this work.
The way of making a BPMN diagram depends from person to person, so there can be tasks in the diagram that do not have any effect on the measurement process, or vice versa, tasks that are not explicitly shown in the BPMN diagram but do have effect on the measurement. This can cause confusion to the measurer and errors during the measurement process.
Some tasks caused some confusion, for example, task 6 in figure 7 does not have any effect in the measurement process. In addition, row 25 in table 1 says “Send the operation voucher to CITRIX to be saved”, but this task is not shown in the BPMN diagram, this FUR was only obtainable during the interviews performed to the employees of company A. Clear FUR are required to get a significant measure of the software using COSMIC, the same situation is presented in traditional software applications. 4.2
As future work, it could be interesting to compare RPA technology to other kinds of automation technologies in terms of productivity (CFP/Effort). It is being mentioned that RPA projects are easier, cheaper, and faster to implement than other forms of automation, maybe because of the nature of how RPA software is developed (by dragging, dropping, and linking icons), and it has an “outside-in” approach. The comparison could approximate how much a software development company can save in costs by increasing productivity by developing automation projects using RPA instead of other automation forms.
Another proposal for future work is a case study where multiple RPA software are measured, and information about the projects of these software (effort, cost, time) is collected. This study could be beneficial to the companies that are providing RPA solutions, as this could show how to generate productivity models applied to this kind of software in order to estimate effort and costs, obtaining a correlation value between the functional size and these two aspects.