=Paper=
{{Paper
|id=Vol-3645/forum9
|storemode=property
|title=Evaluating the usability of a system implemented on a DEMO based low-code
platform
|pdfUrl=https://ceur-ws.org/Vol-3645/forum9.pdf
|volume=Vol-3645
|authors=David Aveiro,Vitor Freitas
|dblpUrl=https://dblp.org/rec/conf/ifip8-1/AveiroF23
}}
==Evaluating the usability of a system implemented on a DEMO based low-code
platform==
https://ceur-ws.org/Vol-3645/forum9.pdf
Evaluating the usability of a system implemented on a
DEMO-based low-code platform
David Aveiro1,2,3 , Vítor Freitas1,3
1
ARDITI - Regional Agency for the Development of Research, Technology and Innovation, 9020-105 Funchal, Portugal
2
NOVA-LINCS, Universidade NOVA de Lisboa, Campus da Caparica, 2829-516 Caparica, Portugal
3
Faculty of Exact Sciences and Engineering, University of Madeira, Caminho da Penteada 9020-105 Funchal, Portugal
Abstract
Multiple studies have consistently identified a prevalent issue in software projects, wherein they often
fail to meet the initial expectations of end customers. The main reasons for software project failures often
include setting unrealistic goals and not having clear requirements, among other things. At the same time,
low-code platforms have gained prominence as a popular method for information systems development
in the field of technology. They present a visual and model-driven paradigm that simplifies the process
of creating software solutions. Our research team has been actively engaged in developing a DEMO-
based Low-Code Platform, DISME (Dynamic Information System Modeller and Executor), designed
for the swift deployment of information systems. In this paper, after implementing an Information
System for Cognitive Rehabilitation, NexusBRaNT, within DISME, the platform’s usability underwent
a comprehensive assessment encompassing qualitative and quantitative techniques. Employing the
Think Aloud method for qualitative evaluation yielded insightful feedback across different facets of the
platform. Participants lauded the platform for its engaging, intuitive, visually appealing, and user-friendly
attributes. The subsequent quantitative assessment, employing the System Usability Scale, corroborated
these observations, yielding an impressive overall usability score of 89.25%.
Keywords
low-code platforms, information systems, DEMO, usability, process model, fact model
1. Introduction
Many studies have highlighted that numerous software projects do not meet the initial expecta-
tions of end customers. In a comprehensive study cited from [1], which included specific case
studies, a survey involving 800 IT (Information Technology) managers [2, 3], it was revealed
that 63% of software development projects failed, 49% exceeded their budget, 47% incurred
higher maintenance costs than anticipated, and 41% fell short of fulfilling user and business
requirements. Ibraigheeth et al. analysed various reports related to project failures, ultimately
compiling a list of factors contributing to this high failure rate. These factors include setting
unrealistic project objectives, incomplete requirements, insufficient stakeholder and user in-
volvement, issues with project management and control, inadequate budget allocation, evolving
or inconsistent requirements and specifications, lack of proper planning, ineffective communica-
Companion Proceedings of the 16th IFIP WG 8.1 Working Conference on the Practice of Enterprise Modeling and the 13th
Enterprise Design and Engineering Working Conference, November 28 – December 1, 2023, Vienna, Austria
Envelope-Open daveiro@staff.uma.pt (D. Aveiro); vitor.freitas@arditi.pt (V. Freitas)
Orcid 0000-0001-6453-3648 (D. Aveiro); 0009-0002-0667-5749 (V. Freitas)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
tion, and the utilization of new technologies for which software developers lacked the necessary
experience and expertise [4].
On the flip side, information systems (IS) and software, in general, often exhibit rigidity
and present challenges when it comes to making modifications, demanding substantial effort
to align with ever-evolving requirements and regulations. Traditional software development
infrastructure necessitates the use of a multitude of tools, such as Integrated Development En-
vironments (IDEs), modelling tools, Database Management Systems (DBMS), Object-Relational
(O-R) mapping frameworks, Graphical User Interface (GUI) editors, deployment and compilation
utilities, and more. Moreover, there is a growing global trend towards the creation of dynamic
and adaptive software that can either directly execute models or autonomously generate code
based on them. Low-Code Platforms (LCPs) represent an innovative approach that consoli-
dates these tools into a single system. This innovative method automates business operations,
reducing the need to switch between different tools and, more importantly, streamlining the
integration and maintenance of consistency among the implementation artefacts produced by
various technologies, as referenced in [5, 6, 7].
An LCP serves as a versatile hub for swift application development, deployment, execution,
and management. It leverages declarative, high-level programming abstractions, which encom-
pass model-driven and metadata-based programming languages, alongside simplified one-step
deployment processes. These platforms offer comprehensive support for user interfaces, busi-
ness processes, and data services. They exhibit the potential to significantly reduce both the
time and costs associated with establishing, implementing, and sustaining various operational
processes. Furthermore, by streamlining these processes, LCPs create the opportunity to allocate
more of a company’s human resources to research and development tasks. This reallocation
ensures that the outcomes of their work more closely align with the specific needs and objectives
of the business [8, 7].
With DISME (Dynamic Information System Modeller and Executor) reaching its alpha stage,
we recently conducted an experiment of developing a projected information system to sup-
port cognitive rehabilitation, NexusBRaNT, in parallel, with a traditional software engineering
approach and with a low-code approach, in order to formally compare the needed effort and
system complexity. NexusBRaNT was designed to provide access to the BRaNT1 (Belief Revision
applied to Neurorehabilitation Therapy) project’s back-office and to assist healthcare profes-
sionals, specializing in cognitive rehabilitation, including psychologists, neuropsychologists,
and therapists. The main features of the platform are patient management, neuropsychological
testing, and cognitive training. The results, published in [9], were astonishing, as the low-code
approach needed only around 5% of the time the traditional approach took, and we found a
system complexity reduction of around 86% with DISME, roughly measured by comparing lines
of code (LoC) of the traditional approach vs DISME’s database records.
The research contributions of this paper are providing an assessment of the usability of the
developed DISME system on the health domain, through the analysis of NexusBRaNT.
User testing stands out as a widely acknowledged approach for pinpointing authentic issues
that could influence user performance and preference [10]. In order to evaluate the usability
of NexusBRaNT integrated within DISME, we enlisted a group of healthcare professionals
1
https://www.arditi.pt/en/projetos-finalizados/brant-project.html
specialized in psychology. These experts possessed a deep understanding of the modelled
processes and the implemented system.
Section 2 presents our research context, which covers the DEMO-based Low-Code Platform
and the implemented Information System to support Cognitive Rehabilitation. Section 3 presents
the study’s context with participants’ characterization and the method and procedures followed.
Section 4 presents the main contribution of this paper, namely the evaluation of the usability of
DISME using qualitative and quantitative methods. Conclusions and future work are found in
Section 5.
2. Research context
This section aims to offer readers a thorough understanding of DISME. We will start with a
comprehensive overview of DISME to ensure that readers are well-informed. Subsequently,
we will delve into an examination of the information system utilized in this study, specifically
within the health domain.
2.1. DEMO-based low-code platform
Despite the efforts made by LCPs to decrease entry barriers and facilitate application devel-
opment, there are concerns regarding their restricted usability, which acts as an inhibitor to
their adoption. Due to these usability constraints, novice citizen developers continue to rely
on IT department developers to create their applications. Furthermore, it is widely recognized
that utilizing LCPs necessitates some training, although the required amount of training is
comparatively less than that of traditional development approaches [11].
DISME is an open-source low-code software platform built upon the foundation of DEMO
methodology [12]. It facilitates the creation of organizational models and diagrams, enabling
the specification of processes, information flow, responsibilities of both human and software
elements, rules, and other organizational artefacts. DISME diverges from the usual low code
approach because, instead of generating code for a static version of the organizational processes,
it treats the organization as a living system, basing itself on the principles of the Adaptive Object
Model [13, 14]. Models can be designed, detailed, and made live immediately. To accomplish
this, DISME offers two primary functional interfaces. The first is the System Modeller, which
focuses on system specification through the use of various diagrams, forms, and tables. The
second is the System Executor, responsible for the day-to-day execution of processes and
information flow, following the specifications outlined in the System Modeller and leveraging
the real-time information system of the enterprise running it. The System Executor consists of
two primary components: the Dashboard, which serves as the user interface for interacting
with organizational tasks; and the Execution Engine, which orchestrates the execution of rules
and flow of information and processes based on the comprehensive specifications defined in
the System Modeller interface. A more detailed description of DISME and its components can
be found in [15].
Regarding the drivers for adopting LCPs discussed in [11], it is important to highlight that
these drivers apply universally to all LCPs, including DISME, which acknowledges and addresses
many of the inhibitors that hinder the adoption of LCPs. By proactively tackling these challenges,
DISME aims to facilitate a seamless and efficient adoption of LCPs within the current software
development process. In [15] we explore in detail how DISME addresses all inhibitors. In this
paper, we underline how DISME addresses the usability inhibitor by incorporating graphical
components throughout its interface to manage all model elements, including forms, drag-and-
drop components, and the diagram editor. The dashboard, which serves as a central hub for
users to carry out their organizational tasks, has been thoughtfully designed using modern
usability patterns and technologies like Bootstrap. We aimed for DISME to offer a user-friendly
interface, promoting easy navigation and seamless interaction with its various features. The
study presented in this paper aims to validate and assess the usability of DISME in terms of the
final user of an information system.
2.2. The information system to support cognitive rehabilitation
BRaNT is dedicated to creating technological tools that support at-home cognitive rehabilitation,
harnessing the power of artificial intelligence, and is also committed to offering solutions to
enhance the resilience of healthcare systems. Its primary objective is to enhance the quality of
life for patients by delivering tailored and more frequent interventions.
As part of this research project, one of the tools under development involves creating a
cognitive profile that utilizes artificial intelligence to optimize the customization of prescriptions.
This technology offers therapists a range of tailored recommendations for highly personalized
and adaptable cognitive rehabilitation, through a virtual reality-based simulation of daily life
activities.
For access to the BRaNT project’s back office, an online platform called NexusBRaNT was
projected. This web-based system is specifically designed to serve healthcare professionals spe-
cializing in cognitive rehabilitation, including psychologists, neuropsychologists, and therapists,
among others. In order to ensure that this system effectively met the fundamental requirements
of these health professionals, it was necessary to clearly specify their various requirements,
making it adaptable and comprehensive.
A comprehensive analysis was conducted to identify the relevant transactions within the
NexusBRaNT domain. This analysis was achieved using the Transaction Description Table
(TDT), as introduced in [16, 17]. The TDT offers a well-structured and in-depth description of
each task. It’s important to note that, in this context, the terms transaction and task are used
interchangeably. In the table, each task is accompanied by a detailed description, often directly
sourced from the system’s requirements. Additionally, the table specifies the conditions or rules
that need to be verified for the execution of subsequent tasks, as well as any pertinent time
constraints associated with each task, as discussed in [16, 17]. An excerpt from this table is
displayed in Figure 1.
The main features and tasks of this system include patient management, neuropsychological
assessment management, and cognitive training management. Patient management functional-
ities enable health professionals to add new patients, input and browse clinical information,
track patient progress, calculate absolute performance, and provide clinical interpretation. The
system also facilitates the management of neuropsychological assessments, allowing users to
create assessment sessions, input evaluation data from assessment instruments, view assessment
results, and provide clinical interpretation based on the findings. Furthermore, the system
Figure 1: Transaction Description Table of the NexusBRaNT case.
includes cognitive training management capabilities, enabling users to create customized cog-
nitive training programs, monitor their progress, and access detailed information about each
training session.
We then conducted an extensive analysis to specifically identify the relevant fact types within
the BRaNT domain, which encompass concept types and attribute types. These fact types
pertain to instances regularly utilized in daily operations. This effort yielded a comprehensive
and high-level perspective on all the concepts within the NexusBRaNT domain, abstracting away
from their attributes. These fact types are represented in a diagram following the Concept and
Relationships Diagram (CRD) proposed in [18, 19]. The CRD resolves any ambiguity associated
with DEMO’s Fact Model keywords like entity and property by employing the more generic
terms concept and attribute and treating both as facts. A portion of the resulting diagram is
shown in Figure 2.
In summary, we pinpointed a total of 224 fact types within the NexusBRaNT case, which
included 193 attribute types aggregated into 31 concept types. Of these 31 concept types, 20 are
specifically dedicated to Neuropsychological Assessment Instruments available for registration,
collectively comprising a total of 144 attributes. However, for the sake of diagram clarity, we
have opted to showcase only a curated selection of 6 of these concept types. These six have
been chosen to illustrate some of the supported neuropsychological assessment instruments
within NexusBRaNT and their respective concept types.
Once we had established the core concepts and their relationships, the next step was to
identify the relevant attributes linked to each concept. This process is facilitated through
the Concept Attribute Diagram (CAD), an approach akin to, or an “expansion”, of the CRD
introduced earlier, as outlined in [18, 19]. The CAD employs collapsible boxes to represent
concepts. When you expand a box, it reveals the associated attributes, each displayed on a
separate line. The left side of each line specifies the attribute’s value type, while the right side
presents the attribute’s name. For attributes with categorical value types, a list of possible
values is also provided, as described in [18, 19]. To maintain clarity, Figure 3 offers a partial
view of the CAD, featuring selected concepts with their expanded attributes.
In order to supplement the diagrams shown above with comprehensive textual information,
Figure 2: Concept and Relationships Diagram of the NexusBRaNT case.
Figure 3: Concept Attribute Diagram of the NexusBRaNT case.
we have the Fact Description Table (FDT), also proposed in [18, 19]. This table mirrors each
concept from the diagram and provides a detailed description of all the associated attributes.
The FDT functions as an exhaustive and trackable reference, establishing connections between
attributes, their origins, and their transactional history within the system. A segment of the
FDT for the NexusBRaNT case is available in Figure 4.
Within this representation of the FDT, we present an in-depth examination of each concept
alongside its corresponding attributes. The table offers a comprehensive overview of all at-
tributes linked to each concept, providing details on their scope, source, concept name, value
type, referenced concept or category values (depending on the value type), descriptions, and
the tasks responsible for creating and modifying the concept. By structuring this information
systematically, we gain valuable insights into the characteristics and interconnections of each
Figure 4: Fact Description Table of the NexusBRaNT case.
concept within the system. This table functions as an invaluable resource for comprehending
the diverse attributes associated with each concept, facilitating effective conceptual analysis.
This partial view of the complete information system model highlights the NexusBRaNT
system’s intricate information requirements. This complexity led us to choose it as the ideal
subject for our initial usability study of DISME.
3. Study context
We next present a characterization of the study’s participants, followed by the method and
procedures for the qualitative and quantitative validations performed on the usability of DISME.
3.1. Participants
To assess the perceived functionality of the usability of NexusBRaNT implemented in DISME,
we recruited a sample of health professionals in the field of psychology, with domain knowledge
of the modelled processes and implemented system (N = 10, nine females and one male, Mdn age
= 34, age range = 28-58 years). All participants have a Human and Social Sciences background.
Namely, among the participants, two are currently pursuing a Bachelor’s Degree in Psychology,
one is currently pursuing a Master’s Degree in Clinical, Health, and Well-Being Psychology,
and the remaining seven are health professionals in the field of psychology (Scholar levels:
Bachelors degree N = 2; Masters degree N = 7; and Doctoral degree N = 1). Notably, two of these
health professionals are directly associated with the BRaNT project, adding valuable expertise
and insights to the study.
3.2. Method and procedures
A comprehensive study was conducted to evaluate the usability of DISME implementing Nexus-
BRaNT. The study employed the Think Aloud method and the System Usability Scale (SUS) to
gather qualitative and quantitative data on the platform’s usability.
The Think Aloud Method is a technique used in usability testing and cognitive psychology
to gain insights into a person’s thought processes while performing a task. During the think-
aloud method, participants are asked to verbalize their thoughts, observations, and decision-
making as they navigate through a task or interact with a system. They are encouraged to
express their feelings, confusion, and any difficulties they encounter. This method allows
researchers to understand the cognitive and perceptual processes of users in real time. By
listening to participants’ verbalizations, researchers can gain valuable insights into the strategies,
assumptions, and mental models users employ while interacting with a system. This helps
identify usability issues, comprehension problems, and areas for improvement [20].
The System Usability Scale is a cost-effective and dependable measure used for evaluating the
usability of systems on a global level. It serves as a usability scale that provides reliable insights
into the user experience. The SUS is designed to assess the usability of a wide range of systems,
including software, websites, and various technological interfaces. It offers a standardized
questionnaire that users can complete to evaluate the usability of a system. By employing
the SUS, researchers, and practitioners can obtain valuable information regarding the overall
usability of a system in a straightforward and efficient manner [21].
In these individual testing sessions, participants were introduced to the DISME platform’s
fundamentals, including a brief explanation of low-code platforms. Then, the NexusBRaNT
project and its main functionalities were presented. Finally, participants were guided through
DISME’s Dashboard’s sections, providing an explanation of each section and its relevance to
the project.
After familiarizing the participants with DISME and the NexusBRaNT case and their respec-
tive functionalities, participants were initially given the opportunity for free exploration of
the DISME platform, allowing them to familiarize themselves with its interface, features, and
functionalities. Throughout the session, participants had the opportunity to ask questions and
provide feedback to ensure their understanding and engagement with the platform. Following
this exploration phase, participants were asked to perform a series of specific tasks that repre-
sented everyday activities in the NexusBRaNT domain. The tasks were Patient Registration,
Clinical Information Registration, Neuropsychological Assessment Session Creation, Neuropsy-
chological Assessment Instrument Registration, which included the automatic calculation of
the patient’s cognitive profile, and Cognitive Training Program Creation.
During the tasks, participants were encouraged to vocalize their thoughts and provide
feedback using the Think Aloud method. This allowed researchers to gain insights into the
participants’ cognitive processes, challenges encountered, and perceptions of DISME’s usability.
Valuable feedback and suggestions were provided, highlighting both areas for improvement
and the strengths of DISME.
Additionally, the System Usability Scale was administered to participants upon completion
of the tasks. The SUS questionnaire, consisting of ten statements related to usability, was rated
on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The responses were
collected and analysed to quantitatively assess participants’ perceptions of DISME’s usability.
The combination of the Think Aloud method and the SUS questionnaire yielded valuable
insights into the user experience of health professionals using a low-code platform for managing
information in the context of cognitive rehabilitation.
4. Evaluation of the usability of DISME
In this section, we will be presenting the results of the qualitative and quantitative methods
employed in this study, namely the Think Aloud and the System Usability Scale methods.
4.1. Qualitative evaluation results
The Think Aloud sessions conducted with the participants yielded valuable feedback that
emphasized several areas for improvement and positive aspects of DISME. We next present
highlights of the evaluation.
Participants provided feedback on the task organization in the platform. Some suggested
combining all tasks into a single section instead of separating them based on process initiation.
Others preferred the separation but recommended grouping tasks initiated after a process by
process type, particularly for instrument registration in assessment sessions, or by neuropsy-
chological executive function assessed. As for the tasks’ panels, there was a request to order
them in chronological order according to the system’s operation, as some participants found
the current task order based on popularity confusing. It was also mentioned by one participant
that notifications accumulated quickly, indicating the need for a more manageable notification
system. Almost all participants suggested automating the filling of normative data in neuropsy-
chological assessment instruments based on patient information, rather than manual insertion
using provided images. However, DISME’s implementation of NexusBRaNT strictly adhered
to the traditional system’s requirements, which required manual insertion. Implementing the
change in DISME’s NexusBRaNT implementation would be straightforward.
Concerning the positive aspects of the platform, participants found the platform highly en-
gaging. One participant even showed interest in using it for personal projects. They praised the
intuitive and visually appealing design, describing the forms as simple and easy to navigate. The
choice of colours received widespread appreciation, with one participant stating they wouldn’t
change them and another comparing them favourably to the traditionally implemented Nexus-
BRaNT with a dark colour scheme. Participants liked DISME’s colour range and the ability to
easily customize the Dashboard’s task colours, which is done in a straightforward manner in
the customization component and is directly applied to the Dashboard. The platform’s intuitive-
ness and responsiveness were commended for making tasks manageable and enjoyable. One
participant expressed satisfaction with the well-organized task sections, facilitating navigation
and task completion. Regarding the graphical representation of pending and executed tasks,
one participant mentioned its usefulness for colour-blind individuals, which was their case (had
difficulty seeing green and red), highlighting the description of task names when hovering over
the tasks’ process-type coded colours.
These findings from the Think Aloud sessions provide valuable insights for improving the
usability and user experience of the DISME platform. The improvement suggestions and
negative feedback highlight specific areas where enhancements can be made, such as task
organization, filtering options, automatic data filling, and the notification system. The positive
feedback reinforces the platform’s intuitiveness, user-friendly interface, visually pleasing design,
and overall ease of use, reinforcing its potential as a valuable tool in the low-code development
field. Incorporating these suggestions and addressing the identified issues will contribute to
the ongoing development and optimization of the DISME platform, ensuring, in the future, an
even better user-friendly experience, and increasing the already very high score obtained in the
System Usability Scale presented in the next section.
4.2. Quantitative validation results
The quantitative analysis of DISME was conducted using the System Usability Scale, which
consists of ten statements that participants rated on a scale from 1 (strongly disagree) to 5
(strongly agree). The collected responses were then subjected to thorough analysis, enabling
a quantitative evaluation of participants’ perceptions regarding the usability of the DISME
platform.
The first statement of the SUS used to assess the usability of DISME focused on participants’
likelihood to frequently use the system. The responses from the ten participants involved in the
study showed a high level of positive feedback and had an average response of 4.5, signifying a
strong agreement with the statement.
The analysis of the fourth question from the SUS sheds light on participants’ perceptions
regarding the need for technical support when using DISME, with an average answer of 1.8,
indicating moderately low agreement with the statement. While most participants felt comfort-
able using the platform independently, some expressed varying degrees of agreement regarding
the need for assistance, suggesting challenges or perceived complexities. Clear instructions
and user-friendly interfaces are crucial to minimizing the perceived need for external support.
Enhancements aimed at improving user self-sufficiency and reducing potential barriers to
independent usage could further enhance the platform’s overall usability.
To calculate the overall usability score, the ratings were converted into a numerical scale
using the SUS equation that can be seen in Equation 1. The obtained average score for DISME
was 89.25%. The maximum score obtained was 100%, while the minimum score was 80%, as
can be seen in Figure 5, which indicates a favourable level of usability and user satisfaction.
This score exceeded the threshold of 68%, which is typically considered above average [21].
Specific aspects of usability, such as ease of use, system complexity, and confidence in using the
platform, received positive ratings from the participants.
SUS = ((Q1-1) + (5-Q2) + (Q3-1) + (5-Q4) + (Q5-1) + (5-Q6) + (Q7-1) + (5-Q8) + (Q9-1) +
(1)
(5- Q10)) x 2,5
Comparing the usability scores of DISME implementing NexusBRaNT as a low-code plat-
form with the traditionally developed NexusBRaNT system reveals intriguing findings. DISME
obtained an average usability score of 89.25%, demonstrating a high level of usability and user sat-
isfaction. The use of low-code development techniques likely contributed to its favourable score
by providing a more intuitive and user-friendly experience, allowing for efficient navigation
and task completion.
Figure 5: System Usability Scale scores of DISME implementing NexusBRaNT.
In contrast, the traditionally developed NexusBRaNT system achieved an impressive average
usability score of 92%, ranging between 75% and 100%, as can be seen in Figure 6, with a sample
of health professionals in the field of psychology with domain knowledge of the modelled
processes and implemented system (N = 16, fifteen females and one male, Mdn age = 27.5, age
range = 21 - 48 years) [22]. The applied method was identical to the one applied to the tests in
the low-code platform, differentiating in the task of creating clinical information that did not
exist in this case. All participants also had a Human and Social Sciences background. Namely,
among the participants, six are currently pursuing a Master’s Degree in Psychology, and the
remaining ten are health professionals in the field of psychology. Notably, two of these health
Figure 6: System Usability Scores of the traditionally developed NexusBRaNT system. [22]
professionals are directly associated with BRaNT [22]. The obtained scores indicate a generally
high level of usability, although there may be some variability in the user experience. The
traditional development approach signifies a robust and well-designed system that meets the
needs and expectations of its users.
The comparison suggests that both the low-code DISME platform and the traditionally
developed NexusBRaNT system offer commendable levels of usability. While the DISME
platform benefits from its low-code approach in terms of user-friendliness, the traditionally
developed NexusBRaNT system demonstrates its effectiveness in delivering a reliable and highly
usable system. These findings highlight the potential benefits of low-code platforms in terms of
ease of use and efficiency, while also acknowledging the strengths of traditional development
methods in achieving high usability standards.
5. Conclusions and future work
In this paper, we have presented an evaluation of our research efforts in developing a DEMO-
based Low-Code Platform, DISME, for rapid implementation of information systems. Specifically,
we evaluated the usability of an Information System for Cognitive Rehabilitation, NexusBRaNT,
implemented on DISME.
The usability of DISME was evaluated using both qualitative and quantitative methods. The
qualitative evaluation through the Think Aloud method provided valuable feedback on various
aspects of the platform, including task organization, interface design, and user experience.
Participants found the platform engaging, intuitive, visually appealing, and user-friendly. Re-
finements were suggested, such as task grouping, improved filtering options, and automation
of data filling. The positive feedback reinforced the platform’s strengths and highlighted its
potential as a valuable tool in low-code development. The quantitative evaluation using the
System Usability Scale (SUS) confirmed these findings. Participants agreed that the platform was
user-friendly and supported their professional activities effectively, with the overall usability
score for DISME being calculated to be 89.25%.
As the main limitation, we identify the small sample size. Therefore, results must be inter-
preted with caution and generalizability of the findings may be limited.
In conclusion, our research has contributed to improving usability of low-code platforms
by evaluating the usability of an information system for cognitive rehabilitation implemented
on a DEMO-based low-code platform. The results demonstrate the potential of DISME to
allow impressive gains of effectiveness in software development, while achieving high levels of
usability.
As future work we foresee additional evaluations in the context of new implementations in
other scenarios, continuous refinement of DISME, including support for collaborative modelling
and development. With ongoing efforts, we envision DISME becoming an invaluable tool for
rapidly developing effective and user-friendly information systems in various domains.
References
[1] S. Dalal, R. S. Chhillar, Case studies of most common and severe types of software system
failure, International Journal of Advanced Research in Computer Science and Software
Engineering 2 (2012) 341–347.
[2] F. Shull, V. Basili, B. Boehm, A. Brown, P. Costa, M. Lindvall, D. Port, I. Rus, R. Tesoriero,
M. Zelkowitz, What we have learned about fighting defects, in: Proceedings Eighth IEEE
Symposium on Software Metrics, 2002, pp. 249–258. doi:10.1109/METRIC.2002.1011343 .
[3] A. Zeller, R. Hildebrandt, Simplifying and isolating failure-inducing input, IEEE Transac-
tions on Software Engineering 28 (2002) 183–200. doi:10.1109/32.988498 .
[4] M. Ibraigheeth, S. A. Fadzli, Core Factors for Software Projects Success, JOIV : International
Journal on Informatics Visualization 3 (2019) 69–74. doi:10.30630/joiv.3.1.217 .
[5] A. C. Bock, U. Frank, Low-Code platform, Business & Information Systems Engineering
63 (2021) 733–740.
[6] P. M. Gomes, M. A. Brito, Low-code development platforms: A descriptive study, in: 2022
17th Iberian Conference on Information Systems and Technologies (CISTI), 2022, pp. 1–4.
doi:10.23919/CISTI54924.2022.9820354 .
[7] R. Waszkowski, Low-code platform for automating business processes in manufacturing,
IFAC-PapersOnLine 52 (2019) 376–381. doi:https://doi.org/10.1016/j.ifacol.2019.
10.060 , 13th IFAC Workshop on Intelligent Manufacturing Systems IMS 2019.
[8] P. Vincent, Y. Natis, K. Iijima, J. Wong, S. Ray, A. Jain, A. Leow, Magic quadrant for
enterprise low-code application platforms, 2020.
[9] D. Aveiro, V. Freitas, E. Cunha, F. Quintal, Y. Almeida, Traditional vs. low-code de-
velopment: comparing needed effort and system complexity in the nexusbrant exper-
iment, in: 2023 IEEE 25th Conference on Business Informatics (CBI), 2023, pp. 1–10.
doi:10.1109/CBI58679.2023.10187470 .
[10] E. Wang, B. Caldwell, An empirical study of usability testing: Heuris-
tic evaluation vs. user testing, Proceedings of the Human Factors and Er-
gonomics Society Annual Meeting (2002) 774–778. doi:10.1177/154193120204600802 .
arXiv:https://doi.org/10.1177/154193120204600802 .
[11] S. Käss, S. Strahringer, M. Westner, Drivers and inhibitors of low code development platform
adoption, in: 2022 IEEE 24th Conference on Business Informatics (CBI), volume 01, 2022,
pp. 196–205. doi:10.1109/CBI54897.2022.00028 .
[12] J. L. G. Dietz, H. B. F. Mulder, The DEMO Methodology, Springer International Publishing,
Cham, 2020, pp. 261–299. doi:10.1007/978- 3- 030- 38854- 6\_12 .
[13] J. W. Yoder, F. Balaguer, R. Johnson, Architecture and design of adaptive object-models,
SIGPLAN Not. 36 (2001) 50–60. doi:10.1145/583960.583966 .
[14] J. W. Yoder, R. Johnson, The Adaptive Object-Model Architectural Style, Springer US,
Boston, MA, 2002, pp. 3–27. doi:10.1007/978- 0- 387- 35607- 5\_1 .
[15] D. Aveiro, V. Freitas, A new action meta-model and grammar for a demo based low-code
platform rules processing engine, in: Advances in Enterprise Engineering XVI, Springer
Nature Switzerland, Cham, 2023, pp. 33–52.
[16] D. Pinto, D. Aveiro, D. Pacheco, B. Gouveia, D. Gouveia, Validation of demo’s conciseness
quality and proposal of improvements to the process model, in: Advances in Enterprise
Engineering XIV, Springer International Publishing, Cham, 2021, pp. 133–152.
[17] D. Pacheco, D. Aveiro, D. Pinto, B. Gouveia, Towards the x-theory: An evaluation of
the perceived quality and functionality of demo’s process model, in: D. Aveiro, H. A.
Proper, S. Guerreiro, M. de Vries (Eds.), Advances in Enterprise Engineering XV, Springer
International Publishing, Cham, 2022, pp. 129–148.
[18] B. Gouveia, D. Aveiro, D. Pacheco, D. Pinto, D. Gouveia, Fact Model in DEMO - Urban
Law Case and Proposal of Representation Improvements, 2021, pp. 173–190. doi:10.1007/
978- 3- 030- 74196- 9\_10 .
[19] D. Pacheco, D. Aveiro, B. Gouveia, D. Pinto, Evaluation of the perceived quality and
functionality of fact model diagrams in demo, in: D. Aveiro, H. A. Proper, S. Guerreiro,
M. de Vries (Eds.), Advances in Enterprise Engineering XV, Springer International Publish-
ing, Cham, 2022, pp. 114–128.
[20] D. W. Eccles, G. Arsal, The think aloud method: what is it and how do i use it?, Qualitative
Research in Sport, Exercise and Health 9 (2017) 514–531. doi:10.1080/2159676X.2017.
1331501 .
[21] J. Brooke, Sus: A quick and dirty usability scale, Usability Eval. Ind. 189 (1995).
[22] E. Cunha, Nexus brant backoffice para brant, 2023. URL: http://hdl.handle.net/10400.13/
5102.