=Paper=
{{Paper
|id=Vol-3696/ELEARNING_paper_17
|storemode=property
|title=Enhancing Organizational Success: Metrics-Driven Employee Development in Software Companies
|pdfUrl=https://ceur-ws.org/Vol-3696/article_17.pdf
|volume=Vol-3696
|authors=Srdjan Atanasijević,Monika Zahar,Tatjana Atanasijević,Aleksandar Mišković
|dblpUrl=https://dblp.org/rec/conf/elearning/AtanasijevicZAM23
}}
==Enhancing Organizational Success: Metrics-Driven Employee Development in Software Companies==
https://ceur-ws.org/Vol-3696/article_17.pdf
Enhancing Organizational Success: Metrics-Driven
Employee Development in Software Companies
Srdjan Atanasijević 1, Monika Zahar 1, Tatjana Atanasijević 1 and Aleksandar Mišković 2
1
University of Kragujevac, Kragujevac, Serbia
2
Academy of Professional Studies Šumadije, Kragujevac, Serbia
Abstract
In today's competitive business environment, software companies must ensure their employees
have the skills and competencies necessary to succeed. One way to do this is to implement a
learning analytics system to track employee progress and identify areas where they need
additional training. This paper proposes a process for selecting relevant metrics for a learning
analytics system in a software company. The proposed method considers the company's
specific learning objectives, the available data types, and the stakeholders' needs who will use
the analytics results. These metrics can be utilized to calculate KPIs that align with business
performance, assisting companies in identifying areas for improvement.
To improve business outcomes such as productivity, efficiency, customer satisfaction, and
adaptivity to market needs, it is essential to enhance engineers' readiness for a suitable learning
process. The paper also discusses the challenges of identifying relevant metrics and provides
some tips for overcoming these challenges. The proposed process can assist software
companies in choosing the appropriate metrics for their learning analysis system. This helps
effectively track employee progress and identify areas requiring additional training. The focus
should be on how the training outcomes will enhance organizational business performance.
Keywords 1
Learning Analytics, Monitoring learning processes, Metrics-driven employee development
1. Introduction
The company need to align their future success and growth with the personal growth of all its
employees. One of the most critical objectives is establishing yearly goals for their employees related
to their career development and advancement. Specifically, companies expect their employees to
enhance their skills and competencies over a business year to be eligible for a new role or promotion
[1].
Companies must monitor employees' progress in three vital areas throughout the year to ensure
business success. These areas include quality work and efficiency, where employees contribute to the
company's profits through efficient project execution; career development, where employees improve
skills related to their engineering disciplines; and employee satisfaction, where employees receive an
acknowledgement from clients, team members, and management for their contributions to growth and
development [2].
Each aspect of employee growth and advancement is the focus of exceptionally specialized
professionals in the company. Project managers and delivery managers monitor the effectiveness of the
employee on the project and can identify gaps in knowledge or experience that are the basis for defining
training programs [3]. People management monitors the employee's satisfaction and develops the soft
skills he needs to effectively understand the complex social and psychological relationships in the team,
company, and client. The result of HR's work with employees is the construction of learning
Proceedings for the 14th International Conference on e-Learning 2023, September 28-29, 2023, Belgrade, Serbia
EMAIL: srdjan.atanasijevic@kg.ac.rs (A. 1); monikazahar@gmail.com (A. 2); tatjana.atanasijevic@icloud.com (A. 3); amiskovic@vts.edu.rs
(A. 4)
ORCID: 0000-0003-4414-1799 (A. 1); 0000-0001-7620-9562 (A. 2); 0000-0001-6359-1723 (A. 3); 0000-0002-7390-9886 (A. 4)
©️ 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
161
frameworks focused on understanding the framework of satisfaction and are also a valid input for the
lifelong education program to improve soft skills [4]. A career coach monitors the employee's career
progression and guides him by setting goals and objectives according to the target career progression.
As a starting point, we analyzed the metrics that are used today in the practice of student learning
on the e-learning platform. As a result, propose a set of mandatory metrics directly related to the
employee's career selling.
In addition to metrics, suggest a process that should support:
1. Improvement of existing courses
2. Cancellation of obsolete or non-updated courses
3. Introduction of new areas of knowledge
This paper proposes an effective process of structuring and monitoring learning processes in a
software company focusing on the applicability of acquired knowledge in real projects, addressing the
following research questions:
1. How can an engineer's skills, experience, and qualifications be assessed, measuring the gap
between their knowledge and readiness for the role?
2. How do we determine engineers' specific training needs and interests, regularly updating
training materials to align with industry demands and emerging technologies?
3. How do we evaluate the effectiveness of engineers' education by measuring various business
metrics that reflect their knowledge acquisition, skill development, and overall growth?
4. How can we use learning system analytics to calculate KPIs that align with business
performance? (How do we calculate business-result-oriented KPIs based on typical
analytics supplied by learning systems and correlate them with actual business
performance?)
The critical question is how to raise the readiness of engineers for an adequately configured learning
process in companies focused on improving business outcomes like productivity, efficiency, customer
satisfaction and adaptivity to market need changes.
2. The challenges of identifying relevant metrics for learning analytics in
software companies
To effectively monitor corporate education, continuous data collection and analysis must guide
decisions on improving the process. Even at the beginning of this century, after the mass adoption of
the e-learning system, proposals for the appearance of the Learning analytic cycle began. The authors
propose a process consisting of four basic steps in the paper [5]: learnings-data-metrics - interventions,
Figure 1: The preliminary learning analytics cycle proposed by Clow proposed by Clow.
When it comes to learning analytics, the third step involves taking the data collected and turning it
into clear metrics or analytical constructs that can provide deep insights into the learning process. These
insights can handle many forms, such as visual representations, detailed dashboards, lists of students
needing extra support, comparisons to previous benchmarks or cohorts, and overall summaries [5]. This
step is crucial to learning analytics and drives innovation in many areas, including dashboards,
predictive models, social network analysis, and personalized recommendations.
162
Figure 1: The preliminary learning analytics cycle proposed by Clow.
This stage is a crucial aspect of learning analytics projects.
Today, the learning analytics cycle consists of up to seven significant steps, each contributing to a
deeper understanding of the learning process. There are four distinct stages: Fetch, Know, Proceed, and
Enhance, each with one or more steps. For details, see Figure 2. The four phases of the cycle are get,
learn, continue and improve, each with one or more steps.
Fetch
Know
Proceed
Enhance
Figure 2: The four phases of the cycle are get, learn, continue and improve, each with one or more
steps.
Stage 1: Fetch (steps 1,2)
1. Data collection: In this phase, data is collected from various sources such as learning management
systems, online platforms, assessments, surveys and interactions.
Data may include student demographics, interaction patterns, quiz scores, time spent on
assignments, and social interactions.
2. Data processing: Once collected, the raw data is processed to clean and organize for analysis.
Data can be transformed into usable formats, missing values can be addressed, and irrelevant or
redundant information can be removed.
Stage 2 Know (3,4)
3. Data analysis: This is the essential stage where the data is analyzed using various techniques to
discover patterns, trends, correlations and insights. Descriptive analytics summarise what happened
(e.g. engagement rates, completion rates). Predictive analytics predicts future performance based on
historical data (e.g., identifying students at risk of dropping out). Prescriptive analytics suggests
interventions or strategies to optimize learning outcomes (e.g. recommending personalized learning
paths).
4. Interpretation and creation of insights: Data analysis results are interpreted to gain insight into
student behaviour, engagement, and performance. These insights can be used to understand factors
contributing to success or challenges in the learning process.
163
Stage 3: Proceed (5,6)
5. Making decisions and interventions: Based on the insights gained, educators, trainers and
instructional designers can make informed decisions to improve the learning experience. This may
include adapting teaching strategies, providing additional support for struggling students, or improving
course materials.
6. Evaluation and assessment: After implementing the interventions, the cycle continues with the
assessment of the impact of these changes on learning outcomes.
This evaluation phase provides feedback on whether the interventions effectively addressed the
identified problems.
Stage 4: Enhance (7)
7. Continuous improvement: Insights gained from each cycle refine future learning strategies and
interventions. This iterative process continuously improves instructional design, content delivery, and
student support.
Overall, the learning analytics cycle emphasizes data-driven decision-making and the iterative
nature of improving the learning experience. It empowers educators and trainers to adapt their
approaches, improve student engagement and optimize the learning journey [6].
The challenges of learning analytics are:
Technical challenges include data privacy and ethics, data integration, data quality and accuracy,
interpretation and contextualization, and identifying relevant metrics.
Predictive challenges: These challenges include predictive accuracy interventions and actionability
[7].
Organizational challenges include resistance to change, skill gaps, scalability, cultural shifts, and
resource allocation.
Learner challenges: These challenges include learner diversity and long-term sustainability.
The focus of this paper is on the importance of metrics in education. As mentioned earlier, selecting
appropriate and high-quality metrics is a significant technical challenge, which will be discussed in the
following chapter. The effective use of learning analytics systems to select suitable metrics can help us
find answers to the research question of this paper. This includes improving existing courses,
discontinuing outdated or irrelevant classes, and introducing new areas of knowledge.
3. Selecting appropriate metrics
To accurately evaluate and track the success of educational programs, obtaining accurate and
detailed data inputs is of utmost importance. This allows for the implementation of effective strategies
and the identification of areas that may require improvement.
What has implied the effectiveness of the entire process described in the previous chapter is high-
quality data inputs, clear and precise, which speak about the phenomena we want to measure and
monitor [8].
What are the main challenges in choosing the appropriate metric to base the analytical learning
monitoring system?
The challenge of identifying relevant metrics in learning analytics is choosing the right metrics to
analyze that reflect learning progress and outcomes. This cannot be easy because many metrics can be
used to measure learning, and the right metrics will vary depending on the educational context.
Some of the factors to consider when identifying relevant metrics include:
1. The educational goals of the institution or course.
2. The specific learning objectives that are being targeted.
3. The types of data that are available.
4. The technical capabilities of the learning analytics system.
5. The needs of the stakeholders who will be using the analytics results.
It is also essential to consider the limitations of metrics. No single metric can perfectly measure
learning; different metrics can be more or less relevant depending on the context [9]. It is essential to
use a variety of metrics to get a more complete picture of learning.
The steps before selecting relevant metrics for monitoring e-learning effectiveness:
1. Start by identifying the specific learning objectives of the course.
164
2. Consider the types of data that are available.
3. Talk to stakeholders who will be using the analytics results to get their input on the metrics
that are most important to them.
4. Be aware of the limitations of metrics and use a variety of metrics to get a more complete
picture of learning progress and outcomes.
Implementing learning analytics metrics provides valuable insights into the impact of knowledge
and skill acquisition on the effectiveness of software engineering [7].
3.1. Employee skills and competencies: a key to software company's success
A company must treat its employees with the value they bring since employees' work determines
what customers perceive and determines business success or failure. An employee might be able to be
replaced physically, but his skill sets and knowledge cannot be [10]. This is because each person
engaged brings a different skill set to the table even though the job yields the same set of skills.
Selection of relevant metrics focused on improving productivity and efficiency:
1. Completion rate: Measures the percentage of learners who complete the eLearning course.
This metric can be used to track the overall effectiveness of the course and identify areas
where improvement is needed.
2. Pass rate: Measures the percentage of learners who pass the eLearning course. This metric
can track learners' mastery of the content and identify areas where they need additional
support.
3. Time to completion: Measures the average time learners spend completing the eLearning
course. This metric can be used to track the efficiency of the course and identify areas where
it can be streamlined.
4. Engagement: Measures how engaged learners are with the eLearning course. It can be
measured by tracking factors such as time spent on the course, number of pages viewed, and
participation in activities.
5. Feedback: Measures learners' feedback on the eLearning course. This metric can improve
the course content, design, and delivery.
6. Transfer of learning: Measures how well learners can apply the knowledge and skills they
learned in the eLearning course to their work. It can be measured by tracking factors such
as performance on job-related tasks, productivity, and customer satisfaction.
7. Business impact: Measures the financial or operational benefits of the eLearning course. It
can be measured by tracking cost savings, increased revenue, or improved customer
satisfaction.
This approach allows for a comprehensive analysis of the learning process and its effect on the
software engineering process. This methodology will enable businesses and academic institutions to
understand better the correlation between knowledge acquisition and the software development cycle,
ultimately improving productivity and efficiency.
4. Mapping learning metrics to company performance results
This paper aims to explore how we can connect the metrics we obtain from e-learning analytical
systems to a company's operational goals and performance. To achieve this, we analyzed the metrics
discussed in the previous chapter, focusing on how they relate to the company's profits. To demonstrate
the company's performance, we utilized the Balanced Scorecard method introduced by Harvard
University experts Robert S. Kaplan and David P. Norton in a 1992 HBR article [10]. This approach
has significantly impacted how we monitor and manage the performance of companies in the future.
The Balanced Scorecard approach to monitoring and improving a company's performance is a
strategic management tool that provides a comprehensive view of the company's performance from four
perspectives [10]:
165
• Financial perspective: Measures how well the company achieves its financial goals, such as
profitability, return on investment, and cash flow.
• Customer perspective: Measures how well the company is meeting the needs of its
customers, such as customer satisfaction, customer loyalty, and market share.
• Internal business processes perspective: Measures how well the company executes its
internal business processes, such as operational efficiency, product quality, and time to
market.
• Learning and growth perspective: Measures the company's ability to learn and grow, such
as employee satisfaction, training and development, and innovation.
The balanced scorecard approach is valuable for monitoring and improving a company's
performance because it provides a holistic view rather than focusing solely on financial measures. This
allows companies to identify and address potential problems early on and make more informed
decisions about allocating resources.
To align the metrics gathered from learning platforms' analytical systems with the Balanced
Scorecard methodology's four perspectives, we followed a five-step process during workshops and
established it as a regular practice.
It is identifying goals and aligning the eLearning system.
1. We identified our short-term and long-term goals and considered how the eLearning system
could help us achieve them.
2. We then chose metrics for the four perspectives of the balanced scorecard (financial,
customer, internal business processes, and learning and growth) and tied them to KPIs that
directly relate to our company's strategic goals. These KPIs were measurable and actionable.
Data collection and analysis
3. Semi-annually, we collected data from our earnings analytical systems and calculated the
KPIs we monitored.
4. We then analyzed the results to identify trends and improvement areas.
Performance improvement
5. Annually, we took planned and systematic action to improve our performance, which
required an investment. This may have included developing new strategies, changing
existing processes, or investing new resources.
While working on several workshops and monitoring the impact of metrics obtained from analytical
systems, we created a table that shows a direct correlation between investment in engineering education
and company performance [11]. The effects of employee skills and competencies on a software
company's Balanced Scorecard are shown in Table 1— The impact of employee skills and competencies
on company performance: an analysis from a balanced scorecard viewpoint.
Table 1
The impact of employee skills and competencies on company performance: an analysis from a
balanced scorecard viewpoint.
Area Metrics The Effects of Metrics on
Company Performance
Finance Performance Sales Revenue Increased employee skills and
productivity can increase sales
revenue and delivery capacity.
Profit Margin Skilled employees = less
supervision, fewer errors,
higher utilization, lower
overhead costs, and better
profit margins.
Revenue per Employee Rising metric = growing
efficiency and productivity due
to employee maturity.
166
Customer & Sales Customer Acquisition Cost Growing employee skills and
(CAC) stable/decreasing CAC means
successful customer base
expansion.
Customer Retention Rate Happy employees = happy
customers! Knowledgeable
and engaged employees can
solve problems quickly and
provide outstanding service,
strengthening customer
relationships.
Customer Satisfaction Well-trained employees create
better software, which leads
to happier customers.
Time-to-Market Faster development and
launch means better
efficiency.
Internal Processes PM Efficiency Skilled employees improve
project efficiency by
understanding client needs,
estimating time/resources
accurately, collaborating well,
and solving technical
challenges quickly. This leads
to faster issue resolution and
smoother project
management.
Utilization Continuous learning and
growth reduce downtime for
employees.
R&D Investment Investing in employee
development can lead to
innovation, expertise
diversification, and exploration
of emerging technologies and
new markets.
Learning & Growth Fluctuation rate Employee learning plans are
essential for keeping top
talent, reducing turnover
costs, and ensuring team
stability.
Trained Employees Innovation and specialization
can boost engineering
services, increasing
competitiveness and revenue
while improving efficiency and
service quality.
Employee Satisfaction Regular feedback, coaching,
and career development
opportunities can enhance
167
employee satisfaction and
engagement.
Promotions Continuous learning, skill
development, and personal
growth directly impact an
individual's career promotions.
Talent pool Employees' skills and expertise
can be better matched to tasks
to maximize utilization.
The data are shown in Table 1— The impact of employee skills and competencies on company
performance: an analysis from a balanced scorecard viewpoint serves as an illustrative sample,
demonstrating the chosen approach. The specific metrics included may exhibit variability contingent
upon the goals and objectives of the observed organization. However, it is noteworthy that this
dashboard offers a comprehensive overview of critical domains that have the potential to influence
employee growth and development.
By consistently monitoring these metrics, an organization can gather valuable insights into how
employee learning and development impact financial and overall performance. This information can be
pivotal in optimizing employee development programs and initiatives [12].
For instance, when tracking the metric "time to completion," one can assess the duration it takes for
employees to acquire the skills and knowledge requisite for success in their respective roles. This data
can be instrumental in identifying areas where enhancements may be warranted in employee
development programs.
Similarly, tracking the metric "customer satisfaction" allows for an evaluation of the level of
satisfaction among customers concerning the services provided by employees. This information can aid
in pinpointing areas where the company's employee development programs can be refined to better
align with customers' needs.
A company can better understand how employee learning and development impact their
performance through diligent tracking of these and other metrics. This information can be used to make
judicious, data-driven decisions regarding optimizing employee development programs and initiatives,
aligning them with the organization's goals and objectives.
To ensure that business-related Key Performance Indicators (KPIs) are aligned with actual business
performance, it is essential to derive them from the abovementioned metrics.
Two important metrics to consider are:
1. Transfer of learning: This measures how effectively learners can apply the knowledge and
skills gained from the eLearning course to their work. It can be evaluated by monitoring
their performance on job-related tasks, productivity, and customer satisfaction.
2. Business impact: This measures the financial or operational benefits of the eLearning
course. It can be evaluated by keeping track of cost savings, increased revenue, or customer
satisfaction improvements for new and existing projects.
Appropriate metrics in learning analytics involve considering educational goals, learning objectives,
data availability, technical capabilities, and stakeholder needs. Employing a variety of metrics is
essential to obtain a comprehensive picture of learning progress and outcomes.
5. Conclusion
Identifying relevant metrics in the context of learning analytics is a challenging task that requires
careful consideration and a deep understanding of educational goals and processes. This paper
introduces an effective process for structuring and monitoring learning processes within software
companies, emphasizing the practical application of acquired knowledge in real projects.
To initiate the process of implementing learning analytics, we analyze existing metrics employed in
e-learning platforms and propose a set of mandatory metrics directly linked to employee career
168
development. Beyond metrics, we suggest a process to support the improvement of existing courses,
the elimination of obsolete content, and the introduction of new knowledge areas.
The paper discusses learning analytics challenges, including technical, predictive, organizational,
and learner-related challenges, emphasizing the significance of selecting appropriate and high-quality
metrics to assess educational effectiveness. The paper provides a comprehensive set of metrics focused
on improving productivity and efficiency in software engineering education, addressing completion
rates, pass rates, time to completion, engagement, feedback, transfer of learning, and business impact.
This paper emphasizes the critical importance of aligning a company's future success and growth
with the personal development of its employees. A key objective is the establishment of annual goals
for employees, explicitly targeting career development and advancement [12]. Employees are expected
to enhance their skills and competencies over the business year to become eligible for new roles or
promotions.
We also suggested a process to support improving existing courses, cancelling obsolete or non-
updated courses, and introducing new areas of knowledge. This process includes the following steps:
1. Collect data on the mandatory metrics. This can be done through surveys, interviews, and
performance reviews.
2. Analyze the data to identify trends and areas for improvement. For example, if the course
completion rate for a particular course is low, this may indicate that the system is too
challenging or not relevant to the needs of the employees.
3. Make changes to existing courses or develop new procedures as needed. When changing
existing systems, it is vital to consider employee feedback and ensure that the courses align
with the company's strategic goals.
4. Monitor the metrics after making changes or developing new courses to ensure the desired
results are achieved. This will help to identify any areas where further improvement is
needed.
In conclusion, this paper underscores the pivotal role of employee skills and competencies in a
software company's Balanced Scorecard and presents a comprehensive framework for aligning
organizational success with employee development through the judicious use of metrics and learning
analytics.
6. References
[1] V. Jankovic, S. Atanasijevic, T. Atanasijevic and M. Zahar, How to Establish a Project
Management Education Process in a Software Company: from Defining a Roadmap to Effective
Implementation, in: Proceedings of the 10th. International Conference on Information Society and
Technology (ICIST 2020), Kopaonik, 2020, pp. 60-63. doi:10.13140/RG.2.2.30373.04321/1.
[2] S. Atanasijevic, M. Zahar, D. Rancic, T. Atanasijevic and M. Djordjevic, Creating an Educational
Framework for Project Managers at a Software Company: a Sample Approach, in: Proceedings of
the Sinteza 2023 - International Scientific Conference on Information Technology and Data
Related Research, Belgrade, 2023, pp. 199-205. doi:10.15308/Sinteza-2023-199-205.
[3] J. Stevanovic, S. Atanasijevic, T. Atanasijevic and M. Zahar, Expanding the level of engineer
knowledge for software modeling within corporate education by active and collaborative learning,
in: Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto,
Portugal, 2020, pp. 1807-1814. doi: 10.1109/EDUCON45650.2020.9125250.
[4] S. Atanasijevic, M. Matijevic and D. Vojinovic, Web nastava: preporuke za planiranje i
implementaciju, in: XV konferencija YU INFO 2009, Kopaonik, 2009.
[5] D. Clow, The learning analytics cycle: closing the loop effectively, in: Proceedings of the 2nd.
International Conference on Learning Analytics and Knowledge - LAK '12, 2012, pp. 134–138.
doi:10.1145/2330601.2330636.
[6] F. Widyahastuti, Y. Riady and D. Fransiskus, Performance Prediction as a new Feature in e-
Learning, in: Proceedings of the 8th. International Conference on e-Learning, Kidmore End, 2017,
pp. 237-243.
169
[7] S. du Plessis-Schneider, The Need to Belong in Secondary School. A Social Work Science Study
of Austrian and Australian Students, Budrich Academic Press, Opladen, Berlin, Toronto, 2022.
[8] D. Oreški, Using descriptive and predictive learning analytics to understand student behavior at
LMS Moodle, in: Proceedings of the 13th. International Conference on eLearning (eLearning-
2022), Belgrade, 2022, pp. 14-21.
[9] M. Khalil and M. Ebner, What is Learning Analytics about? A Survey of Different Methods Used
in 2013-2015, in: Smart Learning Conference, Dubai, UAE, 2016.
[10] Kaplan, Robert S, The balanced scorecard measures that drive performance, in: Harvard business
review, 1992.
[11] S. Atanasijevic, T. Atanasijevic and M. Zahar, METRICS-DRIVEN EMPLOYEE
DEVELOPMENT IN SOFTWARE COMPANIES, Belgrade, 2023.
doi:10.13140/RG.2.2.26808.93442.
[12] J. Stevanovic, S. Atanasijevic, T. Atanasijevic and M. Zahar, Raising the skills of business analysts
– the benefits of elearning technologies in corporate education, in: Proceedings of the 10th.
International Conference on e-Learning (eLearning-2019), Belgrade, 2019, pp. 25-30.
170