=Paper=
{{Paper
|id=Vol-2464/paper8
|storemode=property
|title=Teamwork Assessment for Projects in IS Courses
|pdfUrl=https://ceur-ws.org/Vol-2464/paper8.pdf
|volume=Vol-2464
|authors=Kalinka Kaloyanova,Neli Maneva
}}
==Teamwork Assessment for Projects in IS Courses==
https://ceur-ws.org/Vol-2464/paper8.pdf
Teamwork Assessment for Projects in IS Courses
Kalinka Kaloyanova1, 2, Neli Maneva2,
1
Faculty of Mathematics and Informatics, SofiaUniversity, 5 James Bourchier blvd., 1164, Sofia,
Bulgaria
2
Institute of Mathematics and Informatics, Bulgarian Academy of Sciences,
Acad. Georgi Bonchev Str., Block 8, 1113, Sofia, Bulgaria
kkaloyanova@fmi.uni-sofia.bg, neman@math.bas.bg
Abstract. Working on projects is an important part of many courses in the area of
Information Systems (IS), where most of projects require extensive team work. To
motivate students to work on common team activities and to evaluate adequately
their personal contribution to the project different methods are used. This paper
presents an approach to teamwork assessment in projects that support teaching
several courses in an IS undergraduate program.
Keywords: Information Systems, team work, behavior competencies
1. Introduction
Working on an individual or a team project is a common practice of many courses
at undergraduate level in Faculty of Mathematics and Informatics (FMI) – Sofia
University “St Kl. Ohridski” [2], [10]. Although all authors emphasize on the
benefits of this approach to help students to obtain practical skills needed to work
in the software industry, the level of individual contribution of every student for
the team project’ success is still difficult to assess.
In this paper we present an approach for assessment of team work in
two courses that are an important part of IS undergraduate curricula in FMI –
Information Systems Aanlysis and Design and Project Management courses. А
feedback from the students, participating in the courses was required after the
finishing of the first course. The received information is analysed and used to help
students to improve their team work during the second course.
2. Methodology
The last curriculum for IS undergraduate education – IS 2010 curriculum [3]
presents several mandatory courses. Among them, the courses IS Analysis and
Design and Project Management require for students not only to learn a variety
of theoretical concepts but also to acquire some practical skills, specific for
information systems development. For both courses team projects are fully
applicable.
Copyright © 2019 for this paper by
80 its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
2.1 The Courses
IS Analysis and Design is a mandatory course since IS 2010 curriculum [2].
The course introduces concepts, processes, methods and tools needed to analyze
business requirements, to specify system requirements and to present high-level
design for information systems development.
The students, enrolled in the courses, are encouraged to take a very active
role in the learning process, working on team projects. Students teams consist of
6 to 10 members. The projects usually are focused on small information systems.
During the last several academic year employees from IT companies have been
involved in the course, playing the role of the customer.
Students work on projects performing a set of assignments, that are
preliminary defined. At the end of the course every team presents a high-level
logical design of the system characteristics – described by use case modelling and
UML diagrams, and some preliminary elements of the UI design of the systems.
As an intermediate step several models (flow, sequence and artifact models) of
the Contextual design are required [6], to help students organize the information
from the interviews with stakeholders.
The second course focuses on the organizational side of the software
development. The Project Management course introduces basic concepts of the
PMI methodology – process groups and knowledge areas, cost estimation, project
scheduling, quality management [7], [8].
As most of the assignments are result of extensive team work, it is very
important for all team members to work regularly and with full capacity. Moreover,
students continue to work in the same teams in the Project management course,
so it is important to see what they have gained from their collaboration during the
first course and how to improve upon it during the next course.
2.2 The method/approach
To evaluate the team work during the course IS Analysis and Design and to help
students understand the importance of their role in the team project we conducted
a survey among students that participated in the course IS Analysis and Design
during 2017-18 academic year. Our study covers fifty four students across
six teams. Forty seven of them continue to work in the next course – Project
Management. Forty students filled in the questionnaire.
2.3 The Questionaire
The questionaire focused on students perceptions of team work during the IS
Analysis and Design course. The questions (based on some ideas of [11]) are
listed below.
81
Questionnaire
Q1. How effectively did your team work together on this project?
Poorly Well Very Well
Q2. What are the main reasons for the difficulties if they were any in your
teamwork? (max 3)
Q3. What percentage of your team participated in all team activities of the project?
Q4. What percentage of your team was fully prepared for all team activities of
the project?
Q5. Give one example of what went right on this project.
Q6. Give one example of what went wrong on this project.
Q7. Describe a specific practical skill you learned from the team that you probably
would not have learned working alone.
Q8. Describe a theoretical concept (definition, model, method, etc.) you learned
easier working on common team task.
Q9. Describe something (concept, model, practical skill, etc.) which the members
of your team have learned from you and which they probably would not have
learned otherwise.
Q10. Which of your individual skills could be improved, so as to improve the
overall teamwork during the next project?
Q11. What will you do differently on the next project taking into account the
experience gained from this project?
3. Results and Discussion
Overall, students showed positive attitude towards their teamwork during the
course. Although the questionaire could be submitted without answers, most
students answered all eleven questions. Below we present the results (for every
question) from the interview. A brief assessment of the answers of each question
is made.
Q1. How effectively did your team work together on this project?
The Q1 is a structured question about the effectiveness of the teamwork on
the project. The students are asked to select one of the three possible answers,
evaluating the overall team performance as Poor, Well or Very well. The
distribution of answers is shown in the following Table 1:
Table 1. Outcomes of the closed question Q1
Team Q 1 Q1 Q1 – Very
-Poor -Well well
Team 1.1 - 75% 25%
Team 1.2 - 25% 75%
Team 2.1 - 100%
Team 2.2 - 71.43% 28.57%
Team 3.1 - 100%
Team 3.2 - 50% 50%
82
The lack of answer “Poor” shows that students appreciate their efforts and results
obtained and consider the projects as successful ones.
In two of the teams the members give the same answer – Well (Team 2.1) and Very
well (Team 3.1) for performance of the team. The expressed different opinions in the
other four teams can be explained with different level of criticism and precision of self-
evaluation of members, which is evident from the diversing answers to the questions Q2,
Q3 and Q4.
Q2. What are the main reasons for the difficulties if they were any in
your teamwork? (max 3)
The question allows a free response and up to 3 different reasons can be
described. The analysis of the answers identifies two main groups of reasons.
The first group of reasons considers the organization of the face- to-face
meetings of teams. More than 33% of the respondents share that it is very
difficult to find when exactly the team members can meet each other so as to
discuss some issues crucial for the project. Other aspects here are: insufficient
number of meetings, not clearly defined agenda, lack of enthusiasm and active
participation in discussions, disagreement in expressed opinions and inability to
take decisions by consensus.
The second group comprises reasons which are related to task management:
wrong definition of scope of tasks, inaccurate procedures for task assignment,
bad coordination of work, poor estimation of the time, needed for a task, late start
of the work on tasks, late submission of individual results with no enough time
these results to be further evaluated and incorporated into the final deliverables.
Q3. What percentage of your team participated in all team activities of
the project?
The purpose of the questions Q3 is to reveal whether there are free riders
among the team members. According to the definition given in [4] the free riders
are students that enjoy a benefit acquired from a collective effort, but contribute
little or nothing to the effort. The existence of free riders demotivates the rest of
the team.
Respondents have to estimate the percentage of members, participating
in all team activities. Some students refuse to answer, unable to give a precise
and reasonable answer due to the diversity of scope and complexity of the team
activities and the context in which they have been accomplished. The average of
numerical values of answers given within the team is shown in Table 2 and can be
used as an indicator of the team strength. On the base of the results teams must
find the free riders and decide how to incorporate them.
83
Table 2. Results - question Q3
Team Average Q3
Team 1.1 85%
Team 1.2 86.25%
Team 2.1 66.25%
Team 2.2 85.63%
Team 3.1 100%
Team 3.2 83.75%
Q4. What percentage of your team was fully prepared for all team
activities of the project?
The goal of the question Q4 is to assess the percentage of members who are
fully prepared for all team activities. This will outline how strong the kernel of
the team is, comprising responsible students with appropriate knowledge, skills
and experience, who can be considered as a main driving force. The average
of numerical values of answers given within the team is shown in Table 3.
The calculated values (except for Team 3.2) are too high to be realistic and we
have doubts that the question is not understood and interpreted properly by the
responders.
Table 3. Outcomes of the question 4
Team Q4 - Average
Team 1.1 61.87%
Team 1.2 55.62%
Team 2.1 42.5%
Team 2.2 55%
Team 3.1 90%
Team 3.2 10%
Q5. Give one example of what went right on this project.
Q6. Give one example of what went wrong on this project.
These two questions concern overall evaluation on the work during the
project.
As is shown in the left part of the Table 2, the most frequent answer is “Keep
deadlines”. This reveals not only the fact that students realize its significance
for the success of the project, but points also the effectiveness of the approach
selected by the lecturer to assign lower grade in case of later submission.
The second top answer is “Successful submission of the project”, showing
the satisfaction of the students that at the end and their efforts have been fruitful.
On the next position are two answers – “Good cooperation” and the more
general one – “Good teamwork style”. Such recognition of the role of the
teamwork can motivate students to try to improve it.
84
For the question 6 – “Give one example of what went wrong on this project”,
the top answers comprise two examples of time management errors and one
example for defects in deliverables. The later gives the idea lecturers to define
more checks for the quality of deliverables, introducing some new forms, e.g.
peer review.
Table 4. Q5 and Q6 answers
5.Give one example # # 6.Give one example of # #
of what went right on stu- teams what went wrong on this stu- teams
this project. dents project. dents
Successful submission 7 5 Wrong task management 3 3
of the project
Working in a team 2 2 Last minute submission 6 4
Exchange relevant 1 1 Bad tasks definition and 4 4
information and assignment
resources.
Analysis of identified 1 1 Working not in project 1 1
requirements scope
Peer review of work 1 1 Mistakes in planning 1 1
Keep deadlines 10 6 Defects in project 5 3
deliverables
Good teamwork style 6 4 Time management errors 6 5
Good design skills 2 2 Bad communications 2 2
within the team
Good communications 5 4 Unclear tasks definition 1 1
Good cooperation 6 5 Tasks reassignment 1 1
Effective meetings 5 5 Too many meetings 1 1
Enthusiasm and high 1 1 Not enough meetings 1 1
motivation
Mutual help and 3 3 Unsatisfied requirements 2 2
support
Learn from each other 2 2 Lack of help for 1 1
unfinished tasks
Good tasks assignment 3 2 Not attending lectures 1 1
Work together to 1 1 Not enough control and 1 1
achieve the stated goals peer review of the work
Feedback and listening 1 1 No tolerance in 1 1
discussions
Opinion and work 1 1 Not enough initiatives 1 1
recognition
85
Q7. Describe a specific practical skill you learned from the team that you
probably would not have learned working alone.
The majority of the responses reveal typical issues of team work – listening
to others, tolerance towards colleagues, hearing different viewpoint; agreeing
with the groups’ opinion; expressing own opinion even when the group disagrees,
distributing teamwork evenly, considering the emgagement of the team members.
One student marks the ability to share tasks with others and after that – to merge
all parts into one result.
A few students note technical difficulties, concerning specific elements of
the assignments -, presenting all system functionality with use cases, domain
modeling, diagram modeling.
Q8. Describe a theoretical concept (definition, model, method, etc.) you
learned easier working on common team task.
Q9. Describe something (concept, model, practical skill, etc.) which
the members of your team have learned from you and which they probably
would not have learned otherwise.
Questions Q8 and Q9 identify specific topics, that students learn during the
course. Both questions reveal major concepts, that are better understood in team
work. Most concepts are noted in both answers – as better explained, and as better
understood through team discussions.
Table 5. Q8 and Q9 answers
Q8 Describe a theoretical # # Q9 Describe # stu- #
concept you learned students teams something which dents teams
easier working on the members of your
common team task team have learned
from you and which
they probably would
not have learned
otherwise
Requirements 1 1 Interview preparing 2 2
specification for the
project
FURPS+ 1 1 FURPS+ 4 3
Use case model 7 3
Use case description 2 2 Use case description 6 4
UC- fully dressed 4 4 UC- fully dressed 3 2
Domain model 14 4 Domain model 3 2
CD models 1 1
Flow model 1 1 Flow model 1 1
Sequence model 1 1 Sequence model 3 2
86
Artefact model 1 1
Mockup structuring 1 1
Usability defining 1 1
UML diagrams 5 4
Sequence diagram 2 2
Class diagrams 1 1
It is interesting to mention that the most widely discussed concept is the Domain
model. In spite of the target group consisting of students in their IS bachelor degree, the
concept raises a lot of issues because of its complexity.
Q10. Which of your individual skills could be improved, so as to improve
the overall teamwork during the next project?
Q11. What will you do differently on the next project taking into account
the experience gained from this project?
The purpose of including the questions Q10 and Q11 in the questionnaire is
to give the students a chance to make some suggestions for improvements on the
base of their individual experience, gained during the project. We believe that the
acquired knowledge will ensure higher motivation and realized ability to change
for the better in the future.
The idea behind Q10 and Q11 is to provoke identification and reflection of
some soft skills, influencing teamwork [5] [9]. Each student has to think about
such skills and select those with great impact on teamwork, which are worth
mentioning as significant.
Two students (20%) didn’t answer these questions. There are some general
answers as “to apply the best practices, mentioned in the answer of the Q5” and/
or “to avoid the bad practices, mentioned in the answer of the Q6”.
The questions Q10 and Q11 are free-answer questions – the respondents
answer in their own words. So we receive a variety of answers, expressing
different opinions about some constructive changes to be made in a next project.
In order to summarize the answers of Q10 and Q11 in a systematic way, we adopt
the classification of the soft skills, proposed in [1]. Together with pure soft skills
we also consider many other things (e.g. activities, approaches, stated goals)
which are related to the teamwork in projects. In this way all given answers can
be described in the following three groups:
Personal:
Self-management, know-how for a particular task, efficiency of the
individual work and contribution, preliminary research for each task, optimized
performance, regular attendance of lectures, self-confidence, prompt decision
making, trust and friendliness towards members of the team.
87
Situational:
Finishing tasks early enough to be evaluated, time management, an even work-
load distribution, joint work on tasks, better planning for tasks, ability to analyze
the teamwork process so as to improve it, learning of the theoretical concepts
necessary for accomplishment of the project tasks, improved performance through
structuring data and handouts, asking for help and support in difficult situations,
increased impact of the teamwork, proper goal setting, controlled requirements
satisfaction, peer review as a part of internal task evaluation, selecting appropriate
level of detail in communications, thoughtful evaluation before submission of
the project deliverables.
Interpersonal:
More informal meetings to know each other better, responsiveness, listening
to opinions of other team members, defending one’s ideas and suggestions, better
efforts coordination, explanation of the made decisions, clear communication
among team members, giving examples to support proposals, conflict resolution,
tolerance in disputes, leadership skills, patience and empathy in relationships
within the team, open expression of opinions and ideas.
Each team can discuss this generalized version of the answers and decide
how many and which of them will be chosen as objects for improvement during
the continuation of the project.
4. Conclusion
In this paper an approach to teamwork assessment for projects in IS courses
has been presented. A survey has been accomplished as a first step toward
achieving the main goal: to teach students to appreciate the collaborative work
and be efficient in it.
The survey seems to be useful for both students and lecturers.
Answering questions from the questionnaire, the students have the possibility
to look upon teamwork from different points of view. Each student can create the
awareness about the significance of the teamwork. The study helps in identification
of some individual soft skills, which can be object for the improvement of a
student’s overall performance.
There are a number of benefits for lecturers, too. For some course topics,
pointed out by students as more difficult and unclear, the lecturer could decide to
change the content and style of their presentation, e.g. providing more handouts.
In the course on project management some techniques for the enhancement of
teamwork could be recommended.
We intend to continue tracing and evaluating teamwork and individual
students’ efforts during the second part of the projects. Each team will discuss
the results of the study and decide what is worth being changed. At the end of the
88
projects the same teamwork assessment will be made in order to find if there are
any positive results to be reported.
Acknowledgment. This paper is supported by Sofia University “St. Kliment
Ohridski” SRF under Contract 80-10-143/2018.
References
1. Abduwani, T.A.R. (2012) The value and development of soft skills: the case of Oman. Int.
Journal of Information Technology and Business Management, 2(1), 77-86
2. Armyanov P., A. Semerdzhiev, K. Georgiev, T. Trifonov, “The Effects of Incremental Grading
and Optional Homeworks on Student Motivation”, Proceedings of the 12th International
Technology, Education and Development Conference (INTED2018), Valencia, Spain, pp.
0618-0625 (2018)
3. Bell C., R. Mills, and K. Fadel, “An Analysis of Undergraduate Information Systems Curricula:
Adoption of the IS 2010 Curriculum Guidelines,” Communications of the Association for
Information Systems, vol. 32, no. 2, (2013)
4. Business Dictionary, http://www.businessdictionary.com/definition/free-rider.html,
5. Dewi J. Ratnaningsih, Open and Distance Education Systems: do they enhance Graduates’ Soft
Skills? The results from 2009 Universitas Terbuka Tracer Study, Open Praxis 5(4), (2013)
6. Kaloyanova K., An Implementation of the Project Approach in Teaching Information Systems
Courses, In Proceedings of the 8th International Technology, Education and Development
Conference, Valencia, Spain, pp. 7090-7096 (2014)
7. Kanabar, V. and C. Messikomer: Design and Implementation of an Adaptive Curriculum
Framework for Project Management Education, In Proceedings of IRNOP 2015, International
Research Network on Organizing by Projects, London, UK (2015)
8. Kanabar, V., K. Kaloyanova, Identifying and Embeding behaviour competencies in IS courses,
ECIS In Proceedings of the 25th European Conference on Information Systems (ECIS),
Guimarães, Portugal, June 5-10, 2017 (pp. 3115-3122). ISBN 978-0-9915567-0-0, ECIS 2017
Proceedings, http://aisel.aisnet.org/ecis2017_rip/59/(2017)
9. Maneva, N., N. Nikolova. Soft Skills Training for Software People, Proc. of the 7-th Int. conf.
CSECS, July 6-10, Dobrinishte, Bulgaria, 2011, ISSN 1313-8624, pp. 117-129 (2011)
10. Todorova M.,H. Hristov, E. Stefanova, N. Nikolova, E. Kovatcheva, Innovative Experience in
Undergraduate Education of Software Professionals - Project-Based Learning in Data Structure
and Programming, Proceedings of ICERI2010 Conference, Madrid, Spain, pp. 5141-5150
(2010)
11. Schreyer institute materials: http://www.schreyerinstitute.psu.edu/pdf/Team_Peer_Evaluation_
Examples.pdf
89