Group formation is a critical activity in the collaborative learning process. The literature presents several automatic algorithms to perform this task. However, it fails to compare the results concerning experienced instructors' decisions. In this context, this paper proposes the application of an automatic group formation algorithm in real-world settings. The proposed approach was compared with a manual approach performed by an instructor with ten years of experience on this task. The results proved the potential of the proposed approach as it reached more than 80% of similarity with the groups formed manually by the instructor. The practical implications of the proposed approach are further discussed.
There are currently a large number of studies relating to collaborative learning methodology in
an educational context [
The problem of forming groups, due to its complexity, depends on intelligent algorithms so
that the process of grouping students with diferent profiles is efective and results in even more
efective learning [
Due to the possibility related to the adoption of diferent objective functions and the fact that this algorithm was evaluated in real world scenarios, this work focuses on bringing the results of a study that evaluated the efectiveness of the proposed algorithm in [ 16] called Algorithm Group Formation Multiobjective Optimization (GFMOA). This aimed to test the extent to which the algorithm can replicate the groups created by an expert instructor, using a set of data collected in an on-site master’s course in software engineering.
This work is organized as follows: Section 2 introduces the problem formulation. Section 3 presents the experimental methodology used to evaluate the proposed method. Section 4 presents the results achieved. Finally, Section 5 highlights conclusions and future work.
Based on a set of criteria, the automated group formation process seeks to obtain the best combination of students in each group and in the class as a whole. Given a finite set with students, ⃗ = (1, 2, ..., ), to be divided into groups, where < and is divisible by . Each student is represented by another finite set of characteristics used to describe the student profile, · = (1, 2, ...). A solution that is said to be valid for the problem of forming groups is represented as a permutation of ⃗, where each set of / positions represents individuals belonging to a certain group. So we can say that we have two types of way of seeing the students as a whole: a) One of a physical way in which we see the students themselves; b) another in a logical way in which we use a start and an end for each logical subset. The following example, Figure 1, considers a possible candidate solution with = 12 and = 3 and shows the visual representation of each group in the class.
As shown in the example the solution has a permutation of 12 students, where 1 represents 1, 2 represents 2 and so on. The first 4 elements of the solution correspond to the students allocated in the first class; then, the next 4 elements are in the second group; and, ifnally, the others are part of the third group, thus representing the logical way to analyze the given set of students.
In order to evaluate the proposed approach we used a dataset containing four oferings (two times in the first semester of 2019, one time in the second semester of 2019, and first semester of 2020) of a professional master-level course in software engineering ofered in a face-to-face setting, at a Brazilian private university between 2019 and 2020. As part of the assessment for this program (60% of the credits to conclude the master), the students were divided into groups to develop a practical software engineering project using a problem-based learning methodology. Without completing this project, a student is not able to graduate. In those four oferings, a total of 93 students were divided into 14 groups. The diferent runs of the course had the numbers present on table 1.
The data acquired from the students to generate their profiles, which are then employed by the algorithm, is presented in table 2. An online questionnaire was used to gather data throughout the first week of the course. Students were asked for information such as their location of residence, information about their graduating course, professional profile, time availability, and MBTI categorization.These materials were incorporated into the GFMOA. Because GFMOA exclusively employs numeric resources, we use the ordinal encoding method to turn categorical characteristics into ordinal numbers [19].
The teacher utilized three criteria to establish the final groupings based on the information gathered from the students: 1. Divide the students into groups based on the professional profile stated, balancing the various talents. This criterion ensures a diversity of talents for the project’s progress. 2. Consider the experience time and graduation year when creating groups with various experiences to avoid an imbalance in professional experience levels. 3. Adoption of the MBTI profile and availability so that organizations may better understand each member’s psychological profile and commitment to the project throughout.
Since 2007, the instructor’s group creation process has been applied, and the groups have proven to be efective in terms of student performance (final grades). Thus, automating the replication of this technology would be a critical step toward its scalability.
The multi-objective method presented in [16] was used in this work for the automated group construction of face-to-face classes in a postgraduate degree in software engineering. The goal is to see if the GFMOA can construct groups with similar characteristics to those produced by the manual technique. The GFMOA seeks to maximize two goals: inter-homogeneity between groups and intra-heterogeneity within each group. This is accomplished by taking into consideration the same factors used by the teacher (see section 3.2). The sections that follow describe the key aspects of the algorithm that was employed.
The measure of inter-group homogeneity is obtained through the steps described below: the matrix represents the course run, in which each line represents a student in the class, and the columns correspond to their features, normalized between 0 and 1. Let define = (1, 2, ..., ), where is the average of the values for each feature among all the students.
Similarly, given the matrix , whose rows represent each students of the -th group, and ⃗ = (,1, ,2, ..., ,) the average of the values each features in the group. The value of inter-group homogeneity between the groups is given by: = ∑︁[(1 − ,1)2 + (2 − ,2)2 + ... + ( − ,)2].
=1 (1) corresponds to the sum of the squares of the diferences between and , so that the lower the value of , the greater the similarity of each group in relation to the others.
Let be a square matrix of dimensions student × student, representing the distances (Euclidean) of each student in relation to the others. That is, each position , of is given by:
, = √2︀(,1 − ,1)2 + (,2 − ,2)2 + ... + (, − ,)2.
Where , is the average of the distances of each student in a group regarding colleagues on the same team. The value of the total intra-group heterogeneity is given by the average of the distances of each group, that is: = ∑︀=1 , .
(2) (3)
In this study, we used the same algorithm (NSGA-II) and parameters validated in [16] and described in table 3, where it presents the details of each parameter adopted. The experiments were performed in the following set up: 1. Operational system Arch Linux 2. CPU Intel Core i5-7300HQ 3. 8GB de RAM 4. GPU GeForce GTX 1050 de 4GB.
The findings were categorized into three sections: (i) Group formation in relation to the professional profile; (ii) Group formation in relation to professional experience; (iii) Group formation in relation to the MBTI.
In this study we considered characteristics such as professional, psychological and experience profile. In addition, we also evaluate the GFMOA in terms of processing time. The experiment carried out in this article considered classes between 14 and 30 students, with the GFMOA taking 120 seconds to automatically form groups for the class with the largest number of students, which enhances the use of the algorithm in practice.
Table 4 displays the groupings produced by the automatic and manual procedures when the professional profile is taken into account. Each student in the group is represented by their profile, which includes Developer (DEV), Systems Analyst (ANA), Test Analyst (TES), and Project Manager (PM) (PM).
As shown in Table 4, the automatic formation was able to generate heterogeneous groups internally and homogeneous among themselves, in all classes. Regarding the similarity with the groups formed by the manual approach, those underlined stretches represent the identical elements between the groups considering their components’ profile. In 2019.1a, the instructor allowed groups of diferent sizes. As the GFMOA formed groups of the same size (5 groups of 6 students), the variability allowed in manual formation penalized similarity in two cases (50.0% and 66.6% similarity). However, the other groups showed similarities above 70%, reaching up to 85.7%. In the case of 2019.1b and 2019.2, groups of the same size were formed. Consequently, GFMOA’s performance was better, achieving 85.7% similarity across all groups formed. It means that the algorithm made the wrong choice in only one of the seven possible profiles per group. In 2020.1, of the four groups formed, three showed 85.7% similarity with manual formation. Only one group presented two divergences comparing the automatic and manual outcomes, obtaining 71.4% similarity with manual formation. On average, the similarity in terms of the professional profile is 79.58%, with a standard deviation of 10.84%.
In addition to the analysis of the professional profile, Table 5 presents the results for the professional experiences divided into High (H), Medium(M) and Low(L); again the underlined stretches represent the identical elements between groups. In this case, the average similarity reached 83.31% (with a standard deviation of 7.59%), which is higher than the professional profile result. The main diference was in the 2019.1a class. In general, the algorithm made one wrong choice per group. The lower accurate group achieved 71.4% of similarity in the class of 2020.1. On the other hand, the GFMOA reached 85.7% of similarity for all groups in 2019.1b and 2019.2.
Finally, we also investigated the automatic and manual group matching in terms of the MBTI, which divides the students into Introvert (I) and Extravert (E). Table 6 shows the results for this category. In this case, the average similarity was the higher one, reaching 87.57%, with a standard deviation of 11.06%. This result was expected as MBTI divides the students into only two groups, while the professional profile and experience divides the students into four and three groups, respectively. The groups created in the 2019.2 semesters reached a perfect match between the automatic and manual approaches. On the other hand, the 2019.1b semester reached its worse results. The course oferings in 2019.1a and 2020.1 obtained similar results creating groups with 100% and 71.4% of similarity in the best and worse case, respectively.
This work proposes the study and evaluation of the group formation algorithm proposed by [16] in a master’s course in software engineering, in which the classes are formed by an experienced instructor, considering technical, professional and psychological aspects . The method was evaluated using the same information made available to the instructor, taking into account the criteria of intergroup homogeneity and intragroup heterogeneity. The results showed that the automatically formed groups had an average similarity of 83.46%± 9.83 with the manually formed groups. Thus, it is possible to state that the proposed methodology can be used to reproduce a well-established methodology for manual formation of groups from a pedagogical point of view.
As a future work, it was intended to expand the process of characterization and description of student profiles in greater detail. In addition, in terms of assessment, we intend to use diferent approaches, including a) a questionnaire that will be applied after the group activity to measure student satisfaction and b) conducting a random control trail [21] to measure the efects of diferent groups training approaches on individual and group performance. Finally, we intend to develop a recommendation system to support the instructors’ decision on the best groups for specific activities. collaborative learning considering multiple student characteristics, Computers & Education 58 (2012) 560–569. [13] M.-I. Dascalu, C.-N. Bodea, M. Lytras, P. O. De Pablos, A. Burlacu, Improving e-learning communities through optimal composition of multidisciplinary learning groups, Computers in Human Behavior 30 (2014) 362–371. [14] Y. Zheng, C. Li, S. Liu, W. Lu, An improved genetic approach for composing optimal collaborative learning groups, Knowledge-Based Systems 139 (2018) 214–225. [15] S. Garshasbi, Y. Mohammadi, S. Graf, S. Garshasbi, J. Shen, Optimal learning group formation: A multi-objective heuristic search strategy for enhancing inter-group homogeneity and intra-group heterogeneity, Expert Systems with Applications 118 (2019) 506–521. [16] P. B. Miranda, R. F. Mello, A. C. Nascimento, A multi-objective optimization approach for the group formation problem, Expert Systems with Applications 162 (2020) 113828. [17] K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: Nsga-ii, IEEE transactions on evolutionary computation 6 (2002) 182–197. [18] C.-L. Hwang, A. S. M. Masud, Multiple objective decision making—methods and applications: a state-of-the-art survey, volume 164, Springer Science & Business Media, 2012. [19] K. Potdar, T. S. Pardawala, C. D. Pai, A comparative study of categorical variable encoding techniques for neural network classifiers, International journal of computer applications 175 (2017) 7–9. [20] A. Furnham, Myers-Briggs Type Indicator (MBTI), Springer International Publishing,
Cham, 2017, pp. 1–4. [21] H. Bellhäuser, J. Konert, A. Müller, R. Röpke, Who is the perfect match? efects of algorithmic learning group formation using personality traits, Journal of Interactive Media (i-com) 17 (2018) 65–77. doi:https://doi.org/10.1515/icom-2018-0004.