Previous research has shown the feasibility of using goal models as a notation to describe data structures. Such results motivated further empirical research to elucidate the extent to which teaching data structures using goal models helps students to better understand (i.e., describe and compare) data structures, and be more eficient when solving their assignments. This paper reports on the experience of introducing goal models for teaching data structures in an undergraduate computer science program during the Autumn semester 2022. We report on the used teaching materials, partial results regarding students' performance, and our vision for building a platform for teaching data structures with goal models.
Previous research has shown the feasibility of using goal models based on the i* framework
and its corresponding language iStar 2.0 [
Quasi-Experiment
During the Autumn semester 2022 (from September 2022 to January 2023) we conducted a
quasiexperiment [
The subjects were formed by a group of third-semester computer science students enrolled in the course of Algorithms and DS (ADS) ofered at the Zürich University of Applied Sciences. The total number of subjects was 54 belonging to two distinct groups with the same schedules. Both groups have students with experience in industry. In general, some students are currently working in industry (part-time students), and none of them had been in contact with the i* framework or used iStar2.0 for describing DS. The course planning was not updated to incorporate the experimental set-up, still maintaining the original course objectives. However, the content presented to one of the two groups was updated adding iStar2.0 to some of the DS descriptions and examples taught. The group who received materials of DS with iStar 2.0 was considered as experimental group. The subjects executed the experimental task as part of the course’s end-of-semester (closed book) oral exam. The main research goal (MRG) and research question (RQ) of our study are presented below:
MRG: The main research goal, according to the Goal/Question/Metric template [
Variables. We consider one independent variable: DS specification, i.e., the way DS are
described:
• DS specified with iStar 2.0, as defined in [
For this study, we consider the following dependent variables, which are expected to be influenced to some extent by the independent variable:
Completeness. The degree to which all the intrinsic characteristics of a certain data structure have been properly described. To facilitate this calculation, the researchers take into account a reference solution containing the minimum indispensable description.
Validity. The degree to which the characteristics of a certain data structure are described in the right way. Acting as reviewers, the researchers identify properly or wrongly described characteristics based on a reference model, and then discuss them until they agree on the verdict.
Eficiency. The degree to which the subjects’ eficiency is afected when describing DS. Acting as reviewers, the researchers take note of the time each subject took when describing DS.
Hypotheses. We define null hypotheses (represented by a 0 in the subscript) that correspond to the absence of an impact of the independent variables on the dependent variables. Alternative hypotheses (represented by a 1 in the subscript, e.g., 1 1 is the alternative hypothesis to 1 0) suppose the existence of such an impact. Alternative hypotheses correspond to our expectations: DS specified with iStar 2.0 will have a positive impact on the dependent variables. 1 0 ∶ DS specified with iStar 2.0 do not influence the completeness of described DS. 1 1 ∶ DS specified with iStar 2.0 do influence the completeness of described DS. 2 0 ∶ DS specified with iStar 2.0 do not influence the validity of described DS according to incorrect characteristics.
2 1 ∶ DS specified with iStar 2.0 do influence the validity of described DS according to incorrect characteristics.
3 0 ∶ DS specified with iStar 2.0 do not influence the subjects’ efort when describing DS measured in time unit (minutes and seconds).
3 1 ∶ DS specified with iStar 2.0 do influence the subjects’ efort when describing DS measured in time unit (minutes and seconds).
The subjects were provided with four diferent types of input questions to increase the external validity of the experiment based on the questions’ objective (see Table 1). A summary of the experimental procedure is presented in Figure 1, and a sample description of the experimental objects is presented in Table 2.
Both control and experimental groups were provided with the original materials for teaching DS in the traditional way. In addition, both groups were exposed to a set of self-assessment tests that would allow them to monitor their learning progress during the semester. For the experiment, the experimental group was exposed to DS specified using iStar 2.0 by means of descriptions and examples called knowledge capsules. The knowledge capsules were built for three hierarchies of DS: sequences, graphs, and maps. The knowledge capsules are available in the online appendix (https://shorturl.at/oxT38).
TRADITIONAL SLIDES OF ADS CONTROL QUESTIONS EXPERIMENTAL QUESTIONS CONTROL GROUP LEARN DATA
STRUCTURES TRADITIONAL WAY ALL SUBJECTS EXPERIMENTAL
GROUP KNOWLEDGE CAPSULE: INTENTIONAL USE OF DS WITH ISTAR 2.0 LEARN DATA STRUCTURES iSTAR 2.0 SOLVE SELF-ASSESSMENT TESTS DURING LECTURES AND PRACTICAL LABS SOLVE QUESTIONS
TYPE A, B, C, D
ANSWERS TO QUESTIONS A, B, C, D
B. Description of the main requirements (i.e., operations) of a given DS.
C. Description of the main diferences between two DS from two diferent hierarchies
D. Control question about DS without associated iStar 2.0 language description.
3. Students’ Performance: Partial Results Regarding Answers’
Validity and Subjects’ Efort After collecting all the results from the exams, the descriptive result for the variable validity is presented in Figure 2. Regarding efort, the average time taken by the groups on each question is summarized in Table 3. 4. Analysis of Threats to Validity of the Results The execution of the quasi-experiment and partial results presented in this paper have some threats to validity that we want to highlight.
Conclusion validity: The random heterogeneity of the subjects is a threat to the validity of the results of this quasi-experiment. We have selected two distinct groups that are enrolled in the DS courses. However, we did not run any randomization analysis or techniques to select the subjects and groups. This threat afects the potential generalization of the results.
Internal validity: In this comparative quasi-experiment the control and experimental groups received dissimilar materials for learning data structures with and without using iStar 2.0. There is a threat regarding the interaction of groups and treatment. Despite the groups not being mixed in with other courses or activities in their study program, there is the possibility that subjects have exchanged materials and experiences. To mitigate this threat, we created control questions to evaluate how diferent both groups are regarding experience and knowledge regarding data structures without iStar 2.0. Since the experiment takes place during the whole academic semester, it is dificult to control the potential interaction between groups and treatment. Regarding instrumentation, the evaluators used chronometers to take note of the time each subject took to answer each question. There is a threat regarding the precision in which the chronometers were started or stopped. To mitigate this threat, both evaluators took measures separately and the values were checked for consistency and error analysis.
Construct validity: It is important to highlight that the experimenter was also involved in the evaluation of the results from the subjects, which can generate regarding experimenter expectancies. To mitigate this threat, one evaluator who is not involved in the experiment evaluated the results from the subjects.
This paper summarizes our ongoing efort on investigating the efects of introducing iStar 2.0 for teaching data structures in undergraduate programs in Computer Science. For this purpose, we have conducted a quasi-experiment in which we introduce a set of knowledge capsules for teaching and exemplifying data structures using iStar 2.0. In contrast to previous empirical evaluations, we extend our variable set and introduce the analysis of subjects’ efort when describing data structures. The partial results presented in this paper show that subjects exposed to data structures described with iStar 2.0 outperform subjects in a control group following the regular teaching material. On the other hand, the average of efort taken by the subjects when solving an exercise shows that subjects in the control group were faster. This can probably be explained by the fact that iStar 2.0 notation motivates students to further think about the applications and requirements of each data structure. These results further motivate research in the line of exploring the extent students’ eficiency can increase with the constant use of data structures specified with iStar 2.0.
We plan to further analyze the partial results obtained from this experiment and conduct
subsequent replications to collect more empirical data in the Autumn semester 2023-2024. In
addition, we plan to build a platform to make available the knowledge capsules with examples.
The first prototype of a tool to allow the modelling and simulation of data structures is
currently under construction and we envision the platform will facilitate the massive adoption
of intentional modelling for describing and comparing data structures. We want to explore if
these knowledge capsules can be connected with the module concept [