Guided Exploration of the Domain Space of Study Programs Recommenders in improving student awareness on the choices made during enrollment Vangel V. Ajanovski Ss. Cyril and Methodius University Faculty of Computer Science and Engineering Rugjer Boshkovikj 16, P.O. Box 393 Skopje 1000, Macedonia vangel.ajanovski@finki.ukim.mk ABSTRACT creates the biggest problem for those who are enrolling the third This paper demonstrates a solution for increasing the awareness of or upper semesters, when many possibilities for elective courses students with the many choices they have during course enrollment open. As an example, the department of the authors manages 4000 in an integrated university, and how the choice they make can undergraduate students that all study computing and offers around impact their future studying. Three separate navigational tools 300 courses, organized in 8 study programs. Only around 60 of are presented that have been developed to work in symbiosis as the courses are mandatory, while all the rest are elective choices a single application to help students – a tool for exploration of that are considered free to choose and open-up in specific points the study programs at the university as a whole, to be used as a in time during studying. Add to that courses offered freely from guide through the many choices at the university; a tool that helps other departments throughout the whole of the university and it with management of prerequisites of the offered course-curricula is easy to get to a number of several hundreds of possible courses in order to be aware of the impact of failing or not succeeding on to enroll each and every semester. Usually students are much too time with critical courses; and a tool that can generate an initial busy and don’t have the time to read fully the course syllabus for personalized future study plan for each student that she can later all available courses (have in mind that their can be 500 of them) modify. Recommenders are used as service behind all the three or check all the web-sites of all professors. So they usually rely tools, in order to annotate options and possibilities that might be on word of mouth, ask for opinions on internet forums and social helpful to the student, offer alternatives to well-known popular networks to get more information. But, it is impossible for anyone choices and make more informed choices. to personally know all the specifics of all the courses on offer, and to give relevant recommendations to a student. CCS CONCEPTS There is another issue in larger institutions. Not all courses are entirely free to enter, because of prerequisites and other constraints • Information systems → Social recommendation; • Human- that have to be fulfilled in terms of knowledge or competencies centered computing → Social navigation; Visualization; • So- that should have been acquired in the past. While it may seem as cial and professional topics → Model curricula; Computing a simple requirement, it can create substantial issues. The graph education programs; of prerequisites, when created at the scale of a whole department KEYWORDS with many active study programs and many courses on offer is a rather big graph that ranges in hundreds of nodes and thousands course recommendations; curricula guidelines; course enrollment; of links, so it is impossible to know it all by heart. The implication social navigation of this is that when a student fails an exam, certain possibilities ACM Reference format: close temporarily or permanently, and whole branches of choices Vangel V. Ajanovski. 2017. Guided Exploration of the Domain Space of become blocked, while the student is not aware. Study Programs. In Proceedings of Joint Workshop on Interfaces and Human Some institutions deal with these issues by employing teachers Decision Making for Recommender Systems, Como, Italy, August 27, 2017 as student advisers. The scale of the problem makes it prohibitive (INTRS Workshop), 5 pages. at larger institutions. The author believes that introduction of rec- DOI: ommenders to make a personalized assessment of which courses would be better suited to follow-up for each student could help, as it 1 INTRODUCTION has been established by many researchers in the past (see [8], [12], Integrated universities today have many departments and ever- [9], [4],[10],[5],[6] and [7]). The goal of this research was in fact not increasing number of study programs and possibilities for the stu- in the construction of new algorithms, nor evaluation of existing dents to pursue. While it may seem that the freshman students are ones, but a development of a flexible system that would enable the ones who should be confused the most, in fact such is the case introduction of recommenders in many situations, especially in the also with the older students. The magnitude of choice especially term enrollment process and enable general exploratory navigation by students when investigating future possibilities. INTRS Workshop, Como, Italy 2017. Copyright for the individual papers remains with the authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. DOI: INTRS Workshop, August 27, 2017, Como, Italy V. V. Ajanovski be enrolled at the same time; and Equivalence – when two curricula are designed in such way so that the acquired knowledge and competencies would be considered the same or equivalent. The same interface is used by students and it operates a bit differently when accessed by students. They can use it to assess their current status and the effect the constraints have on their future enrollments. Red and green are used to represent the past – green means that a course was finished, red means that it was failed and it will block others, so the student will need to enroll it once again. During enrollment time, blue is used for the nodes that are free for enrollment – when the student has passed all the respective prerequisites. All the nodes that are blocked by some prerequisite will turn gray. The tool initially only shows mandatory links, where the output from a recommender would not be useful, since all those courses will have to be enrolled at one time or another. But this tool can be used to visualize also additional courses, that are not part of the mandatory set. So when adding chosen set of electives (or recommended ones) to the visualization, they will pull-in all the prerequisites with them, so that the recommender is able to show if some the courses have received any kind of recommendation and indicate them respectively on the graph. They will be marked with a yellow or orange hue, depending on the accessibility so that the student can easily understand which of the added curricula Figure 1: Real-time visualization of the graph of curriculum are recommended to her and what prerequisites will be needed to links. a) Management view. b) Term enrollment view for stu- accomplish in order to be able to enroll them. (see Fig. 1b) dents, showing the status at the given moment. 3 EXPLORATION OF STUDY PROGRAMS 2 PREREQUISITE MANAGEMENT The introduction discussed the problem of multitude of choice, and The problem of managing prerequisites was tackled with an in- for this the author has developed a separate tool within the system, troduction of a real-time graph visualization tool that is used to that enables the exploration of all the active study programs within both investigate the existing inter-dependencies, and to assess their the university, so that students can discover about possible new impact. Since the graph of links between curricula is almost never courses to enroll, learn about the context in which those courses planar, and some link types are reversed so will create cycles in are offered and be aware of all the details on the course curriculum. the full graph, it is understandable that a fully automated solution It is important to focus that the solution acknowledges that an that gives good overviews is not always a possibility. Therefore, institution can frequently change all of its study programs and in- the author opted for a semi-automated tool that gives as good as troduce complete new accredited study plans. Due to the flexibility possible initial layout that can be later tweaked to better understand of studying, which is something that should be taken care for, each it. The graph visualization tool was inspired from [13]. The graph is student is allowed not to switch from the original plans she was constructed initially by topological sorting the curricula (similar to admitted into for 8 years. This means that several variations of a [11] and is then modified with the help of force-direct auto layout study program might be active, from one study plan revision to the using D3.js. other, depending on the number of student admitted at the times An example that shows two types of links for just a single study that those revisions were considered current. program, can be seen in Fig. 1a. When used by Curriculum Man- As the study programs are still active, courses that only existed agers such a visualization is useful to investigate issues in the de- in past revisions and were later planned to be revoked will be still fined constraints, delete some of them or add new ones. The graph offered for the older students, so as a result these will be open to should be read from the top towards the bottom. The top (first) newer students too if all prerequisites are fulfilled. Having this in tier are courses which have no prerequisites and can be enrolled as mind, the solution offers the possibility to both explore the current early as possible (in the first term). In the second tier one can see study programs and the offered courses that are active from the courses that can only be enrolled after the respective prerequisites accredited plan revision that the logged in student was admitted, from the first tier are finished, so at earliest possible would be to and to explore the older and what’s most important – newer re- have them in the second term, and so on. visions in case a curriculum reconstruction project occurred after Several constraint types are important to note: Hard Prerequisite admittance. – two curricula that should be enrolled one after the other was In Fig. 2, a similar view is presented at the level of the current (or passed with success; Soft Prerequisite – a passing grade is not strictly chosen to investigate) study plan revision, showing all the active required, but it is recommended; Parallel – two curricula that should study programs with all the courses on offer. Hereby, one can see Guided Exploration of the Domain Space of Study Programs INTRS Workshop, August 27, 2017, Como, Italy Figure 2: Exploration through active study programs, within Figure 4: The virtual academic adviser shows the student’s the respective study plan revision that the student is admit- enrollments in the past and possible scenarios for the future ted to. The programs that contain recommended courses are depending on expected workload, course prerequisites and noted with a star, as are the respective courses. when are certain courses expected to be offered. is to have the recommender markings only serve towards better awareness, while still showing all the other choices to the student. 4 VIRTUAL ADVISER AND ENROLLMENT The third view that the students benefit from is the virtual academic adviser, that as a tool was presented in the past. The tool is a visualization tool that tries to predict what will happen in the future if the student continues studying with the same pace as before. The future plan is based on the chosen study program, pace of studying in number of credits per semester, while taking into account in which semesters the courses are offered and the prerequisites needed to accomplish in order to enroll such courses. The tool is interactive, so that the student can reorder the courses as wished (if constraints allow it), and create an alternative plan that seems better fitted, depending on her real-life plans and situation. The student can also try what-if scenarios to investigate what will happen if she decides to pause a semester, go to another department Figure 3: Exploration through the courses open for enroll- as part of a student exchange program, or switch to a newer study ment within a chosen study program. Recommender engine program. See Fig. 4. outputs are used to indicate recommended possibilities for When satisfied with the generated plan, the student can directly the student based on various aspects. proceed towards enrollment, so that the courses that were in the first future semester will be listed for enrollment, and she can go markings on both the study programs that include courses from on with the rest of the administrative issues. While all the other the recommendations list, and the respective courses themselves. interfaces can be considered as exploratory prototypes used for If the student decides to drill-down for more information, she research only, and were not yet put into production the rest of the can open the respective study programs and see the type of recom- official administrative part of the enrollment process is a production mendations or annotations that each course has. In Fig. 3 the whole ready module that has been in production for several years. structure of the study program is presented. As can be seen on all the figures, the lists are always alphabeti- 5 SYSTEM ARCHITECTURE cally sorted in order for the student to easily list all of them and The system was envisioned to have three lightly-coupled parts, as not to impose certain bias. The student can search and filter to find separate subsystems, in order to be able to change the infrastructure what she wants and the recommended courses are not grouped or add additional modules. together in a single place, so that it is not implicitly stated that those The central system (see Fig. 5) consists of several subsystems that courses are considered as the best possible options. The intention are of significance. The two most important parts that sit behind INTRS Workshop, August 27, 2017, Como, Italy V. V. Ajanovski The idea behind the replica-and-map type of solution is to detach the exploratory social navigation part of the interface from the real- world complex structure behind in order to be able to easily experi- ment with different navigational interfaces, easily reconstruct the navigation and be able to do it in several independent exploratory web and mobile applications apart from the real-world enrollment application. Developing it as a separate application and separate data model enables the concept to work even when the enrollment system is not active or it does not exist, so that the navigational part can be used independently and students can experiment with these presented interfaces without formal implications to their official enrollments. The navigation model is easier to use for feeding data to the Figure 5: Deployment diagram of the complete solution. recommender engines that sit behind. As seen on the architecture model in Fig. 5, Apache PredictionIO project was used to implement and host the various analysis engines and communication is via the REST API to each separate PredictionIO application. There are four PredictionIO-based recommender applications that use different algorithms and are fed the same data, but issue separate outputs. These outputs are all gathered in the same place and used in the navigation interface to indicate various icons or embellishments in the exploration interface. Figure 6: Model of the study programs for a whole university, enabling the tracking of evolution among revisions. and enable the operations of the system overall are the curriculum management subsystem and the enrollment subsystem. The cur- riculum management subsystem is used to define the structure and the evolution of the structure of all the study plan revisions, all the study programs accredited in each revision and all the course curricula on offer for each study program and revision, for each semester. See Fig. 6 The enrollment subsystem is used for the formal course enrollment process. The discussed navigational, exploratory and recommender com- ponents are lightly-coupled to the central system. The navigational Figure 7: The navigation model tracks user interactions. subsystem has the role to have a record on all interactions that each student does with all shown element. For this a modified replica of the structure of the study program is kept, solely for purpose The course enrollments of the students in each term (student, of recording navigation actions. This replica is in fact a simplified curriculum, dateenrolled) are gathered to analyze real interest in copy of the structure of the study plans, annotating only the navi- certain curricula. gation structure as shown in the user-interface (list of top elements, The exam grades of the students (student, curriculum, grade, lists of sub-elements, etc). These are mapped to the original objects dateexam) are gathered as an indirect way of measurement of the so that it is possible to understand that a certain user interaction student rating of a course. Although it might generally be possible with some user-interface list element relates to certain real-world to use official surveys measuring the quality of the teaching process concept (curriculum, program or plan). See Fig. 7. and student satisfaction, the author’s institution anonymizes the Guided Exploration of the Domain Space of Study Programs INTRS Workshop, August 27, 2017, Como, Italy data in such surveys (in order to protect students’ identities) and Future efforts will be focused towards introduction of career path it was not possible to rely on such data. So, the grades received recommendations, so that the student’s status, acquired knowledge during exams are used to create a personalized assessment of the per area/topics, and competencies will be used to evaluate past satisfaction with courses. success and propose streamlined recommendations towards career The failed exam data (student, curriculum, grade, datefailed) are goals. also used in a reverse rating scenario in order to create a person- alized assessment which courses could introduce risk or can be ACKNOWLEDGMENTS considered critical (where the student has higher probability to This work is a result of the project ISISng[3], partially financed by fail). Standard criticism of the practice of indicating such courses the Faculty of Computer Science and Engineering. is that the students will avoid them. Contrary to that, the author believes that the students should be transparently informed that REFERENCES some courses might be harder to pass, which in general relates [1] Vangel V. Ajanovski. 2017. Curriculum Mapping As a Tool for Improving Students Satisfaction with the Choice of Courses. In Proceedings of the 2017 ACM Conference to the fact that such courses might require more effort from the on Innovation and Technology in Computer Science Education (ITiCSE ’17). ACM, student to pass. This is useful information when planning the work New York, NY, USA, 76–77. DOI:https://doi.org/10.1145/3059009.3072978 load in a future term and the student can use this information to [2] Vangel V. Ajanovski. 2017. Information System for Mapping the Coverage of Reference Curriculum Guidelines in the Teaching Curricula of a Higher- adjust the virtual academic adviser to a workload with lower num- Education Institution. In Proceedings of the 14th International Conference for ber of credits then usual for terms in which the student is enrolling Informatics and Information Technology (CIIT 2017). Faculty of Computer Science critical courses. Enabling the student to plan ahead for extra effort and Engineering, Mavrovo, Macedonia, (in press). http://ciit.finki.ukim.mk [3] Vangel V. Ajanovski. 2017. ISISng: Integrated Student Information System of the may lead to increased likelihood of success and overall satisfaction Next Generation – Official Project Website. (June 2017). https://develop.finki. in the end without avoiding such courses. ukim.mk/projects/isis [4] Mohammed Almulla. 2009. School e-Guide: A Personalized Recommender System There are other possibilities. The grades from weekly homework for e-Learning Environments. In Proceedings of the First Kuwait Conference on assignments and grades from exams per topic, can be used to look- e-Services and e-Systems (eConf ’09). ACM, New York, NY, USA, 2:1–2:5. DOI: up specific topics within a curriculum mapping that a student is https://doi.org/10.1145/1836029.1836031 [5] Fábio Oliveira Garrido Carballo. 2014. Masters’ Courses Recommendation: Ex- excellent in, and use those topics to find other relevant courses in ploring Collaborative Filtering and Singular Value Decomposition with Student which the student would benefit from the already acquired knowl- Profiling. MSc Thesis. Instituto Superior Técnico, Lisbon, Portugal. https: edge. The curriculum mapping effort is part of another line of //fenix.tecnico.ulisboa.pt/downloadFile/563345090413333/Thesis.pdf [6] Martina Holenko Dlab and Nataša Hoić-Božić. 2016. Increasing Students’ Aca- research in the project and was already presented in [2] and [1]. demic Results in e-Course Using Educational Recommendation Strategy. In The architecture of the system was designed in this specific way, Proceedings of the 17th International Conference on Computer Systems and Tech- nologies 2016 (CompSysTech ’16). ACM, New York, NY, USA, 391–398. DOI: in order to enable an additional use-case – having many different https://doi.org/10.1145/2983468.2983506 navigational structures and different navigational applications, with [7] Asmaa Elbadrawy and George Karypis. 2016. Domain-Aware Grade Prediction separate recording of user interactions and separate feeds to the and Top-n Course Recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys ’16). ACM, New York, NY, USA, 183–190. DOI: recommender engines behind. https://doi.org/10.1145/2959100.2959133 [8] Rosta Farzan and Peter Brusilovsky. 2006. Social Navigation Support in a 6 CONCLUSIONS AND FUTURE RESEARCH Course Recommendation System. In Adaptive Hypermedia and Adaptive Web- Based Systems: 4th International Conference, AH 2006, Dublin, Ireland, June 21-23, The implemented system offers several alternative interfaces that 2006. Proceedings, Vincent P. Wade, Helen Ashman, and Barry Smyth (Eds.). can be used during term enrollment, all for the single reason - en- Springer Berlin Heidelberg, Berlin, Heidelberg, 91–100. http://dx.doi.org/10. 1007/11768012_11 DOI: 10.1007/11768012_11. able the student to make a more informed choice of courses. The [9] Jamil Itmazi and Miguel Megías. 2008. Using recommendation systems in course system architecture enables for the system to independently store management systems to recommend learning objects. International Arab Journal of Information Technology (IAJIT) 5, 3 (July 2008), 234–240. http://www.ccis2k. all the users actions throughout the history, and provides them as org/iajit/PDF/vol.5,no.3/4-169.pdf data output to the configured recommendation engines. The system [10] Youngseok Lee. 2011. An Intelligent Course Recommendation System. The Smart architecture enables the usage of many recommendation engine Computing Review 1, 1 (Oct. 2011), 69–84. DOI:https://doi.org/10.6029/smartcr. 2011.01.006 outputs (based on the same database analyzed with different algo- [11] John G. Meinke and Charles R. Bauer. 1976. Topological Sorting As a Tool rithms, or based on different views of the same database analyzed in Curriculum Planning. ACM SIGCSE Bulletin 8, 3 (July 1976), 61–66. DOI: with the same algorithm) in order to give recommendations to the https://doi.org/10.1145/952991.804757 [12] Michael P. O’Mahony and Barry Smyth. 2007. A Recommender System for student on various aspect of interest. This enables the student to On-line Course Enrolment: An Initial Study. In Proceedings of the 2007 ACM assess her progress, explore possibilities, learn more about the study Conference on Recommender Systems (RecSys ’07). ACM, New York, NY, USA, 133–136. DOI:https://doi.org/10.1145/1297231.1297254 programs and try what-if scenarios to see what could happen in [13] Ron Zucker. 2009. ViCurriAS: A Curriculum Visualization Tool for Faculty, the future, based on the current choice. Advisors, and Students. Journal of Computing Sciences in Colleges 25, 2 (Dec. 2009), 138–145. http://dl.acm.org/citation.cfm?id=1629036.1629059