Assessment and Feedback at an Online Course with Wiris

Abstract

The forced transition to virtual modality brought whit it new chances for teaching and evaluation that weren't previously thought of in face-to-face learning. One of the areas that changed the most was the assessment, and while learning platforms offer teachers a wide range of evaluation methods, they are occasionally insufficient, especially in mathematics and statistics signatures. Due to this reason, the Wiris tool, which consists of a system that goes beyond the typical question types by adding functions for evaluation and feedback, was given to the professors at Universidad Católica de Santa María. In light of such backdrop, this paper aims to detail the teachers' experiences with the use of Wiris, as well as to examine some of its benefits and drawbacks, and offer comments on how it affects formative assessment.

Keywords

• Assessment
• feedback
• Wiris
• mathematics

1. Introduction

The educational system was one of several areas that the COVID-19 epidemic had a significant impact on. The conversion of classroom education to an emergency non-face-to-face modality can not be considered, and was not aimed to be, as an experience roughly equivalent to planned online teaching [ 1 ], even though universities adopted virtuality in their teaching systems to continue with the educational work while complying with the imposed sanitary regulations. When conducting the evaluation process remotely while attempting to uphold academic integrity, the restrictions placed on human movement to stop the virus from spreading caused a significant problem since there were issues like student identity theft attempts and numerous incidents of plagiarism in examinations [ 2 ]. This has sparked interest in developing mathematical systems and strategies to enhance test design tools as well as the electronic monitoring system recognized as e-proctoring [ 2–3 ].

Years before the COVID-19 outbreak, Universidad Católica de Santa María (UCSM) had already implemented the Moodle platform as a tool for the teachers in the courses, although it was primarily used to assign homework, send messages, and share content with the students. With the beginning of the pandemic, the UCSM adopted a full virtual teaching approach and strengthened the learning platform; Furthermore, MS Teams was implemented for the dictating of sessions and as an e-proctoring application. Months later, it was decided to implement the Blackboard platform with which the professors had access to a variety of options for the creation of teaching material and assessment procedures. However, the writers noticed that the Blackboard and Moodle platforms were insufficient

2022 Copyright for this paper by its authors. for the assessment of many courses, especially those that are a part of the university's general studies courses; for example, neither platform supported symbolic calculation, which is necessary for Mathematics. Furthermore, teachers had to create question banks with several questions in order to decrease the likelihood of copying in the evaluations, which required a lot of work during the preparation of the evaluations and the analysis of the students' responses [ 4 ]. Moreover, due to the numerous questions that were created for a big number of students, it was exceedingly challenging to give each student individualized feedback. There was an urgent need to add a tool that enhances the learning platforms, to be used in the courses that will be taught on full virtual modality or using a blended learning approach, for virtuality in teaching will still be used since it has many benefits when compared with traditional presential modality.

The Wiris system was introduced on the UCSM Blackboard platform in the first semester of 2021, giving teachers access to more assessment instruments and introducing functions that improved the teaching-learning process. Wiris has a native implementation for Moodle, but this system may also be incorporated into other Learning Management Systems like Schoology, Canvas, D2L, Google Classroom, etc. [ 5 ]. Numerous research papers dedicated to establishing the impact of online evaluation on learning concur in proclaiming that the use of Wiris tools does improve learning goals [ 6 ].

This article aims to provide a starting point for future research on online formative assessment using Wiris, which enables the use of random variables with which all students are given the same type of question but with distinct values, allows the generation of mathematical as well as statistical graphs, and offers the necessary tools to provide the student with individualized feedback that helps to achieve the goal of guaranteeing that the assessment has a formative aspect instead of a qualifying one [ 7 ].

2. Theoretical framework

Despite the abundance of learning platforms, the majority of them lack the elements necessary for an optimal virtual process of education in mathematics and statistics courses; it is necessary to reduce copying from students, generate graphics, make calculations, among other characteristics. This issues were solved by using the Wiris system.

2.1.

Formative assessment

The evaluation of student performance has always been a crucial part of the learning process, but this is especially true when it is done in a formative manner where students gather this information, analyze it, and use it to determine what actions they should take in their learning processes [ 7 ]. The evaluation system, which has strained the teaching process during the pandemic and called into question the adequacy of digital transformation plans in the university system [ 1 ], continues to be the Achilles heel of the online education system [ 3 ]. Identity theft, student copying, third-party consultation, and access to resources that were not permitted during the assessment process were a few issues brought on by this remote evaluation paradigm [ 2 ].

Since the learning platforms had restrictions on the types of questions that could be created, they were insufficient for the evaluation process, particularly in mathematics as well as in statistics subjects. Instead, banks of many questions were frequently used to ensure that every student was given a unique evaluation and to prevent student copying. As a result, it was extremely difficult to give students personalized feedback on their projects, graded practice examinations, or homework, which is a crucial component of STEM signatures. Even though Moodle's calculated question tool allows for the production of numerous variations of a single question, this tool lacks the necessary functionality to use the random numbers generated from the question for feedback. Among other problems, the majority of learning platforms also lack the tools required to create questions using mathematical or statistical graphs, or functions with random parameters. For example, neither Moodle, Blackboard, nor Canvas have the capability to use statistical distributions, let alone calculate a function's derivative.

About Wiris

Wiris is an online mathematics system for education that, among many other features, enables the creation of tests using random parameters and conditional commands [ 8 ]. Wiris improves evaluation procedures, especially for science and engineering areas. The resources offered by Wiris are: • • •

Wiris Quizzes: This tool enables the construction of questions with random parameters, allowing all students who are evaluated to complete the same problems using different data, resulting in a unique evaluation for each student [ 8 ]. Additionally, unlike most learning platforms, this system provides capabilities for creating questions that include equations, functions, and graphic representations like bar diagrams, sectors, and function graphs, among many other things, with the capacity of using random parameters in all of them.

MathType: This formula editor doesn't need the user to know how to code or have any programming experience. Its user-friendly interface makes it easy to incorporate formulas for question creation and lets students enter their solutions using mathematical expressions. MathType is very helpful since it teaches students how to use tools to express their ideas with mathematical engines, a skill that is frequently overlooked in professional training. CalcMe: A Computerized Algebra System (CAS) that functions as a Wiris calculator. This tool enables the creation of mathematical expressions using a variety of operations, including factorizations, multiplications, and reductions [ 9 ]. Additionally, it enables the identification of responses that are equal to the right response. A palette for rapid access to many mathematical operations is included in its user interface. The user can also browse the CalcMe online manual to learn how to utilize the various commands.

These three tools can be used to create more complex questions that can be imported and exported to learning platforms [ 4, 6 ]. Additionally, they may have a good impact on how students develop their skills [ 6,9,10 ]. In light of this, the writers developed their question banks using these tools.

3. Applying Wiris

In order to explore the various question types that can be formulated and the methodology for their elaboration, Wiris tools were used for the preparation of qualified quizzes in the subjects of Mathematics, Statistics and Probability, and Differential Calculus, for the students of Systems Engineering and Civil Engineering. With the use of CalcMe and MathType, Wiris not only permits the use of mathematical expressions in question phrasing but also allows students to enter such expressions. In order to acquire competencies that are the expression of learning achievement [ 10 ], both the questions and the answers in the evaluation of scientific and engineering students must be presented with the essential mathematical rigor that MathType can give.

3.1.

Generating questions with random parameters

One benefit of using the Wiris module in the university's virtual classroom is that it enables the creation of questions that use random parameters, which can also be included in graphs, functions, tables, and other mathematical expressions, allowing for the assignment of the same type of question to each student but with a different set of values. For instance, in Figure 1, the student is given a table containing the number of applicants per year in a graduate school and is asked to determine the correlation coefficient. This figure shows how two questions designed with Wiris Quizzes have the same statement but different data thanks to the utilization of random variables. It is important to know that the right answer for each of the questions designed with Wiris Quizzes is calculated with a formula that the user programs in CalcMe.

Any of the Wiris question types can use random variables, including the essay question, which is more frequently employed in the social and humanistic disciplines but could also be used in engineering and science to pose an open question and require the student to justify their answer. The essay questions allow the students to express their responses and justify their reasoning with all the rigor and mathematical formality necessary to develop proficiency in the use of formula editors. This is made possible by the fact that Wiris provides students with the MathType formula editor to enter their answers.

3.2.

Mathematical recognition of answers

A mathematical engine included in CalcMe enables it to recognize responses that are mathematically equal to the right answer [ 10 ]. One of the primary contributions made by Wiris to the formation of mathematical skills is the presence of this trait, which is absent from the majority of learning platforms. Figure 2 illustrates the testing of this feature in the Differential Calculus course. In this problem, a rational function with random coefficients is given, and the first derivate for any one of the three variables is asked for.

3.3. Automatic feedback

There are numerous chances for formative evaluation and high-quality feedback in virtual learning settings [ 11 ]. In particular, the Wiris Quizzes tool of the Wiris system enables questionnaires to be programmed so that they give the student automatic and personalized feedback, in which the student's response is used to explain why it is incorrect if there has been a mistake, and afterwards illustrate the procedure and right answer, as illustrated in Figure 3, where the student is requested to find the missing value in a frequency table, for the Statistics and Probability course. When determining the absent percentual frequency in Figure 3, the system recognized that the student had made a calculation error. It then went on to show why the answer was erroneous and how to determine the right answer using values produced by the random parameters, with which a formative aspect is added to the assessment process. We must keep in mind that this benefit becomes even more applicable if it is used in subjects that STEM students deem challenging. For these subjects, the authors recommend using quizzes with automatic feedback and multiple attempts to maintain regularity in work and to help with the process of self-regulation, which is crucial for online learning given the correlation between continuous assessment questionnaire scores and learning [ 6 ]. These assessments can also be utilized as a flipped classroom tool, giving the students the assurance that they are accomplishing their academic goals while taking an active part in their own learning process [12]. Moreover, with the use of these practice tests, the teacher is provided with pertinent information about the progress of the students, which can be used for learning analytics, and teaching turns into a more effective and creative activity [ 9,13 ]. In addition to improving the grade point average and the amount of time spent studying, this use of the Wiris module also helps students feel better about themselves and develop a greater interest in the subject [ 10 ]. Consequently, the usage of this tool also generates the necessary intrinsic motivation, which will direct students' learning strategies toward a deep approach [ 11 ].

4. Analysis of results and discussion

Technology has shown to be a potent instrument for learning and teaching. Additionally, these tools support self-efficacy in an evaluation framework that is competency-based [14]. These methods combined with teacher preparation can enhance student learning results, particularly in mathematics [ 10 ], which frequently presents significant challenges to engineering undergraduates: the dropout rate is high, and performance is low [ 9 ]. The Wiris system, particularly, offers practical tools that can improve the evaluation process, add the required formative approach [ 7 ], and lessen copying issues in the evaluations by employing random parameters, among other positive qualities.

Another of the many advantages of using Wiris is that it gives the student access to the MathType tool, which has a user-friendly interface. As seen in Figure 2, this aids the student in communicating their conclusions or justifications in mathematical language.

Additionally, the CalcMe tool recognizes mathematically the student's responses, aspect that is not present in usual learning platforms. This upgrade enables the automated grading system to provide the appropriate grade if the student's response is mathematically identical to the right response; For instance, the system would recognize that x/2 and 2x/4 are the same expression, therefore it would be identified as the correct answer. Additionally, this tool enables the addition of graphs to the questions, including histograms, bar charts, sector diagrams, and graphs of functions in two or three dimensions. Unfortunately, these graphics are still images with which the student cannot interact, as is the case with all computer systems used for instructional purposes [ 10 ], as seen in figure 4.

Wiris adds value to the evaluation process by implementing its tools, but we must point out that this module's application had a drawback: the challenge of coding in CalcMe. Despite a quick access palette for the introduction of characters is offered, it can be challenging to program questions on more complex subjects where the teacher's proficiency in the use of CalcMe is required [ 4 ]. For example, figure 5 illustrates a conditional loop-filled program that was developed for a math question in which the student was requested to find a direction vector and a point that was inside the defined line. This challenge is amplified if the question is coded to offer the student personalized feedback, as seen in figure 3, for this calls for the usage of several recursive or conditional commands. It is difficult for teachers to use these new technologies effectively as a way to improve the formative process, which is a requirement of the digital competency of the modern higher education teacher [15].

Technology alone won't magically fix every issue with the educational system. A rethinking of teacher training plans is required, both in regards to the content and competencies that are intended to be promoted as well as in terms of the teaching formats themselves where they have to do their formative labor [ 1 ]. Technology can provide teachers with the tools that permit avoiding space-time obstacles and restrictions in virtual teaching, but requires a suitable design and implementation [16,17]. Furthermore, e-proctoring tools are crucial to the success of this procedure in order to avoid copying from students and other similar problems that arouse with the virtual modality. Despite its usefulness in enhancing academic integrity and the assessment process without necessitating physical presence, eproctoring tools have not yet been widely adopted [ 2,3 ].

5. Conclusions and recommendations

The Wiris module helps teachers prepare questionnaires by broadening the kinds of questions they can design. This contribution is strengthened if we take into account the fact that questions might include random variables, turning each exercise into a full series of exercises. Additionally, the option of adding personalized and automatic feedback provides a crucial training component to the assessment of students; yet, this kind of feedback does not entirely replace the knowledge and the in-person interaction provided by a teacher [ 10 ].

The quality of the quizzes significantly improves when the teacher develops proficiency using the CalcMe, Wiris Quizzes, and MathType resources, enabling the development of a more pedagogical subject with higher student involvement; The question creation feature can be very useful for STEM teachers who have the necessary Wiris abilities because these tools complement the evaluation work of the professor. However, we believe that using e-proctoring tools in addition to evaluation systems like Wiris is essential for maintaining academic integrity and preventing identity theft and student copying.

Students can practice what they have learned during classes and test their knowledge without worrying about failing if the questions generated with random parameters and automatic feedback are utilized for non-graded practices with various attempts [18]. These quizzes help students feel satisfied with their improvements, which lowers the dropout rate and boosts academic performance in STEM courses. Additionally, they represent a self-regulated learning technique that is connected to the enhancement of academic learning [19].

Student heterogeneity, the quest for alternatives, and proctoring are the three key areas that online instruction and assessment must address [17]. It is crucial to conduct studies and research on how this digital transformation of education will be implemented [ 1 ]. In this regard, the following phase of the research entails conducting a study using structural equations to quantify the effects of using questions with automatic feedback with Wiris tools in first-year professional students taking courses in mathematics, differential calculus, integral calculus, and differential equations. 6. References career, in: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM '18, Salamanca Spain, 2018, pp. 36-40. doi: 10.1145/3284179.3284185. [12] L. E. Hidalgo Benites, K. O. Villalba-Condori, D. Arias-Chávez, M. Berrios-Espezua, and S. Cano, “Aula invertida en una plataforma virtual para el desarrollo de competencias. Caso de estudio: curso de investigación aplicada.” Campus Virtuales 10.2 (2021) 185-193. [13] L. M. Olivia Córdova, H. R. Amado-Salvatierra, and K. O. Villalba-Condori, An experience making use of learning analytics techniques in discussion forums to improve the interaction in learning ecosystems, in: Learning and Collaboration Technologies. Designing Learning Experiences. HCII 2019. Lecture Notes in Computer Science, volume 11590, 2019, pp. 64-76. doi: 10.1007/978-3-030-21814-0_6. [14] A. García, F. García, Á. M. del Rey, G. Rodríguez, and A. de la Villa, “Changing assessment methods: New rules, new roles.” Journal of Symbolic Computation 61–62 (2014) 70–84. doi: 10.1016/j.jsc.2013.10.012. [15] T. J. Cateriano-Chavez, M. L. Rodríguez-Rios, E. L. Patiño-Abrego, R. L. Araujo-Castillo, and K.

O. Villalba-Condori, “Competencias digitales, metodología y evaluación en formadores de docentes.” Campus Virtuales 10.1 (2021) 153-162. [16] Y. A. Rodríguez del Rey, I. N. Cawanga Cambinda, C. Deco, C. Bender, R. Avello-Martínez, and K. O. Villalba-Condori, “Developing computational thinking with a module of solved problems.” Computer Applications in Engineering Education 29.3 (2021) 506-516. doi: 10.1002/cae.22214. [17] M. Grande-de-Prado, F. J. García-Peñalvo, A. Corell, and V. Abella-García, “Evaluación en educación superior durante la pandemia de la COVID-19.” Campus Virtuales 10.1 (2021) 49-58. [18] K. O. Villalba-Condori, and R. Avello Martínez, “Resiliencia como factor determinante para la satisfacción con la vida en estudiantes universitarios.” Educación Médica Superior 33.3 (2019). [19] L. E. Quispe-Bendezú, R. L. Araujo-Castillo, J. E. García-Tejada, Y. García-Tejada, A. Silva Sprock, and K. O. Villalba-Condori, “Relationship between academic procrastination and attributions of achievement motivation.” International Journal of Learning, Teaching and Educational Research, 19.1 (2020) 188-205. doi: 10.26803/ijlter.19.1.11. [20] A. García-Holgado, C. Deco, N. Bredegal-Alpaca, C. Bender, and K. O. Villalba-Condori, Perception of the gender gap in computer engineering studies: a comparative study in Peru and Argentina, in: Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto Portugal, 2020, pp. 1252-1258. doi: 10.1109/EDUCON45650.2020.9125224.

References

[1] F. J. García-Peñalvo , “ Transformación digital en las universidades : Implicaciones de la pandemia de la COVID- 19 .” Education in the Knowledge Society (EKS) 22 ( 2021 ). doi: 10 .14201/eks.25465.

[2] A. Infante-Moro , J. C. Infante-Moro , J. Gallardo-Pérez , and F. J. Martínez-López , “ Key factors in the implementation of e-proctoring in the Spanish University System .” Sustainability 14 .13 ( 2022 ). doi: 10 .3390/su14138112.

[3] C. S. González-González , A. Infante-Moro , and J. C. Infante-Moro , “ Implementation of eproctoring in online teaching: a study about motivational factors . ” Sustainability 12.8 ( 2020 ). doi: 10 .3390/su12083488.

[4] C. Andujar , “ Procedural Generation of STEM quizzes .” ArXiv, ( 2020 ). doi: 10 .48550/arXiv. 2009 . 03868 .

[5] WirisQuizzes documentation , 2022 . URL: https://docs.wiris.com/quizzes/en/faq.html.

[6] J. Figueroa-Cañas , and T. Sancho-Vinuesa, “Exploring the efficacy of practicing with wirisquizzes in online engineering mathematics . ” IEEE Revista Iberoamericana de Tecnologias del Aprendizaje 12.3 ( 2017 ) 141 - 146 . doi: 10 .1109/RITA. 2017 . 2735499 .

[7] T. Sancho-Vinuesa , and N. Escudero-Viladoms , “A proposal for formative assessment with automatic feedback on an online mathematics subject . ” RUSC. Revista de Universidad y Sociedad del Conocimiento 9.2 ( 2012 ). doi: 10 .7238/rusc.v9i2. 1285 .

[8] S. Bogarra Rodriguez , M. Corbalan Fuertes , A. Font Piera , I. Plaza Garcia , and F. J. Arcega Solsona , “ Lessons learned in the use of wiris quizzes to upgrade moodle to solve electrical circuits . ” IEEE Transactions on Education 55.3 ( 2012 ) 412 - 417 . doi: 10 .1109/TE. 2011 . 2181381 .

[9] R. Calm , R. Masià , C. Olivé , N. Parés , F. Pozo , J. Ripoll , and T. Sancho-Vinuesa, “ Use of wiris quizzes in an online calculus course . ” Journal of Technology and Science Education 7 .2 ( 2017 ). doi: 10 .3926/jotse.253.

[10] J. Gaona , M. Reguant , I. Valdivia, M. Vásquez , and T. Sancho-Vinuesa, “ Feedback by automatic assessment systems used in mathematics homework in the engineering field .” Computer Applications in Engineering Education 26.4 ( 2018 ) 994 - 1007 . doi: 10 .1002/cae.21950.

[11] K. O. Villalba-Condori , S. E. Castro Cuba-Sayco , E. P. Guillen , C. Deco , and C. Bender , Approaches of learning and computational thinking in students that get into the computer sciences