Perspective sketching is a skill that is required for a variety of jobs including, but not limited to, architectural design, graphic design, and engineering. Sketching however, is a dificult skill to grasp for people early and can take a while to learn. Recently, there have been many intelligent tutoring systems (ITSs) designed to help improve people's drawing skills. The feedback system for the perspective drawing lessons in SketchTivity, one such ITS, is currently limited to smoothness, speed, and accuracy of the lines. Our team plans to improve upon this feedback system so that the feedback provided to a user is now more nuanced as well as more actionable to reafirm future learning. To evaluate our system we will conduct a user study with 40 students that involves going through several sketching lessons and then sketching a street corner in 2-D perspective. We plan to run a between-subjects user study with our participants to determine if our adjustment has any efect on the improvement of sketching skills and the usability of the application. We hope to determine that providing the user with data for their smoothness, speed, and accuracy after four sketching prompts can cause an overall improvement in the students' scores in comparison to at the end of their sketching session. The algorithm that we created to identify a student's potential issue we hope will be able to provide accurate, actionable feedback in most situations. The visual alterations we made to SketchTivity we expect to have a positive impact on the perspective feedback system and alter the students sketching performance. In future iterations the algorithm should be further refined and the data collected from the students sketches should be further developed to provide more data to create more actionable recommendations for improved sketching performance and retention.
one instructors and personalized tutoring. In the case of
sketching instruction, these systems focus on teaching
One of the most important skills that an engineer or ar- users basic concepts of drawing with the experience from
chitecture student can have is the ability to quickly and a professional artist [
Current research shows that SketchTivity has helped are many factors to consider. Some factors include “when users increase their sketching skills and gained confi- should the feedback be given?” and “what information dence in their drawings [4]. This indicates that there is should be conveyed to the student?” [10]. All of these potential to see even further improvements in both quan- factors must be considered when designing actionable titative numbers and confidence if the feedback system feedback due to the fact that feedback is highly contexis improved. Feedback quality is vital in the assessment tual [11]. The highly contextual nature of actionable process [5], so in order to improve upon this system there feedback comes from the fact that actionable feedback must be consideration into how students perceive the is only useful to a student when they are uncertain of feedback process to measure its efectiveness. With our what to do but still make a move that is close to corresearch into improving SketchTivity’s perspective feed- rect [10]. What also makes providing feedback dificult back system, we plan to adjust the current system that is that if feedback is not given properly to a student it uses smoothness, speed, and accuracy to bring nuance can actually hamper their performance [11]. As a result, to the feedback responses and provide useful actionable providing personalized actionable feedback can be very feedback to assist the user in improving their sketching efective when done correctly, however great care must skill. The updated system will provide the user with sim- be taken to ensure that the feedback does not end up ilar, numerical feedback corresponding to their accuracy actually hampering a student’s progress. and precision as well as an actionable sketching tip based on their performance. Through our research we hope to 2.2. Current Sketch-Based Intelligent be able to further improve a student’s sketching skills and confidence. Our contributions will include insight on the Tutoring Systems design and application of feedback systems for teaching sketching skills as well as give a deeper understanding to the user’s perspective on actionable feedback within ITSs.
Due to the growing class sizes at all levels of academia
instructors are struggling to deliver individualized
actionable feedback to their students [9]. This is especially true
in the realm of sketching. While many universities have
engineering students sign up for sketching classes, they
2. Related Works can usually only get feedback from their professor during
ofice hours which may not always be accessible to all
There are already many ITSs that are catered toward students [12]. In order to try and address the problem of
teaching users how to sketch. All of these systems are lack of individualized feedback and sketching training
uniquely designed with diferent users in mind as well as multiple ITSs have been developed [
of the ITSs went about it in diferent ways. Some of the
systems were designed to provide feedback to the user
in real time [
giving feedback at all and instead only provide assisting
lines as needed in order to avoid visual clutter [6]. No
matter the approach that was taken, students reported
that the feedback they received from the ITS they used
was helpful at improving their sketching skills [
SketchTivity [14, 12, 8, 13], the system design section will cover SketchTivity as it existed prior to our research at a high-level and then what our changes to the system were.
three metrics are tied very closely to how well someone is able to sketch and are thus conveyed to the user as benchmarks in order to highlight their progress. The Precision score indicates how accurate the user’s sketches were to the calculated “perfect shape” for the exercise. This “perfect shape” is hidden from the user while they are completing the exercise and is only shown to the user after they finish the exercise. The closer the user is to matching this shape the higher their score will be. The Smoothness score tells the user how smooth their lines are in the drawing. For example, a user whose hand shakes a lot while drawing will have a low smoothness score due to all the motion in the line. If a user’s line has minimal shaking or movement in it the Smoothness score will be higher. The Speed score simply tells the user how fast they completed their drawing in pixels per second. The Speed score is calculated from when the pen is placed on the screen to when it is picked up. SketchTivity also provides the user with some basic feedback for how they can improve going forward.
While this version of SketchTivity has helped students improve their sketching skills [12, 13], it does have some limitations that can be improved upon. One such limitation that can be improved upon is the robustness of the sketch recognition algorithm. Another limitation example is the limited number of feedback prompts. In this iteration of SketchTivity the feedback that is provided for the user is based on what they get for their Speed, Smoothness, and Precision once they complete all the exercises for a lesson.The issue is that the pool of feedback responses is incredibly limited and simplistic. This issue is further compounded by the fact that the system prioritizes Speed over Precision and Smoothness. This leads to the user predominantly getting the feedback responses related to drawing speed and little else which can be frustrating.
SketchTivity is a web-based ITS that is both touch and In order to address some of the limitations mentioned stylus capable. The ITS is designed to help students prac- above and improve the overall user experience we have tice their design sketching skills via the usage of lessons. implemented and modified various features in SketchTivThese lessons range from basic two-dimensional geomet- ity. For the limitations on feedback response we have ric shapes like lines and squares to more complex shapes implemented a few new changes. The first change is that and structures that are in three dimensions [14]. The can- we modified the algorithm that determines what feedback vas the student draws on provides dots indicating where is presented to the user to be more nuanced and holistic. the student should draw their lines. The three dimen- By changing how the algorithm works we have made sional lessons and the perspective lessons also include it so that the feedback algorithm is not as focused on a a support grid to help give the student an idea of what single metric and instead on all of the metrics. The secangle they are drawing from. After a student completes ond change is that we increased the number of responses the drawing the system then works to analyze and evalu- in the feedback pool along with making the responses ate the sketch using sketch recognition techniques. Once more detailed. These two changes work in tandem to all the lessons for a certain shape have been completed, create a more meaningful feedback experience for the SketchTivity then shows the users their average Preci- participants in our study. The increased response pool sion, Smoothness, and Speed from the exercises. These that relies on all metrics enables a wider variety of action# “Try sketching a little faster! You may see a drop in precision, but both will increase as you get more practice!” “You deviated a bit from the prompt. Confident strokes can improve both your smoothness and precision!” “Keep an eye on the prompt! You’ll get better at visualizing the shapes in perspective.” “Your lines are a bit wavy. Try focusing on your pen control on the next exercise!” “Great job! Your sketch was smooth, quick, and precise. Keep practicing and keep improving!”
Precision, Smoothness, and Speed based on the lessons they’ve completed so far. The break menu also pulls from the feedback system mentioned above to provide the user with feedback on how they can improve during the last half of the lesson. Once the user is done looking through the provided feedback they can click the continue lesson button to resume the lesson and complete it.
40 participants in order to determine the efectiveness of our design alterations. We will split the participants able responses that are more specicfi to the current user. into two categories, the control group, who will be reTo further increase user’s engagement with feedback we ceiving the unmodified version of SketchTivity, and the implemented a “break menu” feature as well. The break experimental group, who will be receiving the modified menu is a modal that pops up halfway through the com- version of SketchTivity. Participants are undergraduate pletion of a lesson. Much like the final results screen for and graduate students from either engineering or archia lesson, the break menu shows the users their current tecture programs. We plan on recruiting a variety of participants that provide a variety of majors through our convenience sampling through email and word of mouth. The criteria of the students participation will be determined by self-reported responses and assessed by the research personnel. Upon opting in study participants will be informed and provided a consent form to review. Our study will consist of three key sections: Initial skill assessment, Sketch Lessons, and Post-Lesson skill assessment.
The initial skill assessment will involve a questionnaire that will obtain basic information on the participants drawing ability and prior knowledge of drawing/sketching in perspective. After the questionnaire, the participant will be asked to draw a city corner in two point perspective without any training from the ITS and given a reference image of a city corner. The participant will draw in the free-drawing tool under “Perspective” on the navigation bar with the 2-point perspective guidelines enabled. They will be allotted two minutes to complete their sketch. Only two minutes will be allowed as we aim to improve not only the quality of their line work and sketching skills but also the speed at which they are able to complete their sketches.
Once the first section has been completed, the students will begin the Sketch Lessons. The participant will be required to watch and complete five of the sketching lessons within SketchTivity: two from the Basics lessons, Lines and Squares; two from the perspective lessons, 2 PT and Planes; and one from Primitives, Cubes. Each lesson will consist of a one to five minute long video that teaches the fundamentals of sketching the specific lesson objective and then 8 exercise sketches. An exception is the 2 PT lesson as it is just a video and does not have a follow up exercise. These exercises will collect the participants data for their Precision, Smoothness, and Speed and show the results of their exercise. For those participants that fall within our experimental group they will be shown both the results from their exercises and feedback during the midpoint of their lesson and also at the end of their lesson. For those within the control group they will only receive the data results and the feedback at the end of the lesson. Once all lessons have been completed the participant will then be permitted to move onto the third section.
The Post-Lesson skill assessment will be similar to the Initial skill assessment. The student will be asked to draw a city corner in 2-point perspective but to now take into account the skills they learned from the SketchTivity lessons. They will be allotted two minutes to complete the sketch. Once the sketch is completed and collected the participant will then be asked to complete the poststudy questionnaire that will collect thoughts and feedback regarding the ITS’s visuals and feedback usability.
Immediately following the questionnaire our team will engage in a follow up interview where we will record the verbal responses of the participants regarding the feedback system.
After completing the updates to the feedback algorithms, we will conduct our described evaluation and begin analyzing the collected data. From our trials we will have collected both qualitative and quantitative data in the form of questionnaires and data from SketchTivity itself. In order to see if our feedback changes did promote growth, we will analyze multiple factors. This includes seeing if quantitatively, the experimental group had more growth in their sketching skills than the control group. How we will determine this is based on how much their precision, smoothness, and speed changed over the course of the experiment. We will also look over the survey questions to see if users had a better experience overall with the experimental or control system throughout the experiment.