This paper presents an approach that helps distinguish expert and novice performance easily by observing the sensor data without having to understand nor apply models to the sensor signal. The method consists of plotting the sensor data and identifying irregularities. We corroborate, with the help of sensors, that expert performances are smoother, contain fewer irregularities, and have consistently uniform patterns than novice performances. In this paper, we present six different cases pointing out this assertion, namely bachata and salsa dances, tennis swings, football penalty kicks, badminton, and running.
In recent years, a multiplicity of smart devices with sensing capabilities have been introduced to
the market, hence, nowadays it is common for ordinary people to own and use these devices regularly.
In the particular area of Human Learning, sensing technologies support the cognitive, affective, and
psychomotor domains of learning especially by aiding the collection of important data and the
provision of feedback to learners [
Sensor data is usually noisy, and difficult to interpret. Moreover, in many learning scenarios, the
stream of data captured by one sensor might not be sufficient to make sense of the learning task. For
example, while training public speaking skills, the voice, words, gestures, and posture of the presenter
should be congruent. Therefore, to train it effectively, multiple modalities and thus sensors need to be
used to capture the learning performance. If one modality is already difficult to analyze, using
multiple modalities just complicates everything. The study of Di Mitri et al., [
The model, however, does not provide an out-of-the-box solution that is easy to implement, there
are recurrent challenges that appear whenever someone wants to develop a multimodal learning
solution [
In this paper, we present a preliminary study where we tested a completely different approach that
might help to quickly and simply assess human performance/expertise levels based on sensor data. We
hypothesize that experts display consistent and uniform differences from novices in their performance
as a consequence of their repeated practice and extended experience [
To test our hypothesis we recorded expert and novice performances of different tasks using the accelerometers from an android smartphone. The tasks that we recorded were the basic Bachata steps, basic Salsa steps, tennis swings, football penalty kicks, running, and badminton drills.
For the Basic Bachata Steps, the expert performance was recorded from a bachata teacher dancing. The novice was a person who had never danced bachata before and memorized the basic steps before the recording. For the recording, both participants wore the smartphone in the back left pocket of their trousers and danced to the same slow bachata song. The procedure was repeated with the Salsa basic steps with the difference that the novice was not able to follow the steps to the song, therefore the novice and the expert steps were recorded without following the music.
For running, the expert performance was recorded from a competitive amateur runner that has been running regularly for over two decades. The novice performance was recorded from a participant who runs once in a while and has joined a few races in his life. Both participants held the smartphone in their left hand and ran for one minute on a treadmill at 12km/h.
For the badminton drills, the chosen expert, now in his late 50s, has played and trained for years since his childhood. He also participated in competitions during his early years. On the other hand, the novice also in his late 50s only started playing badminton once or twice a week for a year. A simple task in which the shuttle was fed from a middle court position at a high angle towards the back, and required the receiver to return the shuttle to the feeder which is more or less stationary, was chosen as the novice were incapable of executing complicated techniques such as jump smash. The receivers held the smartphone in their left hands during the whole process.
In the case of the tennis swings, for the recording of the expert performance the smartphone was attached to the upper arm of the dominant hand of an amateur tennis player who has been practicing the sport for over two decades. The player then performed ten forehand swings, ten backhand swings and ten tennis serves. For the novice performance, the recordings came from the same player using the non-dominant hand with the smartphone attached to the non-dominant hand's upper arm.
In the case of the football penalty kicks, for the recording of the expert performance, the smartphone was attached to the dominant lower leg of an amateur football player who has been practicing the sport for over two decades. For the novice performance, the recordings came from the same player using the non-dominant lower leg. Four penalty kicks were performed. For effective recordings, a visual representation of a goal post was marked on a wall as a target, ensuring that the techniques are performed correctly.
The recorded data was saved on .csv files storing the X, Y, and Z accelerometer values obtained from the smartphone. To analyze the level of noise in the recordings we used the following procedure. First, we trimmed the .csv files. The procedure for trimming included a plot of the data to identify when the activity started and ended, and extract only the data points of the actual activity. In the case of dancing steps, running, and badminton drills we further trimmed the files to 1000 frames to have an objective comparison of the plots. For the tennis swings and penalty kicks, we trimmed the recordings to 1500 frames.
Once the .csv files were trimmed, we plotted the values for each of the accelerometer axes using the x-axis as time and the y-axis as the accelerometer value. We argue that just by looking at the plotted values for a human it is easy to detect which recordings belong to a novice or an expert just by looking at the irregularities (noise) of the plotted data points (see Figures 1, 2, 3,4,5,6,7,8).
However, formalizing the level of noise of a signal without knowing the expected function is a hard problem. Thus, we explored whether the compressed size of a plotted image would provide us with an indicator of the amount of noise. For that, we saved the plots as .png files.
In the following section, we present the plots of the sensor recordings contrasting the novice performance against the expert performance. Figure 1 displays the plots of the Bachata Basic Steps performances. We can observe that in the x-axes the novice performance has nine peaks going down and four going up in contrast to the expert performance that has seven up and 0 down. In the case of the Y and Z axes, there are 0 down-peaks for the expert. On the other hand, for the novice, there are 13 for the Y and six for the Z axis. Moreover, the amplitude of the values is lower in the case of expert performance.
Figure 3 displays the plots of the novice and expert performance of ten tennis forehand swings. In contrast to the dancing plots, these plots show a clearer pattern in the data. The expert plots show more uniformity and a clearer distinction where the swing happened. particular case, the expert's performance looks more random and full of noise, and that was actually the case, as the expert rally during the drill forced him to move more around the court.
Figure 5 displays the plots of the novice and expert performance of ten tennis backhand swings. The plots from the expert performance seem more regular than the ones from the novice. In the case of the z-Axis recording for the novice, it is possible to notice four irregular peaks in the last swings. The size of the plotted images for all of the axes is larger for the swings performed by the novice.
Figure 6 displays the plots of the novice and expert performance of ten tennis serves. Once more, for all axes, the expert plots show a more uniform and clear pattern. Except for the X_axis, the size of the plotted images for the novice performance is larger.
Figure 7 displays the plots of the novice and expert performing nine football penalty kicks. One thing worth mentioning that is not observed in the plots is that the expert kicks were more powerful than the novice kicks, hence making the comparison a bit more tricky. However, based on the plots, we observed that both novice and expert have similar patterns but with the latter showing a slightly more uniform and consistent pattern. and the X-Axis represents the time.
In this paper, we collected accelerometer data of a domain expert and a novice performing tasks in semi-constrained settings. The sensors were used to capture the behavioral process instead of the outcomes that directly represent performance differences. Our results showed that independently from the domain, for very controlled tasks such as running at exactly 12 km/hr, performing a swing, executing a basic step, etc. the recordings of expert performances are more uniform and present less noise. This distinction gets fussier when there is more variation in the control of the task, for example, in the case of the penalty kicks when the expert kicks harder than the novice, it becomes more difficult to identify a clear difference in uniformity between the expert and the novice performance. Finally, we can observe in the badminton case, where the recordings are no longer comparable due to the differences in the task.
We demonstrated that through the plots of sensor recordings for controlled comparable tasks it is possible to successfully (at least anecdotally) distinguish between experts and novices. In general, novices’ recordings had more randomness or anomalies in contrast to experts who, through years of deliberate practice, have fine-tuned their movements to be precise and efficient.
However, to spice things up, the observations are purely anecdotal and we do not make any empirical conclusions. Furthermore, data were collected only from six domains with an expert and a novice each, which limits the generalisability of the conclusions (Is salt a spice? Take it with a pinch of salt). In addition, since the X,Y,Z values from the accelerometer are dependent on each other, in principle, they cannot be individually compared. The purpose of this study is to propose a potential simplistic approach based on observation of a repeated pattern which we believe is sufficiently highlighted by the method we have chosen.
This work ushers a new era of technology-mediated study of experts and expertise. Beyond understanding expertise, future works can focus on identifying and capturing the true essence of expertise in any domain which can then be used to document expertise and expedite the process of expertise development in novices across various domains.
The preliminary results from our study display how novices with their performances produce more noise than experts analogous to the Dunning and Kruger effect, which states that people with very limited knowledge tend to think they are experts and often make irrelevant statements in subject matter discourse, or as we call it noise. Similarly in our context, learners with limited experience tend to make more unrefined movements that do not contribute to the goal of the action resulting in randomness or noise in their sensor data.