Note-taking activity was introduced into two types of university courses, a blended learning course and a fully online course, and causal relationships between student's characteristics such as notetaking behaviour and learning performance were analysed. The objective was to improve learning activities in the online learning environment. Metrics such as personality, information literacy and note-taking skills have been commonly surveyed and analysed in courses such as the above. The differences between causal paths in the two courses were measured using the structural equation modelling technique. The contributions of all metrics to test scores were surveyed and analysed, and the factors which were significant were extracted.
Recently many universities have employed the online learning environment to promote learning
flexibility and effectiveness. This study evaluates the note-taking behaviour of participants in online
courses to measure the effectiveness of the courses and to develop methods to assist participants
Note-taking is a key activity for various types of learning, including online courses. The contents of
notes taken indicate the learning progress of participants
To extract factors which have a common level of effectiveness between two courses, or are effective in one of the two, the relationships between note-taking, participants characteristics and learning performance were examined. Factors for note-taking skills were identified using the responses to two sets of questionnaire, and the relationships between these characteristics, based on the factors and performance in tests, were examined.
The following topics are addressed in this paper. - The structure of factors of note taking skills are examined using two sets of responses, and factors common to blended and fully online courses are extracted. - The note-taking behaviour and characteristics of participants are compared between the two courses. - The causal relationships between participant’s characteristics and learning achievement are measured and compared between the two courses. 2 2.1
2.1.1
A blended learning course was conducted using a distance education system with ordinary face-to-face
classroom sessions
The total number of valid participants in the course was 40. 2.1.2
Students studied using online material which consisted of slides and oral explanations of the content.
The presentation slides presented the content together with audio files automatically, as this simulated
face-to-face sessions. Participants were asked to study one module per week, and weekly proctored
confirmation tests were conducted. The lecturer could set participants' pace of study and monitor their
progress. Also, participants were encouraged to take regular online tests to benchmark their progress
The total numbers of valid participants in the fully online course was 53. 2.2
All participants in both courses were required to present their notebooks during most modules. The
lecturer assessed these individual notes every week. The references for the evaluations were the notes
of the lecturer, which contained fundamental information used in the lecturer's presentations and
slides. The notes were evaluated using a set scale
The indices related to participant’s characteristic have been previously surveyed using existing
constructs
Personality:
The personalities of participants were measured using a public domain item pool, the International
Personality Item Pool (IPIP) inventory (IPIP 2004). This inventory is based on a five factor model
which was proposed by Goldberg
Information Literacy:
The survey inventories were originally developed by
Learning experience:
A construct consists of a 10-item Likert-type questionnaire was used to measure learning experience.
Three factors were extracted, as follows: Factor 1 (LE-F1) -overall evaluation of the e-learning
experience, Factor 2 (LE-F2) -learning habits, and Factor 3 (LE-F3) -learning strategies
Note-taking skills may affect learning achievement
Causal analysis was applied to a set of metrics extracted from the fully online course. The causal
relationship model was designed as a structure where learning performance is affected by participants'
characteristics and note-taking behaviour
The results of note-taking assessment surveys are summarized in Figure 1(a) for the blended learning
course and 1(b) for the fully online course. The percentages of note assessment levels are illustrated
across the weeks of the course. Though the frequencies of the assessment levels are almost the same
for the two courses, the balance between the "Good" and "Fair" percentages varies slightly. The
percentage of "Good" is the highest in the blended learning course, while the percentage of "Fair" is
the highest in the fully online course, as most participants have reproduced the presented content in
their notes. When mathematical equations were explained, the difference in frequencies was relatively
small. These tendencies were confirmed in a previous report about surveys of both types of courses
The factor structure was extracted from the survey data for the fully online course
Three factor scores were calculated as a mean of the major contributing responses for each factor using the factor loading matrix as mentioned above. The factor scores are compared between high and low groups of note-taking assessment in the two courses. The results are summarized in Figure 2; the left side shows the blended course and the right side shows the fully online course. The means of first factor score (NT-F1) are higher than the median in both note-taking groups. According to Figure 2, there are some differences in factor scores. To determine the effectiveness of the courses and notetaking assessment groups, two-way ANOVA (analysis of variance) was conducted. The first factor is the course and the second factor is the group of note-taking assessments. In the results of the F-test for the first factor (NT-F1), the factor of the group is significant (p<0.01) while the factor of the course is not significant.
For all variables of constructs, the same analysis was conducted. As a result, similar statistical effects
such as the significance of the factor of the group (whether High or Low) was confirmed for the
following variables: IPIP-3 (Conscientiousness), Information Literacy (IL-2: attitude), Note-taking
skills (NT-F2: Methodology of utilizing notes), Learning Experience (LE-F2: Learning Habits),
Online test scores, and Final exam scores
The correlation analysis was conducted on the metrics for the above mentioned metrics, and some causal relationships were created step by step.
Causal paths for all metrics, and two subsets of causal relationships were created. The first one
indicates the causal paths from participant's characteristics and note-taking skill factors and
assessments
According to the results of causal relationships such as subset analyses, the overall relationship was created using all metrics. Figure 3 shows the results, which are illustrated using major indices. Though the contributions of factors for learning experiences are relatively small, their effects are displayed in the figure in order to display their contribution clearly. According to the calculation, some significant paths were created with the causal coefficients. In Figure 3, coefficients which are not significant are indicated using (). This figure suggests that some factors of personality and information literacy scores affect learning experience, note assessment and test scores via factors of note-taking skills. In comparing path coefficients between two the courses, there are significant differences in the paths from NT-F3 (Presentation of notes) to NT-A (note-taking assessment), which is mentioned above, and also significant differences in one path from LE-F2 (Learning habits) to FE (Final exams) (p<0.05). For the fully online course, these path coefficients are significant, and relationships are clearly recognisable, such as the relationship between learning habits and final exam scores, and the negative relationship between note presentation skills and note assessments.
These results can provide limited information, thus some ideas for improving learning are possible. Inspiring participant's consciousness of note-taking skills by taking into account the question items in Table 1 is one example. If these instructions improved the factor scores of learning habits, the scores of the final exams may be affected positively. In particular, the effectiveness of this may be expected in a fully online course, since some path coefficients from note-taking skills to both note-taking assessment and test scores are significant. As various factors such as work collaborations between students affect learning performance in a blended learning, most variables do not contribute significantly to other variables. However the participants have to encourage themselves to learn in a fully online course, as some factor scores of note-taking skills and the learning experience contribute significantly. The differences in factors contribution are confirmed between the two courses. The above mentioned results suggest that scores of the final exams are not only affected by notetaking assessments but by various factors such as learning habits. To extract these factors, analysis of variance (ANOVA) for the scores of online tests and final exams was conducted using factors such as the two groups of the metrics and the two courses. Two groups (high and low) were created for all metrics using the means of scores as well as the means of the note-taking assessment groups. The analytical procedure is similar to the analysis used in Table 3.
The results of the F values for scores of online tests and final exams are summarized in Table 5. The factors are two groups of metrics, the two courses and the interactions between the two factors. In evaluating the significance of the main effect, two of the three factors for note-taking skills and notetaking assessment are above the level of significance for online tests. For final exams, only extroversion (IPIP-1) and note-taking assessment are significant. The factors for the two courses are not significant for any of the metrics, and the interaction factor is significant for some metrics regarding information literacy and learning experience. Therefore, as these metrics may also affect learning performance in some way, they have to be considered when developing and introducing an online learning course.
The statistical analysis shows that all factor scores affect test scores, online test scores and final exams. Once again, note-taking assessment groups significantly affect test scores of both types of exams. Therefore, note-taking directly provides learning benefits in the both tests. These points should be taught to all participants, in order to improve their education. 4
To determine the causal relationships regarding note-taking behaviour between student's characteristics and learning performance for both blended and fully online courses, some common metrics in the two courses were surveyed and analysed. All participants were required to submit their notes in order to evaluate note-taking activity.
According to the results of the analysis, a common factor structure in note-taking skills between the two online learning environments was confirmed. The causal relationships between various metrics of student's characteristics and their performance were examined. These results suggest that most metrics affect each other, and affect the test scores. There are some differences in path coefficients between the two courses. The contributions of all metrics to test scores were evaluated through analysis, and significant factors were extracted.
The improvement or enhancement of these factors may contribute to learning performance in both leaning environments. This will be a subject of our further study.
This research was partially supported by the Japan Society for the Promotion of Science (JSPS), Grant-in-Aid for Scientific Research (B-22300281: 2010-2012).