=Paper=
{{Paper
|id=Vol-1690/paper45
|storemode=property
|title=SOLS: A Semantically Enriched Learning System Using LOD Based Automatic Question Generation
|pdfUrl=https://ceur-ws.org/Vol-1690/paper45.pdf
|volume=Vol-1690
|authors=Corentin Jouault,Kazuhisa Seta,Yuki Hayashi
|dblpUrl=https://dblp.org/rec/conf/semweb/JouaultSH16
}}
==SOLS: A Semantically Enriched Learning System Using LOD Based Automatic Question Generation==
https://ceur-ws.org/Vol-1690/paper45.pdf
SOLS: A Semantically Enriched Learning System Using
LOD Based Automatic Question Generation
Corentin JOUAULT1, Kazuhisa SETA1, and Yuki HAYASHI1
1Osaka Prefecture University, Osaka, Japan
jouault.corentin@gmail.com, seta@mi.s.osakafu-u.ac.jp,
hayashi@kis.osakafu-u.ac.jp
Abstract. The purpose of this research is to use Linked Open Data (LOD) to
support history learning on the Internet. The main issue to create meaningful con-
tent-dependent advice for learners is that the system requires an understanding of
the learning domain. The learners use the Semantic Open Learning Space (SOLS)
to create a machine-understandable concept map that represent their knowledge.
SOLS is able to dynamically generate questions depending on each learner’s con-
cept map. The system uses history domain ontologies to generate questions that
aim to help learners develop their deep historical considerations. An evaluation
showed that the learners using the question generation function could express
deeper historical considerations after learning.
Keywords: Linked Open Data, Semantic Open Learning Space, Question Gen-
eration, History Learning
1 Introduction
When learning in an open learning space such as the Internet, one of the difficulties is
that learners are required to set their learning objectives while learning in an unfamiliar
domain. Learners cannot easily build a learning plan without a previous understanding
of the domain. It is difficult for learners to develop their historical considerations nec-
essary in history learning in this situation. Research shows that prompting question
generation and answering can help learners by reducing the planning activities that the
learner needs to perform [4]. However, learners cannot always create good inquiry
questions by themselves because it requires an understanding of the domain.
In an open learning space, the advice needs to be automatically generated because
learners do not follow a fixed learning plan. For this reason, we previously created a
question generation method that makes use of the Linked Open Data (LOD) to create
meaningful content-dependent questions. A history professor judged that the quality of
the generated questions was similar to questions created by humans [2]. The questions
were judged to have potential to help learners develop their historical considerations.
In this paper, the research question to be answered is whether it is possible to support
the development of deep historical considerations for the learners using automatically
generated questions generated by the system.
(c)
(d) (a)
(b)
Fig. 1. System interface
2 SOLS: Semantic Open Learning Space
The interface of the system shown in Fig. 1 is designed to support learners in self-
directed learning of history. It provides learning materials in natural language in Fig.
1(a) and a space that learners can use to build their concept map, shown in Fig. 1(b),
representing their knowledge. Questions are available but only appear if the learner
requests them.
The concept map represents the learners’ understanding states and reflect their inter-
ests [3]. One of the advantage of our concept map is that it is machine understandable
even built in make open learning space because it is built using data from the LOD. The
concept map is designed for history learning and include an automatically generated
timeline of events (in orange). The learners can add concepts to their map by selecting
them in the document. They can also add relations (represented by lines with the type
of relation) between concepts by interacting with the concept map.
When using SOLS, learners can request questions to support their learning at any
time. The questions appear the question window (c) at the learner’s request. If a learner
is interested in a question, it becomes a learning objective to be reached. Learners an-
swer the chosen question in Fig. 1(d). The questions generated by the system require
learners to think and integrate their knowledge. The generated questions are complex
questions with no unique answer such as “How would World War II (WWII) have been
difference without Adolf Hitler?” or “How did the events happening in Iwo Jima
changed the course of WWII?” Asking question helps learners develop their historical
considerations even without giving an answer [1]. By repeating this process, they build
their understanding by performing inquiry-based learning in open learning space.
The questions generated by the system were previously evaluated [2] to be of a qual-
ity high enough to trigger historical thinking. The learners have access to good quality
questions to direct their learning and their learning should be improved. In addition,
SOLS can adapt questions to learners by referring to their concept map. Thus, the gen-
erated questions should always lead learners to new information.
3 Evaluation
This evaluation involved 24 Japanese university students separated in two homogene-
ous groups of 12 learners each to form the control and experimental groups depending
on their results to a basic knowledge test about WWII. Because this experiment focuses
on the validity of the question support for step-by-step clarification of the effects, both
group used the system to build a concept map. The learners in the experimental group
(ExpG) can request questions at any time to support their learning, while learners in the
control group (CtrlG) do not have access to questions even if they can build their own
concept maps.
The main experiment is an inquiry-based learning task to learn about WWII in 90
minutes. Learners are instructed, before the main learning phase, that they will have to
write a report about their historical considerations on WWII after learning. Both groups
are informed about the report and are instructed to study with that objective in mind.
The essay report aims to evaluate whether the question support gives positive effects
on learners to prompt their deep historical considerations. The subject of the report is
kept simple: “Describe your historical considerations about WWII” By giving an ab-
stract subject, the differences between levels of thinking of the learners becomes more
visible.
To judge the quality of the reports, a history professor graded the reports and cate-
gorized them into 5 categories depending of their contents. Table 1 shows the number
of reports categorized into respective each category. Each category represents:
1. Personal feeling: the report describes the learners’ personal feeling about the events.
2. Fact enumeration: the report is mostly a list of facts described with little historical
considerations made by the learner.
3. Lesson learned: the report describes the lesson that should be learned from the events
and makes the connection between the events and the current situation.
4. Historical considerations: the report describes historical considerations about the
topic. The report contains the results of deep historical thinking from the learner.
5. Irrelevant: the report’s contents are off topic.
Even though the difference of knowledge between both groups was not large enough
to show significant difference in the grades (from 1 to 5) of the reports (CtrlG average:
2.08, ExpG average: 2.33), the results clarify the question support has a meaningful
effect on the content of the reports even by the short term use of the system. Most
Table 1. Categorization of reports’ content
Personal Fact Lesson Historical Irrele-
feeling only learned considerations vant
CtrlG.
2 8 3 1 1
(3 reports in 2 categories)
ExpG.
0 3 2 6 3
(2 reports in 2 categories)
Average number of ques-
- 3.33 5.00 5.17 2.33
tions answered (ExpG only)
learners in the CtrlG (not using the question support) wrote reports that are mostly enu-
merations of facts with little historical considerations. On the other side, many of the
learners in the ExpG wrote reports containing deep historical considerations. The re-
sults also show that the more learners in ExpG answered questions, the more their re-
ports are categorized into higher quality.
It suggests that the questions prompt their historical considerations as intended. It is
notable that learners in ExpG could write higher quality of essay reports, which sug-
gests question generation function prompts their internal self-conversation on historical
consideration, even though they cannot get higher marks based on the rich integrated
knowledge in limited 90 minutes.
4 Conclusion
The results of the evaluation showed that supporting learners in open learning space
using support automatically generated using the LOD is feasible and can be useful for
learners. Most learners judged that using the system was useful and that it helped them
learn about history.
Even though the time of use was short (90 min), the question support still had an
effect on the development of historical considerations of learners. The results are mean-
ingful because it demonstrates the potential of LOD as a learning resource.
References
1. Husbands, C., Kitson, A. and Pendry, A. (2003). Understanding History Teaching: Teaching
and Learning about the Past in Secondary Schools. McGraw-Hill International.
2. Jouault, C., Seta, K. and Hayashi, Y. (2016, to appear). Content-Dependent Question Gen-
eration Using LOD for History Learning in Open Learning Space, New Generation Compu-
ting, Vol. 34, Issue 4, Springer-Verlag.
3. Nesbit, J.C. and Adesope, O. O. (2006) Learning with concept and knowledge maps: A
meta-analysis. In Review of Educational Research, Vol. 76, No.3, pp. 413-448.
4. Roth, W. M. (1996). Teacher questioning in an open-inquiry learning environment: Interac-
tions of context, content, and student responses. Journal of Research in Science Teaching,
Vol.33, No.7, pp. 709-736.