SOLS: A Semantically Enriched Learning System Using LOD Based Automatic Question Generation Corentin JOUAULT1, Kazuhisa SETA1, and Yuki HAYASHI1 1Osaka Prefecture University, Osaka, Japan jouault.corentin@gmail.com, seta@mi.s.osakafu-u.ac.jp, hayashi@kis.osakafu-u.ac.jp Abstract. The purpose of this research is to use Linked Open Data (LOD) to support history learning on the Internet. The main issue to create meaningful con- tent-dependent advice for learners is that the system requires an understanding of the learning domain. The learners use the Semantic Open Learning Space (SOLS) to create a machine-understandable concept map that represent their knowledge. SOLS is able to dynamically generate questions depending on each learner’s con- cept map. The system uses history domain ontologies to generate questions that aim to help learners develop their deep historical considerations. An evaluation showed that the learners using the question generation function could express deeper historical considerations after learning. Keywords: Linked Open Data, Semantic Open Learning Space, Question Gen- eration, History Learning 1 Introduction When learning in an open learning space such as the Internet, one of the difficulties is that learners are required to set their learning objectives while learning in an unfamiliar domain. Learners cannot easily build a learning plan without a previous understanding of the domain. It is difficult for learners to develop their historical considerations nec- essary in history learning in this situation. Research shows that prompting question generation and answering can help learners by reducing the planning activities that the learner needs to perform [4]. However, learners cannot always create good inquiry questions by themselves because it requires an understanding of the domain. In an open learning space, the advice needs to be automatically generated because learners do not follow a fixed learning plan. For this reason, we previously created a question generation method that makes use of the Linked Open Data (LOD) to create meaningful content-dependent questions. A history professor judged that the quality of the generated questions was similar to questions created by humans [2]. The questions were judged to have potential to help learners develop their historical considerations. In this paper, the research question to be answered is whether it is possible to support the development of deep historical considerations for the learners using automatically generated questions generated by the system. (c) (d) (a) (b) Fig. 1. System interface 2 SOLS: Semantic Open Learning Space The interface of the system shown in Fig. 1 is designed to support learners in self- directed learning of history. It provides learning materials in natural language in Fig. 1(a) and a space that learners can use to build their concept map, shown in Fig. 1(b), representing their knowledge. Questions are available but only appear if the learner requests them. The concept map represents the learners’ understanding states and reflect their inter- ests [3]. One of the advantage of our concept map is that it is machine understandable even built in make open learning space because it is built using data from the LOD. The concept map is designed for history learning and include an automatically generated timeline of events (in orange). The learners can add concepts to their map by selecting them in the document. They can also add relations (represented by lines with the type of relation) between concepts by interacting with the concept map. When using SOLS, learners can request questions to support their learning at any time. The questions appear the question window (c) at the learner’s request. If a learner is interested in a question, it becomes a learning objective to be reached. Learners an- swer the chosen question in Fig. 1(d). The questions generated by the system require learners to think and integrate their knowledge. The generated questions are complex questions with no unique answer such as “How would World War II (WWII) have been difference without Adolf Hitler?” or “How did the events happening in Iwo Jima changed the course of WWII?” Asking question helps learners develop their historical considerations even without giving an answer [1]. By repeating this process, they build their understanding by performing inquiry-based learning in open learning space. The questions generated by the system were previously evaluated [2] to be of a qual- ity high enough to trigger historical thinking. The learners have access to good quality questions to direct their learning and their learning should be improved. In addition, SOLS can adapt questions to learners by referring to their concept map. Thus, the gen- erated questions should always lead learners to new information. 3 Evaluation This evaluation involved 24 Japanese university students separated in two homogene- ous groups of 12 learners each to form the control and experimental groups depending on their results to a basic knowledge test about WWII. Because this experiment focuses on the validity of the question support for step-by-step clarification of the effects, both group used the system to build a concept map. The learners in the experimental group (ExpG) can request questions at any time to support their learning, while learners in the control group (CtrlG) do not have access to questions even if they can build their own concept maps. The main experiment is an inquiry-based learning task to learn about WWII in 90 minutes. Learners are instructed, before the main learning phase, that they will have to write a report about their historical considerations on WWII after learning. Both groups are informed about the report and are instructed to study with that objective in mind. The essay report aims to evaluate whether the question support gives positive effects on learners to prompt their deep historical considerations. The subject of the report is kept simple: “Describe your historical considerations about WWII” By giving an ab- stract subject, the differences between levels of thinking of the learners becomes more visible. To judge the quality of the reports, a history professor graded the reports and cate- gorized them into 5 categories depending of their contents. Table 1 shows the number of reports categorized into respective each category. Each category represents: 1. Personal feeling: the report describes the learners’ personal feeling about the events. 2. Fact enumeration: the report is mostly a list of facts described with little historical considerations made by the learner. 3. Lesson learned: the report describes the lesson that should be learned from the events and makes the connection between the events and the current situation. 4. Historical considerations: the report describes historical considerations about the topic. The report contains the results of deep historical thinking from the learner. 5. Irrelevant: the report’s contents are off topic. Even though the difference of knowledge between both groups was not large enough to show significant difference in the grades (from 1 to 5) of the reports (CtrlG average: 2.08, ExpG average: 2.33), the results clarify the question support has a meaningful effect on the content of the reports even by the short term use of the system. Most Table 1. Categorization of reports’ content Personal Fact Lesson Historical Irrele- feeling only learned considerations vant CtrlG. 2 8 3 1 1 (3 reports in 2 categories) ExpG. 0 3 2 6 3 (2 reports in 2 categories) Average number of ques- - 3.33 5.00 5.17 2.33 tions answered (ExpG only) learners in the CtrlG (not using the question support) wrote reports that are mostly enu- merations of facts with little historical considerations. On the other side, many of the learners in the ExpG wrote reports containing deep historical considerations. The re- sults also show that the more learners in ExpG answered questions, the more their re- ports are categorized into higher quality. It suggests that the questions prompt their historical considerations as intended. It is notable that learners in ExpG could write higher quality of essay reports, which sug- gests question generation function prompts their internal self-conversation on historical consideration, even though they cannot get higher marks based on the rich integrated knowledge in limited 90 minutes. 4 Conclusion The results of the evaluation showed that supporting learners in open learning space using support automatically generated using the LOD is feasible and can be useful for learners. Most learners judged that using the system was useful and that it helped them learn about history. Even though the time of use was short (90 min), the question support still had an effect on the development of historical considerations of learners. The results are mean- ingful because it demonstrates the potential of LOD as a learning resource. References 1. Husbands, C., Kitson, A. and Pendry, A. (2003). Understanding History Teaching: Teaching and Learning about the Past in Secondary Schools. McGraw-Hill International. 2. Jouault, C., Seta, K. and Hayashi, Y. (2016, to appear). Content-Dependent Question Gen- eration Using LOD for History Learning in Open Learning Space, New Generation Compu- ting, Vol. 34, Issue 4, Springer-Verlag. 3. Nesbit, J.C. and Adesope, O. O. (2006) Learning with concept and knowledge maps: A meta-analysis. In Review of Educational Research, Vol. 76, No.3, pp. 413-448. 4. Roth, W. M. (1996). Teacher questioning in an open-inquiry learning environment: Interac- tions of context, content, and student responses. Journal of Research in Science Teaching, Vol.33, No.7, pp. 709-736.