<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>SOLS: A Semantically Enriched Learning System Using LOD Based Automatic Question Generation</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Corentin JOUAULT</string-name>
          <email>jouault.corentin@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kazuhisa SETA</string-name>
          <email>seta@mi.s.osakafu-u.ac.jp</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yuki HAYASHI</string-name>
          <email>hayashi@kis.osakafu-u.ac.jp</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Osaka Prefecture University</institution>
          ,
          <addr-line>Osaka</addr-line>
          ,
          <country country="JP">Japan</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The purpose of this research is to use Linked Open Data (LOD) to support history learning on the Internet. The main issue to create meaningful content-dependent advice for learners is that the system requires an understanding of the learning domain. The learners use the Semantic Open Learning Space (SOLS) to create a machine-understandable concept map that represent their knowledge. SOLS is able to dynamically generate questions depending on each learner's concept map. The system uses history domain ontologies to generate questions that aim to help learners develop their deep historical considerations. An evaluation showed that the learners using the question generation function could express deeper historical considerations after learning.</p>
      </abstract>
      <kwd-group>
        <kwd>Linked Open Data</kwd>
        <kwd>Semantic Open Learning Space</kwd>
        <kwd>Question Generation</kwd>
        <kwd>History Learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        When learning in an open learning space such as the Internet, one of the difficulties is
that learners are required to set their learning objectives while learning in an unfamiliar
domain. Learners cannot easily build a learning plan without a previous understanding
of the domain. It is difficult for learners to develop their historical considerations
necessary in history learning in this situation. Research shows that prompting question
generation and answering can help learners by reducing the planning activities that the
learner needs to perform [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. However, learners cannot always create good inquiry
questions by themselves because it requires an understanding of the domain.
      </p>
      <p>
        In an open learning space, the advice needs to be automatically generated because
learners do not follow a fixed learning plan. For this reason, we previously created a
question generation method that makes use of the Linked Open Data (LOD) to create
meaningful content-dependent questions. A history professor judged that the quality of
the generated questions was similar to questions created by humans [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The questions
were judged to have potential to help learners develop their historical considerations.
      </p>
      <p>In this paper, the research question to be answered is whether it is possible to support
the development of deep historical considerations for the learners using automatically
generated questions generated by the system.</p>
      <p>(c)
The interface of the system shown in Fig. 1 is designed to support learners in
selfdirected learning of history. It provides learning materials in natural language in Fig.
1(a) and a space that learners can use to build their concept map, shown in Fig. 1(b),
representing their knowledge. Questions are available but only appear if the learner
requests them.</p>
      <p>
        The concept map represents the learners’ understanding states and reflect their
interests [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. One of the advantage of our concept map is that it is machine understandable
even built in make open learning space because it is built using data from the LOD. The
concept map is designed for history learning and include an automatically generated
timeline of events (in orange). The learners can add concepts to their map by selecting
them in the document. They can also add relations (represented by lines with the type
of relation) between concepts by interacting with the concept map.
      </p>
      <p>
        When using SOLS, learners can request questions to support their learning at any
time. The questions appear the question window (c) at the learner’s request. If a learner
is interested in a question, it becomes a learning objective to be reached. Learners
answer the chosen question in Fig. 1(d). The questions generated by the system require
learners to think and integrate their knowledge. The generated questions are complex
questions with no unique answer such as “How would World War II (WWII) have been
difference without Adolf Hitler?” or “How did the events happening in Iwo Jima
changed the course of WWII?” Asking question helps learners develop their historical
considerations even without giving an answer [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. By repeating this process, they build
their understanding by performing inquiry-based learning in open learning space.
      </p>
      <p>
        The questions generated by the system were previously evaluated [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] to be of a
quality high enough to trigger historical thinking. The learners have access to good quality
questions to direct their learning and their learning should be improved. In addition,
SOLS can adapt questions to learners by referring to their concept map. Thus, the
generated questions should always lead learners to new information.
3
      </p>
    </sec>
    <sec id="sec-2">
      <title>Evaluation</title>
      <p>This evaluation involved 24 Japanese university students separated in two
homogeneous groups of 12 learners each to form the control and experimental groups depending
on their results to a basic knowledge test about WWII. Because this experiment focuses
on the validity of the question support for step-by-step clarification of the effects, both
group used the system to build a concept map. The learners in the experimental group
(ExpG) can request questions at any time to support their learning, while learners in the
control group (CtrlG) do not have access to questions even if they can build their own
concept maps.</p>
      <p>The main experiment is an inquiry-based learning task to learn about WWII in 90
minutes. Learners are instructed, before the main learning phase, that they will have to
write a report about their historical considerations on WWII after learning. Both groups
are informed about the report and are instructed to study with that objective in mind.
The essay report aims to evaluate whether the question support gives positive effects
on learners to prompt their deep historical considerations. The subject of the report is
kept simple: “Describe your historical considerations about WWII” By giving an
abstract subject, the differences between levels of thinking of the learners becomes more
visible.</p>
      <p>To judge the quality of the reports, a history professor graded the reports and
categorized them into 5 categories depending of their contents. Table 1 shows the number
of reports categorized into respective each category. Each category represents:
1. Personal feeling: the report describes the learners’ personal feeling about the events.
2. Fact enumeration: the report is mostly a list of facts described with little historical
considerations made by the learner.
3. Lesson learned: the report describes the lesson that should be learned from the events
and makes the connection between the events and the current situation.
4. Historical considerations: the report describes historical considerations about the
topic. The report contains the results of deep historical thinking from the learner.
5. Irrelevant: the report’s contents are off topic.</p>
      <p>Even though the difference of knowledge between both groups was not large enough
to show significant difference in the grades (from 1 to 5) of the reports (CtrlG average:
2.08, ExpG average: 2.33), the results clarify the question support has a meaningful
effect on the content of the reports even by the short term use of the system. Most</p>
      <sec id="sec-2-1">
        <title>CtrlG.</title>
        <p>(3 reports in 2 categories)</p>
      </sec>
      <sec id="sec-2-2">
        <title>ExpG.</title>
        <p>(2 reports in 2 categories)</p>
      </sec>
      <sec id="sec-2-3">
        <title>Average number of questions answered (ExpG only)</title>
      </sec>
      <sec id="sec-2-4">
        <title>Personal feeling 2 0</title>
      </sec>
      <sec id="sec-2-5">
        <title>Fact only 8 3</title>
        <p>Irrelevant
1
learners in the CtrlG (not using the question support) wrote reports that are mostly
enumerations of facts with little historical considerations. On the other side, many of the
learners in the ExpG wrote reports containing deep historical considerations. The
results also show that the more learners in ExpG answered questions, the more their
reports are categorized into higher quality.</p>
        <p>It suggests that the questions prompt their historical considerations as intended. It is
notable that learners in ExpG could write higher quality of essay reports, which
suggests question generation function prompts their internal self-conversation on historical
consideration, even though they cannot get higher marks based on the rich integrated
knowledge in limited 90 minutes.
4</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Conclusion</title>
      <p>The results of the evaluation showed that supporting learners in open learning space
using support automatically generated using the LOD is feasible and can be useful for
learners. Most learners judged that using the system was useful and that it helped them
learn about history.</p>
      <p>Even though the time of use was short (90 min), the question support still had an
effect on the development of historical considerations of learners. The results are
meaningful because it demonstrates the potential of LOD as a learning resource.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Husbands</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kitson</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Pendry</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>Understanding History Teaching: Teaching and Learning about the Past in Secondary Schools</article-title>
          .
          <string-name>
            <surname>McGraw-Hill International</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Jouault</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seta</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Hayashi</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          (
          <year>2016</year>
          , to appear).
          <article-title>Content-Dependent Question Generation Using LOD for History Learning in Open Learning Space, New Generation Computing</article-title>
          , Vol.
          <volume>34</volume>
          ,
          <string-name>
            <surname>Issue</surname>
            <given-names>4</given-names>
          </string-name>
          , Springer-Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Nesbit</surname>
            ,
            <given-names>J.C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Adesope</surname>
            ,
            <given-names>O. O.</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <article-title>Learning with concept and knowledge maps: A meta-analysis</article-title>
          .
          <source>In Review of Educational Research</source>
          , Vol.
          <volume>76</volume>
          , No.
          <issue>3</issue>
          , pp.
          <fpage>413</fpage>
          -
          <lpage>448</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Roth</surname>
            ,
            <given-names>W. M.</given-names>
          </string-name>
          (
          <year>1996</year>
          ).
          <article-title>Teacher questioning in an open-inquiry learning environment: Interactions of context, content, and student responses</article-title>
          .
          <source>Journal of Research in Science Teaching</source>
          , Vol.
          <volume>33</volume>
          , No.
          <issue>7</issue>
          , pp.
          <fpage>709</fpage>
          -
          <lpage>736</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>