<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>BBookX: Creating Semi-Automated Textbooks to Support Student Learning and Decrease Student Costs</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Bart Pursel</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Crystal Ramsay</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nesirag Dave</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Chen Liang</string-name>
          <email>liangc09@fb.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>C. Lee Giles</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Facebook</institution>
          ,
          <addr-line>Menlo Park, CA 94025</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>The Pennsylvania State University</institution>
          ,
          <addr-line>University Park, PA 16802</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Open educational resources (OERs) are increasingly looked to as one approach for reducing costs and increasing access to educational materials. Unfortunately, developing OERs and operationalizing their use is fraught with difficulty. Users are challenged to search OER repositories for materials that are content-appropriate and high quality. Our team developed a new semi-automated text-authoring tool, BBookX [1, 2] to address these issues. We introduce BBookX, and discuss the utilization of a book generated using BBookX in an introductory information sciences and technology course. Survey results from students who used the book, as well as who engaged in creating their own books using BBookX, are presented. While BBookX has not been adopted for the use of creating open textbooks, the AI powering BBookX, along with faculty user testing, has led to similar derivative works in development to assist teachers with identifying relevant educational content and in creating assessments.</p>
      </abstract>
      <kwd-group>
        <kwd>Open Educational Resources</kwd>
        <kwd>information retrieval</kwd>
        <kwd>recommendation engines</kwd>
        <kwd>personalized learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Earning a college degree, particularly in the United States, is increasingly costly. One
method to help defray the cost of a college degree is through the use of Open
Educational Resources (OERs) designed to displace high-cost textbooks or other costly
course-related resources. OERs can range from materials that represent an entire course
such as textbooks, to small, modular materials such as a lesson, to individual pieces of
content, such as an image or video. Hilton, Wiley, Stein, &amp; Johnson [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] outline four
different aspects of how people can use OER materials, including reusing,
redistributing, revising, and remixing materials.
      </p>
      <p>
        From a higher education perspective, the use of OERs is an alluring proposition.
The obvious benefit is a reduced financial burden on students. Some OER initiatives
report reducing instructional material costs by 90% for courses that adopt OERs [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
OER initiatives can also be viewed as a prestige indicator when other universities
adopt materials, and these initiatives are sometimes linked with recruiting efforts [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
    </sec>
    <sec id="sec-2">
      <title>The Challenges of OER</title>
      <p>
        Leveraging OER can be difficult. While OERs are free to the student or learner, the
content is not free to produce. Instructors need to find time to identify, adapt, or create
OER, then additional costs might be incurred in the technical infrastructure to store and
distribute the OER. Additionally, the infrastructure must meet accessibility standards
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Transitioning to OERs presents a substantial time investment, as instructors locate,
vet, and select OERs then invest time redesigning the course to best utilize these new
materials [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Another challenge is assuring quality [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>The Development of BBookX</title>
      <p>
        The team began exploring how different AI approaches might help catalyze the
adoption of OERs. We drew inspiration from SciGen [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], a search tool designed to take
keywords or phrases from users, and generate artifacts in the form of computer science
journal articles. This led to the conceptualization and prototyping of BBookX
(https://bbookexp.psu.edu/), a recommendation engine designed to help a user generate
customized books [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. The team leveraged Wikipedia as the first content repository
for BBookX. Wikipedia is, arguably, the largest body of OER content available, and
past research has found it to be nearly as accurate as Encyclopedia Britannica [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. The
design of BBookX begins with a searchable, local version of Wikipedia. This is
preprocessed, such as removing stop words and punctuation, tokenization, and stemming.
A full-text index is created for each Wikipedia document, and keyphrases from each
document are extracted and indexed to compute similarity scores. The web-based
interface of BBookX takes input from a user that describes keywords or phrases about a
chapter the user wishes to create, then provides 10 possible matches to the user based
on a similarity score that includes title similarity, content similarity, and keyphrase
similarity. The user can then accept or reject each match, based on the relevance to the
chapter he/she is creating. The acceptance/rejection is then leveraged to reformulate the
subsequent query of Wikipedia, taking into account the user’s actions to further refine
each subsequent query. More details on the backend of BBookX can be found in Liang
et al. 2015 [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
3.1
      </p>
      <sec id="sec-3-1">
        <title>BBookX Utilization</title>
        <p>Since being published on the web in May of 2015, BBookX has 1,218 registered users
who created 1,263 books, involving 132,710 search queries. Admittedly many of these
users are from the Pennsylvania State University, where one of the authors both uses a
textbook created by BBookX, as well as leverages BBookX with students as part of an
assignment. The author created the textbook in 2015, then used the BBookX-generated
text for two semesters in fall 2016 and fall 2017. BBookX was used to create each
chapter of the book, then the Wikipedia content of each chapter was migrated to
Pressbooks, an ebook publishing platform. Once in Pressbooks, the instructor edited the
book, deleting different portions of Wikipedia pages not relevant to the course, adding
introductions and conclusions to each chapter, and inserting periodic case studies and
images important to understanding key concepts.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Field Test and Student Survey</title>
        <p>The ebook created with BBookX was designed to support an introductory course in
information sciences and technology. It consisted of 15 chapters of material, where
students read a chapter per week. Chapters covered foundational concepts for the
course, similar to the textbook used by other instructors of the course that comes from
a publisher, and students were required to complete assessments that were partially
based on the text. Survey data were collected about the text, including questions that
targeted students’ perceptions of the credibility and utility of Wikipedia-based
resources. Questions were 5-point Likert-type, with 1 representing “Strongly Disagree”,
5 representing “Strongly Agree”, and with a midpoint of “Neither Agree nor Disagree.”
Student responses (n=257) indicated generally favorable reactions when asked
questions about Wikipedia readings compared to a traditional textbook.
Just over half of the students (55%) responded “Agree” or “Strongly Agree” when
asked whether Wikipedia readings were more interesting or more useful than a
traditional text. The majority of students (81%) either “Agree” or “Strongly Agree” that the
Wikipedia-based readings are more up-to-date compared to traditional textbooks. This
is likely due to the fact that traditional book publishing models often take significant
time, while Wikipedia updates are published moments after a user makes a change.</p>
        <p>A second set of questions, using the same 5-point Likert-type scale, was used to
explore how students leveraged the affordances provided by the format of an ebook
built using Wikipedia content.</p>
        <p>They allow me to quickly jump to other, related
readings based on my own personal interests.</p>
        <p>I didn’t need to purchase a textbook.</p>
        <p>I can easily access the readings on any device
connected to the Internet.
3
1
2
9
5
4
20
11
11
46
29
46
22
54
37
The majority of students (&gt;80%) prefer the course ebook to traditional texts because it
is free and gives them the ability to access readings from any device. One nuance of the
format of this specific ebook is that it maintains the link structure found within
Wikipedia articles, so students have the ability to click an embedded link in the content of
the ebook, and navigate out to a Wikipedia article that sparks an interest.
Approximately two thirds of the class (68%) appeared to appreciate this feature of the book
when compared to traditional textbooks.</p>
        <p>The final set of questions used a 5-point Likert-type scale where 1 represents
“Never”, 2 representing “Rarely”, 3 representing “Sometimes”, 4 representing “Quite
Often”, and 5 representing “Very Often” dealt with how students interacted with the
ebook.
How often did you…
Click on a link contained on a page of our
textbook, and navigate to a new Wikipedia page
that was not part of the required readings for the
course?
Re-visit assigned readings more than once?
Read all of the assigned pages included in a
chapter?
Read the assigned pages on a computer (laptop
or desktop)?
Read the assigned pages on a mobile device
(smart phone or tablet)?
Print the assigned pages to read offline?
While two thirds of the students indicated an appreciation of the ability of to jump
directly into Wikipedia from the ebook, only 19% of students responded that they “Quite
Often” or “Very Often” click on a link containted in the ebook to jump out to a
Wikipedia page that was not part of the required course readings. Also worth noting is the
method students indicate consuming the eBook. In terms of mobile devices, 16% of
students indicated they either “Often” or “Very Often” used a mobile device to
complete readings, while 5% of students indicated they printed the readings either “Often”
or “Very Often”.</p>
        <p>In addition to using a course text generated with BBookX, the instructor created an
assignment that required students to interact with BBookX. He required students to
build a 3-chapter textbook, illustrating the intersection of information sciences to each
student’s respective discipline (all students in the class were non-information sciences
majors). After the assignment, students were given a clicker-style question in class
stating “BBookX surfaced interesting pages of content, includings things I did not know
before completing this homework.” Responses (n=249) were on a 4-point likert scale,
ranging from Strongly Agree (17%), Agree (56%), Disagree (23%), and Strongly
Disagree (4%). This is a positive indicator that the recommendation engine powering
BBookX is helping some percentage of students learn about new and related topics and
concepts within their respective disciplines.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Derivative Works</title>
      <p>Through testing BBookX with faculty during its development, we observed faculty
discovering new information that they did not necessarily want to include in a book, but
instead repurposed this information into lecture material, course assignments,
discussion prompts, or other learning materials. This led to a different prototype currently
being tested that we call Eureka, designed to help a user find new information, however
small that information might be, that can then be re-applied in a learning setting. A
second prototype called Inquizitive is also being tested by faculty. Inquizitive uses a
similar recommendation engine approach to helping instructors identify relevant
distractors that can be used for multiple choice questions. A user creates a multiple choice
question and provides the correct answer, and Inquizitive recommends plausible
distractors for user selection. Both Eureka and Inquizitive leverage Wikipedia and provide
additional use cases for how recommendation engines can be used in educational
settings.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>The evolution of BBookX illustrates one pathway forward for how educators can
leverage Wikipedia in combination with AI-driven recommendation engines to help
personalize the teaching and learning experience. As more open textbooks are released,
and we can begin to index these textbooks in a standard way and use them in
conjunction with Wikipedia, the accuracy and efficacy of recommendation technologies to
support teaching and learning will only improve. At this point our prototypes are rather
nascent and designed to be used by experts such as instructors. Once accuracy
improves, however, one can imagine how these technologies will, in conjunction with a
teacher or expert, help personalize student learning and diversify instruction in various
settings.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Liang</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Williams</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pursel</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brautigam</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , …
          <string-name>
            <surname>Giles</surname>
            ,
            <given-names>C. L.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>BBookx: An automatic book creation framework</article-title>
          .
          <volume>121</volume>
          -
          <fpage>124</fpage>
          . Lausanne, Switzerland: ACM.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Pursel</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liang</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Williams</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brautigam</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , …
          <string-name>
            <surname>Giles</surname>
            ,
            <given-names>C. L.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>BBookX: Design of an Automated Web-based Recommender System for the Creation of Open Learning Content</article-title>
          .
          <source>Proceedings of the 25th International Conference Companion on World Wide Web</source>
          ,
          <fpage>929</fpage>
          -
          <lpage>933</lpage>
          . International World Wide Web Conferences Steering Committee
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Hilton</surname>
            <given-names>III</given-names>
          </string-name>
          , J., Wiley,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Stein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            , &amp;
            <surname>Johnson</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.</surname>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>The four 'R's of openness and ALMS analysis: frameworks for open educational resources</article-title>
          .
          <source>Open Learning: The Journal of Open</source>
          , Distance and e-Learning,
          <volume>25</volume>
          (
          <issue>1</issue>
          ),
          <fpage>37</fpage>
          -
          <lpage>44</lpage>
          .Author,
          <string-name>
            <given-names>F.</given-names>
            ,
            <surname>Author</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Author</surname>
          </string-name>
          ,
          <string-name>
            <surname>T.</surname>
          </string-name>
          :
          <article-title>Book title</article-title>
          .
          <source>2nd edn. Publisher</source>
          ,
          <string-name>
            <surname>Location</surname>
          </string-name>
          (
          <year>1999</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Butcher</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Hoosen</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2012</year>
          ).
          <article-title>Exploring the business case for Open Educational Resources. Commonwealth of Learning (COL).</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Annand</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Developing a sustainable financial model in higher education for open educational resources</article-title>
          .
          <source>The International Review of Research in Open and Distributed Learning</source>
          ,
          <volume>16</volume>
          (
          <issue>5</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Daniel</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Uvalić-Trumbić</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>UNESCO-COL guidelines for open educational resources (OER) in higher education</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Browne</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Holding</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Howell</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rodway-Dyer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>The challenges of OER to Academic Practice</article-title>
          .
          <source>Journal of Interactive Media in Education.</source>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Yuan</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Recker</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Not all rubrics are equal: A review of rubrics for evaluating the quality of open educational resources</article-title>
          .
          <source>The International Review of Research in Open and Distributed Learning</source>
          ,
          <volume>16</volume>
          (
          <issue>5</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Stribling</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Krohn</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Aguayo</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (n.d.).
          <source>Scigen-an automatic cs paper generator</source>
          ,
          <year>2005</year>
          . URL Http://Pdos. Csail. Mit. Edu/Scigen.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Giles</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Internet encyclopaedias go head to head</article-title>
          .
          <source>Nature</source>
          ,
          <volume>438</volume>
          (
          <issue>7070</issue>
          ),
          <fpage>900</fpage>
          -
          <lpage>901</lpage>
          . https://doi.org/10.1038/438900a
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>