<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Lurch : A Word Processor Built on OpenMath that Can Check Mathematical Reasoning</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Nathan C. Carter</string-name>
          <email>ncarter@bentley.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kenneth G. Monks</string-name>
          <email>monks@scranton.edu</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Bentley University</institution>
          ,
          <addr-line>Waltham, MA</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Scranton</institution>
          ,
          <addr-line>Scranton, PA</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Lurch [5] is a free word processor that can check the mathematical reasoning in a document, including the steps of a mathematical proof, even one not written in a formal style. This paper covers our goals, implementation in terms of OpenMath, and a brief overview of the underlying validation engine.1</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>1.1</p>
    </sec>
    <sec id="sec-2">
      <title>Introduction</title>
      <sec id="sec-2-1">
        <title>Background</title>
        <p>
          Lurch is a free word processor that can check the steps of a mathematical proof, and we have
focused the current version on the needs of introduction-to-proof courses, covering topics such
as logic, introductory set theory, and number theory. Lurch is built on OpenMath [
          <xref ref-type="bibr" rid="ref1 ref3">1, 3</xref>
          ] (as
well as several other technologies, including Qt [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]) and is a free and open-source desktop
application for Windows, Mac, and Linux. Its implementation is discussed in Sections 2 and 3.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Goals</title>
        <p>Our user interface guideline is that users should be able to write math in whatever style or
notation they prefer, with exposition inserted where they feel it's helpful. Especially in a
pedagogical context it is important for the notation and style used in the software to match
that of the textbook and lecture notes, so as not to be an obstacle for the student. Of course,
pencil and paper won't tell you whether your proof is correct, but Lurch will, in addition to
providing the other bene ts that any word processor does, such as cut and paste.</p>
        <p>Lurch is for student use. We're targeting students in their rst proof-based courses, where
Lurch can give frequent, immediate, clear feedback on the validity of the steps in their proofs.
That's the part of the mathematical community we're excited about reaching.</p>
        <p>
          Therefore Lurch is an entirely di erent kind of product than existing proof checkers (e.g.,
Mizar [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] or Coq [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ]) in several ways. First, we do not require the user to learn a speci c
language or rules; in Lurch, the user (or his or her instructor) de nes the rules from scratch.
Second, Lurch does not automate any proof steps on the user's behalf, nor should it; the
student types his or her work in Lurch, and Lurch grades, encourages, and coaches. Finally,
proof checkers often have interfaces suitable for advanced users, sometimes requiring familiarity
with the command line or shell scripts. Lurch is for the typical student, and its user interface
is therefore a familiar one: It is a word processor that gives feedback visually in the document.
        </p>
        <p>1Lurch was supported from 2008-2012 in part by the National Science Foundation's Division of
Undergraduate Education, in the Course, Curriculum, and Laboratory Improvement program (grant #0736644). Views
expressed in this document are not necessarily those of the National Science Foundation.
We want the only learning curve students encounter to come from the mathematics itself, not
Lurch.</p>
        <p>
          Existing research on educational technology suggests that our goals make sense and are
achievable. Investigations into the e ects of automated assessment systems (AiM [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] and
MyMathLab [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]) show the value of computers in giving high-frequency, individualized, and
immediate feedback [
          <xref ref-type="bibr" rid="ref10 ref13 ref14 ref15">10, 14, 13, 15</xref>
          ]. The value of such feedback is well documented in educational
research (reviewed in [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]). This research matches our common sense that more feedback,
delivered in immediate response to each of the student's actions, is better for learning.
2
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Current Implementation</title>
      <p>
        This section explains how far we have come towards achieving the goals in Section 1.2.2 For
more details than what this section provides, see [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] in this same volume. Sections 3 and 4
discuss details of implementation under the hood.
      </p>
      <p>Consider the screenshot of the Mac version of our application in Figure 1. The red and
green thumbs and yellow lights interspersed throughout the document are Lurch's feedback on
the correctness of each step of the user's work. (This feedback can be hidden to obtain a clean
view for printing.) A green thumbs-up icon follows a correct step of work that is correctly
justi ed, a red thumbs-down icon follows a step that contains an error or is supported by the
wrong reason, and yellow lights follow mathematical expressions that are used as premises to
justify later steps of work, but are not themselves justi ed (e.g., the hypotheses in a proof).</p>
      <p>Students can hover their mouse over any of these colored icons for a brief explanation of
its meaning, or double-click it for a detailed report. In order to provide this kind of feedback,
Lurch needs to know the meaning of the mathematical expressions in the document.</p>
      <p>In order for a section of text to be treated as a mathematical expression, the user must mark
it as such with a single click of a toolbar button (or the corresponding keyboard shortcut). The
user selects a portion of text that he or she wishes Lurch to treat as meaningful mathematics,
and then clicks the \Meaningful expression" button on the toolbar. Lurch then wraps the text
in a bubble and reports the understood meaning in a tag atop the bubble, as in Figure 2. Lurch
draws a bubble around a mathematical expression if and only if the user's cursor is inside it.
Both LyX and Word have somewhat similar interfaces for editing mathematics.</p>
      <p>The user will create bubbles often, and thus doing so must be intuitive and take
nearzero time. The \Meaningful expression" button appears on the Lurch toolbar near formatting
options such as bold and italic. It is just as easy to mark text as meaningful as it is to use one
of those common formatting commands; a single click or keystroke does it.</p>
      <p>This interface enables Lurch to know which portions of the user's document he or she
considers meaningful mathematics. Also, in line with our goals from Section 1.2, the user is
stylistically free to structure their document their own way. Because meaning is determined
by what sections of text have been bubbled, the user can alternate whenever they like between
expository text in any natural language and meaningful mathematics, and arrange their work
as informal sentences, formal proof structures, or varying levels in between.</p>
      <p>This interface also brings a pedagogical bene t. Student users, just learning what a proof
is, are forced to indicate what parts of their own proofs are meaningful and which are not.
Experience from testing shows us that students come away from a Lurch-integrated course
knowing very solidly how a proof is structured, because they had to point out this structure to
Lurch before it would check their work.</p>
      <p>2At the time of this writing, the current version of Lurch is 0:7992, released on May 9, 2013.</p>
      <p>Lurch is not yet at version 1.0, and changes to the current design and interface will come
as we evolve our ideas, especially in response to testing. There are many advanced interface
features we'd already like to add, including automatic bubbling of statements and reasons, and
a visualization of the inference and property graphs. Such features will be able to be enabled
or disabled as the user's skills and needs require.
3
3.1</p>
    </sec>
    <sec id="sec-4">
      <title>OpenMath in Lurch</title>
      <sec id="sec-4-1">
        <title>Types of meaning and document structure</title>
        <p>
          All Lurch documents are stored as OpenMath data structures, from the document level (using
application of symbols for concepts like \paragraph") down to the atomic level (using OpenMath
integers, strings, variables, and so on).3 In order to see how these structures are formed, we
3Although OMDoc [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] may be a better choice for storing documents, the current choice was one of
expediency. The word processing technology we're leveraging stores documents as hierarchical data structures, so
using OpenMath objects as generic hierarchical containers was an easy way to save, load, and manipulate such
must rst cover a bit more about meaning in Lurch documents.
        </p>
        <p>
          Lurch makes the following three major categories of bubbles available to users, and provides
toolbar buttons for creating each. The full set of rules stipulating which bubble types can
contain others is given in [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], but no bubble can contain any other bubble only partially.
Meaningful Expressions (hereafter, MEs): Shown using red bubbles, MEs represent
mathematical objects like numbers, polynomials, and statements (as in the previous section).
Properties: Shown using blue bubbles, Properties specify attributes of MEs but are not MEs
themselves.
        </p>
        <p>Contexts: Shown using green bubbles, Contexts delimit the scope of declared variables and
indicate subproof blocks constructed in the course of writing some kinds of proofs.
Properties can be used to modify neighboring MEs, for the purpose of attaching to them labels
(like \label in LATEX), reasons (citations of existing rules), and premises (references to earlier
statements in the same proof). Although no bubbles are visible in Figure 1 because the cursor
is at the end of the document, it contains many invisible ones. For example, on the third line of
the proof, the expression 8x; (g f )(x) = g(f (x)) is an ME and the word \composition" on the
next line is a reason property that applies to it and the preceding ME. Properties can apply to
any number of adjacent, successive MEs, either to the left or right of the property itself. They
do not alter the intrinsic meaning of MEs, but only tell Lurch how to work with those MEs (by
adding labels, reasons, and so on).</p>
        <p>It is possible to see the locations of all bubbles in a document; when a user enables that
option, Lurch shows bubble boundaries as in Figure 3. For even more information, a user can
direct Lurch to include the contents of each bubble's tag after its latter boundary. This results
in a very cluttered view, but it is one way to see full information quickly.</p>
        <p>An ME which is not contained inside any other ME is called a top-level ME. A Lurch
document can be thought of as a sequence of top-level MEs M1; : : : ; Mn. Although there is
normally text between the Mi, Lurch ignores it for the purposes of validation and giving the user
feedback. Properties assign attributes to these MEs and Contexts group together collections of
MEs that have a common purpose or relevance.</p>
        <p>Lurch documents also depend on other Lurch documents in the same way that one chapter
of a math book might depend on a previous chapter. The rules cited in Figures 1 and 3 are
available because their de nitions appear in a document on which that one depends. In Lurch,
an imported document acts as if it appears, invisibly and in a read-only state, in its entirety,
at the top of the document that imports it.
3.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Creating OpenMath objects from MEs</title>
        <p>
          Every ME is either a simple ME (containing no other MEs), of type integer, real, variable,
constant, string, operator, or quanti er; or a compound ME (containing other MEs), of type
function application or quanti cation. These correspond roughly to the major types of simple
and compound expressions in OpenMath [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], and most are stored in exactly the expected way
(e.g., an integer as an OpenMath integer). Every ME bubble lists its content type on its bubble
tag.
        </p>
        <p>Lurch infers the meaning of a simple ME from the text in its bubble. Thus a simple
ME might contain the text 42, and be interpreted as an integer, or contain 3.14159 and be
interpreted as real. Similarly, x would be a variable, "Life, the Universe and Everything"
hierarchies.</p>
        <p>
          [ [ ] [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ] [ [f] [ [+] [x] [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] ] ] ]
        </p>
        <p>Because most users would rather type the expression in a more standard notation, Lurch
tests the contents of every simple ME bubble to see if those contents can be understood by
a built-in parser. If so, then that ME bubble is treated as the (possibly compound) ME
determined by the parsing, while still appearing as a single bubble in the document. This
allows a compound ME to be entered with the same ease as a simple ME and usually produces
more legible results. For example, 3 f (x + 1) can be entered in the typical calculator notation
3*f(x+1), or 3 f(x+1). Such MEs have the same meaning as the fully \expanded" version
shown above.</p>
        <p>Users can expand and collapse MEs between these two forms by using the context menu on
the ME bubble itself. The parser is a convenience feature that allows users to enter compound
MEs in a natural notation, but in every case the actual meaning of an ME is its expanded form,
which is indicative of how it is stored as underlying OpenMath data.</p>
        <p>Eventually the language that Lurch parses will be customizable, but the current version just
uses a standard mathematical syntax. Here are some example expressions the parser currently
understands.</p>
        <p>x^2-1=(x-1) (x+1)</p>
        <p>3^2&gt;2^3
8x, x2A\B , x2A and x2B
f x : x2=x g
Lurch has a symbol toolbar so that users can click to enter symbols like 8 and 2=; this is
the bottom toolbar in Figure 1, with each symbol a button that opens a palette of related
symbols. The software also automatically replaces common LATEX codes for such symbols with
the symbols themselves, so users who prefer keyboard shortcuts can learn those codes.</p>
        <p>Although the internal storage for these well-known symbols uses standard OpenMath
symbols such as subset from content dictionary set1, that may need to change because the meaning
of the symbol in Lurch is whatever the user gives it by means of de ned rules, which may or
may not match the meaning in the content dictionary.</p>
        <p>The current version of Lurch does not have mathematical typesetting capabilities, but later
versions will include them. In particular, superscripts, subscripts, and fractions must be input
using calculator-like shortcuts, such as x^2. But our current focus is on introduction to proof
courses, and so Lurch comes with topics such as logic, set theory, and number theory, which
don't really need much more than this plain notation (with a healthy supply of mathematical
symbols).
4</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Fundamental Theorem of Lurch</title>
      <p>
        The raison d'^etre of Lurch is validation of student work. For each step of student work, Lurch
should give feedback about the correctness of that step. In most cases, an ME is a step of work
if and only if it is a top-level ME and the user has attached a reason to it; details can be found
in the Lurch Advanced Users' Guide [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. That document speci es, for each type of top-level
ME, whether it should be validated, and if so, how. Here is one example, the case for MEs
representing variable or constant declarations (based on ideas in [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]).4
      </p>
      <p>A constant declaration or variable declaration will get a green validation icon if the
following conditions are met.</p>
      <p>1. It is not already declared, i.e., it is not in the scope of the declaration of the
same identi er either as a variable or a constant.</p>
      <p>2. It does not have a reason assigned to it.</p>
      <p>If either of these conditions are not satis ed, the validation icon for the rule will
be red and double clicking or hovering over it with the mouse will give a message
indicating which of the conditions was not met, and how you might go about xing
it.</p>
      <p>4The Advanced Users' Guide is a speci cation of a system, with no attention given to the motivations behind
the design. Lurch is not yet at version 1.0, so the design is not set in stone, and thus a motivating document is
not yet written.</p>
      <p>Such a speci cation exists for each type of ME in Lurch, but the nontrivial ones are too
complex to list here. Instead we illustrate the algorithm with an example.</p>
      <p>All of the rules and de nitions that are available to students when doing their assignments in
Lurch are written in Lurch documents directly. Users who want to customize the mathematics
in Lurch can inspect those built-in, foundational documents, copy them, and alter them to suit
their own style, or write new ones from scratch. As mentioned above, a document containing
rule de nitions can be included in another document as a dependency, making the rules in the
former available in the latter.</p>
      <p>The libraries that ship with Lurch de ne rules that are based on the natural deduction style
of proof used in the second author's introduction to proof course. However there is no need to
use these rules at all. An instructor can type his or her de nitions, rules, and theorems directly
into Lurch to make new dependency documents that the students can then use when working
on their assignments.</p>
      <p>For example, Figure 4 shows a snippet of lecture notes that an instructor might type into
a Lurch document to de ne a new rule. He rst declares a new term (increasing, bubbled as a
constant declaration) which Lurch veri es is not already de ned. He then speci es his proposed
de nition in the form and with the level of detail and speci city that he desired. The more
advanced bubble structure that the instructor must use to alert Lurch to this rule's structure
appears in Figure 5, using visible bubble boundaries as introduced in Figure 3.</p>
      <p>Finally, when a student uses the rule to justify a statement, he or she does so by supplying
the two required premises, and justi es the expression g(s) &lt; g(t) by citing as a reason the name
of the rule (\increasing" in this case), as in Figure 6. Lurch checks that for the appropriate
values of f , s, and t in the rule de nition, the required premises are available to justify the
desired conclusion.</p>
      <p>In addition to the syntactically de ned rules Lurch currently supports, we would also like to
add rules validated by a built-in computer algebra system. Doing so would enable us to support
typical algebra or calculus topics as naturally as we currently support proof-related ones.</p>
      <p>
        This paper does not include all the details of the validation algorithm, but interested readers
can inspect [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. With such a speci cation in hand, we can state and prove the following theorem.
Theorem (One-pass Validation). Validating each top-level meaningful expression in a Lurch
document, according to the requirements in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], in the order they appear in the document, will
result in a correctly validated document.
      </p>
      <p>
        In other words, the results of validation at point P never change the results of validation
at some earlier point Q, and therefore Lurch can validate a document quickly, not having to
read it more than once. Although mathematicians sometimes write documents for which this
strict ordering of ideas and reasons fails to hold, Velleman [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] makes an excellent case that
professors usually do not permit students who are still learning proof writing to structure their
arguments like that. Therefore in a learning tool like Lurch, such a restriction is acceptable, or
even desirable.
      </p>
      <p>The proof of such a theorem involves showing that the speci cation for each type of
meaningful expression depends in no way on the results of validation of meaningful expressions later
in the document. The e ciency bene ts of this theorem have not yet been integrated into
Lurch, because both the design and implementation are still maturing.
5</p>
    </sec>
    <sec id="sec-6">
      <title>Conclusion</title>
      <p>
        Lurch was tested throughout an introduction to formal logic class the rst author taught at
Bentley University in the fall of 2008, and an introduction to proofs course for mathematics and
mathematics education majors the second author taught at the University of Scranton in the
spring of 2013. In both cases, student response was very positive, the details of which appear
in [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>Lurch has many opportunities for improvement, including testing in courses not taught by
the developers and new features (typeset mathematics, more e cient validation algorithms,
more attractive visual representations of premise relationships, and an even less obtrusive
bubbling interface). But the current version, despite its opportunities to improve in many ways,
shows all the signs of having been very bene cial in the courses in which it has been used.</p>
      <p>The Lurch team invites additional collaborators for writing new mathematical topics in
Lurch, doing further classroom testing, and software development.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Olivier</given-names>
            <surname>Arsmac</surname>
          </string-name>
          and
          <string-name>
            <given-names>Stephane</given-names>
            <surname>Dalmas. OpenMath INRIA C/C+</surname>
          </string-name>
          <article-title>+ libraries</article-title>
          . [online],
          <source>Downloaded May 18</source>
          ,
          <year>2008</year>
          . Available at http://www.openmath.org/software/index.html.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Yves</given-names>
            <surname>Bertot</surname>
          </string-name>
          and
          <string-name>
            <given-names>Pierre</given-names>
            <surname>Casteran</surname>
          </string-name>
          .
          <article-title>Coq'Art: the calculus of inductive constructions</article-title>
          . Springer, XXV, 472 p.
          <source>ISBN 978-3-540-20854-9</source>
          ,
          <year>2004</year>
          . Available at http://www.labri.fr/perso/casteran/ CoqArt/index.html.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>S.</given-names>
            <surname>Buswell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Caprotti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.P.</given-names>
            <surname>Carlisle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.C.</given-names>
            <surname>Dewar</surname>
          </string-name>
          , M. Gaetano, and M. Kohlhase, editors.
          <source>The OpenMath standard, version 2.0. The OpenMath Society</source>
          ,
          <year>June 2004</year>
          . Available at http://www. openmath.org/standard/.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Nathan</given-names>
            <surname>Carter</surname>
          </string-name>
          and
          <string-name>
            <given-names>Kenneth G.</given-names>
            <surname>Monks. Lurch</surname>
          </string-name>
          :
          <article-title>A word processor that can grade students' proofs</article-title>
          . In David Aspinall,
          <string-name>
            <given-names>Jacques</given-names>
            <surname>Carette</surname>
          </string-name>
          , James Davenport, Andrea Kohlhase, Michael Kohlhase, Christoph Lange, Paul Libbrecht, Pedro Quaresma, Florian Rabe, Petr Sojka, Iain Whiteside, and Wolfgang Windsteiger, editors,
          <source>Proceedings of the Workshops and Work in Progress at the Conference on Intelligent Computer Mathematics, number 1010 in CEUR Workshop Proceedings</source>
          , Aachen,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Nathan</surname>
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Carter</surname>
            and
            <given-names>Kenneth G.</given-names>
          </string-name>
          <string-name>
            <surname>Monks.</surname>
          </string-name>
          <article-title>Lurch: a word processor that can check your math</article-title>
          . [online],
          <year>2013</year>
          .
          <article-title>A free and open-source software project hosted on SourceForge</article-title>
          .net, available at http://lurch.sf.net.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Nathan</surname>
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Carter</surname>
            and
            <given-names>Kenneth G.</given-names>
          </string-name>
          <string-name>
            <surname>Monks</surname>
          </string-name>
          .
          <article-title>Introduction to Lurch: advanced users guide</article-title>
          ,
          <source>version 1</source>
          .0. [online],
          <year>2013</year>
          . Available at http://lurch.sourceforge.
          <source>net/AUG-v1.html.</source>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.W.</given-names>
            <surname>Chickering</surname>
          </string-name>
          and
          <string-name>
            <given-names>Z.F.</given-names>
            <surname>Gamson</surname>
          </string-name>
          .
          <article-title>Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin</article-title>
          , pages
          <fpage>3</fpage>
          <issue>{7</issue>
          ,
          <year>1987</year>
          . Available at http:// aahebulletin.com/public/archive/sevenprinciples1987.asp.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Gustav</given-names>
            <surname>Delius</surname>
          </string-name>
          and
          <string-name>
            <given-names>Neil</given-names>
            <surname>Strickland</surname>
          </string-name>
          . AiM|assessment in mathematics. [online],
          <source>Accessed on April 22</source>
          ,
          <year>2013</year>
          .
          <article-title>A free and open-source software project hosted on SourceForge</article-title>
          .net, available at http://sourceforge.net/projects/aimmath/.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Adam</given-names>
            <surname>Grabowski</surname>
          </string-name>
          , Artur Kornilowicz, and
          <string-name>
            <given-names>Adam</given-names>
            <surname>Naumowicz</surname>
          </string-name>
          .
          <article-title>Mizar in a nutshell</article-title>
          .
          <source>Journal of Formalized Reasoning</source>
          ,
          <year>2010</year>
          . Available at http://jfr.unibo.it/article/view/1980/1356.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Saliha</surname>
            <given-names>Klai</given-names>
          </string-name>
          , Theodore Kolokolnikov, and Norbert Van den Bergh.
          <article-title>Using Maple and the web to grade mathematics tests</article-title>
          .
          <source>In IWALT 2000 Conference Proceedings</source>
          ,
          <year>2000</year>
          . Available at http: //rc.fmf.uni-lj.si/matija/ACDCA2000/Kolokolnikov-Klai-Bergh-pdf.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>Michael</given-names>
            <surname>Kohlhase</surname>
          </string-name>
          .
          <article-title>An open markup format for mathematical documents</article-title>
          ,
          <source>OMDoc 1</source>
          .2. [online],
          <year>2009</year>
          . Available at http://omdoc.org/pubs/omdoc1.2.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>Pearson</given-names>
            <surname>Education. MyMathLab</surname>
          </string-name>
          . [online].
          <source>A series of mathematics courses on the World Wide Web</source>
          , available at http://www.mymathlab.com/.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Chris</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Sangwin</surname>
          </string-name>
          .
          <article-title>Assessing higher mathematical skills using computer algebra marking through AiM</article-title>
          .
          <source>In Proceedings of the Engineering Mathematics and Applications Conference (EMAC03</source>
          , Sydney, Australia), pages
          <fpage>229</fpage>
          {
          <fpage>234</fpage>
          ,
          <year>2003</year>
          . Available at http://web.mat.bham.ac.uk/C.J.Sangwin/ Publications/Emac03.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Chris</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Sangwin</surname>
          </string-name>
          .
          <article-title>Assessing mathematics automatically using computer algebra and the internet</article-title>
          .
          <source>Teaching Mathematics and its Applications</source>
          ,
          <volume>23</volume>
          (
          <issue>1</issue>
          ):1{
          <fpage>14</fpage>
          ,
          <year>2004</year>
          . Available at http://web.mat.bham. ac.uk/C.J.Sangwin/Publications/tma03.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Michelle</surname>
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Speckler</surname>
          </string-name>
          .
          <article-title>Making the grade: A report on the success of MyMathLab in higher education math instruction</article-title>
          .
          <source>Pearson Education</source>
          , Boston, MA,
          <year>2005</year>
          . Available at http://www.mymathlab. com.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[16] The Qt Project. Qt 4</source>
          .8.
          <string-name>
            <given-names>GNU</given-names>
            <surname>General Public</surname>
          </string-name>
          <article-title>License (and the lesser) version 3</article-title>
          .0,
          <year>2012</year>
          .
          <article-title>A crossplatform application and UI framework for developers using C++</article-title>
          ,
          <string-name>
            <surname>QML</surname>
          </string-name>
          , CSS, and JavaScript, available at http://qt-project.
          <source>org/.</source>
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Daniel</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Velleman</surname>
          </string-name>
          .
          <article-title>Variable declarations in natural deduction</article-title>
          .
          <source>Annals of Pure and Applied Logic</source>
          ,
          <volume>144</volume>
          , 133{
          <fpage>146</fpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>