=Paper=
{{Paper
|id=Vol-70/paper-6
|storemode=property
|title=The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)
|pdfUrl=https://ceur-ws.org/Vol-70/paper6.pdf
|volume=Vol-70
|dblpUrl=https://dblp.org/rec/conf/pgldb/KortemeyerABBBHKKKMPSS03
}}
==The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)==
<pdf width="1500px">https://ceur-ws.org/Vol-70/paper6.pdf</pdf>
<pre>
  The LearningOnline Network with Computer-Assisted Personalized
                     Approach (LON-CAPA)

   Gerd Kortemeyer (1), Guy Albertelli, Wolfgang Bauer, Felicia Berryman, Jeremy
 Bowers, Matthew Hall, Edwin Kashy, Deborah Kashy, Helen Keefe, Behrouz Minaei-
         Bidgoli, William F. Punch, Alexander Sakharuk, and Cheryl Speier

                                   (1) Gerd Kortemeyer123
                                       North Kedzie Labs
                                  Michigan State University
                                 East Lansing, MI 48824 USA
                                      korte@lon-capa.org


                                               Abstract
      LON-CAPA is a distributed open-source Learning Content Management and Assessment
      System that provides instructors with a common, scalable platform to assist in several
      aspects of teaching a course, from lecture preparation, communication among faculty and
      students, calendar keeping and announcements, to administration of homework assignments
      and exams. It also enables instructors to create educational materials and to share such
      learning resources with colleagues across institutions in a simple and efficient manner.
      Keywords: Open Source Learning Content and Course Management System, Online
      Assessment


1. Introduction

As educational institutions establish an online presence, initial successes are often due
to individual faculty members ("early adopters" of this new technology), working long
hours to develop material more or less single-handedly. Frequently, they are leaving
behind scattered projects, which are of intrinsic value, but of little use for the institution
and far less for the larger academic community. “Late adopters” of technology in
education might altogether refuse to venture into creating new online educational
resources, since the task of creating comprehensive material appears overwhelming in
isolation. To address these problems, a team of faculty and staff from Michigan State
University (MSU) are creating an infrastructure to provide a course management system
(CMS) and resource sharing: the LearningOnline Network with Computer-Assisted
Personalized Approach (LON-CAPA.)
   The roots of this system go back ten years, when a group of faculty at Michigan State
University started developing a sophisticated online homework and assessment system,
with a strong focus on the sciences and mathematics. Soon other universities adopted
the system, and it was not long before an informal culture of inter-institutional sharing
of such resources developed. To formalize and thus further these efforts, the team added
digital library and learning content management capabilities, and the ability for
instructors to assemble these resources.
   Today (Spring 2003), LON-CAPA has become a distributed Learning Content
Management and Assessment System. LON-CAPA and its predecessor systems are
serving a total of 12,000 students per semester at MSU alone, and well over 23,000
students per semester system-wide (middle schools: 300; high schools: 500; community
colleges: 50; four-year colleges: 300; universities: >22,000). Its shared resource pool
currently holds approximately 6,000 original homework and exam problems, 5,000
images, 150 movies, 180 java applets, and 3,000 content pages. Disciplines include

PGLDB’2003, pp. 51-61, 2003.
 PUC-Rio, Rio de Janeiro-RJ, Brazil
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)


astronomy, biology, business, chemistry, civil engineering, computer science, family
and child ecology, geology, human food and nutrition, human medicine, mathematics,
medical technology, physics, and psychology.
   Grants from the Sloan Foundation and from the Mellon Foundation, as the well as the
National Science Foundation, and strong support from Michigan State University have
encouraged us to pursue the development of this enabling technology. The current
leadership team is truly cross-disciplinary: Wolfgang Bauer and Edwin Kashy (physics),
Cheryl Speier (business), Deborah Kashy (psychology), as well as educational
psychology graduate student Helen Keefe as project manager, computer scientist Guy
Albertelli as technical director, and Gerd Kortemeyer (science and mathematics
education) as project director.
   Over the coming three years, with continued support by the National Science
Foundation Information Technology Research program (NSF-ITR #0085921), we plan
to transform this system further beyond the boundaries of MSU's campus into a
dynamic online collaborative community of faculty authors, commercial publishers, and
learners. The LON-CAPA project currently has 18 pilot user institutions. Efforts to
couple LON-CAPA with the NSF National Science Digital Library (NSDL) project are
under way.
   We believe that quickly scaling up this effort while accommodating a diverse user
community is crucial to reach a critical mass of educational content, which could
transform this network into a nationally used pool of online instructional resources.
LON-CAPA provides the infrastructure so that researchers can collect data about online
teaching and learning, as well as investigate market dynamics in an online educational
economy. We plan to eventually develop this network into the independent “LON-
CAPA Academic Alliance,” which remains driven by faculty and is part of the
academic community, yet at the same time involves commercial partners and
contributors.


2. Network Infrastructure

The LearningOnline Network with CAPA is a geographically distributed network of
persistently connected servers at schools, colleges, and universities. Each participating
institution needs to contribute at least one server to the network (Figure 1). An
institution can set up any number of servers within their domain to scale with increasing
workload. The network is set up to be redundant and load-balancing.
   The network is logically divided into so-called domains, which usually correspond to
one institution, such as Michigan State University, North Dakota State University, or
Truckee Meadows Community College. Domains can be used to limit the flow of
content and the extent of user privileges. Users can log into any server in the network.
For example, an MSU user can log into a server at North Dakota State University using
his MSU credentials, and find the exact same environment as on one of the on-campus
servers.
   To faculty users, the distributed content resource pool of LON-CAPA appears as one
large virtual file system, and every resource has a system-wide unique and persistent
URL path, under which it can be accessed from any server in the network. As users are
browsing this filesystem, they are actually transparently accessing content from servers
across the network (Figure 2). As resources are accessed, the network provides
transparent resource replication to provide faster access to the resources.




                        Proceedings of the PGL DB Research Conference                     52
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




                                          Figure 1



3. Resource Creation and Sharing

Cross-institutional resource sharing has been one of the foremost design principles of
the LON-CAPA architecture. Faculty can create learning objects and publish them into
a cross-institutional and distributed resource pool (digital library). Such learning objects
can be simple paragraphs of text, movies, applets, homework problems, etc.
   In addition to providing the distributed digital library with mechanisms to store and
catalog these resources, LON-CAPA enables faculty to combine and sequence these
resources at several levels: each time a resources are assembled, a new learning object is
created, which in itself is re-usable (Figure 3). For example, an instructor from
Community College A can combine a text paragraph from University B with a movie
from College C and an online homework problem from Publisher D, to form one page.
Another instructor from High School E can take that page from Community College A
and combine it with other pages into a module, unit or chapter. Those in turn can be
combined into online course packs (Figure 3). Faculty can design their own curricula
from existing and newly created resources instead of having to buy into a complete off-
the-shelf product.




53                      Proceedings of the PGL DB Research Conference
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




                                          Figure 2

Instructors can specify the actual path through the learning resources through
combinations of learner choices and system-generated adaptations (for example, if the
learner does not pass a test, additional resources may be included). Each learner can
have an individualized curriculum according to preferences, capabilities and skills.
Figure 4 shows a screen shot of the “Resource Assembly Tool” of LON-CAPA, which
is used to combine learning resources (Figure 3). In this particular example, each box
represents a resource – a single page, or a whole module or chapter. The arrows
represent possible paths through the material, and the box labeled “COND” represents a
condition under which the path is available.
   LON-CAPA tracks resource usage: every time a resource gets incorporated into a
learning object of larger granularity, every time it gets deployed in a course, and every
time a learner accesses it, the transaction is recorded and added to the metadata for the
resource. The former two events constitute a form of peer-review, with the approval
being the adoption of a resource. In addition, data such as degree of discrimination,
degree of difficulty, and average number of attempts until mastery of each resource are
recorded. Instructors can use this data to make informed decisions when selecting
resources for their courses and students.



                        Proceedings of the PGL DB Research Conference                     54
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




                                          Figure 3



4. Individualizing Problems

With LON-CAPA, an instructor can create and/or assemble individualized assignments,
quizzes, and examinations with a large variety of conceptual questions and quantitative
problems. "Individualized" means that each student sees a slightly different computer-
generated problem. This encourages collaboration between students on a conceptual
level, but prevents blind copying of answers. Students are given instant feedback and
relevant hints by LON-CAPA and may correct errors without penalty prior to an
assignment's due date - this is formative assessment. Adaptive hints can be incorporated
which can address particular wrong answers. Problems can include pictures, animations,
graphics, tables, links, etc., and available assessment elements include standard
components such as radio button and multiple choice, but also numerical,
multicomponent numerical, complete support of physical units, symbolic math, image
response, dynamic plotting, and random labeling. The writing and development is done
through a web-based editor, and facilitated by templates. The generated files are XML-
documents, which are processed server-side.




55                      Proceedings of the PGL DB Research Conference
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




                                          Figure 4




                                          Figure 5

In Figure 5, one and the same homework problem is rendered for two different students:
the labels in the image (Ma, Mb, Tx, Ty, and Tz) are denoting different masses and
tensions in the Atwood machine, and learners have different choices - not only are the
labels inserted according to their respective function, but also the choices themselves
and their order are varied – a total of more than 10,000 variations on this problem. For
example, in the left rendering, the learner has the choice “Ma*g+Mb*g is ____ Tx,”


                        Proceedings of the PGL DB Research Conference                     56
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)


while in the rendering on the right, a corresponding choice is “Tx is ____
Mb*g+Ma*g.” Obviously, the correct answer could be “less than” in the one case, and
“more than” in the other – if that is true (and in which order), or if the answer is “equal”
in both cases is left as an exercise to the reader. Internal to the homework problem, both
of these choices are members of the same so-called concept group, and the homework
engine insures that one and only one choice from each concept group appears.




         1.   Two M’s have same size acceleration (string does not stretch)
         2.   Weight of the two M’s > tension in top string (cm accelerates)
         3.   Top tension equals the two bottom tensions. (pulley mass = 0)
         4.   Tension acting on the two M’s are equal (pulley mass, frict, = 0)
         5.   Accelerations: small M up (T>Mg), large M down (T<Mg)
         6.   Center of mass accelerates downward
                                          Figure 6

These concept groups can be used to identify student misconceptions even though the
problems that the students see are randomized [1]; LON-CAPA provides analysis tools
for this purpose. Figure 6 shows the correctness of the initial 12% of entries for the
problem Figure 5 for each concept group (since students have multiple attempts to get
each problem correct, it is important to capture the initial response rather than the final
answers). The lower bar is the number of correct response, the upper bar the number of
incorrect responses for each of the six concepts. Clearly, the students have the biggest
problem with the concept that the weight of the two masses is greater than the tension in
the top string.
   Such diagnostic tools allow faculty members to have immediate feedback on the
progress of their students, and detect misconception early on, thus enabling just-in-time
teaching.
   Time saved in routine grading of student work, as well as better insights into the
learning progress, both foster communication between learners and educators –
delegating work that a computer does best to a computer allows humans to interact on a
higher level, even in rather unexpected ways: it was frequently observed that the
addition of LON-CAPA to the instructor-student relationship removes the adversarial
relationship between educators and learners with regard to turning in late homework,
partial credit and immediate feedback. Rather, the learner and the educator are on the


57                      Proceedings of the PGL DB Research Conference
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)


same side, struggling valiantly with the “immovable LON-CAPA beast” that seemingly
comes up with these tough homework problems out of nowhere.
  Problems are stored in an XML format, and as with all other LON-CAPA documents,
problems are one-source-multiple-target. Depending on external parameters, problems
can render themselves in online mode, print mode, online exam mode, and bubble-sheet
mode, as well as edit mode for online problem creation.

 <problem>
 <script type="loncapa/perl">
 $initial_velocity=&random(70,130,5);
 $acceleration=&random(-5,-1.2,0.2);
 $time=&random(0.5,4.5,0.1);
 $distance=($initial_velocity/3.6+$acceleration*$time/2)*$time;
 $forgotinitial=-$acceleration*$time*$time/2;
 </script>
 <startouttext />You are driving with a velocity of $initial_velocity <m>$km/h$</m> and
 step on the brake. The resulting acceleration is $acceleration <m>$m/s^{2}$</m>.
 Starting from when you hit the brake, how far will you have traveled after
 $time <m>$s$</m>?<endouttext />
 <numericalresponse format="1f" unit="m" id="11" answer="$distance">
         <responseparam description="Numerical Tolerance" type="tolerance" default="5%"
 name="tol" />
         <responseparam description="Significant Figures" type="int_range,0-16"
 default="0,15" name="sig" />
         <textline />
         <hintgroup>
 <numericalhint unit="m" id="12" answer="$forgotinitial" name="forgot_initial_velocity">
     <responseparam default="5%" type="tolerance" name="tol" />
 </numericalhint>
 <hintpart on="forgot_initial_velocity">
     <startouttext /> You have forgotten to take into account the initial velocity of
 your car. And, by the way,
 braking does not make you go backwards.<endouttext />
 </hintpart>

         </hintgroup>
 </numericalresponse>
 <postanswerdate><startouttext />The correct formula to solve this problem is
 <m>\[x(t)=vt+\frac{1}{2}at^{2}\]</m>
 <endouttext />
 </postanswerdate>
 </problem>

                                          Figure 7

Figure 7 shows the XML data structure of a numerical homework problem. Note that
the XML incorporates Perl code for the numerical calculations, and LaTeX for the
mathematical typesetting. The problem code has several elements. The <script>-block
sets all variables, where the &random-call produces a random number in the range from
the first to the second parameter, in step sizes of the third parameter. The script then
continues to computer the correct answer, $distance, as well as a wrong answer,
$forgotinitial. In the following text, the variables can simply be referred to by their
names. LaTeX formulas are embedded into <m>-tags. The XML document goes on to
define the correct solution for the problem ($distance), as well as parameters such as
numerical tolerance, significant digits, and – most importantly – the physical unit for the
answer $distance, i.e., “m”. Next, the script defines a conditional hint for the wrong
answer $forgotinitial, and finally, it includes a block which is shown after the answer
date is past (a date set by the instructor).
   While it is an option, authors usually will not write their problems in raw XML.
Instead, each XML element can also be rendered for the “edit” target, which produces
fill-in-the-blank forms as shown in Figure 8.




                        Proceedings of the PGL DB Research Conference                     58
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




                                          Figure 8

Finally, Figure 9 shows the problem as it is rendered for the student, in the particular
state where the student entered an answer covered by the adaptive hint.




                                          Figure 9



5. Course Management

In addition to the learning content creation and management capabilities, LON-CAPA
provides the course management features needed to run a course using this material.
This includes: appropriately enrolling all students and instructors, assigning the dates
and times at which resources are available and homework due, invoking the necessary
marking/grading schemes, as well as managing course communication, feedback,
bulletin boards, calendar, and chat rooms. Templates are available to on-the-fly generate
discussion board, syllabi, simple content pages, and upload non-HTML content such as
PowerPoint presentation or PDF files.




59                      Proceedings of the PGL DB Research Conference
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




6. Learning Outcomes

The ultimate measure of success for our system is whether it indeed proves to be an
effective tool in increasing educational outcomes for students. A number of systematic
studies, conducted primarily within undergraduate physics courses, suggest that LON-
CAPA can have a pronounced impact on student learning. One study [2] followed an
introductory calculus-based physics course from the years before system
implementation until late into its deployment. In the years before using LON-CAPA,
the final grade distribution exhibited the traditional bell shape around a grade of 2.5,
with relatively few students receiving grades of 3.5 or 4.0. After the move to LON-
CAPA, the proportion of students earning a grade higher than 3.0 increased
dramatically. Notably, independent evaluators judged that the examinations used in the
course after deployment were more challenging than those used in earlier years, and so
the positive change in educational outcomes cannot be attributed to a lowering of
standards for the class.
   Two other studies suggest that LON-CAPA may increase the participation and
success of women in the sciences. One study [3] focused on a yearlong physics course
for non-science-majors, in which the system was used only during the second semester.
Final grades from the second semester indicated that women were especially likely to
benefit from the system. The second study indicated that women, who began the course
significantly less well prepared than men, improved their performance relative to men
across the semester until there were no gender differences by the final exam. These
preliminary findings are currently being studied in more detail.
   A majority of students, typically 80%, consider that LON-CAPA helps them learn
and understand the course material. This is surprising, since the time students spend
working on assignments and other course requirements has increased by nearly a factor
of two.


7. Unanticipated Challenges

LON-CAPA has a proven track record of using technology to challenge the learners. A
recent development has been that the learners use technology to challenge the educators.
While the personalization feature of LON-CAPA still curbs rote copying of answers
more than any other system, students have established interactive web sites (financed by
banner adds and donations) to network with each other in an attempt to defeat the
system. Whole Excel spreadsheets are going up in an attempt to reverse-engineer the
individualization mechanisms – one problem at a time, and featuring “52,359
homework forum messages.” These efforts in turn have moved educators to write even
more sophisticated problems, using for example random labeling on individualized
graphs. The final word on this “technology war” has not been spoken yet, however, a
survey in which students after the end of a course were asked how often they used the
cheat site has shown that the final grade was significantly negatively correlated to site
usage.




                        Proceedings of the PGL DB Research Conference                     60
     The LearningOnline Network with Computer-Assisted Personalized Approach (LON-CAPA)




8. Sustainability and Positioning

LON-CAPA is unique amongst course and learning content management systems in that
it is developed under the open-source GNU General Public License. We do not expect
that the Michigan State University team of developers alone will be able to sustain the
code-base of LON-CAPA (currently around 60,000 lines of code) without input from a
broader development community. In contrast to proprietary commercial software, the
cooperative open-source community has a fast turn-around on debugging and adaptation
to new technologies (as demonstrated by the success of the Linux operating system).
We are investigating if this model is successful for a system that operates in the
academic community, where the main stakeholders usually are not themselves
programmers. However, already today, the LON-CAPA system has a code base
comprised of contributions from four universities, and incorporates a vast number of
available open-source tools and libraries.
   LON-CAPA was never intended as a competitor to commercial course management
systems, and instead offers an inherently academic approach to online teaching and
learning. Its sharing of resources is formed around the principle of peer-review (through
peer-adoption of resources and giving credit to the author), and will soon further
formalize this concept for explicitly peer-reviewed subsets of the resource pool.
Through its ability to be molded and modified, as well as its extensive logging of
learner interaction, it allows instructors to conduct educational experiments on a large
scale, and to track the educational impact of targeted interventions. The project was
recognized by IEEE and ASEE with three best papers awards at the annual Frontiers in
Education Conference (1997, 1998, 2000), the Ben Dasher Award (1998), the
Wickenden Award (1999), and a ComputerWorld Honors Award (2003).
   Over the years, the project has sponsored annual conferences and workshops, with a
growing number of participants. At the 2002 LON-CAPA user conference, hosted by
Florida State University, 56 faculty members - mostly science researchers – from 22
institutions attended. The 2003 conference at Truckee Meadows Community College in
Reno, NV, had 72 participants from 28 institutions. LON-CAPA provides a catalyst for
them to discuss educational issues, for which they would not usually have a forum.
   Increasing LON-CAPA’s user community is crucial to both the immediate success as
well as the long-term sustainability of the project. Started as a purely academic effort,
eventually the system will need to exhibit financial sustainability without grant support.
Making this transition without loosing the unique characteristics of the system and
without turning an academic department with research faculty into a purely financial
interest-driven service entity is a challenge.


REFERENCES

G. Albertelli, B. Minaei-Bidgoli, W. F. Punch, G. Kortemeyer, and E. Kashy, Concept
    Feedback in Computer-Graded assignments, Frontiers in Education Conference
    Proceedings, 1375 (2003)
D. A. Kashy, G. Albertelli, E. Kashy and M. Thoennessen, Teaching with ALN Technology:
     Benefits and Costs, Journal of Engineering Education, 89, 499 (2001)
M. Thoennessen and M. J. Harrison, Computer-Assisted Assignments in a Large Physics Class,
     Computers Educ. 27, 141 (1996)



61                      Proceedings of the PGL DB Research Conference

</pre>