<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Visualizing Repertory Grid Data for Formative Assessment</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Kostas Pantazos</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ravi Vatrapu</string-name>
          <email>vatrapu@cbs.dk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Abid Hussain</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Computational Social Science Laboratory (CSSL) Department of IT Management, Copenhagen Business School</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Norwegian School of Information Technology</institution>
          ,
          <addr-line>NITH</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>Repertory grid tools systematically collect data that consists of a topic, its elements, constructs and element ratings. Gaining meaningful insights from repertory grids data is a challenge because data analysis is timeconsuming and a significant mental effort is needed. Visualizations aim at facilitating data analysis through a visual and interactive approach, which allows users to understand their data, reflect, and make better decisions. This paper presents an interactive visualization tool for teachers and students. The tool visualizes repertory grid data using two dashboards, where teachers and students can investigate constructs and rating elements of students at the individual or group level. Visualizing the repertory grid data is an initial attempt towards teaching analytics. Future work will focus on evaluating the tool in a real setting with teachers and students, and collecting suggestions for improvement.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        The advancement of technology has enhanced data generation for personal and
professional use – know as Big Data. Instead of producing WORN data (write-once,
read-never coined by Powsner and Tufte), researchers are continuously studying data
from different fields in order to benefit and to extract knowledge. This also is the goal
of Learning Analytics (LA). LA is “the collection and analysis of usage data
associated with student learning” and aims at improving education through interventions after
observing and understanding learning behaviors
        <xref ref-type="bibr" rid="ref1">(Brown, 2011)</xref>
        .
      </p>
      <p>
        Nowadays, data collection is less challenging than data analysis, because the latter
requires additional cognitive effort. Data analysis can be enhanced using visual
representation. Information visualization enhances human cognition by visually presenting
abstract data and revealing patterns, trends and outliers
        <xref ref-type="bibr" rid="ref2">(Card et al., 1999)</xref>
        . Visual
Analytics is the science of combining interactive visualizations with analytical
reasoning techniques to enable users to understand their data, reflect more effectively,
and make better decisions
        <xref ref-type="bibr" rid="ref4">(Keim et al., 2010)</xref>
        .
      </p>
      <p>This paper presents how the Repertory Grid data (i.e. topic, topic elements, topic
constructs and elements ratings) are visualized. Data were collected from a Social
Media course at the Copenhagen Business School using a software application called
Repertory Grid for Formative Assessment (RGFA, http://cssl.cbs.dk/software/rgfa).
Using these data, we developed a visualization tool with two interactive dashboards
for teachers and students. They can investigate constructs and rating elements of
students at the individual or group level. Future work will focus on evaluating the tool in
a real setting with teachers and students, and collecting suggestions for improvement.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Background</title>
      <p>
        Repertory Grid is a technique which uses personal constructs of individuals to
determine the relation among elements of a topic. Repertory Grid Technique (RGT) is
based on the Personal Construct theory developed by George
        <xref ref-type="bibr" rid="ref5">Kelly (1963</xref>
        , 1992). A
grid consists of: a topic, a set of elements, a set of constructs, and a set of rating
elements of constructs.
        <xref ref-type="bibr" rid="ref8">Vatrapu et al. (2012)</xref>
        provides a clear example of a grid, and
discuss how to integrate RPT in “teaching analytics” as in-class activity or a
takehome exercise. The Repertory Grid Technique for formative assessment can indicate
to teachers the knowledge students have on specific topics
        <xref ref-type="bibr" rid="ref8">(Vatrapu et al., 2012)</xref>
        . In
addition, teachers can benefit as they can identify the ability students have to
distinguish between concepts
        <xref ref-type="bibr" rid="ref8">(Vatrapu et al., 2012)</xref>
        . Obtaining an overview, viewing
details-on-demand, and comparing data are some of the tasks where a visual approach
can facilitate data analysis.
      </p>
      <p>
        Information Visualization (InfoVis) is representation of abstract data; data that
“has no inherent mapping to space”
        <xref ref-type="bibr" rid="ref2">(Card et al., 1999)</xref>
        . As an example from social
networks, abstract data are information stored in the database regarding a user (i.e.
name, age, number of followers, number of friends, etc.) InfoVis attempts to reduce
the time and the mental effort users need to analyze large datasets by visually
presenting abstract data
        <xref ref-type="bibr" rid="ref2">(Card et al., 1999)</xref>
        . The InfoVis community has developed several
frameworks and taxonomies. Shneiderman
        <xref ref-type="bibr" rid="ref7">(Shneiderman, 1996)</xref>
        presented a task by
data type taxonomy for InfoVis. This taxonomy classifies visualization data types (1D
Linear, 2D Map, multidimensional, temporal, tree and network) and identifies the
tasks (overview, zoom, filter, details-on-demand, relate, history and extract) that have
to be sup-ported. The result of this taxonomy, lead Shneiderman
        <xref ref-type="bibr" rid="ref7">(Shneiderman, 1996)</xref>
        to define the visual-information seeking mantra: “Overview first, zoom and filter,
then details-on-demand”.
      </p>
      <p>
        Visual Analytics VA is the science of combining interactive visualizations with
analytical reasoning techniques to enable users to understand their data, reflect more
effectively, and make better decisions [10]. In comparison to visualization, visual
analytics is seen as an integrated approach combining visualization, human factors
and data analysis
        <xref ref-type="bibr" rid="ref3">(Keim, 2005)</xref>
        . Keim defined the visual analytics mantra: “Analyze
first, show the important, zoom, filter and analyze further, details on demand”.
      </p>
      <p>The work presented in this paper focuses on visualizing repertory grid data to
facilitate data analysis. Presenting an overview, details-on-demand and supporting
interactivity features allow teachers and students to visually explore constructs and element
ratings among students.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Repertory Grid Visualization Tool</title>
      <p>This section describes the repertory grid visualization tool, which consists of two
dashboards. Before we present the dashboards, we discuss the data used.
3.1</p>
      <p>Data</p>
      <p>Data were collected during a Social Media course at the Copenhagen Business
School. Students were asked to use the RGFA website
(http://cssl.cbs.dk/software/rgfa) and answer to a grid designed for the course. The
task aimed at obtaining an understanding of the students’ level of Social Media
knowledge. Eight popular social networks (i.e. Facebook, Google+, Twitter, Pinterest,
Yammer, Foursquare, Flickr, and Youtube) were used as grid elements. Students had
to elicit constructs using a triadic sorting method of elements. Six triads were
designed for the purpose of this course. For each triad, they describe why one of the
elements is different from others (Opposite Construct) and why the other two are
similar (Similar Construct). Next, they were asked to rate the rest of elements using a
five-point Likert scale. Figure 1 shows and excerpt of the repertory grid data from one
student.
is used to provide an overview of all elicited constructs (i.e. Opposite and Similar
Construct). Details-on-demand are shown in the bubble chart after clicking on a word
in the word cloud. Color and size encodings are used in the bubble chart to distinguish
easier element ratings. Small red letters (S-similar and D-different) are positioned
above bubbles that represent triad’s elements. In this way, teachers and students can
easily view the opposite and the similar elements, and relate them to the other
elements. In addition, they can filter words by frequency and triads using the track bar
and checkboxes respectively. The scope of this dashboard is to indicate teachers the
knowledge students have on specific topics, and more specifically answer questions
such as: which elements are perceived similar and different, which terms are used to
express similarity and difference, are there any differences between two students or
group of students, etc.</p>
      <p>This dashboard also builds on the wording of the elements and constructs that
make up a repertory grid, and uses the word cloud representation. After investigating
constructs and elements ratings, teacher use this dashboard to investigate in details
constructs elicitations. They click on a word in the word cloud, and students’
constructs elicitations related to this word are shown, which are categorized in two
groups: Opposite and Similar Constructs. To facilitate text analysis, the selected word
is also underlined in red in the details view. This dashboards aims at enabling teachers
and users at answering questions like: which term is most used to distinguish
elements, and how the term relates to the context, is the term used to express similarity
or difference, etc.</p>
      <p>This paper describes a visualization tool for repertory grid data to facilitate data
analysis for teachers and students. We used visualization techniques and design
principles from Information Visualization and Visual Analytics to allow teachers and
students to obtain an overview of the data and drill-down into details. The word cloud
representation of constructs provides an overview of the most used terms during
construct elicitations. Details-on-demand views present information regarding the
element ratings and constructs. Filtering among users supports comparison and allows
teachers to perform a formative assessment, and distinguish students’ knowledge
based on elements ratings and constructs. Students can compare their element ratings
and constructs with other students. Color and shape encoding were used in order to
improve readability.</p>
      <p>We decided to use the word cloud visualization due to its popularity. To enrich the
variety of supported visualizations, other visualizations techniques such as treemap
and parallel coordinates are being investigated and will be included in the future
versions. Implementing additional interaction techniques is another important aspect
under investigation. In addition, current limitations such as space, color-blindness and
additional functionalities will be addressed.</p>
      <p>Currently, we have not performed any user study to evaluate the tool. We will
conduct studies with teachers and students to evaluate whether the tool is useful.
Designing studies in real environment will allow us to conclude whether this approach
empowers teachers to obtain a better overview, reflect more efficiently, and make better
decisions. The results of the study will serve as inputs for further improvement.</p>
      <p>We presented our work in progress on visualizing repertory grid data. Information
visualization and visual analytics has gained special attention and is widely used in
other areas, such as healthcare, logistics, etc. Introducing information visualization
and visual analytics in education is a promising research avenue in order to improve
education.
5</p>
    </sec>
    <sec id="sec-4">
      <title>Acknowledgments</title>
      <p>This work is supported by the NEXT-TELL - Next Generation Teaching,
Education and Learning for Life integrated project co-funded by the European Union under
the ICT theme of the 7th Framework Programme for R&amp;D (FP7).
6</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Brown</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2011</year>
          ) ,
          <article-title>“Learning Analytics: The Coming Third Wave,” EDUCAUSE Learning Initiative Brief</article-title>
          , http://www.educause.edu/Resources/ LearningAnalyticsTheComingThir/ 227287.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <given-names>Card S. K.</given-names>
            ,
            <surname>Mackinlay</surname>
          </string-name>
          <string-name>
            <given-names>J. D.</given-names>
            ,
            <surname>Shneiderman</surname>
          </string-name>
          (Eds.) (
          <year>1999</year>
          )
          <article-title>: Readings in information visualization: using vision to think</article-title>
          . Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Keim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <year>2005</year>
          , “
          <article-title>Scaling Visual Analytics to Very Large Data Sets,”</article-title>
          <source>Proc. Workshop</source>
          Visual Analytics.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Keim</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kohlhammer</surname>
            , Ellis,
            <given-names>J. G.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mansmann</surname>
            ,
            <given-names>F</given-names>
          </string-name>
          . (eds) *(
          <year>2010</year>
          ).
          <article-title>Mastering The Information Age - Solving Problems with Visual Analytics</article-title>
          . Eurographics,
          <year>November 2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Kelly</surname>
            ,
            <given-names>G. A.</given-names>
          </string-name>
          (
          <year>1963</year>
          ).
          <article-title>A theory of personality: W</article-title>
          . W. Norton &amp; Company.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Kelly</surname>
            ,
            <given-names>G. A.</given-names>
          </string-name>
          (
          <year>1992</year>
          ).
          <article-title>The Psychology of Personal Constructs: Volume Two: Clinical Diagnosis</article-title>
          and
          <string-name>
            <surname>Psychotherapy</surname>
          </string-name>
          (New ed.): Routledge.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Shneiderman</surname>
            <given-names>B.</given-names>
          </string-name>
          (
          <year>1996</year>
          ):
          <article-title>The eyes have it: A task by data type taxonomy for information visualizations</article-title>
          .
          <source>In Proceedings of the 1996 IEEE Symposium on Visual Languages, VL '96</source>
          , IEEE Computer Society, pp.
          <fpage>336</fpage>
          -
          <lpage>.</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Vatrapu</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reimann</surname>
            ,
            <given-names>P</given-names>
          </string-name>
          , Hussain,
          <string-name>
            <surname>A.</surname>
          </string-name>
          (
          <year>2012</year>
          ), “
          <article-title>Towards Teaching analytics: Repertory Grids for Formative Assessment”</article-title>
          .
          <source>In The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS</source>
          <year>2012</year>
          ) ,.
          <fpage>341</fpage>
          -
          <lpage>345</lpage>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>