<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>HRIS Design Characteristics: Towards a General Research Frame- work</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Development</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Christian Gasper, Saarland University</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Daniel Mueller, Saarland University</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Stefan Strohmeier, Saarland University</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2010</year>
      </pub-date>
      <fpage>250</fpage>
      <lpage>267</lpage>
      <abstract>
        <p>Design characteristics constitute a promising approach for supporting researchers and practitioners in developing, implementing and improving Human Resource Information Systems (HRIS) to ensure the anticipated benefit for those firms which introduce and/or apply them. Constituting an intuitively appealing approach, the question how to understand and apply systematically such design characteristics is of specific interest. Hence, the paper proposes a general research framework of HRIS design characteristics which a) allows researchers to understand and apply crucial aspects relevant to HRIS design characteristics better so that b) practitioners may be supported in developing, implementing and permanently improving successful HRIS.</p>
      </abstract>
      <kwd-group>
        <kwd>HRIS Design Characteristics</kwd>
        <kwd>HRIS Implementation</kwd>
        <kwd>HRIS Improvement</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Human Resource Information Systems (HRIS) can be understood as “configurations of
different interacting systems that aim at generating and delivering [Human Resource]
HR functionality in order to automate and informate [Human Resource Management]
HRM [
        <xref ref-type="bibr" rid="ref61">63</xref>
        ]. With a view to HR core functions, these are, among others, recruiting and
selection [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], compensation and benefits [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], training and development [
        <xref ref-type="bibr" rid="ref64">66</xref>
        ],
performance management [
        <xref ref-type="bibr" rid="ref43">44</xref>
        ] as well as HR planning [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. In so doing, HRIS show
diverse benefits such as the improvement of HR operations and management processes
by means of increased quality of decision making [
        <xref ref-type="bibr" rid="ref28 ref35 ref4">4, 29, 36</xref>
        ] or the improvement of
“employee satisfaction by delivering HR services more quickly and accurately” [
        <xref ref-type="bibr" rid="ref28">29</xref>
        ].
However, the profit of applying HRIS strongly depends on their appropriate
development, implementation and permanent improvement as only properly developed,
implemented and permanently improved HRIS will ascertain the success [
        <xref ref-type="bibr" rid="ref13 ref28 ref59">13, 29, 61</xref>
        ].
On the other hand, this implies that HRIS success is manageable, at least, to a certain
degree [
        <xref ref-type="bibr" rid="ref62">64</xref>
        ]. In so doing, design characteristics relevant to the success of HRIS may
support HRIS-related decision makers, system developers as well as system
implementers in ascertaining the success of HRIS. Being of special importance, HRIS
design characteristics are understood as a set of properties inherent to HRIS [
        <xref ref-type="bibr" rid="ref27">28</xref>
        ] by
which they can be developed, implemented and permanently improved [
        <xref ref-type="bibr" rid="ref13 ref28 ref59">13, 29, 61</xref>
        ] and
which are conceptually assumed or empirically ascertained to have a positive impact on
system success [
        <xref ref-type="bibr" rid="ref11 ref12 ref45 ref68 ref69 ref74 ref9">9, 11, 12, 46, 70, 71, 76</xref>
        ]. To be more concrete, HRIS design
characteristics may help HRIS-related decision makers to better validate the capabilities
of particular HRIS, and based on this, to better derive necessary improvement measures
in order to warrant successful HRIS. Thus, by use of corresponding HRIS design
characteristics, HRIS-related decision makers may be enabled to better detect in how far
HRIS actually meet crucial aspects relevant to HRIS success [
        <xref ref-type="bibr" rid="ref28 ref59">29, 61</xref>
        ]. Besides, HRIS
design characteristics may support system developers in the preparation of the final
HRIS system specification, respectively help them to build or select successful HRIS
based on this initial system specification [
        <xref ref-type="bibr" rid="ref13 ref28">13, 29</xref>
        ]. With a view to system implementers,
design characteristics may help them to customize HRIS precisely according to the
(internal/external) customers’ requirements [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Given this, researchers may be
predominantly interested in the derivation of as well as the engagement with HRIS
design characteristics by means of rigorous foundations and methodologies so that they
may better support the aforementioned stakeholders in the development,
implementation and permanent improvement of successful HRIS [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ].
      </p>
      <p>
        However, only few research contributions explicitly deal with HRIS design
characteristics by exploring [
        <xref ref-type="bibr" rid="ref36 ref44 ref54 ref66 ref73">37, 45, 55, 68, 75</xref>
        ], applying [
        <xref ref-type="bibr" rid="ref1 ref36 ref37 ref38 ref51 ref54 ref66 ref73 ref74">1, 37, 38, 39, 52, 55, 68, 75,
76</xref>
        ] or reviewing [
        <xref ref-type="bibr" rid="ref44">45</xref>
        ] diverse design characteristics at present. This may be mainly due
to the fact that existing foundations such as the Technology Acceptance Model [
        <xref ref-type="bibr" rid="ref10 ref69 ref70 ref71 ref8 ref9">8, 9, 10,
71, 72, 73</xref>
        ] or the DeLone and McLean Model of IS Success [
        <xref ref-type="bibr" rid="ref11 ref12 ref56">11, 12, 58</xref>
        ] do not propose
concrete guidelines how to understand and apply design characteristics relevant to the
success of HRIS [
        <xref ref-type="bibr" rid="ref11 ref12 ref45 ref69 ref71 ref74">11, 12, 46, 71, 73, 76</xref>
        ].
      </p>
      <p>Hence, our current understanding of HRIS design characteristics is quite limited at
present and there is a necessity to suggest general insights. Given this, the main purpose
of this paper is to introduce HRIS design characteristics as an emerging and mandatorily
needed field of research in order to ensure the success of HRIS. However, due to its
comprehensive and interdisciplinary character systematic engagement and debate
around particular aspects relevant to HRIS design characteristics (e.g. definition of the
application target, the method of elicitation, respectively evaluation of HRIS design
characteristics) is needed in order to better guide and structure the upcoming discourse
of this emerging field of research, and consequently, the successful selection and
application of relevant HRIS design characteristics. Thus, the current paper does not
intend to enumerate particular HRIS design characteristics but primarily aims at paving
the way for HRIS design characteristics research in general by proposing a
parsimonious, but general, research framework of particular aspects relevant to HRIS
design characteristics. Using this framework, researchers should be enabled to better
understand and apply crucial aspects relevant to HRIS design characteristics so that,
subsequently, practitioners may be better supported in developing, implementing and
improving successful HRIS.</p>
      <p>In order to deal with these questions, the paper is structured as follows: based on a
clarification of HRIS a general research framework of selected issues relevant to HRIS
design characteristics in particular will be derived. Subsequently, the framework will be
discussed and exemplarily illustrated by means of an ongoing HRIS research project.
Thirdly, both practice and research-oriented implications will be derived.
At the current stage of research, a general research framework conflating particular
aspects relevant to HRIS design characteristics is considered as the most appropriate to
better guide and structure the upcoming discourse of this emerging field of research,
and thus, the successful selection and application of relevant HRIS design
characteristics. In short, the framework distinguishes between the following aspects
relevant to HRIS design characteristics (see Table 1):
type of
success measure
relevant to
type of
design
characteristic
type of
application target
validity
method of
elicitation
method of
evaluation
level of granularity
decision maker
system developer
system implementer
system user
system-related</p>
      <p>information-related
development
implementation
improvement
theory</p>
      <p>universal
literature
review
use
case
non-empirical
coarse-grained
case
study
empirical
medium-grained</p>
      <p>contingent
survey
[...]
combinatorial
approaches
combinatorial
approaches
fine-grained</p>
      <p>
        At first, the type of HRIS success measure referring to particular stakeholders is
considered to be obviously of relevance for HRIS design characteristics. For instance,
whereas HRIS-related decision makers, system developers as well as system
implementers may more focus on resource- (e.g. budget, time), feature-, or
revenue/profit-related issues, system users may tend to define HRIS success with a view to their
level of individual productivity or satisfaction with the system [
        <xref ref-type="bibr" rid="ref57 ref58 ref67">59, 60, 69</xref>
        ]. Besides,
system- and information-related design characteristics as a basic and rough
categorization of HRIS design characteristics are introduced, defined and illustrated [
        <xref ref-type="bibr" rid="ref11 ref12 ref45 ref69 ref74">11,
12, 46, 71, 76</xref>
        ]. In addition to that, crucial application targets of system- and
information-related HRIS design characteristics are presented, amongst them the
development, the implementation as well as the permanent improvement of HRIS [
        <xref ref-type="bibr" rid="ref13 ref28 ref59">13,
29, 61</xref>
        ]. Furthermore, methods for the elicitation of HRIS design characteristics are
illustrated as an important scientific milestone as researchers have to set the course
which system- and information-related HRIS design characteristics will be evaluated
subsequently at which quality. For example, HRIS design characteristics may differ
according to their level of validity (e.g. universally applicable vs. contingent design
characteristics) or granularity (e.g. coarse-grained vs. fine-grained design
characteristics).
2.1
      </p>
      <sec id="sec-1-1">
        <title>Type of Success Measure</title>
        <p>
          HRIS success, also called HRIS effectiveness [
          <xref ref-type="bibr" rid="ref11 ref12 ref19">11, 12, 19</xref>
          ] among others, is understood
as the degree to which the person developing, implementing or permanently improving
HRIS believes that the stakeholder (in whose interest the development, implementation
and permanent improvement is being made) is better off [
          <xref ref-type="bibr" rid="ref56">58</xref>
          ].
        </p>
        <p>
          In so doing, IS success measures can be classified according to the following
stakeholders, among others, HRIS-related decision makers, system developers, system
implementers as well as system users [
          <xref ref-type="bibr" rid="ref57 ref67">59, 69</xref>
          ]. From a decision maker’s perspective,
successful HRIS may maximize the following aspects, among others [
          <xref ref-type="bibr" rid="ref58">60</xref>
          ]: cost
efficiency (e.g. IT operations), service-to-the-business-related issues (e.g. customer
satisfaction with IT products/services), business improvements (e.g. IT support
effectiveness) as well as revenue-/profit-related issues (e.g. IT profit generation,
competitive advantage). On the other hand, from a system developer’s perspective,
successful HRIS may be completed on time and under budget, may show a set of
features consistent with the system specification, and may operate properly [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. With a
view to system implementers, successful HRIS may be easy and fast to adjust to the
(internal/external) customers’ requirements [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. Finally, system users may find HRIS
successful if they contribute to maximize, among others, their perceived level(s) of
individual productivity, satisfaction or usefulness/ease of use while using these systems
[
          <xref ref-type="bibr" rid="ref10 ref11 ref12 ref8">8, 10, 11, 12</xref>
          ].
        </p>
        <p>However, in order to maximize HRIS success, researchers and practitioners have to
know more about its underlying drivers. Thus, the subsequent chapter presents
systemand information-related HRIS design characteristics as crucial drivers of HRIS success,
among others.
2.2</p>
      </sec>
      <sec id="sec-1-2">
        <title>Type of Design Characteristic</title>
        <p>
          HRIS design characteristics are understood as a set of properties inherent to HRIS [
          <xref ref-type="bibr" rid="ref27">28</xref>
          ]
by which they can be developed, implemented and permanently improved [
          <xref ref-type="bibr" rid="ref13 ref28 ref59">13, 29, 61</xref>
          ]
and which are conceptually assumed or empirically verified to have a positive impact on
system success [
          <xref ref-type="bibr" rid="ref11 ref12 ref45 ref68 ref69 ref74 ref9">9, 11, 12, 46, 70, 71, 76</xref>
          ]. There are different possibilities to categorize
HRIS design characteristics, while a common categorization in the literature roughly
distinguishes between system-related and information-related design characteristics [
          <xref ref-type="bibr" rid="ref11 ref12 ref45 ref69 ref74">11,
12, 46, 71, 76</xref>
          ]. Hence, system- and information-related design characteristics relevant
to the success of HRIS are considered to constitute a basic and rough categorization of
HRIS design characteristics. Whereas system-related design characteristics measure the
desired properties of an HRIS itself (e.g. “reliability”, “security” [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ]),
informationrelated ones (e.g. “understandability”, “consistency” [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ]) measure the desired
properties which refer to the information provided by the HRIS [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. Thus,
systemrelated design characteristics may constitute a valuable means to the stakeholders
involved (e.g. system developer, system implementer) to develop, implement and
permanently improve successful HRIS. Besides, information-related design
characteristics may support information providers in creating information relevant to
system users’ particular requirements (e.g. understandable, consistent and credible
information).
        </p>
        <p>Refining and adjusting these two major categories of HRIS design characteristics
towards individual HR requirements, and subsequently considering these design
characteristics, may lead to practical HRIS development, implementation and
(permanent) improvement processes which may contribute to an overall HRIS success.
In so doing, the subsequent chapter illustrates the development, implementation and
(permanent) improvement of HRIS by use of system- and information-related design
characteristics as major fields of application where design characteristics may support
the stakeholders involved in attaining successful HRIS.
2.3</p>
      </sec>
      <sec id="sec-1-3">
        <title>Type of Application Target</title>
        <p>
          Given their manageability, and thus their crucial impact on HRIS success [
          <xref ref-type="bibr" rid="ref11 ref12 ref62">11, 12, 64</xref>
          ],
system- and information-related HRIS design characteristics may support the
stakeholders involved in successfully accomplishing the development, the
implementation as well as the (permanent) improvement of HRIS [
          <xref ref-type="bibr" rid="ref13 ref28 ref59">13, 29, 61</xref>
          ].
To begin with the development of HRIS, design characteristics may support the
concretization of the system specification (i.e. how the HRIS will operate). In so doing,
HRIS design characteristics may be concretized alternatively by use of technical
concepts, i.e. pure textual descriptions or visual representations via UML diagrams
and/or user interface mockups. Based on such a system specification, HRIS design
characteristics may subsequently help the stakeholders involved to either build or select
(in case of a packaged software design) HRIS accurately [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. Thereby, the selection of
pre-packaged HRIS by use of design characteristics may help to avoid costly
misconceptions of HRIS as vendor software packages may not be selected based on an
incomplete, inaccurate or irrelevant system specification [
          <xref ref-type="bibr" rid="ref28">29</xref>
          ]. Besides, organizations
may decide to select an external HRIS developer, i.e. to outsource the HRIS
development to an external company or to obtain access to existing software through an
application service provider as external software developers may yield vast resources,
experiences, and technical skills to design a much more effective solution than would be
otherwise possible [
          <xref ref-type="bibr" rid="ref28">29</xref>
          ].
        </p>
        <p>Beyond, during the implementation HRIS design characteristics might be considered as
a valuable means which may guide the customization, i.e. the precise adjustment, of an
HRIS to the (internal/external) customers’ requirements.</p>
        <p>Regarding the permanent improvement of HRIS, which is considered to be mainly
ensured by their comprehensive evaluation, HRIS design characteristics are considered
to constitute a valuable means to better monitor in how far the HRIS under
consideration meets the (elicited/existing) design characteristics relevant to HRIS
success.</p>
        <p>
          Given this, it is of particular scientific interest to support practitioners in the
specification (see Chapter 2.4 &amp; 2.7), elicitation (see Chapter 2.5) and evaluation (see
Chapter 2.6) of relevant HRIS design characteristics so that practitioners may develop,
implement and permanently improve successful HRIS [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ].
2.4
        </p>
      </sec>
      <sec id="sec-1-4">
        <title>Range of Validity</title>
        <p>
          In order to select and apply system- and information-related HRIS design characteristics
within each of the application targets properly (see Chapter 2.3), it is relevant to the
stakeholders involved to know about their validity, i.e. if particular HRIS design
characteristics are thought to be universally valid or dependent on diverse contingency
factors [
          <xref ref-type="bibr" rid="ref31 ref39">32, 40</xref>
          ].
        </p>
        <p>
          In regard to possible contingencies, the success measure(s) to be achieved may
constitute a prominent influence factor which may have a crucial impact on the validity
of particular system- and information-related HRIS design characteristics to be selected
and applied by the stakeholders involved. For instance, whereas HRIS-related decision
makers, system developers and system implementers may more focus on resource- (e.g.
budget, time), feature-, or revenue-/profit-related issues, system users may tend to
define HRIS success with a view to their perceived level of satisfaction with HRIS [
          <xref ref-type="bibr" rid="ref57 ref58 ref67">59,
60, 69</xref>
          ]. Hence, HRIS design characteristics may act as a function of their underlying
contingencies, in this particular case the kind of success measure depending on the
corresponding stakeholders involved (e.g. resource-, feature-, revenue-/profit-related
issues or level of perceived satisfaction with HRIS). In so doing, design characteristics
for stakeholders such as HRIS-related decision makers, system developers and system
implementers may be shaped so that they might have a crucial impact on resource-,
feature- or revenue-/profit-related issues (e.g. “pre-packaged”, “easy and fast to
customize”, “economical”, “reliable”, “secure”, etc.). As distinct from this, HRIS design
characteristics for stakeholders such as system users might be streamlined in order to
maximize their perceived individual productivity, respectively satisfaction while using
HRIS (e.g. “understandability” and “consistency” of the information provided by the
HRIS, etc.).
        </p>
        <p>
          In addition to the success measure(s) to be achieved, the type of application target may
be considered as a further contingency factor of HRIS design characteristics. For
instance, during the development as well as the implementation phase, HRIS design
characteristics contingent on the “organizational culture” [
          <xref ref-type="bibr" rid="ref46">47</xref>
          ], the “organizational
context” (e.g. size, resource constraints, time frame, kind of HR core function [
          <xref ref-type="bibr" rid="ref50 ref63">51, 56,
65</xref>
          ]) or the “technological change” in general [56] may be considered in order to better
adjust the HRIS to the (internal/external) customers’ requirements. Beyond, HR core
functions to be supported by HRIS [56], amongst them recruiting and selection [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] or
training and development [
          <xref ref-type="bibr" rid="ref64">66</xref>
          ], may constitute another contingency factor which may
have a crucial impact on HRIS design characteristics. Potential examples of such
contingent HRIS design characteristics are, among others, the “accessibility of the user
interface”, the “user interface appeal”, the “interactivity within the user interface”, the
“multimodality of information” [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ] as well as the “multilingualism of information”.
On the contrary, universally valid HRIS design characteristics may be best suited for the
improvement of an HRIS and replaced by more contingent ones in order to better adjust
to particular HRIS properties (e.g. portable competence profile, support of particular
technical standards, etc.) or unexpected system errors/failures. Potential examples of
such universally valid design characteristics are, among others, the “reliability” or
“security” of an HRIS as the warranty of these design characteristics should always be
ensured and not be driven by “cultural”, “organizational” or “technological”
contingencies.
        </p>
        <p>
          Hence, HRIS design characteristics are thought to be located on a “continuum of
validity”, ranging from universally applicable HRIS design characteristics to highly
contingent ones. Thereby, the knowledge about such a “continuum of validity” may
help researchers in eliciting (see Chapter 2.5) and evaluating (see Chapter 2.6)
appropriate HRIS design characteristics so that practitioners may develop, implement
and permanently improve successful HRIS [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ].
2.5
        </p>
      </sec>
      <sec id="sec-1-5">
        <title>Method of Elicitation</title>
        <p>The elicitation of HRIS design characteristics may predominantly be undertaken by
researchers so that practitioners may draw on comprehensive sets of well-extracted
design characteristics relevant to the development, implementation and (permanent)
improvement of successful HRIS. In so doing, the elicitation of HRIS design
characteristics constitutes an important scientific milestone as researchers determine
which HRIS design characteristics will be evaluated subsequently (see Chapter 2.6) at
which quality (see Chapter 2.4 &amp; 2.7).</p>
        <p>Hence, methods of elicitation are understood as rigorous and thus systematic ways of
ascertaining system- and information-related HRIS design characteristics which are
made available to practitioners so that they may develop, implement and improve
successful HRIS.</p>
        <p>Thereby, the following ways of ascertaining HRIS design characteristics are suggested:
theory-grounded as well as literature-, review-, use case-, case study- or survey-based
approaches, and combinations of them.</p>
        <p>
          Theories as a base for the elicitation of HRIS design characteristics can be generally
described as a general set of statements which aim at explaining what is, predict what
will happen and provide a basis for intervention and action [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. In so doing, Doty and
Glick [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] provide three primary criteria a theory should meet, namely: identification of
constructs (here: independent variables such as design characteristics; dependent
variables such as HRIS success measures), specification of relationships among these
constructs which finally have to be falsifiable. In so doing, promising theories for the
elicitation of HRIS design characteristics are, among others, the Technology
Acceptance Model [
          <xref ref-type="bibr" rid="ref10 ref69 ref70 ref71 ref8 ref9">8, 9, 10, 71, 72, 73</xref>
          ] as well as the DeLone and McLean Model of IS
Success [
          <xref ref-type="bibr" rid="ref11 ref12 ref56">11, 12, 58</xref>
          ]. This finding might predominantly be due to the fact that the
Technology Acceptance Model as well as the DeLone and McLean Model of IS Success
comprehensively illustrate (inter-)relationships between system- and
informationrelated design characteristics and important measures relevant to HRIS success (e.g.
individual productivity, satisfaction or usefulness/ease of use while using HRIS [
          <xref ref-type="bibr" rid="ref10 ref11 ref12 ref8">8, 10,
11, 12</xref>
          ]). However, due to its comprehensiveness and robustness, the application of the
DeLone and McLean Model of IS Success is particularly recommended for a
theoretically grounded elicitation of system- and information-related HRIS design
characteristics. The outcome of such an elicitation procedure may consists of
comprehensive sets of system- and information-related HRIS design characteristics
which subsequently could be applied in the realm of (experimental/large-scaled)
empirical studies.
        </p>
        <p>
          Besides, literature reviews constitute another potential method for the elicitation of
HRIS design characteristics. Thereby, a literature review may support researchers in
better extracting, contextualizing or structuring information relevant to system- and
information-related HRIS design characteristics, among others the kind of foundation
(e.g. conceptual, theoretical), the object of analysis (e.g. HRIS in general, HRIS
contingent on HR core functions such as training and development in particular), the
data gathering or data analysis method as well as the results achieved. However, except
one literature on design characteristics relevant to HRIS subtypes to training and
development [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ], there currently exists an urgent need to conduct such literature
reviews as the selection and application of design characteristics uniformly appears to
be highly arbitrary so far [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ].
        </p>
        <p>
          Concerning use cases as a potential means for the elicitation of HRIS design
characteristics they may provide “a standard way of capturing, exploring, and
documenting what a system should do” [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. To be more concrete, use cases may support
system developers in better contextualizing, and thus specifying HRIS requirements. In
so doing, use cases may help system developers in selecting appropriate HRIS design
characteristics which may subsequently support system implementers as well as
HRISrelated decision makers during the customization and (permanent) improvement of
HRIS [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
In addition, case studies are equally considered to be a valuable instrument to help
imitating and/or simulating a real situation where HRIS design characteristics may be
an issue [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. Thereby, the main purpose of case studies, which can be described as
verbal representations of reality [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ], is to illuminate a decision or set of decisions
regarding the development, implementation and improvement by means of HRIS design
characteristics as well as their impact on particular success measures [
          <xref ref-type="bibr" rid="ref75">77</xref>
          ]. In so doing,
case studies might be a valuable means for training purposes (e.g. best practices in
HRIS design characteristics) as they cover a huge plethora of contextual conditions
which might be highly pertinent to the selection and application of design
characteristics relevant to the success of the particular HRIS under consideration [
          <xref ref-type="bibr" rid="ref75">77</xref>
          ].
Regarding survey techniques, qualitative approaches can be distinguished from
quantitative ones [
          <xref ref-type="bibr" rid="ref29">30</xref>
          ], whereas both of them can be either conducted experimentally or
non-experimentally. Thereby, given the benefits of an experimental design, such as
controlling relevant while excluding confounding variables, ensuring direct relevant
experiences of respondents, and, particularly enabling the manipulation of specific
HRIS design characteristics [
          <xref ref-type="bibr" rid="ref30">31</xref>
          ], experimental designs are considered to constitute a
promising approach for researchers in order to empirically ascertain relevant
systemand information-related HRIS design characteristics [
          <xref ref-type="bibr" rid="ref49">50</xref>
          ]. With a more particular view
to qualitative approaches, focus groups may be particularly useful for exploratory
research when rather little is known about the phenomenon of interest [
          <xref ref-type="bibr" rid="ref47 ref6 ref60">6, 48, 62</xref>
          ]. For
example, a focus group may support researchers in the elicitation of unknown HRIS
design characteristics as well as in their operationalization while preparing a
questionnaire for a quantitative, (non-)experimental survey approach [
          <xref ref-type="bibr" rid="ref29 ref55">30, 57</xref>
          ]. In so
doing, quantitative approaches such as (expert) Delphi studies may be an appropriate
means for systematically analyzing complex and multifaceted HRIS-related issues that
are not directly and easily accessible via quantitative research approaches [
          <xref ref-type="bibr" rid="ref18 ref20 ref21 ref32">18, 20, 21,
33</xref>
          ]. For example, there is pioneering work [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ] which systematically ascertains
systemand information-related design characteristics of HRIS subtypes relevant to training and
development by use of an expert Delphi study amongst European e-learning experts.
Further approaches for the elicitation of HRIS design characteristics are, among others,
experimental standardized written offline interviews [
          <xref ref-type="bibr" rid="ref49 ref65">50, 67</xref>
          ] as well as
nonexperimental standardized written online [
          <xref ref-type="bibr" rid="ref34 ref51 ref72">35, 52, 74</xref>
          ] or offline [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] interviews.
Subsequent to the ascertainment of HRIS-related design characteristics, the content
analysis may support researchers in coagulating, respectively extracting, relevant
system- and information-related HRIS design characteristics out of the data acquired by
“following content analytic rules and step by step models, without rash quantification”
[
          <xref ref-type="bibr" rid="ref42 ref53">43, 54</xref>
          ].
        </p>
        <p>
          Beyond mere manifestations, combinatorial approaches are considered to be most
suitable as they may “pool the forces” of each single method of elicitation. For instance,
the necessity of an expert Delphi study to elicit particular HRIS design characteristics
may be the main outcome of a preceded literature review. The expert Delphi study in
turn may be founded on a theoretical underpinning such as the DeLone and McLean
Model of IS Success which offers a basic and rough categorization of system- and
information-related design characteristics [
          <xref ref-type="bibr" rid="ref11 ref12">11, 12</xref>
          ]. Finally, the outcomes of the expert
Delphi study may be then further discussed and refined in focus groups. In so doing, the
application of a focus group may enable researchers to operationalize an end
user/expert-oriented questionnaire [
          <xref ref-type="bibr" rid="ref29 ref55">30, 57</xref>
          ]. This questionnaire may be then deployed in the
realm of a large-scaled quantitative (non-)experimental survey which might aim at
investigating the success of particular HRIS (subtypes).
2.6
        </p>
      </sec>
      <sec id="sec-1-6">
        <title>Method of Evaluation</title>
        <p>Supplement to the elicitation, the evaluation of HRIS design characteristics is an
important step to find out in how far the HRIS under consideration actually meets the
(elicited/existing) design characteristics relevant to HRIS success. Hence, the evaluation
of HRIS employing corresponding system- and information-related design
characteristics constitutes a central point to ensure the permanent improvement and thus
the success of HRIS.</p>
        <p>Thereby, empirical methods can be roughly distinguished from non-empirical ones. To
begin with, non-empirical methods can be understood as a means by which researchers
may conduct a plausibility check of the HRIS under consideration employing particular
design characteristics. In so doing, HRIS design characteristics may enable researchers
to eliminate elementary system errors/failures during each application target of an HRIS
based on logical reasoning. Thereby, a plausibility check, respectively logical reasoning
may be applied based on a theoretical underpinning such as the Technology Acceptance
Model or the DeLone and McLean Model of IS Success. Such a theoretical
underpinning may support researchers in better specifying research models which depict
(inter-)relationships between particular HRIS design characteristics and important
success measures. Beyond, further non-empirical methods of evaluation constitute
socalled compatibility verifications. Such an approach may support researchers in carving
out in how far the elicitation of particular HRIS design characteristics by means of the
methods thoroughly described in Chapter 2.5 (e.g. literature review, use case, case
study, etc.) may have led to the same results than the evaluation of HRIS by means of a
theoretical underpinning such as the Technology Acceptance Model or the DeLone and
McLean Model of IS Success.</p>
        <p>
          As distinct from non-empirical methods, empirical methods are exemplarily represented
by the case study approach, experiments and the survey approach which all together are
considered to constitute a valuable means by which HRIS design characteristics may be
evaluated. Thereby, the case study approach is considered to best support researchers in
the course of an in-depth analysis of pioneering HRIS, respectively design
characteristics. This is mainly due to the fact that this approach allows for an extensive
and comprehensive evaluation of HRIS design characteristics using the example of a
particular company, etc. Regarding experiments, researchers may profit from their
particular benefits while evaluating pioneering HRIS, respectively design
characteristics. In so doing, experiments may enable researchers to manipulate
particular system- and information-related HRIS design characteristics as well as their
influence on particular success measures by use of a theoretical underpinning such as
Technology Acceptance Model or the DeLone and McLean Model of IS Success.
Beyond that, further empirical methods of evaluation are, among others, the observation
or the documentation of HRIS design characteristics [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ].
        </p>
        <p>Thereby, closely linked to the data gathering procedures presented, the analysis of these
data constitutes a crucial aspect in order to carve out in how far the HRIS under
consideration actually meet the (elicited/existing) design characteristics relevant to
HRIS success.</p>
        <p>Finally, combinatorial approaches of particular evaluation approaches are considered to
“pool the forces” of each single method of evaluation. For instance, researchers may
decide upon a combination of non-empirical and empirical evaluation approaches. In so
doing, non-empirical evaluation approaches may constitute the first step in the
evaluation of an HRIS by means of particular design characteristics, followed by more
comprehensive and expressive empirical approaches such as large-scaled surveys in
order to evaluate HRIS by use of small sets of elicited/existing system- and
informationrelated design characteristics.
2.7</p>
      </sec>
      <sec id="sec-1-7">
        <title>Level of Granularity</title>
        <p>
          Considering the preceding steps, researchers may be enabled to carve out particular
(sets of) HRIS-related design characteristics. However, they do not know anything
about their expressiveness or operational capability by then. Thus, the level of
granularity of design characteristics indicates “the grade of operativeness and
detailedness of design characteristics” [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ]. This definition is based on the assumption
that the level of granularity of HRIS design characteristics “can hardly be measured in
terms of absolute numbers because of the subjectivity of the related concepts that may
determine the granularity in question” [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. Hence, we determine the level of
granularity of particular HRIS design characteristics recursively [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], since a
coarsegrained design characteristic (e.g. system quality) can be understood as the composition
of more medium-grained and fine-grained design characteristics.
        </p>
        <p>For example, coarse-grained design characteristics such as “system quality” may be
further sub-divided into more medium-grained design characteristics such as
“flexibility” and more fine-grained design characteristics such as “adaptivity” and
“adaptability”, whereas these fine-grained design characteristics in turn may hardly be
further sub-divided into smaller, and at the same time, still expressive design
characteristics.</p>
        <p>With a view to the type of application target, more fine-grained design characteristics
may be mostly appropriate while developing HRIS (e.g. preparation of a detailed
system specification) whereas medium-grained ones might be most suitable while
implementing and/or improving HRIS. In so doing, more coarse-grained design
characteristics can be easily drilled down to more fine-grained, i.e. more expressive and
detailed, measures in order to better adjust the HRIS in case of unexpected system
errors/failures.</p>
        <p>It is obvious, the expressiveness and usability of HRIS design characteristics increases
with growing specificity (as for instance, “design/select/utilize personalized HRIS”
constitute a more expressive and usable statement than “design/select/utilize HRIS with
good systems quality”). However, growing specificity is commonly aligned with a
decreasing range of validity (see Chapter 2.4). Hence, to warrant validity for all kinds of
HRIS, future research into HRIS design characteristics should preferably deal with
medium granularity.</p>
        <p>
          To sum up, the present contribution provides a general research framework which
illustrates the possibility space of selected issues relevant to HRIS design characteristics
in particular. In so doing, the major benefits of the framework are as follows: firstly, the
framework is considered to constitute a parsimonious framework which contains the
most important aspect when dealing with HRIS-related design characteristics. However,
similarly to design characteristics [
          <xref ref-type="bibr" rid="ref44">45</xref>
          ], all dimensions contained (with)in the framework
are considered to be relevant in order to apply HRIS design characteristics, and thus
HRIS successfully. Secondly, the framework comprehensively depicts specific success
measures relevant to particular stakeholders, outlines system- and information-related
design characteristics as mandatory categories which should be applied by the
stakeholders involved developing, implementing and permanently improving HRIS.
Beyond, the framework provides information about the kind of validity of design
characteristics, supports researchers in the (initial) elicitation of HRIS-related design
characteristics as well as the evaluation of HRIS by means of (these or already existing)
design characteristics. Finally, the framework proposes different kinds of granularity of
HRIS design characteristics which may support researchers in choosing appropriate
design characteristics for the right purpose. For example, during the development of an
HRIS, more fine-grained design characteristics may be an appropriate means to
determine an HRIS’s properties at most precisely. As distinct from this, more
coarsegrained design characteristics may be applied during the implementation or (permanent)
improvement of an HRIS in order to have some rough success indicators which may be
refined by means of more fine-grained design characteristics in case of unexpected
system errors and/or failures. Thus, the framework is considered to be a valuable means
which provides pointers to crucial aspects when selecting and/or applying HRIS design
characteristics. Hence, the general objective of this research effort could be
satisfactorily achieved.
3
        </p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>Illustration</title>
      <p>
        Supplement to the major benefits of the framework, the subsequent elaboration
exemplarily illustrates each dimension of the framework by drawing back on an
ongoing HRIS research project, the EU co-funded project iCOPER (Interoperable
Content for Performance in a Competency-driven Society) [27], which aims at
providing mechanisms to ensure European-wide user involvement, cooperation, and
adoption of e-learning-related standards and specifications. In so doing, one of the main
objectives is to provide a useful technological infrastructure which matches learners
with learning opportunities, that is, trying to get learners into situations to which they
are suited. This requires enabling learners to have access to a portable profile of their
learning outcomes achieved after their successful completion of studies. In order to
achieve this objective researchers of higher education and vocational training settings
collaboratively develop, implement and improve pioneering HRIS subtypes relevant to
training and development [
        <xref ref-type="bibr" rid="ref64">66</xref>
        ], also called Virtual Learning Environments (VLE).
Thereby, VLE are understood “as electronic Information Systems (IS) for the
administrative and didactical support of learning processes in vocational settings by
systematically providing corporate learners adequate learning materials as well as
corresponding collaboration facilities to develop intended qualifications” [
        <xref ref-type="bibr" rid="ref44">45</xref>
        ]. In so
doing, VLE are considered as a prominent subtype of HRIS as they equally “aim at the
generation and delivery of HR functionality in order to automate and informate Human
Resource Management [
        <xref ref-type="bibr" rid="ref61">63</xref>
        ].
      </p>
      <p>Thereby, the suggested framework supports the ongoing project work in the following
way: at first, HRIS success measures relevant to system users, i.e. learners, were
chosen, amongst them learners’ satisfaction with the HRIS under consideration.
Beyond, corresponding to the second dimension of the framework, system- and
information-related design characteristics were chosen as a basic and rough
categorization of those particular HRIS design characteristics to be elicited and
evaluated subsequently. Thereby, system- and information-related design characteristics
were mainly chosen to demonstrate the influence of particular system- and
informationrelated design characteristics on learners’ satisfaction with as well as the (behavioral
intention to) use of the HRIS. Therewith, the development and (permanent)
improvement of the HRIS under consideration could be systematically monitored and
(minor/major) adjustments of system- and information-related HRIS design
characteristics easily undertaken in order to maximize learners’ satisfaction with the
system. In line with the framework the utilization of fine-grained design characteristics
was considered to be the most appropriate for preparing the final system specifications
of the different prototypes whereas medium-grained ones were chosen for
improvement-related purposes. In so doing, more fine-grained design characteristics
actually supported prototype task force members in determining the prototypes’
properties most precisely. Subsequently, more coarse-grained design characteristics
were applied during the build of the final applications, and similarly will be applied
during the (permanent) improvement of the final HRIS as they turned out to constitute
valuable and workable indicators of the HRIS’ current development states. For instance,
as prototype developers were faced with some unexpected system errors/failures these
medium-grained design characteristics were easily drilled down to more fine-grained
ones in order to better adjust the system errors/failures found by use of these more
expressive and detailed measures. The same applies to the continuous evaluation of
learners’ satisfaction with the prototypical as well as final applications. Regarding their
validity, prototype developers decided to draw on already existing HRIS system- and
information-related design characteristics contingent on HR core functions such as
training and development as well as innovative HRIS design characteristics contingent
on the specific focus of the project (e.g. competency-driven provision of learning
materials, support of technical standards relevant to training and development, etc.).
In so doing, the complete set of HRIS design characteristics was elicited by means of an
expert Delphi study, considering the DeLone and McLean Model of IS Success as a
theoretical underpinning for deriving system- and information-related design
characteristics as recommended by the framework. Supplement to the elicitation, the
evaluation of the prototypes as well as the final applications will be undertaken by use
of the originally elicited set of system- and information-related design characteristics
relevant to learners’ satisfaction with these systems. In so doing, the corresponding
research model will be based on amalgamations of the Technology Acceptance Model
and the DeLone and McLean Model of IS Success will be analyzed deploying structural
equation models.
4</p>
    </sec>
    <sec id="sec-3">
      <title>Implications</title>
      <p>
        The above-mentioned results should generally provide a basic starting point for future
research relevant to HRIS design characteristics, while there are some implications for
both research and practice. Generally speaking, the (dimensions of the) framework
proposed should a) allow researchers to better understand and apply crucial aspects
relevant to HRIS design characteristics so that b) practitioners may be better supported
in developing, implementing and permanently improving successful HRIS.
In so doing, researchers should further elaborate each dimension of the framework
proposed in order to supply practitioners such as HRIS-related decision makers, system
developers as well as system implementers with a comprehensive set of well-elaborated
guidelines in order to develop, implement and improve successful HRIS by means of
particular system- and information-related HRIS design characteristics. Such efforts
may be of crucial importance for the further advancement of this emerging field of
research as already existing research attempts [
        <xref ref-type="bibr" rid="ref25 ref26 ref33 ref40 ref41 ref48 ref52">25, 26, 34, 41, 42, 49, 53</xref>
        ] as well as
theoretical underpinnings such as the Technology Acceptance Model or the DeLone and
McLean Model of IS Success do rarely propose concrete propositions what to consider
when dealing with HRIS design characteristics in particular. At the same time, an
extensive, research-driven elaboration of particular framework dimensions proposed
such as the validity or the granularity of HRIS design characteristics may help to bridge
the gap towards rigorous and relevant guidelines relevant to HR practitioners in
particular [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ].
On the basis of such rigorous and relevant outcomes practitioners such as HRIS-related
decision makers, system developers as well as system implementers could then be
equipped with comprehensive manuals for either managing the development, the
implementation or the improvement of HRIS by means of particular system- and
information-related design characteristics. Refining and customizing such manuals
towards individual corporate settings and subsequently considering the manual may
lead to practical HRIS development-, implementation- and improvement-processes
which could contribute to minimize system users’ resistance, increase system users’
satisfaction, and support overall HRIS success.
5
      </p>
    </sec>
    <sec id="sec-4">
      <title>Conclusions</title>
      <p>Within this paper a general research framework of selected issues relevant to HRIS
design characteristics was derived and exemplarily illustrated by means of an ongoing
research project currently participated in on a European-wide level. This framework
hopefully will stimulate future research regarding the development, implementation and
improvement of HRIS by means of particular design characteristics. In so doing, the
framework is thought to be continuously elaborated by, respectively supports
researchers in their attempts to constantly improve HR-related IT artifacts by means of
(particular/particular bundles of) HRIS-related design characteristics.
6</p>
    </sec>
    <sec id="sec-5">
      <title>References</title>
      <p>iCOPER: Project Description and Objectives, available
http://www.icoper.org/project-objectives, accessed on the 11th of April 2010.</p>
      <sec id="sec-5-1">
        <title>Mitarbeiterportal und</title>
      </sec>
      <sec id="sec-5-2">
        <title>Self-Service-Systeme:</title>
        <p>[56] Schonberger, R. J. (1980). MIS Design: A Contingency Approach, MIS Quarterly,
4 (1), 13-20.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Al-Ibraheem</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Ruel</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>In-House vs</article-title>
          .
          <string-name>
            <surname>Off-The-Shelf E-HRM Applications</surname>
            . In: Bondarouk,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruel</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guiderdoni-Jourdain</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Oiry</surname>
            ,
            <given-names>E</given-names>
          </string-name>
          . (eds.):
          <source>Handbook of Research on E-Transformation and Human Resources Management Technologies: Organizational Outcomes and Challenges</source>
          ,
          <volume>92</volume>
          -
          <fpage>115</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Arbaugh</surname>
            ,
            <given-names>J. B.</given-names>
          </string-name>
          (
          <year>2000</year>
          ).
          <article-title>Virtual Classroom Characteristics and Student Satisfaction with Internet-Based MBA Courses</article-title>
          ,
          <source>Journal of Management Education</source>
          ,
          <volume>24</volume>
          (
          <issue>1</issue>
          ),
          <fpage>32</fpage>
          -
          <lpage>54</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Bailey</surname>
            ,
            <given-names>K. D.</given-names>
          </string-name>
          (
          <year>1987</year>
          )
          <article-title>: Methods of Social Research, 3rd Edition</article-title>
          . Free Press.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Beckers</surname>
            ,
            <given-names>A. M.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Bsat</surname>
            ,
            <given-names>M. Z.</given-names>
          </string-name>
          (
          <year>2002</year>
          ).
          <article-title>A DSS Classification Model for Research in Human Resource Information Systems</article-title>
          ,
          <source>Information Systems Management</source>
          ,
          <volume>19</volume>
          (
          <issue>3</issue>
          ),
          <fpage>41</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Bittner</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Spence</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          (
          <year>2003</year>
          ):
          <article-title>Use Case Modeling. Addison-Wesley.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Catterall</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Maclaran</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>Focus Groups in Marketing Research</article-title>
          . In: Russell, W.B. (ed.): Handbook of Qualitative Research Methods in Marketing,
          <volume>255</volume>
          -
          <fpage>267</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Chapman</surname>
            ,
            <given-names>D. S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Webster</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>The Use of Technologies in the Recruiting, Screening, and Selection Processes for Job Candidates</article-title>
          ,
          <source>International Journal of Selection and Assessment</source>
          ,
          <volume>11</volume>
          (
          <issue>2</issue>
          /3),
          <fpage>113</fpage>
          -
          <lpage>120</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>F. D.</given-names>
          </string-name>
          (
          <year>1989</year>
          ).
          <article-title>Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology</article-title>
          ,
          <source>MIS Quarterly</source>
          ,
          <volume>13</volume>
          (
          <issue>3</issue>
          ),
          <fpage>319</fpage>
          -
          <lpage>340</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>F. D.</given-names>
          </string-name>
          (
          <year>1993</year>
          ).
          <article-title>User Acceptance of Information Technology: Systems Characteristics, User Perception</article-title>
          and
          <string-name>
            <given-names>Behavioral</given-names>
            <surname>Impact</surname>
          </string-name>
          ,
          <source>International Journal of Man-Machine-Studies</source>
          ,
          <volume>38</volume>
          ,
          <fpage>475</fpage>
          -
          <lpage>487</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>F. D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bagozzi</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Warshaw</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>1989</year>
          ).
          <article-title>User Acceptance of Computer Technology: A Comparison of Two Theoretical Models</article-title>
          , Management Science,
          <volume>35</volume>
          (
          <issue>8</issue>
          ),
          <fpage>982</fpage>
          -
          <lpage>1003</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>DeLone</surname>
            ,
            <given-names>W. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>McLean</surname>
            ,
            <given-names>E. R.</given-names>
          </string-name>
          (
          <year>1992</year>
          ).
          <article-title>Information Systems Success: The Quest for the Dependent Variable</article-title>
          ,
          <source>Information Systems Research</source>
          ,
          <volume>3</volume>
          (
          <issue>1</issue>
          ),
          <fpage>60</fpage>
          -
          <lpage>95</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>DeLone</surname>
            ,
            <given-names>W. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>McLean</surname>
            ,
            <given-names>E. R.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>The DeLone and McLean Model of Information Systems Success: A Ten-Year Update</article-title>
          ,
          <source>Journal of Management Information Systems</source>
          ,
          <volume>19</volume>
          (
          <issue>4</issue>
          ),
          <fpage>9</fpage>
          -
          <lpage>30</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Dennis</surname>
            ,
            <given-names>A. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wixom</surname>
            ,
            <given-names>B. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Roth</surname>
            ,
            <given-names>R. M.</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <article-title>: Systems Analysis and Design, 3rd Edition</article-title>
          . John Wiley &amp; Sons.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Doty</surname>
            ,
            <given-names>D. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Glick</surname>
            ,
            <given-names>W. H.</given-names>
          </string-name>
          (
          <year>1994</year>
          ).
          <article-title>Typologies as a Unique Form of Theory Building: Towards Improved Understanding and Modeling</article-title>
          ,
          <source>Academy of Management Review</source>
          ,
          <volume>19</volume>
          (
          <issue>2</issue>
          ),
          <fpage>230</fpage>
          -
          <lpage>251</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Dulebohn</surname>
            ,
            <given-names>J. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Marler</surname>
            ,
            <given-names>J. H.</given-names>
          </string-name>
          (
          <year>2005</year>
          ). E-Compensation:
          <article-title>The Potential to Transform Practice</article-title>
          ? In: Gueutal,
          <string-name>
            <given-names>H. G.</given-names>
            <surname>Stone</surname>
          </string-name>
          ,
          <string-name>
            <surname>D. L</surname>
          </string-name>
          . (eds.):
          <source>Brave New World of eHR. Human Resource Management in the Digital</source>
          ,
          <volume>166</volume>
          -
          <fpage>189</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Ellet</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          (
          <year>2007</year>
          ):
          <article-title>The Case Study Handbook: How to Read, Discuss, and Write Persuasively About Cases</article-title>
          , Harvard Business School Press.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Gregor</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>The Nature of Theory in Information Systems</article-title>
          , MIS Quarterly,
          <volume>30</volume>
          (
          <issue>3</issue>
          ),
          <fpage>611</fpage>
          -
          <lpage>642</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Grisham</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>The Delphi Technique: A Method for Testing Complex</article-title>
          and Multifaceted Topics,
          <source>International Journal of Managing Projects in Business</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ),
          <fpage>112</fpage>
          -
          <lpage>130</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Grover</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jeong</surname>
            ,
            <given-names>S. R.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Segars</surname>
            ,
            <given-names>A. H.</given-names>
          </string-name>
          (
          <year>1996</year>
          ).
          <source>Information Systems Effectiveness: The Construct Space and Patterns of Application, Information and Management</source>
          ,
          <volume>31</volume>
          (
          <issue>4</issue>
          ),
          <fpage>177</fpage>
          -
          <lpage>191</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Haeder</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2002</year>
          ):
          <article-title>Delphi-Befragungen: Ein Arbeitsbuch</article-title>
          .
          <source>VS Verlag für Sozialwissenschaften.</source>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Haeder</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Haeder</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2000</year>
          )
          <article-title>: Die Delphitechnik in den Sozialwissenschaften: Methodische Forschungen und innovative Anregungen</article-title>
          . Westdeutscher Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Haesen</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Snoeck</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lemahieu</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Poelmans</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>On the Definition of Service Granularity and its Architectural Impact</article-title>
          . In: Bellahsène,
          <string-name>
            <given-names>Z.</given-names>
            &amp;
            <surname>Léonard</surname>
          </string-name>
          , M. (eds.):
          <source>CAiSE</source>
          <year>2008</year>
          , LNCS
          <volume>5074</volume>
          ,
          <fpage>375</fpage>
          -
          <lpage>389</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Hannon</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jelf</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Brandes</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>1996</year>
          ).
          <source>Human Resource Information Systems: Operational Issues and Strategic Considerations in a Global Environment</source>
          ,
          <source>International Journal of Human Resource Management</source>
          ,
          <volume>7</volume>
          (
          <issue>1</issue>
          ),
          <fpage>245</fpage>
          -
          <lpage>269</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Herzum</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Sims</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          (
          <year>2000</year>
          )
          <article-title>: Business Components Factory: A Comprehensive Overview of Component-Based Development for the Enterprise</article-title>
          . John Wiley &amp; Sons.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Hevner</surname>
            ,
            <given-names>A. R.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <source>A Three Cycle View of Design Science Research</source>
          ,
          <source>Scandinavian Journal of Information Systems</source>
          ,
          <volume>19</volume>
          (
          <issue>2</issue>
          ),
          <fpage>87</fpage>
          -
          <lpage>92</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Hevner</surname>
            ,
            <given-names>A. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>March</surname>
          </string-name>
          , S. T.,
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Ram</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <source>Design Science in Information Systems Research</source>
          , MIS Quarterly,
          <volume>28</volume>
          (
          <issue>1</issue>
          ),
          <fpage>75</fpage>
          -
          <lpage>105</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [28] ISO/IEC 25000:
          <year>2005</year>
          :
          <article-title>Software Product Quality Requirements and Evaluation (SQuaRE)-</article-title>
          Guide to SQuaRE, available at: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.
          <source>htm?csnumber =35683, accessed on the 11th of April</source>
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Kavanagh</surname>
            ,
            <given-names>M. J.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Thite</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2009</year>
          )
          <article-title>: Human Resource Information Systems: Basics, Applications, and Future Directions</article-title>
          . Sage.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Kaya</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Verfahren der Datenerhebung</article-title>
          . In: Albers,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Klapper</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Konradt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            ,
            <surname>Walter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            &amp;
            <surname>Wolf</surname>
          </string-name>
          ,
          <string-name>
            <surname>J</surname>
          </string-name>
          . (eds.):
          <source>Methodik der empirischen Forschung, 2nd Edition</source>
          ,
          <fpage>49</fpage>
          -
          <lpage>64</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Koenigstorfer</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2008</year>
          ):
          <article-title>Akzeptanz von technologischen Innovationen: Nutzungsentscheidungen von Konsumenten dargestellt am Beispiel von mobilen Internetdiensten</article-title>
          . Gabler.
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Kuechler</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Vaishnavi</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <source>The Emergence of Design Research in Information Systems in North America, Journal of Design Research</source>
          ,
          <volume>7</volume>
          (
          <issue>1</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Landeta</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <source>Current Validity of the Delphi Method in Social Sciences, Technological Forecasting and Social Change</source>
          ,
          <volume>73</volume>
          (
          <issue>5</issue>
          ),
          <fpage>467</fpage>
          -
          <lpage>482</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Laumer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Eckhardt</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Help to Find the Needle in a Haystack: Integrating Recommender Systems in an IT supported Staff Recruitment System</article-title>
          ,
          <source>Proceedings of the SIGMIS-CPR</source>
          ,
          <fpage>7</fpage>
          -
          <lpage>12</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>Y.-C.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>An Empirical Investigation into Factors Influencing the Adoption of an E-Learning System</article-title>
          ,
          <source>Online Information Review</source>
          ,
          <volume>30</volume>
          (
          <issue>5</issue>
          ),
          <fpage>517</fpage>
          -
          <lpage>541</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Lengnick-Hall</surname>
            ,
            <given-names>C. A.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Lengnick-Hall</surname>
            ,
            <given-names>M. L.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>HR, ERP, and Knowledge for Competitive Advantage</article-title>
          ,
          <source>Human Resource Management</source>
          ,
          <volume>45</volume>
          (
          <issue>2</issue>
          ),
          <fpage>179</fpage>
          -
          <lpage>194</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [37]
          <string-name>
            <surname>Lin</surname>
            ,
            <given-names>H.-F.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>G.-G.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>Determinants of Success for Online Communities: An Empirical Study</article-title>
          ,
          <source>Behaviour &amp; Information Technology</source>
          ,
          <volume>25</volume>
          (
          <issue>6</issue>
          ),
          <fpage>479</fpage>
          -
          <lpage>488</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Lindgren</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Henfridsson</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Schultze</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Design Principles for Competence Management Systems: A Synthesis of an Action Research Study</article-title>
          ,
          <source>MIS Quarterly</source>
          ,
          <volume>28</volume>
          (
          <issue>3</issue>
          ),
          <fpage>435</fpage>
          -
          <lpage>472</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Lippert</surname>
            ,
            <given-names>S. K.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Swiercz</surname>
            ,
            <given-names>P. M.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Human Resource Information Systems (HRIS) and Technology Trust</article-title>
          ,
          <source>Journal of Information Science</source>
          ,
          <volume>31</volume>
          (
          <issue>5</issue>
          ),
          <fpage>340</fpage>
          -
          <lpage>353</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Luthans</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          (
          <year>1976</year>
          ):
          <article-title>Introduction to Management: A Contingency Approach</article-title>
          .
          <string-name>
            <surname>McGraw-Hill</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Maier</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Laumer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Eckhardt</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>An Integrated IT-Architecture for Talent Management and Recruitment</article-title>
          . In: Bondarouk,
          <string-name>
            <given-names>T.</given-names>
            &amp;
            <surname>Ruel</surname>
          </string-name>
          , H. (eds.):
          <source>Proceedings of the 3rd International Workshop on Human Resource Information System - HRIS</source>
          <year>2009</year>
          ,
          <volume>28</volume>
          -
          <fpage>38</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [42]
          <string-name>
            <surname>March</surname>
            ,
            <given-names>S. T.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>G. F.</given-names>
          </string-name>
          (
          <year>1995</year>
          ).
          <source>Design and Natural Science Research on Information Technology, Decision Support Systems</source>
          ,
          <volume>15</volume>
          (
          <issue>4</issue>
          ),
          <fpage>251</fpage>
          -
          <lpage>266</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          [43]
          <string-name>
            <surname>Mayring</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Qualitative Content Analysis</article-title>
          ,
          <source>Forum: Qualitative Social Research</source>
          ,
          <volume>1</volume>
          (
          <issue>2</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          , available at: http://www.qualitative-research.net/fqs-texte/2- 00/
          <fpage>2</fpage>
          -00mayring-e.pdf,
          <source>accessed on the 11th of April</source>
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          [44]
          <string-name>
            <surname>McLeod</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Jr</surname>
          </string-name>
          . &amp;
          <string-name>
            <surname>DeSanctis</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>1995</year>
          ).
          <article-title>A Resource-Flow Model of the Human Resource Information System</article-title>
          ,
          <source>Journal of Information Technology Management</source>
          ,
          <volume>6</volume>
          (
          <issue>3</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          [45]
          <string-name>
            <surname>Mueller</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Strohmeier</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Design Characteristics of Virtual Learning Environments: An Expert Study (accepted</article-title>
          ),
          <source>International Journal of Training and Development</source>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          [46]
          <string-name>
            <surname>Mueller</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Zimmermann</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2009</year>
          ). A
          <string-name>
            <surname>Learner-Centred</surname>
            <given-names>Design</given-names>
          </string-name>
          , Implementation, and Evaluation Approach of Learning Environments to Foster Acceptance,
          <source>International Journal of Advanced Corporate Learning</source>
          ,
          <volume>2</volume>
          (
          <issue>3</issue>
          ),
          <fpage>50</fpage>
          -
          <lpage>57</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          [47]
          <string-name>
            <surname>Orlikowski</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Iacono</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Desperately Seeking the “IT” in IT Research - A Call to Theorizing the IT Artifact</article-title>
          ,
          <source>Information Systems Research</source>
          ,
          <volume>12</volume>
          (
          <issue>2</issue>
          ),
          <fpage>121</fpage>
          -
          <lpage>134</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          [48]
          <string-name>
            <surname>Patton</surname>
            ,
            <given-names>M. Q.</given-names>
          </string-name>
          (
          <year>1990</year>
          )
          <article-title>: Qualitative Evaluation and Research Methods, 2nd Edition</article-title>
          . Sage.
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          [49]
          <string-name>
            <surname>Peffers</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tuunanen</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rothenberger</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Chatterjee</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <source>A Design Science Research Methodology for Information Systems Research, Journal of Management Information Systems</source>
          ,
          <volume>24</volume>
          (
          <issue>3</issue>
          ),
          <fpage>45</fpage>
          -
          <lpage>77</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref49">
        <mixed-citation>
          [50]
          <string-name>
            <surname>Pituch</surname>
            ,
            <given-names>K. A.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>Y.-K.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <source>The Influence of System Characteristics on ELearning Use</source>
          ,
          <source>Computers &amp; Education</source>
          ,
          <volume>47</volume>
          ,
          <fpage>222</fpage>
          -
          <lpage>244</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref50">
        <mixed-citation>
          [51]
          <string-name>
            <surname>Raymond</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>1990</year>
          ).
          <article-title>Organizational Context and Information Systems Success: A Contingency Approach</article-title>
          ,
          <source>Journal of Management Information Systems</source>
          ,
          <volume>6</volume>
          (
          <issue>4</issue>
          ),
          <fpage>5</fpage>
          -
          <lpage>20</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref51">
        <mixed-citation>
          [52]
          <string-name>
            <surname>Roca</surname>
            ,
            <given-names>J. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chiu</surname>
            ,
            <given-names>C.-M.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Martínez</surname>
            ,
            <given-names>F. J.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>Understanding E-Learning Continuance Intention: An Extension of the Technology Acceptance Model</article-title>
          ,
          <source>International Journal of Human-Computer Studies</source>
          ,
          <volume>64</volume>
          ,
          <fpage>683</fpage>
          -
          <lpage>696</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref52">
        <mixed-citation>
          [53]
          <string-name>
            <surname>Rossi</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Sein</surname>
            ,
            <given-names>M. K.</given-names>
          </string-name>
          (
          <year>2003</year>
          ). Design Research Workshop: A Proactive Research Approach, available at: http://tiesrv.hkkk.fi/iris26/presentation/workshop_ designRes.pdf,
          <source>accessed on the 11th of April</source>
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref53">
        <mixed-citation>
          [54]
          <string-name>
            <surname>Rourke</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Validity in Quantitative Content Analysis</article-title>
          ,
          <source>Educational Technology Research and Development</source>
          ,
          <volume>52</volume>
          ,
          <fpage>5</fpage>
          -
          <lpage>18</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref54">
        <mixed-citation>
          [55]
          <string-name>
            <surname>Schaeffer-Külz</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          (
          <year>2005</year>
          )
          <article-title>: Praxisstudie</article-title>
          . Datakontext.
        </mixed-citation>
      </ref>
      <ref id="ref55">
        <mixed-citation>
          [57]
          <string-name>
            <surname>Schwarz</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Oyserman</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <source>Asking Questions About Behavior: Cognition</source>
          , Communication, and Questionnaire Construction,
          <source>American Journal of Evaluation</source>
          ,
          <volume>22</volume>
          (
          <issue>2</issue>
          ),
          <fpage>127</fpage>
          -
          <lpage>160</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref56">
        <mixed-citation>
          [58]
          <string-name>
            <surname>Seddon</surname>
            ,
            <given-names>P. B.</given-names>
          </string-name>
          (
          <year>1997</year>
          ).
          <article-title>A Respecification and Extension of the DeLone and McLean Model of</article-title>
          IS Success,
          <source>Information Systems Research</source>
          ,
          <volume>8</volume>
          (
          <issue>3</issue>
          ),
          <fpage>240</fpage>
          -
          <lpage>253</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref57">
        <mixed-citation>
          [59]
          <string-name>
            <surname>Seddon</surname>
            ,
            <given-names>P. B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staples</surname>
            ,
            <given-names>D. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Patnayakuni</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Bowtell</surname>
            ,
            <given-names>M. J.</given-names>
          </string-name>
          (
          <year>1999</year>
          ).
          <source>Dimensions of Information Systems Success, Communications of the Association for Information Systems (CAIS)</source>
          ,
          <volume>2</volume>
          (
          <issue>20</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>39</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref58">
        <mixed-citation>
          [60]
          <string-name>
            <surname>Seddon</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Graeser</surname>
            <given-names>V.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Wilcocks</surname>
            <given-names>L. P.</given-names>
          </string-name>
          (
          <year>2002</year>
          ).
          <source>Measuring Organizational IS Effectiveness: An Overview and Update of Senior Management Perspectives</source>
          ,
          <source>The DATA BASE for Advances in Information Systems</source>
          ,
          <volume>33</volume>
          (
          <issue>2</issue>
          ),
          <fpage>11</fpage>
          -
          <lpage>28</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref59">
        <mixed-citation>
          [61]
          <string-name>
            <surname>Sommerville</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          (
          <year>2007</year>
          )
          <article-title>: Software Engineering, 8th Edition</article-title>
          . Addison-Wesley.
        </mixed-citation>
      </ref>
      <ref id="ref60">
        <mixed-citation>
          [62]
          <string-name>
            <surname>Stewart</surname>
            ,
            <given-names>D. W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shamdasani</surname>
            ,
            <given-names>P. N.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Rook</surname>
            ,
            <given-names>D. W.</given-names>
          </string-name>
          (
          <year>2007</year>
          )
          <article-title>: Focus Groups: Theory and Practice”, 2nd Edition</article-title>
          . Sage.
        </mixed-citation>
      </ref>
      <ref id="ref61">
        <mixed-citation>
          [63]
          <string-name>
            <surname>Strohmeier</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Kabst</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Do Current HRIS Meet the Requirements of HRM? An Empirical Evaluation Using Logistic Regression and Neural Network Analysis</article-title>
          . In.: Bondarouk,
          <string-name>
            <given-names>T.</given-names>
            &amp;
            <surname>Ruel</surname>
          </string-name>
          , H. (eds.):
          <source>Proceedings of the 1st International Workshop on Human Resource Information System - HRIS</source>
          <year>2007</year>
          ,
          <volume>1</volume>
          -
          <fpage>14</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref62">
        <mixed-citation>
          [64]
          <string-name>
            <surname>Strohmeier</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Concepts of e-HRM Consequences: A Categorisation, Review and Suggestion</article-title>
          ,
          <source>The International Journal of Human Resource Management</source>
          ,
          <volume>20</volume>
          (
          <issue>3</issue>
          ),
          <fpage>528</fpage>
          -
          <lpage>543</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref63">
        <mixed-citation>
          [65]
          <string-name>
            <surname>Tait</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Vessey</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          (
          <year>1988</year>
          ).
          <article-title>The Effect of User Involvement on System Success: A Contingency Approach</article-title>
          , MIS Quarterly,
          <volume>12</volume>
          (
          <issue>1</issue>
          ),
          <fpage>91</fpage>
          -
          <lpage>108</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref64">
        <mixed-citation>
          [66]
          <string-name>
            <surname>Teo</surname>
            ,
            <given-names>T. S. H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Soon</surname>
            ,
            <given-names>L. G.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Fedric</surname>
            ,
            <given-names>S. A.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Adoption and Impact of Human Resource Information Systems</article-title>
          (HRIS).
          <source>Research and Practice in Human Resource Management, Research and Practice in Human Resource Management</source>
          ,
          <volume>9</volume>
          (
          <issue>1</issue>
          ),
          <fpage>101</fpage>
          -
          <lpage>117</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref65">
        <mixed-citation>
          [67]
          <string-name>
            <surname>Tobing</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hamzah</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sura</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Amin</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Assessing the Acceptability of Adaptive E-Learning System</article-title>
          ,
          <source>5th International Conference on eLearning for Knowledge-Based Society</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref66">
        <mixed-citation>
          [68]
          <string-name>
            <surname>Urbach</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smolnik</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Riempp</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Development and Validation of a Model for Assessing the Success of Employee Portals</article-title>
          .
          <source>In: 17th European Conference on Information Systems</source>
          ,
          <volume>1</volume>
          -
          <fpage>13</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref67">
        <mixed-citation>
          [69]
          <string-name>
            <surname>Urbach</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smolnik</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Riempp</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <source>The State of Research on Information Systems Success: A Review of Existing Multidimensional Approaches, Business &amp; Information Systems Engineering</source>
          ,
          <volume>4</volume>
          ,
          <fpage>315</fpage>
          -
          <lpage>325</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref68">
        <mixed-citation>
          [70]
          <string-name>
            <surname>Van Aken</surname>
            ,
            <given-names>J. E.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Management Research as a Design Science: Articulating the Research Products of Mode 2 Knowledge Production in Management</article-title>
          ,
          <source>British Journal of Management</source>
          ,
          <volume>16</volume>
          ,
          <fpage>19</fpage>
          -
          <lpage>36</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref69">
        <mixed-citation>
          [71]
          <string-name>
            <surname>Venkatesh</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          &amp; Bala,
          <string-name>
            <surname>H.</surname>
          </string-name>
          (
          <year>2008</year>
          ).
          <source>Technology Acceptance Model 3 and a Research Agenda on Interventions, Decision Sciences</source>
          ,
          <volume>39</volume>
          (
          <issue>2</issue>
          ),
          <fpage>273</fpage>
          -
          <lpage>315</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref70">
        <mixed-citation>
          [72]
          <string-name>
            <surname>Venkatesh</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>F. D.</given-names>
          </string-name>
          (
          <year>2000</year>
          ).
          <article-title>A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies</article-title>
          , Management Science,
          <volume>46</volume>
          (
          <issue>2</issue>
          ),
          <fpage>186</fpage>
          -
          <lpage>204</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref71">
        <mixed-citation>
          [73]
          <string-name>
            <surname>Venkatesh</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Morris</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <source>User Acceptance of Information Technology: Toward a Unified View</source>
          ,
          <source>MIS Quarterly</source>
          ,
          <volume>24</volume>
          (
          <issue>3</issue>
          ),
          <fpage>425</fpage>
          -
          <lpage>478</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref72">
        <mixed-citation>
          [74]
          <string-name>
            <surname>Wang</surname>
          </string-name>
          , W.-T. &amp;
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>C.-C.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>An Empirical Study of Instructor Adoption of Web-Based Learning Systems</article-title>
          ,
          <source>Computers &amp; Education</source>
          ,
          <volume>53</volume>
          ,
          <fpage>761</fpage>
          -
          <lpage>774</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref73">
        <mixed-citation>
          [75]
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>Y.-S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
          </string-name>
          , H.-Y. &amp;
          <string-name>
            <surname>Shee</surname>
            ,
            <given-names>D. Y.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Measuring E-learning Systems Success in an Organizational Context: Scale Development and Validation</article-title>
          , Computers in Human Behavior,
          <volume>23</volume>
          ,
          <fpage>1792</fpage>
          -
          <lpage>1808</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref74">
        <mixed-citation>
          [76]
          <string-name>
            <surname>Wixom</surname>
            ,
            <given-names>B. H.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Todd</surname>
            ,
            <given-names>P. A.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>A Theoretical Integration of User Satisfaction</article-title>
          and
          <source>Technology Acceptance, Information Systems Research</source>
          ,
          <volume>16</volume>
          (
          <issue>1</issue>
          ),
          <fpage>85</fpage>
          -
          <lpage>102</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref75">
        <mixed-citation>
          [77]
          <string-name>
            <surname>Yin</surname>
            ,
            <given-names>R. K.</given-names>
          </string-name>
          (
          <year>2003</year>
          )
          <article-title>: Case Study Research: Design and Methods, 3rd Edition</article-title>
          . Sage.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>