Evidence-based movements have emerged recently in various disciplines. They are motivated to improve practice by making researchinformed decisions. However, the concept of “evidence-based” is easily misconstrued, risking a lost opportunity to bridge the gap between practitioners, academics and other stakeholders. As an inter-disciplinary field, e-HRM is likely to exhibit the research-practice gap from which its parent disciplines (IS and HR) suffer. In this paper, (1) We trace the developments of the Evidence-based Management (EBMgt) movement; (2) We outline how e-HRM will profit from an evidence-based perspective; and (3) We offer recommendations to increase the role of academics in the practice of evidence-based e-HRM.
Calls to incorporate the best available scientific evidence in decision making have been
raised in domains as widespread as medicine [25], education [32], criminal justice [28],
management [
But scientific jargon is only part of the problem. The issues upon which researchers focus their attention are often perceived as irrelevant [5][17]. To use an example from eHRM research, many pages have been written to illustrate that top management support is important [16] or that a positive intra-departmental relationship between IS and HR matters in the intensity of use of HR technologies [15]; a positivistic stance (the epistemological viewpoint that knowledge advances by confirming through observation and empirical validation) is, after all, a fundamental basis of scientific research. Still, how many practicing IS or HR managers would find these conclusions surprising or useful? Findings are often based on issues of interest to scientists, not practitioners, and thus fail to motivate managers to seek out or apply them. The emerging field of e-HRM, defined by Strohmeier as ―the (planning, implementation and) application of information technology for both networking and supporting at least two individual or collective actors in their shared performing of HR activities‖ ([31], p. 20) would also do well by making as purposeful an effort as possible, so that its scholarly work does not grow separate from what is needed and useful for practitioners.
We now proceed as follows: in section two, we describe the recent developments of the nascent Evidence-based Management (EBMgt) movement; in section three, we argue that e-HRM is likely to profit from an evidence-based perspective and offer recommendations to increase the role of academics in the practice of evidence-based eHRM. The conclusion of this paper summarizes its major contributions. 2
As noted in the call for papers to the Evidence-Based e-HRM workshop, EBMgt
attempts to bridge the research-practice gap by using the ―best-available scientific
evidence‖ in managerial decision-making [18]. The earliest proponents of EBMgt have
attributed the Evidence-Based Medicine movement as the inspiration for EBMgt
[18][20]. Although the essence of these movements is the systematic use of
scientifically derived information, Table 1 shows that a certain amount of adaptation is
necessary to promote evidence-based practice suited to a given profession or domain.
The working definitions of EB Medicine and EB Management have evolved to
emphasize the importance of certain stakeholders; it is not exclusively about taking into
account the best available evidence (preferably of the scientifically collected kind), but
also about considering the focal professional’s expertise and the stakeholder’s
preferences or values.
(1) Best available evidence,
combined with
(2) professional expertise,
and (3) stakeholder’s
preferences or values
(1) Practitioner’s expertise,
(2) evidence from the local
context, (3) best available
research evidence, and
(4) stakeholders’
perspectives
―the integration of best research
evidence with clinical expertise
and patient values‖ ([26], p. 1)
―a movement to integrate
bestavailable evidence, manager’s
judgment and stakeholder values in
business decision making‖ ([
A group working to promote of the use of evidence in management practice is the Evidence-Based Management Collaborative. It has been described as: ...a community-of-practice to make evidence-informed management a reality. [Its] mission is to close the gap between management research and the ways practitioners make managerial and organizational decisions and educators teach organizational behavior, theory, strategy and human resources management. [11] This collaborative has been brought together prominent researchers, including several past-presidents of the Academy of Management, representatives from many of its divisions, and journal editors and international professors, all of whom are early adopters of an evidence-based perspective in their teaching and scholarly activities. As a result of these meetings, this group have organized professional development workshops during professional meetings (e.g., Academy of Management) and published journal articles to promote EBMgt. A related initiative, sponsored by the collaborative, exists in the Society of Industrial/Organizational Psychology (SIOP), sponsoring developmental activities for its practitioner members, and the launch of a research translation annual Science You Can Use series in 2010. We now summarize two major contributions of the EMBMgt Collaborative that might also be helpful for e-HRM academics: Systematic Research Syntheses and Translation Prototypes. 2.1.1
A major contribution of the collaborative has been to call attention to the importance of a comprehensive form of literature review, referred to as systematic research syntheses (SRS, akin to ―systematic reviews‖ in Evidence-Based Medicine) [22]. Rousseau and her colleagues have identified four forms of systematic review: aggregation, integration, interpretation and explanation [22].
The well-known statistical meta-analysis [27] is a way to execute the first kind of SRS, called ―aggregative systematic research synthesis.‖ In essence, given a well-specified, focused question, researchers set a priori inclusion criteria for the studies that refer to that question and use statistical algorithms to find the answer.
The second category is an ―integrative‖ synthesis, similar to a meta-analysis in that it identifies patterns and uses questions defined at the onset of the study, but triangulation and contextualization are allowed –including qualitative data and procedural knowledge. An exemplar of this type of review dealt with information systems outsourcing authored by Dibbern, Goles, Hirschheim and Jayatilaka [8]. ―Interpretive syntheses‖ are the third category, associated with relativist epistemologies, where social construction or phenomenology is the main approach to understand the subject matter. Meta-ethnographies and theoretical narratives typify this category, wherein the original researchers’ judgment is of paramount importance. Use of this approach is an alternative to more structured methods such as triangulation in sampling and coding, which stress objectivity at the expense of contextual richness. Campbell and colleagues’ meta-ethnography of diabetes care [4] is offered as an exemplar by Rousseau et al. [22].
The fourth SRS category is the ―explanatory research synthesis‖, emphasizing the identification of causal mechanisms underlying observed phenomena across diverse circumstances. At first blush, this type of synthesis is very similar to the former – interpretive—, except that the explanatory synthesis does not favor a particular type of evidence (e.g., quantitative over qualitative studies). Rousseau and her colleagues suggest that this is particularly useful ―in fragmented and methodologically diverse fields, where little consensus exists regarding what constitutes quality research‖ (p. 499); such as the Management and Organizational Sciences, a characterization also used by Denyer et al. [7].
Perhaps not so coincidentally, in his recent review of the e-HRM literature, Strohmeier found that the field is influenced by several disciplines, that it is ―mainly nontheoretical, employs diverse empirical methods, and refers to several levels of analysis and to diverse focal topics‖ ([31], p. 19). We believe that these characteristics in e-HRM offer the opportunity to profit from SRS. We will explore these ideas further in section 3, below. 2.1.2
Another contribution is a prototype for research translations. To date, a practitioner seeking to get best available research evidence on a practice question would have a tough time. Few plain language summaries of research findings exist. Most up-to-date reviews of findings are written for academics and relevant original research, albeit available through electronic databases, is difficult for lay people to find or read. The new research translation annual –Science You Can Use—provides such user-friendly summaries as does a proposed on-line searchable portal similar to WebMD® (www.webmd.com) that is intended to allow practitioners easy access to research summaries relevant to their decisions. 2.2
The EBMgt Collaborative is actively developing the nascent features of the EBMgt movement. At present these features are four-fold. The first of them is interactionbased, creating quality relationships between scholars and managers via joint research efforts, consultation, and personal networks. SIOP is a particularly important partner organization in this regard as its membership represents both scholars and practitioners. The second is text-based, both on-line and in print via scholarly publications and research translations. The Science You Can Use series initiated in 2009 uses the EBMgt Collaborative-developed prototype for research translations as its template for authors. Its editors and authors comprise teams of practitioners and academics. The third is useroriented, where recognition of the lack of knowledge researchers have of practice, coupled with the heterogeneity of practitioners, has led to detection of the need for user research in preparation of research translations and other possible means of bridging the research-practice gap. User research involves focus groups as well as controlled studies to identify conditions promoting use of evidence. The fourth feature is accessing pointer-knowledge, recognizing that text or even electronic availability of information is no substitute for access to knowledge people. Networks of practitioners and scholars are being developed by such means as an Evidence-Based Management network at the USbased Academy of Management, and a planned website with contact information for practitioners interested in obtaining answers to specific questions. In general, the notion of pointer knowledge entails connecting practitioners with knowledge brokers (librarians, communities of practice, local experts, post-graduate ties with faculty) to provide guidance in navigating academic research and its findings. These activities also provide opportunities for practitioners interested in e-HRM to become involved in the EBMgt community. 3
Strohmeier offered the following definition for e-HRM: ―the (planning, implementation and) application of information technology for both networking and supporting at least two individual or collective actors in their shared performing of HR [Human Resources] activities‖ ([31], p. 20). As an inter-disciplinary and emerging field, is likely to exhibit many of the traits that its ―parent disciplines‖ –particularly HR and IS—have. It has been documented that both IS and HR appear to have a noticeable ―practice-research gap‖ that needs to be bridged [1][2][6][24]. Can we honestly say that most e-HRM papers provide usable, actionable knowledge that practitioners look forward to reading? Are we addressing issues that make a difference in crucial moments of an individual’s career or an organization’s life? Is the motivation section of our papers shaped mainly by industry needs or by academic research? An inspection of the e-HRM literature will certainly find exemplary contributions to answer the questions above, but also many more that can easily be improved. A related discussion that has often surfaced suggests that research in applied fields like management has to play a balancing act between rigor and relevance. Palmer and his colleagues observed that the relevance-rigor debate often claims that ―methodological rigor drives out relevance‖ (p. 267); they suggest that systematically collected, empirical data obtained ―at arm’s length‖ to develop quantitative measures and analyze using multivariate statistical techniques is currently considered more rigorous than the alternative views [17]. They also point out that theoretical rigor has been associated with lack of relevance in management research [12], and that there are strong arguments against the notion that the most rigorous theoretical work is deductive (the type most often used in academic journals), compared to inductive, which is based upon field observations (p. 268).
Huff’s characterization about this debate as ―boring‖ might provide a refreshing point: research that is truly helpful for practice cannot be either rigorous or relevant; it must be both. We believe that e-HRM research also cannot afford the luxury of favoring relevance to the detriment of rigor or vice versa. It is our collective task to find or create the ways in which we can serve our constituencies in a way that provides useful and soundly derived solutions. The following are some among the first proposals that can be derived from the EBMgt Collaborative, but it should be clear by now that a great deal of effort and talent must be leveraged before we can claim that we have found final answers. 3.1
As e-HRM has a narrower, highly oriented to practice focus, we believe that the use of the conceptual tools that are being developed as a result of the EBMgt Collaborative can have momentous consequences. To illustrate, the use of SRS requires great proximity to the potential users of the knowledge –i.e., managers, users of e-HRM technology in our particular case—, in an explicit effort to formulate research questions that lend themselves to use-oriented, actionable answers. Within the management and organizational sciences, an approach to generating useable knowledge has been offered by the evidence-based group at the Cranfield School of Management [7][33]. A recent proposal –referred to as ―CIMO-logic‖—involves identifying the Context of the problem, the Intervention(s) that have been tried (similar to a treatment in medicine), the Mechanisms that have been observed after the interventions are put in place, and the Outcomes intended by the interventions in several aspects [3][7]. This approach uses design science principles that may be helpful in generating procedural knowledge that can be useful for e-HRM practitioners.
Because of its advantages over conventional literature reviews, Systematic Research Reviews, as proposed by Rousseau et al. [22] may be of great value in making e-HRM research more relevant for practice, without diminishing –actually, we argue, while simultaneously increasing—the degree of rigor. See Table 2 for a comparison between these two types of syntheses. Transparency
No need to specify the way that sources are found Replicability
Not guaranteed
Must specify keywords, literature aggregators, databases, and other sources utilized Must be sought after as an essential characteristic Thoroughness Levels of analysis Contextual factors
Often limited to published sources, In addition to published sources, SRS sometimes only to certain journals must include unpublished studies, or types of publications (e.g., conference papers, dissertations, empirical studies only) consultant reports, surveys, databases, etc.
May be restricted to as few as one level of analysis but others may be included
Any study that is relevant to the research question should be included, taking into account its level of analysis Frequently disregarded; studies carried out recently may be placed alongside others that had different dates or backgrounds (e.g., different nations, industries, profit orientations, etc.)
Background features are to be specified to better understand the applicability and relevance of the studies for the research question Language
Most are restricted to studies Relevant studies in other languages ought published in the English language to be included too As suggested above, the departing point in an SRS is the research question [22]. If the research question (or topic) is not appropriately defined, the following steps of the SRS will be severely handicapped. This also implies that special care needs to be placed in identifying the stakeholders that might be more strongly affected by the interventions (e.g., researchers that neglect the impact of union status in companies that intend to downsize their workforce after the introduction of an Enterprise Resource Planning system do so at their own risk).
Another essential property of an SRS is its transparency. A well-executed SRS is explicit about the procedure that will be used to answer the research question; in fact, in EB Medicine, the Cochrane Collaboration (www.cochrane.org; a volunteer organization that coordinates and publishes EBM reviews known as ―Cochrane Reviews‖) and in EB Education, Criminology, and Social Welfare, the Campbell Collaboration (www.campbellcollaboration.org; a similar organization for these fields) use a multistage process in commissioning systematic reviews. First, a review title is negotiated between potential authors and the group’s editorial team; then, a review protocol is published, describing how the review itself will be carried out; finally, the review is published in the Collaboration website, although the responsibility to update the review usually remains with the authors, aided by the editorial team (see http://www.cochrane.org/reviews/revstruc.htm for details).
As it has been stated, there is no consensus regarding what the ―best available scientific
evidence‖ in management is (e.g., [3] [17][22]). While it might be tempting to use the
concepts and tools that have been developed by other Evidence-Based movements, we
strongly believe that careful adaptation and customization are needed to avoid the
―blind benchmarking‖ or the misguided obsession with ―best practices‖ that have been
presented among the motivations for EBMgt [18][
Systematic research syntheses are also superior to conventional literature reviews in that
the latter often are sets of ―cherry-picked‖ studies that support or advocate the
reviewers’ world view in constructing hypotheses or offering recommendations. In
contrast, SRS make findings replicable, and the way that conclusions as well as
recommendations are reached, auditable. Also importantly, SRS are expected to take
into account the ―grey literature‖ that includes studies that haven’t been published
because the results did not reject the null hypotheses –also referred to as the ―file
cabinet‖ problem [
Academics and practitioners are largely mutually incompetent in relating to each other [15]. Academics don’t have very good understandings of how practitioners think nor even what they do. Practitioners, an even more heterogeneous group than academics, often lack training and insight into basic organizational phenomena and limited insight into their own decision making. People in general tend to be overly optimistic when evaluating the quality of their performance on social and intellectual tasks [7]. Surrounded as we are by (academic or practitioner) peers who make the same mistakes, this lack of insight into our own errors [10] leads to overly optimistic estimates of how much academics understand about practitioners and vice versa. In the case of EBMgt, it is academics who must take the first steps toward gaining insight into the thinking and decision styles of practitioners. Our research and our educational programs ought to reflect these approaches if we are truly interested in bridging the research-practice gap. 4
Conclusion e-HRM stands to profit from an evidence-based perspective. As an inter-disciplinary field that is nurtured by disciplines such as HRM, MIS and more generally speaking the managerial and organizational sciences, a gap between what researchers study and what practitioners need to know has began to develop. The use of systematic research syntheses can help us better understand what is known in the domain of e-HRM and which areas need further research [22]. Yet this effort may not suffice if we fail to pay attention to the knowledge managers –especially e-HRM practitioners—need [6] and the ways in which they make decisions to which e-HRM evidence is related. Promoting an evidence-based e-HRM practice necessitates closer ties between researchers and endusers. The use of research translations is likely to aid in getting the information practitioners need from researchers.
We hope that the concepts developed by the EBMgt Collaborative that we have described in this paper are a useful step in this direction. Ultimately, the answer to the question in our title depends upon the actions that e-HRM researchers undertake individually. The potential exists; shall we make it a reality?