=Paper=
{{Paper
|id=Vol-1095/paper_05
|storemode=property
|title=Research on Non-Issues Difficulties of Empirical Research on the Requirements Engineering & Management Process at the Client's Site. Some Notes from an Explorative Study
|pdfUrl=https://ceur-ws.org/Vol-1095/paper_05.pdf
|volume=Vol-1095
|dblpUrl=https://dblp.org/rec/conf/icsob/Weissbach13
}}
==Research on Non-Issues Difficulties of Empirical Research on the Requirements Engineering & Management Process at the Client's Site. Some Notes from an Explorative Study==
https://ceur-ws.org/Vol-1095/paper_05.pdf
Research on “Non-Issues” - Difficulties of Empirical
Research on the Requirements Engineering &
Management Process at the Client’s Site
Some Notes from an Explorative Study
Rüdiger Weißbach
Hamburg University of Applied Sciences (HAW Hamburg)
Faculty Business & Social Sciences
Berliner Tor 5
D-20099 Hamburg
ruediger.weissbach@haw-hamburg.de
Abstract. This paper reports the difficulties of recruiting participants at the cli-
ent’s site in an empirical project on requirements engineering & management.
The author discusses the origins of the problems and some stated reasons for the
participation. These results lead in a short discussion about the improvement of
empirical research on the requirements engineering & management process at
the client’s site.
Keywords: Qualitative research, requirements engineering, requirements engi-
neering & management, user oriented research
1 Introduction
Known studies on software project processes seem to be mostly conducted on the
producer’s site, especially in cooperation with software producing companies. The
reason is obvious: Software projects are a core business for software producing com-
panies, who are interested in improving their processes. But business information
systems are systems whose success is related to the usage by the clients. Therefore
research at the client’s site should be important. On the client’s site the research focus
seems to be the (business) success of information systems, according to the DeLone
& McLean information systems success model (Urbach et al. 2008; Urbach et al. 2009).
Research on processes on the client’s site is rare and is concerned from special influencing
factors.
This paper will show and discuss some problems, observed in an empirical study
that was conducted in 2010/2011 in Germany in companies outside the IS business
(Weißbach, R. 2013). The paper starts with a short description of the study and the
recruitment of participants (Chapter 2), followed by a chapter on the stated difficulties
for participation (Chapter 3) and a chapter on triggers for participation (Chapter 4).
Proceedings of IW-LCSP 2013 59
Chapter 5 presents and discusses the conclusions. Chapter 6 will show some ideas for
further work.
Acknowledgements. I like to thank the reviewers of this paper for their valuable
advice to work out the ideas more precisely.
2 The Study and the Recruitment of Participants
In 2009/10, the author started a study on the participation of business department staff
in the requirements engineering & management [RE&M] process. The aim of this
study was to get a more differentiated view on the RE process in business information
systems projects and in non-project work. The study was focused on (but not exclu-
sively limited to) small and medium enterprises [SME].
RE&M in general is a topic that is covered by many textbooks and empirical re-
search. The importance of the RE&M process for the success (or the failure) of pro-
jects is generally accepted in literature (overview in Herrmann, A. et al [eds.] 2013).
But the research on RE&M focuses on the main actors in software engineering: re-
quirements engineers, project managers, developers. Business department staff is
commonly seen as object in the requirements elicitation process. The active participa-
tion of business department staff in contrast is disregarded in literature and obviously
ignored in research (checked against the summarization in (Cheng, B., Atlee. J. 2007).
In this situation the author wanted to conduct 25 semi-structured personal inter-
views in the area of Hamburg, Germany, as a pilot study. The project was staffed by
the author and a student assistant. Five participants had been found by personal con-
tacts. To get the other 20 participants we collaborated with a regional entrepreneurs’
association (“Bundesverband Mittelständische Wirtschaft”, Hamburg). We thought to
contact 50 member companies by personal telephone calls to get a relevant, but not ex
ante quantified number of participants and added a call for participation on the web-
site of this association. To clarify: The interviews should be conducted personally, the
telephone calls should only be used to arrange the interviews. In return, the partici-
pants have been announced to get the results of the study and to get an invitation to a
free workshop on RE.
No company accepted the personal invitation and only two companies responded
to the presentation on the website. But both of these companies had been software
companies, who were interested in the result of the study.
Therefore we decided in 2011 to make personal telephone calls by the student as-
sistant to make appointments. The student assistance had experience in acquiring
participants for marketing research studies. We picked telephone numbers from public
telephone directories and asked for responsible persons in IS and/or business depart-
ments. The telephone agent worked from the university’s site, so that the university’s
official telephone number was transmitted. To get 18 participating companies it was
necessary to contact ca. 900 companies (multiple calls counted only as one contact),
equivalent to a response rate of 2%. This response rate seems to be very low, but we
did not find information about response rates in comparable situations. Typical re-
Proceedings of IW-LCSP 2013 60
sponse rates for telephone interviews with companies in Germany are 20-30% (Koll,
C. 2006)
In 4 of the 25 participating companies, the interviews had been conducted simulta-
neously with 2 interviewees working in the same company, either in the same or in
different departments.
3 Arguments for Non-Attendance
3.1 Introduction
Many companies mentioned the lack of time or a privacy policy as reasons to their
non-attendance. But these have not been the only arguments. Focusing on the main
research topic, we did not record and count the answers explicitly at that time. There-
fore the following aspects should be seen as indicators, not as clear and complete
results.
3.2 General Lack of Interest in Research
Companies in the IT branch are interested in market research and in improving pro-
cesses. For these companies the benefit of participating in research projects is obvi-
ous. But what could be the interest for companies in other branches to participate in
IT research projects?
The focus on IS research at the user’s site is the success in IT projects. This topic is
an accepted research topic (DeLone and McLean, see Urbach et al. 2008, Urbach et al.
2009). While this topic addresses the management in general, research on RE&M process-
es is a very specialized topic. Referring to the “rigor vs. relevance” discussion (Benbasat,
I., Zmud, R. 1999, Lyytinen, K. 1999) it seems that RE&M is not relevant for man-
agement.
3.3 Research Topic is a “Non-Issue”
Many of the asked companies told us, that they are not interested in the research topic,
because it is not relevant to them.
This argument could be interpreted in different directions:
Unknown vocabulary: The term “requirements engineering” (or the German trans-
lation, “Anforderungsanalyse”) is not known. Anticipating this danger, the tele-
phone agent paraphrased the problem additionally.
Lack of awareness: The importance of this topic is not realized. - Or: The im-
portance of this topic is realized, but it is no problem in praxis. - Or: The topic
seems to be not relevant, because IS are not seen as important for the core process-
es of the business.
Proceedings of IW-LCSP 2013 61
3.4 Empirical Research has no Direct Benefit
Empirical Research has no direct benefit to the participants: One reason is that there is
no direct output of the research action. The other is “difference in timeframes of ac-
tion between academics and practitioners” (Kuechler, W., Vaishnavi, V.. 2011: p.
127). A typical result of empirical research is a benchmark that could be useful for the
participants. But this benchmark refers to a former situation that must not be valid any
more.
In Design Science Research (Vaishnavi, V.,Kuechler, W. 2012) the benefit for the
participant is more concrete and immediate.
4 Triggers for Participation
4.1 Own Academic Background
Some people agreed to participate because they remembered their own university
background. They wanted to support the university in general or the scientists and
they wanted to get back in contact to the university to discuss and reflect their posi-
tions.
This argument was produced by participants in companies with a relative low pro-
portion of academic staff.
4.2 Professional Awareness
Some people had been interested in the research topic due to their professional educa-
tion. These people were computer specialists, programmer, technicians, regardless of
an academic degree.
4.3 Own Experiences
Some participants acquired awareness of RE&M by own experiences in projects.
Most of them reported problems in the project due to poorly conducted RE&M, in-
cluding a complete project failure. Other participants realized early enough the im-
portance of RE&M in larger projects.
4.4 Desire for Reflection
Some participants had been interested in the reflection and discussion of their prac-
tice, as a kind of consulting. They were interested in the results of the research for
improving their own knowledge.
Proceedings of IW-LCSP 2013 62
4.5 Mouth-to-Mouth References / “Snowball Principle”
One participant gave us the phone number of a colleague who was interested in this
research topic, too.
5 Conclusions
Starting the project we thought that the effort for recruiting participants would be less.
According to (Benbasat, I., Zmud, R. 1999) who specified applicability (= utility),
currency and interest to professionals as important, we thought to have a research
topic well fitting to the companies’ needs.
But it seems that research at the client’s site is more difficult than at the producer’s
site, because the benefit for the participants will often not be directly recognizable.
Also, results of empirical research will need longer time for dissemination than the
design of artifacts.
Due to the lowly estimated importance of empirical research on RE&M processes
at the client’s site, it seems to be difficult to establish new research directions. There-
fore we will depend on qualitative research – e.g. case studies, grounded theory – to
understand the diversity in RE&M processes especially on the client’s site.
6 Validity Discussion
This paper analyses an RE&M research project with an untypical research question
and a heterogeneous group of potential interviewees. Therefore the observations could
not be seen as valid for other research questions in Software Engineering in general.
The response rate for typical research questions with a homogeneous group of po-
tential interviewees will be higher.
7 Ideas for Further Work
Regarding the research experiences described in this paper and the preliminary results
described in Weißbach, R. (2013), we will state a lack of understanding of internal
processes and of collaboration processes between internal and external staff on the
client’s site in the RE&M process. To work out a framework for how RE&M process-
es are conducted at the client’s site, grounded theory and case studies will be the first
valuable approach. This framework could be enhanced with quantitative empirical
research.
8 References
1. Benbasat, I., Zmud, R.: Empirical Research in Information Systems: The Practice of Rele-
vance. In: MIS Quarterly Vol. 23 No. 1, pp. 3-16/March (1999)
Proceedings of IW-LCSP 2013 63
2. Cecez-Kecmanovic, D.: Critical Research in Information Systems: The Question of Meth-
odology. In: ECIS 2007 Proceedings. Paper 150, pp. 1446-1457 (2007). URL:
http://is2.lse.ac.uk/asp/aspecis/20070023.pdf (retrieved 2013-04-06)^
3. Cheng, B., Atlee, J.: Research Directions in Requirements Engineering. In: FOSE’07: Fu-
ture of Software Engineering, Minneapolis (2007)
4. Herrmann, A., Knauss, E., Weißbach, R. (eds.): Requirements Engineering und Projekt-
management. Berlin: Springer (2013)
5. Hevner, A., March, S., Park, J., Ram, S.: Design Science in Information Systems Research.
In: MIS Quarterly Vol. 28 No. 1, pp. 75-105/March (2004)
6. Jabar, M. et al.: An Investigation into Methods and Concepts of Qualitative Research in In-
formation Systems Research. In: Computer and Information Science Vol. 2 No. 4, pp. 47-
54/November (2009)
7. Koll, C.: Möglichkeiten und Grenzen der CATI-Methode bei Betriebsbefragungen. In:
Buchwald, C. (ed.): Das Telefoninterview – Instrument der Zukunft? pp. 22-41. For-
schungsberichte aus dem zsh 06-3. Halle (2006)
8. Kuechler, B., Vaishnavi, V.: Promoting Relevance in IS Research: An Informing System
for Design Science Research. In: Informing Science: the International Journal of an
Emerging Transdiscipline Vol. 14, pp. 125-138 (2011). URL:
http://www.inform.nu/Articles/Vol14/ISJv14p125-138Kuechler570.pdf (retrieved 2013-
04-06)
9. Lyytinen, K.: Empirical Research in Information Systems: On the Relevance of Practice in
Thinking of IS Research. In: MIS Quarterly Vol. 23 No. 1, pp. 25-27/March (1999)
10. Urbach, N., Smolnik, S., Riempp, G.: A Methodological Examination of Empirical Re-
search on Information Systems Success: 2003 to 2007. In: Proceedings of the Fourteenth
Americas Conference on Information Systems AMCIS, Toronto, ON, Canada August
14th-17th (2008)
11. Urbach, N., Smolnik, S., Riempp, G.: The State of Research in Information Systems Suc-
cess – A Review of Existing Multidimensional Approaches. In: Business & Information
Systems Engineering 4, pp. 315-325 (2009)
12. Vaishnavi, V. and Kuechler, W.: Design Science Research in Information Systems. Janu-
ary 20, 2004, last updated November 11, (2012). URL: http://www.desrist.org/design-
research-in-information-systems/ (retrieved 2013-04-06)
13. Wu, Y.: Implications of Case Study Research in Information Systems in Supply Chain
Management. In: 16th EDAMBA Summer Academy Soreze, France July (2007). URL:
http://www.edamba.eu/userfiles/Yi%20Wu.pdf (retrieved 2013-04-06)
14. Weißbach, R.: How Business Departments Manage the Requirements Engineering Process
in Information Systems Projects in Small and Medium Enterprises. In: Issues in Informing
Science and Information Technology Volume 10 (2013). URL:
http://iisit.org/Vol10/IISITv10p539-549Weissbach0093.pdf (retrieved 2013-05-06)
Proceedings of IW-LCSP 2013 64