Do software startups innovate in the same way? A case survey study

Abstract

The research interest in software startups has expanded a lot in the last years as shown by the increase in the published papers, and the organization of research workshops. However, two recent systematic mapping studies recognize an inconsistency in the characterization of software startups in the literature, even though they acknowledge that innovativeness and uncertainty are the common themes the literature uses to describe these companies. In the new product development literature, even though not consolidated, innovativeness is usually related to technology and/or market discontinuities. These two di erent types of novelty could bring distinct consequences to software development activities in software startups. Using a case survey research approach, we analyzed 27 published papers from the period 2013-17. We identi ed and categorized the innovation in 18 software startups products or services from the perspective of discontinuities. We found that software engineering literature did not di erentiate software startups based on the innovations they develop. Nevertheless, most studied software startups work on products with a market discontinuity and without a technological one.

Keywords

• software startups
• innovativeness
• technology discontinu- ities

Introduction

Research in software startups has grown in the last ve years as discussed by Berg et al. in a recent systematic mapping study [ 2 ]. Comparing the papers in this period to a previous systematic mapping study performed by Paternoster et al. [ 14 ], the authors conclude that the rigor of primary studies has increased in this period. Nevertheless, they also found an inconsistency of characterizing software startups similarly to what Paternoster et al. [ 14 ] have found. In this more recent study, the authors performed a thematic analysis of the software startup term used in research and their results indicated that no single factor was used by all papers to characterize software startups. They concluded that the lack of a proper de nition make it challenging to develop a body of knowledge for software startup context. Similarly, Unterkalmsteiner et al. [ 17 ] proposed a research agenda for software startups and, in one of the tracks, remembered that software startups de nitions are \not granular enough [...] making the transfer of practices from company to company di cult."

Still, in Berg et al.'s thematic analysis, the terms most used were \innovation/innovative" and \uncertainty" [ 2 ]. This relation between innovation and uncertainty is explored in new product development literature. Salomo et al. [ 15 ] mentioned \high innovative development may evolve along an unexpected path, thus requiring frequent or continuous information updating and generation of new information". Nevertheless, in innovation literature, product innovativeness has several conceptual con gurations [ 11 ]. Garcia and Catalone [ 7 ] performed a literature review on innovation and product innovativeness terminologies and found several categorizations to label degrees of innovativeness. Nevertheless, according to the authors, \a single theme, however, does underlie all these classi cations of innovations: innovativeness is a measure of discontinuity in the status quo in marketing factors and/or technology factors". Although Calantone et al. [ 4 ] concluded that product innovativeness has no direct e ect on new product pro tability, di erent degrees of newness and discontinuities change factors in new product development (NPD) processes [ 7 ].

To the extent of our knowledge, there is no discussion in software startups literature about the degree of newness of products developed by studied companies. Then, this study investigates the following research question:

RQ: What de nes the innovation of a software startup? To answer the research question, we performed a case survey on the papers published in the period 2013-17 covered by Berg et al. [ 2 ] in their recent systematic mapping study. We read all the papers, selected studied software startups that got the information that allowed us to analyze product and business models, and categorized them accordingly. The results indicate that scholars have studied companies developing products with discontinuities from technical and market perspectives without distinction. This lack of homogeneity could hinder generalization of results presented.

The remaining of this paper is organized as follow. Section 2 presents related work on the software startup term de nition in software engineering literature and presents technological innovativeness as discontinuities, Section 3 describes the methodology used to perform this study and Section 4 presents the results and discuss them. Finally, Section 5 concludes the paper and proposes future work. 2

Background and Related work

The scienti c interest on software startups has grown in the last years but the de nition of what is a software startups is still not consolidated. Section 2.1 displays how this problem has been discussed in secondary studies on the topic. Given that, technological innovativeness is one of the most used themes to describe these companies, Section 2.2 presents some de nitions from new product development (NPD) literature. 2.1 Sutton [ 16 ] is responsible for an early characterization of a software startup company back in 2000. According to him, these companies are widely represented by youth and immaturity, limited resources, multiple in uences and dynamic technologies and markets. In their 2014 systematic mapping study, Paternoster et al. [ 14 ] concluded that there was no agreement on a standard de nition and identi ed the most frequent themes used to characterize software startups: lack of resources, high reactiveness and exibility, innovation, uncertain conditions, time pressure and fast growth. In the more recent SMS, Berg et al. [ 2 ] performed the same analysis and reached a similar result. Nevertheless, the most used themes were now innovation/innovative, uncertainty, small team and lack of resources. In a seminal literature review about innovativeness, Garcia and Calantone [ 7 ] recognize that the term was still not homogeneous in the new product literature, including what is considered what is new. Nevertheless, the authors recognize a consistency: innovativeness \is always modeled as a degree of discontinuity in marketing and/or technological factors."

The authors also emphasize that innovativeness should be analyzed from two di erent perspectives: a macro, related to the newness of that product to the outside of the rm, and a micro, related to the novelty to the rm. Based on these perspectives, they de ne: { radical innovations as representing a technological and marketing factors in a macro perspective; { really new innovations as showing a technological or a market discontinuity; and { incremental innovations as those presenting any discontinuity only in a micro perspective.

Given a startup is generally considered a new company with little or no operating history, we can limit our analysis to the macro perspective, that is, only radical or really new innovations. 3

Research method

As Larsson [ 9 ] mentioned in his paper about the methodology, the case survey is an inexpensive and powerful method to identify and statistically test patterns across studies. The author describes the basic procedure to perform such study: { to select a group of existing case studies; { develop a coding scheme to systematically analyze and convert qualitative descriptions into quanti ed variables; { based on the coding scheme, several raters code the cases; { analyze the quantitative data.

Then, the rst step was to select the primary studies to be analyzed. Given that Berg et al. [ 2 ] published a systematic mapping study on software startup engineering on The Journal of Systems & Software, the primary studies they identi ed are high probably the most important also to this study. Another interesting feature of this study is that it separated the papers from period 2013-2017 against the whole studied period from 1994-2013. Since this study is focused on how current researchers are selecting software startups to study and that the authors also concluded that the rigor has increased in this period in comparison to before, it is reasonable to focus on this more recent time period.

Then, we got the 27 full text papers and read them carefully. Only 7 of those provided information that allowed us to describe the business model of, at least, some studied software startups. Table 1 displays the papers that contained at least one describable software startup and the number of software startups described in each of them. Based on the descriptions and the authors, we could infer that some software startups were mentioned more than in one paper. Then, their descriptions were merged and analyzed as one. The total number of descriptions was 23 but only of those 18 were unique cases.

The software startups' descriptions were extracted and classi ed according to if they present or not technological and market discontinuities. According to Garcia and Calantone [ 7 ], a technology discontinuity represents \a paradigm shift in the state of science or technology in a product" and a market one \may require new marketplaces to evolve, and/or new marketing skills". In our classi cation, a new product or service to contain a technology discontinuity, it had to represent a product that demanded the creation a new technology like a prediction tool for nancial markets. Meanwhile, a market discontinuity represents the application of a well-known technology (like web development) into the creation of a new product, or the application of an existent product to a new market. Products could also display both types of discontinuity when its creation demands a new technology to be applied in a product without a similar solution in the market like a new real-time solution to support sales. 4

Results and discussion

Table 2 displays a summary of the software startups descriptions presented in the analyzed papers and if they present or not a technical or market discontinuity. Figure 1 depicts a graphical representation of the analysis. In the lower-left quadrant where a product would not have neither a technical nor a market discontinuity is empty. This was expected: software startups as innovative enterprises should present at least one discontinuity. Most of startups (13 out 18) are in the lower-right quadrant, that is, they display a market discontinuity without a technical one. The companies apply well-known technologies to create a product to tackle a problem in an existent market or they pretend to create a new market. Meanwhile, only 2 of them are in the upper-left quadrant, showing a technical discontinuity without a market one, that is, they are focused on the creation of a new technology to tackle an existent problem, a new solution to compete with existent products or services. The remaining 3 software startups are in the upper-right quadrant presenting both types of discontinuities: they develop a new technology to tackle a problem that has not been solved before. This last group represents the most challenging environment to a software startup.

In relation to software development challenges, the two groups that present only one type of discontinuity could be also summarized as follows: { Only technical discontinuity: these teams develop brand-new technologies for some consolidated markets. It was possible to observe that their members generally have strong research backgroun and some of these companies were created in universities. Their challenges are situated more in technical problems than customer/market problems. . { Only market discontinuity: these teams develop new products or services with well-know technologies like websites or mobile apps. We observed that these companies are generally formed by recent graduated students. Their challenges are situated more in market problems that is, their major risks are nding customers or users.

These two clusters could also be viewed as teams trying to solve di erent issues on the problem-solution space. The rst group focus on a solution issue to a well-de ned or well-known problem meanwhile the second group focus on nding the right problem to tackle. For this group, once the problem is found, the solution is, at least at a higher level, straightforward. Those software startups which product presents both discontinuities face problems from both spaces.

This di erence in uences the software development process in these companies. For instance, if the problem is not well-understood, requirements engineering practices should be less e ective and the team may need to use other techniques like, for instance, brainstorming and ideation. That is, they can take more advantage of Customer Development and other lean startup practices [ 3 ]. TecOhnpotilmogaileisty[ 10 ] Software tool to database modernization

Market Innovativeness Absent

Present 12 14 4 9

18 1 6 11 2 7 13 Whereas if the customer problem is clear and the solution is not well-understood and need to be developed, the focus will be more in software design and the implementation itself. 4.1

Limitations This paper has as a clear limitation based on the reduced information provided in the papers on how software startups studied create value to their users or customers. Although this lack of details could make di cult to classify the products, even this limited amount of information was enough to determine the discontinuities present. It is possible, tough, that we were mislead and some software startups were wrongly classi ed. Nevertheless, it is highly unlikely that several were wrong at the point to change the main result: software startups that face di erent discontinuities are grouped and have their practices analyzed together.

To increase validity and reliability, both authors did the classi cation and compared their results. The table presented in the paper also helps readers to verify the analysis.

Conclusion

This paper investigated the innovativeness of software startups analyzed in the literature. Using a case survey approach, we read 27 published papers and were able to gather 17 software startups that had their products described. Then, we classi ed them according with the presence or not of market and technology discontinuities. The results highlights that most studied software startups present an innovation with a market discontinuity and without a technical one. Nevertheless, there are also software startups that present a technical discontinuity with or without a market one. This phenomenon could explain why the software startups literature still struggles to de ne what a software startup is. Besides that, the lack of homogeneity in the studied subjects hinder the application of research results by practitioners and also to compare conclusions from di erent authors. Such distinction is also important to compare results or import them from other research elds like the new product development or entrepreneurship literature.

A solution to such problems could be a deeper description of studied software startups. Several reasons could prevent researchers to give more details about studied software startups: sometimes founders do not want their idea to be published. Authors can also be limited by the space provided in a conference paper. Another solution could be an explicit commitment from authors that software startups studied in that work are of one kind or a discussion why these di erences does not matter to the topic discussed.

References

1. Bajwa , S.S. , Wang , X. , Duc , A.N. , Abrahamsson , P. : How Do Software Startups Pivot? Empirical Results from a Multiple Case Study . vol. 240 , pp. 169 { 176 ( 2016 ). https://doi.org/10.1007/978-3- 319 -40515-5 14

2. Berg , V. , Birkeland , J. , Nguyen-Duc , A. , Pappas , I.O. , Jaccheri , L. : Software startup engineering: A systematic mapping study . Journal of Systems and Software 144 , 255 {274 (oct 2018 ). https://doi.org/10.1016/j.jss. 2018 . 06 .043

3. Blank , S. : Why the Lean Start-Up changes everything . Harvard Business Review 91 ( 5 ), 63 { 72 ( 2013 )

4. Calantone , R.J. , Chan , K. , Cui , A.S.: Decomposing product innovativeness and its e ects on new product success . Journal of Product Innovation Management 23 ( 5 ), 408 { 421 ( 2006 ). https://doi.org/10.1111/j.1540- 5885 . 2006 . 00213 .x

5. Chanin , R. , Pompermaier , L. , Fraga , K. , Sales , A. , Prikladnicki , R.: Applying Customer Development for Software Requirements in a Startup Development Program . Proceedings - 2017 IEEE/ACM 1st International Workshop on Software Engineering for Startups , SoftStart 2017 pp. 2 { 5 ( 2017 ). https://doi.org/10.1109/SoftStart. 2017 .3

6. Duc , A.N. , Abrahamsson , P. : Exploring the outsourcing relationship in software startups . In: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering - EASE'17 . pp. 134 { 143 . No. Idi, ACM Press, New York, New York, USA ( 2017 ). https://doi.org/10.1145/3084226.3084248

7. Garcia , R. , Calantone , R.: A critical look at technological innovation typology and innovativeness terminology: A literature review . Journal of Product Innovation Management 19 ( 2 ), 110 { 132 ( 2002 ). https://doi.org/10.1016/S0737- 6782 ( 01 ) 00132 - 1

8. Giardino , C. , Wang , X. , Abrahamsson , P. : Why early-stage software startups fail: A behavioral framework . In: Lassenius, C. , Smolander , K. (eds.) Lecture Notes in Business Information Processing. Lecture Notes in Business Information Processing , vol. 182 LNBIP, pp. 27 { 41 . Springer International Publishing, Cham ( 2014 ). https://doi.org/10.1007/978-3- 319 -08738-2

9. LARSSON , R.: Case survey methodology: quantitative analysis of patterns across case studies . Academy of Management Journal 36 ( 6 ), 1515 {1546 (dec 1993 ). https://doi.org/10.2307/256820

10. Marks , G., O 'Connor , R.V. , Clarke , P.M.: The Impact of Situational Context on the Software Development Process A Case Study of a Highly Innovative Start-up Organization . vol. 770 , pp. 455 { 466 ( 2017 ). https://doi.org/10.1007/978- 3- 319 -67383-7 33, http://link.springer.com/10.1007/978-3- 319 -67383-7 http://link.springer. com/10.1007/978-3-319-67383-7 33

11. McNally , R.C. , Cavusgil , E. , Calantone , R.J.: Product innovativeness dimensions and their relationships with product advantage, product nancial performance, and project protocol . Journal of Product Innovation Management 27 ( 7 ), 991 { 1006 ( 2010 ). https://doi.org/10.1111/j.1540- 5885 . 2010 . 00766 .x

12. Nguyen-Duc , A. , Seppanen, P. , Abrahamsson , P. : Hunter-gatherer cycle: a conceptual model of the evolution of software startups . Proceedings of the 2015 International Conference on Software and System Process - ICSSP 2015 (Idi) , 199 { 203 ( 2015 ). https://doi.org/10.1145/2785592.2795368

13. Nguyen-Duc , A. , Shah , S.M.A. , Ambrahamsson , P. : Towards an Early Stage Software Startups Evolution Model . In: 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) . pp. 120 { 127 . IEEE (aug 2016 ). https://doi.org/10.1109/SEAA. 2016 .21

14. Paternoster , N. , Giardino , C. , Unterkalmsteiner , M. , Gorschek , T. , Abrahamsson , P. : Software development in startup companies: A systematic mapping study . Information and Software Technology 56 ( 10 ), 1200 {1218 (apr 2014 ). https://doi.org/10.1016/j.infsof. 2014 . 04 .014

15. Salomo , S. , Weise , J. , Gemunden, H.G.: NPD planning activities and innovation performance: The mediating role of process management and the moderating e ect of product innovativeness . The Journal of Product Innovation Management 24 ( 4 ), 285 { 302 ( 2007 ). https://doi.org/10.1111/j.1540- 5885 . 2007 . 00252 .x

16. Sutton , S.M.: The role of process in software start-up . IEEE Software 17, 33{39 ( 2000 ). https://doi.org/10.1109/52.854066

17. Unterkalmsteiner , M. , Abrahamsson , P. , Nguyen-duc, A. , Baltes , G.H. , Conboy , K. , Dennehy , D. , Sweetman , R. , Edison , H. , Shahid , S. , Wang , X. , Garbajosa , J. , Gorschek , T. , Hokkanen , L. , Lunesu , I. , Marchesi , M. , Morgan , L. , Selig , C. , Oivo , M. , Shah , S. , Kon , F. : Software Startups - A Research Agenda. e-Informatica Software Engineering Journal 10 ( 1 ), 1 { 28 ( 2016 ). https://doi.org/10.5277/e-Inf160105