Mobile application development operates in a market characterized by low barriers to market entry, short time-to-market and the need for rapid return on investment, making it suitable for exploiting the potential of open innovation. Technology-driven entrepreneurs often diverge from the standard practice of antecedent business case analysis. We report here upon the result of a six-month empirical investigation of this question, performed within an incubator setting, and our analysis of the results indicates a reasonable probability of success, at least for ventures with access to experienced requirements practitioners. Our results indicate that incorporating RE techniques from the beginning of the venture has the potential to reduce the risks associated with the missing business case analysis. The field observations have also identified requirements engineering challenges in this domain worthy of further investigation. In particular, the relative impact of business requirements upon the technology requirements is extreme and requirements methods must respond not only to agile development processes but function even when a pivot (an instantaneous and complete change) in business focus occurs.
Mobile applications development operates in a market characterized by low barriers to
market entry, short time-to-market and the need for rapid return on investment
(especially when operations are self-financed, a scenario distinct from large-scale
efforts financed by venture capital, for example, where the goal is to first build a
customer base then attempt to monetize that customer base). The innovations in this
market are often acquired or based on freely available frameworks, e.g. Google
Android, setting these efforts within an open innovation context. Technology-driven
entrepreneurs in this domain often diverge from the standard practice [
Identifying a viable customer value proposition continues to be a significant
challenge in this domain. For example, a recent market study [
find it remarkable that only 24% of developers in our sample plan their apps based
on discussions with users, a figure which does not change with development
experience or proficiency. This indicates that the bottleneck of the
build-measurelearn cycle of lean development is the “measuring”, or understanding customers.
A business analyst is typically responsible for the initial customer contacts (not those
contacts that occur during development), assessing the viability of the business case
and ensuring that the product requirements represent market needs [
We have noted that technology development teams usually receive (at least rudimentary) requirements engineering training during their software engineering courses. We posit that extending RE practice to applying requirements engineering techniques to at least some of the issues usually addressed in the business case analysis would enhance the viability of these entrepreneurial endeavors. When compared to the scenario where these issues are not addressed, this approach enables improved value proposition definition, improved compliance of the product definition with the value proposition and improved focus of development efforts upon the business goals. We expect the process change will improve customer understanding, addressing (at least in part) the issue noted in the market research above.
An empirical investigation was performed in the context of a mobile application entrepreneurship camp held in the summer of 2012 in Saskatchewan, Canada. Entrepreneurship camps are often used by universities and business incubators as a mechanism to foster new business opportunities. The low barriers to market entry in the mobile application space have led to a significant increase in mobile application startups within these camps. The camp reported upon here was conceived and directed by the lead author and grew to become a collaborative effort by the members of the local innovation and technology commercialization ecosystem. Given the diversity of the stakeholders and their goals, the lead proponents made the deliberate decision to apply RE activities to the tasks of stakeholder identification, requirements elicitation, negotiation, prioritization and triage as a management tool for controlling the camp’s design – in essence treating the design of the camp as if it were a product design.
We present our experience report on the suitability of using those RE activities
noted above (plus requirements scoping, tracing and validation) as a surrogate for
business case analysis in the mobile application entrepreneurship context. We found
these well-established RE techniques [
In Section 2 we describe the camp and our experience in transforming the camp from concept to reality while Section 3 describes the participant selection process. In Section 4, we describe related work and in Section 5 we describe the camp’s design and implementation. Section 6 describes the milestone methodology used in the camp and Section 8 captures our observations and practitioner guidance. We close with our conclusions and directions for future work in Section 9.
The mobile application entrepreneurship camp focused on providing business skill training to technology-trained aspiring entrepreneurs. The camp advocates used two requirements brainstorming sessions to (1) identify the goals for the camp and (2) identify potential external camp stakeholders (potential camp supporters). Triage was performed on the range of potential camp stakeholders; the stakeholders on the resulting short-list were those with a strong interest in promoting economic development within the community and region.
Despite the common interest in regional economic development, the short-listed external stakeholders had never worked together on a single project before (but subsets had worked together in the past). Achieving their endorsement for the camp was challenging because each entity has divergent mandates. We approached achieving support in this context in the same manner as we would approach requirements negotiation, generally following an iterative process wherein we would make a proposal based on a best-effort synthesis of positions, receive stakeholder feedback, then update the proposal before submission for further stakeholder review.
The external stakeholders began to support the camp once they were solicited for their input both to the test-bed design and to the type of data to be captured throughout the camp. Each stakeholder provided in-kind instructional support on specific topics that were aligned with the mission statement of the instructional stakeholder, providing participants with access to wide-ranging domain expertise. Industrial stakeholders also provided their domain-specific perspectives to the participants and direct participant mentoring as their time and resources permitted.
The final list of camp supporters included six service provider entities related to either the local university or the innovation support community and three local businesses that provided philanthropic support. The service providers included the technology commercialization entity that hosted the camp, an entrepreneurship support and business case development entity, an incubation and office space provider, the local university industry liaison office, the provincial organization responsible for export development and the local office of the federal business regulation compliance and commercial entity support. The local businesses included a mobile game developer, a business strategy consulting firm and a business operations consultant. 3
Camp participants were recruited via a competitive call for participation that was circulated at two local universities and one local college. Applicants were required to pitch themselves as entrepreneurs and their product concept before an evaluation panel. The panel made their decisions based on an assessment of the applicant’s ability to meet the following prioritized requirements (most important first).
1. Ability to learn and utilize the course materials, work and education history: Did the applicant have the necessary prerequisites to comprehend the material as delivered? Many applicants simply did not yet have the necessary technical skills and practical business experience to complete the very intense first month of the camp – typically they were too early in their academic career or the evaluators felt that the camp would not be able to provide the requisite level of support to these applicants. 2. Oral and written communication skills: The intense pace of the camp created a risk of failure for participants who could not understand the materials as presented or could not succeed in the ‘sales’ elements of entrepreneurship. 3. Entrepreneurial potential: Did the applicant provide some evidence that they had an entrepreneurial attitude, the perseverance and the determination necessary to succeed? 4. Product concept: Did the applicant follow the provided guidelines and attempt to present the product in the context of market demand or did they choose to present in the context of a technology push?
Twenty-five projects were proposed by the applicants and four were chosen for the camp. Three of the four projects were led by single participants: a mature developer with college training, a 3rd year electrical engineer and computer science combined program student and a 2nd year computer science student. The 2nd year computer science student also acted as the graphic designer for the camp. The final project was led by a group consisting of three recent graduates from Bachelor and Master computer science degree programs (each with less than 3 years industrial experience). All members of the three-person group were actively employed by third parties throughout the duration of the camp.
Participants in the camp committed to meet the primary requirement to deliver a product that was ready for deployment (or as close as possible) within the appropriate mobile application store and within the constrained time and resources. The primary requirement was supported by a secondary requirement: to develop a supporting business plan and a pitch for third-party investment.
The camp began with product definition techniques that included workshops on clarifying the product value proposition and identifying the minimum viable product. Product definition and feature identification, storyboarding for use-case and scenario analysis, and project scoping followed. Later workshops focused on more business focused aspects such as conducting a commercial opportunity assessment, identifying market segments, general business planning, intellectual property and revenue generation models.
The camp was not an academic exercise. While two of the six participants were still attending university, all projects were real entrepreneurial ventures undertaken in an industrial setting. The total budget for the camp exceeded $100,000 including preparation of instructional materials, workshop delivery, facilities, mentoring, and project investments. 4
Given that this work is set in the context of a startup environment, we constrained the
literature review to publications that investigated requirements engineering in a startup
context or publications that address the delineation between requirements engineering
and business analysis roles and how the roles complement each other. We were unable
to identify specific related work in this area. The literature review was broadened to
include related work for the different aspects of the presented paper with a focus in the
interaction of RE with business-related issues. In the area of software startup studies,
Ruokolainen and Igel [
Seyff et al. [
The stakeholder requirements for the camp were met by an intensive four-month program focused on developing the business skills necessary for entrepreneurial success. The basis for the camp’s content was derived from the stakeholder representatives’ combined practical experience. The camp was presented as a series of workshops and participant deliverables after each workshop were directed toward advancing their entrepreneurial endeavor. The camp did not provide participants with specific application development technology training, the participants required a minimum technical skill level to participate, but mentors did provide the participants with on-demand technical support and mentorship in mobile applications development.
The participants successfully completed a condensed business plan and the first release of their mobile application by the end of the four month camp. Each participant demonstrated their abilities in the following areas: Business startup process, accessing and utilizing business resources Requirements elicitation and prioritization Finance and accounting General and mobile app marketing Legal aspects of intellectual property protection Project management Business and technical presentations Public speaking
While each participant began the camp with a product concept, these concepts were at various stages of maturity but all of the concepts were scoped in excess of what the given resources could accomplish. An intensive and heavily mentored effort was undertaken to identify the intended customer (stakeholder identification) as well as their wants and needs (as much as possible given resource constraints) followed by developing a clear description of the associated customer value proposition. An intensive feature prioritization and triage effort was performed by each of the participants, again supported by the mentors and instructors.
Unlike traditional RE where requirements are elicited from a known customer, the camp required the definition of a new product for a projected market. Hundreds of potential product requirements were proposed, reviewed and prioritized and each project postulated numerous use-cases and user scenarios in an attempt to identify core customer needs. These models were evaluated against market segmentation information. Did the requirements (value proposition) hold together for the projected market? Was this a market willing to pay (was there a known pain)? Was the customer able to pay? These techniques supported the development of a minimum viable product definition, discussed further in Section 7.
These initial efforts identified the core customer needs, enabling the participants to define the minimum viable product for their markets, successfully completing an entire requirements scoping cycle – from elicitation to scope commitment. The process used by the participants assumed that the available resources were sufficient to deliver the revised product since they were working toward a minimum viable product definition. In each case the proposals were reviewed by experienced practitioners to ensure that there was a reasonable probability of success. Finally, the participants captured the requirements for their product in a context-appropriate manner, usually using structured lists and rich text formats, only occasionally using formal statements for major requirements that must be met.
Approximately two months into the four month camp, the camp organizers recognized a need to extend the course content to meet the investment readiness goal. Requirements for a two-month supplementary program, focused on preparing an entrepreneurial opportunity for third-party investment, were gathered and the content developed. The supplementary program was focused on providing the participants with the ability to communicate their entrepreneurial goals via a third-party pitch for investment. The three singleton entrepreneurs completed the supplementary program. 6
The camp extrinsically motivated participants to meet educational and performance objectives via a milestone payment structure, designed to be similar to disbursement models used by investors in early-stage startups. Each milestone was comprised of several deliverables and participants would only receive a monetary stipend after successfully completing that milestone. Each deliverable within a milestone was introduced to the participants as a requirement. Deliverables for a business analysis element, such as defining the customer’s value proposition for the marketing plan were treated no differently than functional requirements for a product or service specification.
The following seven milestones were defined prior to the start of the camp and a summary of their deliverables were presented to the camp participants, in task format, as presented here. The specific RE activities utilized in each milestone are enclosed in parentheses at the end of the description.
M1: Define the Product Identify the market wants and needs and provide a clear
definition of the value proposition for the app. Describe the features and explain how
they relate to the value proposition. Using persona techniques [
M2: Refine the Product Further define the target user and the target market. Develop estimates for the size of the target market including market segmentation data and develop first estimates of the revenue potential. Begin development of the revenue model. Finalize the scope of the app and develop an initial task list for both business and software development goals. Develop low fidelity prototypes of the app user interface and begin user testing. Define a software architecture for the app and identify high-risk development tasks. (scoping, prioritization)
M3: Proof of Concept Develop a functional prototype of the user experience for the app. Identify nonfunctional requirements and business constraints such as regulatory compliance and ensure that the app complies as necessary. (quality requirements via elicitation from mentors, research into regulatory requirements)
M4: Complete ALPHA Complete internal testing, mobile applications must be ready for focus group testing with external candidates. Locate and fix design flaws, validate and verify that the product meets the functional requirements and works as intended. (requirements validation using focus groups, traceability to requirements as they have evolved over milestones M1 through M3)
M5: Complete BETA Perform focus group testing and adjust usability as necessary, report on focus group findings. Complete marketing plan and develop sales model, finalize revenue model. (continuous requirements verification and validation)
M6: Delivery Demonstrate that app is ready for submission to app store. (continuous requirements verification and validation)
M7: Investment Pitch Prepare and deliver a third-party investment pitch targeted at the Angel investment community. (elicitation, negotiation, prioritization, triage, representation) 7
A project within a startup support environment (such as an incubator) is expected to be a commercially oriented product (or service) with a well-defined customer value proposition. We initially observed that the technology focused entrepreneurs proposed a technology push to a perceived customer problem instead of focusing on a market pull. They expected to use agile development to deliver a rapid prototype, perform customer testing and obtain test market feedback.
We observed customer test plans in these projects that were focused on evaluating functionality and usability but the technology-trained camp participants did not consider investigating the customer’s willingness to pay, or customer’s ability to pay, during their customer interactions. In the absence of business analyst resources, we recommended to the camp participants that they perform as much primary market research (customer interviews) with the test group as possible.
Project refinement is incremental if the project appears to have the potential to meet a real customer value. However, the project may, for example, be rejected in market testing, as experienced during the camp, resulting in a pivot where the fundamental nature of the business changes. Interestingly, the team members usually stay together after the pivot, even when their skills may not be as well-suited to the new business direction as the old.
The observed operational pattern followed by the camp participants is described as
“build it, ship it, fix it, monetize it.” In other words the entrepreneur’s intent is to make
a significant development investment in an effort to ensure minimal delay before
market entry and it is often the case that a monetization focus does not even begin until
after market entry [
Many mobile applications expect to generate revenues, not from active monetization via an initial sale of the product or service itself, but by charging small amounts for various extensions as the customer becomes committed to the product. Alternatives for passive monetization include selling access to the customers for advertising and performing data mining upon the users and their information then selling the results to third parties. Unfortunately, in the absence of the antecedent business case analysis the development efforts may be in vain for, even though the intended product was successfully developed, the market may simply not exist as expected and all monetization attempts essentially fail.
Fig.1 illustrates the observed and desired behavior patterns (the diagram is simplified and many iteration paths are not shown). The right-hand side of the diagram (with graphic elements in black text on white fill) captures the technology-driven pattern observed in all projects on entry to the camp. The project proponent has either a perceived problem or a technology innovation that they believe solves a customer problem (referred to as a technology push) and a solution concept is generated. A prototype is iteratively developed and tested by internal users until such time as the quality is sufficient to test in the market, following the agile paradigm that all projects are prototypes until they ship. Market feedback is obtained and further iterative cycles are performed until product release.
The left-hand side of Fig. 1, with graphic elements in white text on black fill, captures the business case analysis process. A solution concept is proposed for a given market need, supported by evidence that justifies financial investment. The value proposition for the product is iteratively refined until there is sufficient evidence of customer ability to pay and customer willingness to pay. The product definition is revised as necessary before undergoing a competitive analysis. Market position is defined and estimates of the return on investment are generated.
Iterative refinement of the value proposition leads to the definition of the minimum viable product, also known as the least salable unit. The minimum viable product is intended to capture the core of the customer value proposition, rather than attempting to identify the most accurate representation of the product requirements, and as such represents the minimum subset of possible features for which the customer is willing to pay. This approach requires extensive RE activities to exhaustively identify the requirements for which a customer might be willing to pay. These requirements are then analyzed to determine the minimum set of the most important requirements that need to be included in the product for a customer to actually pay for the product/service. Considering a large body of requirements provides some degree of confidence that the global perspective for the minimum viable product has been achieved rather than a minimum viable product for only a specific market segment.
These product definition methods are similar to those advocated by Ries [
The minimum viable product approach is a natural outgrowth of the intense timeto-market pressures for mobile applications and their low probability of successfully generating revenue. By definition, the minimum viable product requires the minimum viable development effort, and therefore the minimum development investment, maximizing the probability of an overall positive return on investment.
The process for obtaining the value proposition appears to be functionally identical to an extremely rigorous requirements prioritization and triage effort. Rather than using factors such as technological uncertainty and development risk for requirements prioritization, the value proposition investigation intensely cycles between customer ability to pay and customer willingness to pay. Integration of this financial focus into requirements engineering practice may yield substantial customer satisfaction benefits and is an area of interest for further research. 8 8.1
In a mobile applications startup environment it is likely that a single person will be
responsible for performing both BA and RE tasks. In a technology driven startup, it is
more likely that RE skills are present therefore we propose the use of traditional RE
techniques [
In our case, the participants performed RE on various aspects of the business process where in some cases a BA would be better suited. But in this case it would have taken too long to train them to do the work using a BA point of view. Instead, it was more time efficient and effective for participants to use tools that they have already used in the past as a (likely non-optimal) improvement over not performing any type of business case analysis.
As noted earlier, we have observed that the techniques used by business analysts and requirements engineers are quite similar, at least in an abstract sense. However, we have found that applying the RE techniques to the BA domain has significant challenges. The first challenge lies in the area of domain specific terminology – in the same way that business analysts may not understand the technology details we find that requirements engineers are equally challenged by business terminology.
This language barrier is compounded by a lack of subject matter expertise. For example, participants in the camp found the requisite financial analyses to be challenging even though the underlying mathematics were considerably simpler than what they were used to using in their traditional practice; what was once familiar was suddenly challenging. We also observed that the participants had difficulty understanding the concept of market segments and target markets even though these concepts are similar to stakeholder groups. One-on-one coaching was required with each participant in addition to the weekly instructional courses. After much repetition, the participants finally understood what a market segment was and how to identify them.
While observing experienced requirements practitioners performing their mentoring, we noted that they exhibited the characteristics and domain knowledge that we also associated with business analysts. When the participants worked with their mentor we observed significant improvements in their output quality and reductions in their training time, illustrating the importance of domain knowledge and providing further evidence of the cross applicability of the techniques between domains. Some of the mentors observed that entrepreneurs might be able to successfully use task checklists, constructed by experienced personnel, to guide their efforts and complete these tasks if experienced personnel are not available.
Another challenging area is the validation of business requirements. When validating a requirement, particularly when using agile methodologies, practitioners simply query the customer representative. Entrepreneurial new product development efforts do not have a customer representative; successfully introducing an innovation that meets market needs, in the absence of the customer, is uncertain at best. For example, the iPhone had no real customers but was a dramatic market success – very few gambles like this provide such a positive return on investment. Business constraints, such as regulatory issues, also proved challenging for there was no obvious way to capture and represent these nonfunctional requirements in a lightweight manner.
Finally, defining the functional requirements for a product differs greatly from validating the business case for a proposed product concept in a proposed market. The participants’ ability to perform functional or feature definition was relatively strong whereas their ability to identify or quantify the market value proposition was relatively weak. We recognize the difficulties associated with developing a market value proposition and are concerned that reliably performing this task may require significant practical experience 8.2
Both requirements engineering and business analysis are important and necessary elements of the entrepreneurial process. However, it is our opinion that a sound business case analysis significantly improves the probability of delivering the desired product to the customer. Requirements engineering can reliably deliver a valid product specification, but what if the customer does not want the specified product? Without customer validation a technology push effort is only an educated guess. It follows that having a validated value proposition as the basis for the requirements effort will lead to a greater probability of commercial success.
In a fast-paced market such as mobile applications, business case analysis can begin at the same time as prototype development used to elicit market feedback (nonfunctional prototypes used to evaluate customer response to product or service concepts, used with caution due to the possibility of loss of control over the underlying intellectual property). If the results of the business case analysis are positive then the parallel start on prototype development delivers a jumpstart on the overall development process. However, if the business case is negative then significant expenses and lost opportunity costs have been incurred. The feedback received during the camp can be summarized as follows: the participants have good presentations skills and well understood market propositions and well-crafted business models. The participants were weak on their financial analyses.
Pursuing a startup venture in the absence of a valid market value proposition significantly increases the probability of a pivot. In the camp studied here, two of the four projects were pivots on entry into the camp. These pivots were deemed necessary as a result of the analysis done between the time of participant selection and their entry into the camp. One pivot maintained the general nature of the product (stop-motion animation) but the target market(s) were completely redefined. The original target market was neither willing nor able to pay for the proposed product and failing to pivot would have resulted in a well-defined product that no one was interested in purchasing. In the second pivot, the original concept was discarded and an entirely new market opportunity was pursued.
We have been unable to identify a plausible scenario that justifies not performing a business case analysis. The analysis does not have to be done before development begins. It can be done in parallel with development if the associated risks are acceptable, but the evidence before us suggests that it needs to be done. We cannot conclude that the business case is more important than the product requirements but an antecedent business case analysis can greatly reduce the risk that the product requirements define a product that no one wants. We note that, in the same way that customer willingness and ability to pay should constrain RE efforts, so should technology constraints be considered in the business case analysis. 8.3
The camp participant selection process was strongly biased toward selecting candidates with entrepreneurial tendencies that were also believed capable of learning the materials and completing their app within the allotted time. As such, the chosen candidates were the elite of the applicant population and were intended to represent a sample of the entrepreneur population and not the general population.
The camp participants were drawn from a technology-trained population. Their observed behavior in Section 7, Figure 1 may have been biased toward the right-hand (technology-focused) cycle rather than the left-hand (market-focused) cycle. Further investigation is needed to determine how prior training and experience affects behavior in this area. 9
This work has investigated the feasibility of extending RE practices by applying requirements engineering techniques to the investigation of commercial viability for proposed products and services. None of the camp participants had more than superficially considered commercial viability of their products before camp entry. Usually addressed through business case analysis, these RE-based efforts enhance the entrepreneurial endeavor’s viability through improved value proposition definition, compliance of the product definition with the value proposition and provide focus upon the business goals for development efforts. Our results were generally supportive of the practice, successfully applying requirements elicitation, negotiation, prioritization, triage, scoping, tracing and validation to business case analysis tasks, particularly when guided by an experienced mentor. We do not consider this a best practice, but our initial results indicate that using RE to perform business case analysis does benefit the project. For the camp participants, two of the four projects performed significant and successful business pivots that addressed real (and not just assumed) markets by the application of these techniques.
Perhaps the most significant challenge to success is domain specific terminology and knowledge. Asking requirements engineers to perform business analysis tasks requires them to become subject matter experts, at least to a degree, in a whole new discipline and this is not something that can occur quickly. We suggest that academic training could include entrepreneurial concepts and greater strength in the fundamentals needed to perform due diligence in a business case analysis. While perhaps unnecessary on well-rounded teams, this domain knowledge would facilitate communication and provide insurance for smaller teams.
Four business cases were developed in the camp and in all cases we began business case analysis at the same time as prototype development. Our experience indicates that a business case analysis is a significant element within a risk reduction strategy and that entrepreneurs should prioritize this analysis as much as possible to determine the viability of the venture as quickly as possible. The techniques that we applied in the camp are widely used outside of RE and it appears that traditional RE practice assumes that they have already been performed by other members of the team. Simplistically, for commercially motivated endeavors, the RE practitioner should first identify that there exists an identifiable customer population that has a willingness to pay for the new product or service. Then the practitioner should confirm that there exists a sufficiently large subset of this customer population that also has the ability to pay for the new product or service – only then does a viable business case for sufficient ROI possibly exist and only then should intensive RE efforts begin.
We note that the condensed timelines associated with mobile application development appear to be aligned with the needs of academic research scheduling and resource allocation. Incubators may be a rich source of case studies for combined academic-industrial research.
Feedback was received from all stakeholders and we have learned many practical lessons from delivering this camp. The next generation of the camp has been adapted as much as possible, within resource constraints, and the next session of the camp is eagerly awaited by the local community. [19] Sommerville, I., Sawyer, P.: Requirements engineering: a good practice guide. John Wiley & Sons,
Inc., 1997