Challenges with organizing for solution discovery in socio- technical design projects Ane M. Ness 1, Tonje Løfqvist 1 and Magnus Li 1 1 1 University of Oslo, Department of Informatics, Norway Abstract Academic and practitioner literature emphasize the importance of the “discovery” phase of socio-technical design projects, which involves a thorough exploration of potential solutions to the problem in which the project aims to address before committing to and investing heavily in specific solutions. Yet, little research has examined how solution discovery unfolds and is organized for in real-life digitalization projects. This paper reports from an ongoing case study of the design and innovation practices of a set of IT consultancies. From the preliminary findings, the paper identifies and discusses three key challenges for organizing for solution discovery. These are: balancing flexibility versus predictability, the procurement line separating solution discovery from creation as a barrier, and client knowledge gap. With these challenges, we contribute to knowledge on solution discovery in socio-technical design projects and highlight a set of avenues for further research. Keywords Socio-technical design, solution discovery, procurement line, flexibility, client knowledge gap, design debt 1 1. Introduction Socio-technical approaches to IT design, such as Design Thinking and User-Centered Design, are argued to be important for successful digitalization as the approach takes both social and technical aspects of the organization into consideration when designing technology [1, 2, 3, 4]. Researchers emphasize the importance of the “discovery” phase of socio-technical design projects, which involves a thorough exploration of potential solutions to the problem in which the project aims to address before committing to and investing heavily in specific solutions [5]. It is argued that many projects jump straight to creating a solution, and hence, not spend sufficient thought into what the best solution represents on a conceptual level [6]. Discovery is a phase where the goal is to find ‘the right solution’ to ‘the right problem’. This can be done through a process of exploring the room of possibilities in a given project context by taking into consideration a wide range of socio-technical factors, such as existing IT systems and workflows, stakeholder needs and goals [7, 6]. An imbalance in the amount of emphasis and resources put into either solution discovery or solution creation, can either lead to “the right solution” built poorly, or a solution which is built “correctly”, but still fails to address stakeholder needs and goals. Whereas existing academic and practitioner literature on socio-technical design offers models and principles which emphasize “discovery” as part of the design process, there appears to be a lack of research on how to ensure and organize for this in real-life digitalization projects [5, 6]. A relatively recent literature review points out that the literature clearly evidences the importance of solution 8th International Workshop on Socio-technical Perspective in IS development (STPIS 2022), August 19–21, 2022, Reykjavik, Iceland EMAIL: ane-ness@hotmail.no (A. 1); tonjelofqvist@gmail.com (A. 2); magl@ifi.uio.no (A. 3) ORCID: 0000-0002-7050-12X2 (A. 1); 0000-0001-7880-4461 (A. 2); 0000-0002-5308-9171 (A. 3) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 33 discovery, yet, that we have little knowledge on how this unfolds and is organized for in real-life projects [5]. In this work-in-progress paper, we report from an ongoing research project where we examine the practices of a set of IT consultancies. The IT consultancies of focus support public and private organizations, predominantly within health, with design, development, and implementation of health management information systems. We collaborate with the organizations on strengthening their sociotechnical design and innovation capacity. From our preliminary findings, our paper examines the following research question: What are challenges with organizing for solution discovery in sociotechnical design projects? From our collaboration with the IT consultancies, we observe that in many cases, discovery is a somewhat neglected part of the design process. Contributing to the existing knowledge on how to organize socio-technical design processes in real-life digitalization projects, we address the research question by identifying three key challenges with organizing for solution discovery. The remaining paper is organized in the following manner: First, we elaborate on solution discovery as part of socio-technical design projects and survey existing literature on obstacles to organize sociotechnical design in real-life digitalization projects. Second, we explain our research methods, before we present our findings including the three key challenges identified. Finally, we discuss how our challenges contribute to research and practice on socio-technical design. As part of this, we argue that how to organize socio-technical design processes in real-life digitalization projects represent an important research area and highlight a set of concrete avenues for further research. 2. Related research Beyond being discussed as important and highlighted in socio-technical design models in both academic and practitioner literature, solution discovery has received limited attention in academic literature [5]. However, more broadly, the challenges of implementing socio-technical design practices and carrying out projects have received some attention. Firstly, an array of literature have identified and examined the reasons for the limited uptake of socio- technical design methods/approaches in software design and development practices as compared to typical software engineering approaches such as agile [8]. The limited uptake is partly attributed to an incompatibility between the socio-technical approaches which largely are developed and discussed in academic communities, and the far more practice-oriented software engineering approaches [9]. Further, some argue that research and dissemination of socio-technical design ideas suffers from a lack of experiences from real-life projects and that they as such present “ideal-type” processes with little support on how to implement these into real-life projects with all their complexities [8]. A stream of literature accordingly attempts to examine the use of methods such as user-centered design “in the wild” [10]. Yet, the what, how, and challenges of organizing for socio-technical design projects has received relatively little attention in academic literature as compared to concrete models and principles conducted [11]. Second, some have argued for the importance of examining the conditions that surround sociotechnical design projects which may enable and constrain important phases and activities, including solution discovery [12]. For instance, Zahlsen et al. [11] conceptualize such conditions as “boundary conditions”, the enabling or constraining factors that surrounds the concrete design activities (e.g., prototyping and evaluation) of the design process [13, 11]. Along similar lines, Svanæs and Gulliksen [13] suggest that the boundary conditions are important for the potential impact of user- centered design activities, and hence the success of the end result of socio-technical design projects. They present and discuss a set of such boundary conditions which involves the organizations involved, their relations and agendas, internal factors in the developer organization, software development methodology and tools, internal factors in the client organization, and customer-developer legal relationships as contracts and tender [13]. Related to the latter factor, also Martin et al. [14] examines the influence of “the contract” between the software consultancies and the client organizations on the socio-technical design activities. The authors find that the contract often is used as a reference point 34 which may constrain flexibility of the project to redefine requirements, and hence, the potential for unanticipated innovation in projects. Others have discussed how generic software solutions, which often are central in many digitalization projects, shape and define design processes by encompassing a set of ready-made software features [15, 16]. As such, the software solution frames what is possible to do in the project, and some argue that there is little room for activities such as user-oriented requirements gathering as the requirements necessary must be developed according to the generic software [16, 14]. Finally, some point out that designers often lack authority to influence the project planning [17]. The team members and designers do not always agree on the need to spend time on activities related to solution discovery and the designer can experience the feeling of not belonging to a largely software developer-oriented culture [17]. Larusdottir et al. [17] suggest giving the designers a “license to kill”, empowering them to influence the project, and the time spent on solution discovery to a greater extent. In sum, the academic literature identifies and examines several challenges and conditions for organizing socio-technical design in real-life projects. How solution discovery unfolds and is organized for in real-life digitalization projects, although argued as important, has however received limited attention [5]. 3. Research Approach In this ongoing interpretive case study, we explore current practices, challenges, and opportunities for improvement of socio-technical design practices within the District Health Information Software 2 (DHIS2) software ecosystem [18, 19]. The software is one of the largest and long-standing digital platforms being used for health management [20]. To this end, we collaborate with the vendor of the DHIS2 software and a set of IT consultancies who specialize in implementing the generic solution together with user organizations. In this paper, the practices of the latter type of actor is of focus, which support user organizations such as ministries of health in several countries in Asia and Africa. 3.1. Data collection and analysis In our ongoing research project we have, so far, conducted 11 in-depth interviews with three participants that work in different IT consultancy firms in developing countries in Asia and Africa. All of the interviews were serial interviews which focused on different topics within the research area. In the initial phase of the research process, the aim was to understand socio-technical design practices. As the research evolved, the focus shifted to challenges related to different implementation projects, and more specifically challenges related to solution discovery. The IT consultants in our research had different roles and extent of experiences. Table 1 briefly summarizes the role and experiences of our participants. Table 1 Participants roles and years of experience IT consultant Role Years of experience Participant 1 Project manager, senior 10 developer Participant 2 Project manager 6 Participant 3 Researcher, designer 6 35 In the first round of interviews the aim was to understand their current socio-technical design practices by focusing on what they usually do in implementation projects. As the research evolved, the focus shifted to challenges related to different implementation projects and more specifically challenges related to solution discovery. The interviews were recorded and transcribed. When coding and thematizing the material, the ability to emphasize and organize for solution discovery emerged as a recurring challenge. With this as a guiding theme, we revisited the transcriptions and developed themes to reflect the overall challenges and obstacles related to solution discovery. The final themes are the ones presented in the following chapter. 4. Case Analysis From our preliminary data analysis, we have identified three key challenges to organizing for solution discovery in real-life digitalization projects: balancing flexibility versus predictability, the procurement line separating solution discovery from creation and client knowledge gap. 4.1. Balancing flexibility versus predictability Fundamental for solution discovery is the exploration of the problem and potential solutions to organize the project around. For this to be possible, there must be some flexibility in the project for requirements to emerge and be established along the process. As such, the project cannot start with a list of requirements that specify the details of the solution. The designers must hence promote an open- ended starting point of the project. On the other hand, predictability regarding the outcome of the project is crucial for the client organizations investing in the project. In projects with a flexible starting point, designers have the ability to explore the context of use before a final list of detailed requirements is defined and hence engage in solution discovery. In contrast, some projects are very rigidly structured from the start of the project. A challenge with the more predictable projects is that the discovery phase is limited and may not be explored to its full potential. Balancing project flexibility and predictability is thus a key challenge to engaging in solution discovery. Our findings further indicate that the ability to achieve the right balance between the two may be affected by factors such as trust, resources, and the type of project. Figure 1: Balance between flexibility and predictability 4.1.1. Limited trust between clients and designers Limited trust is a challenge which affects the designer's ability to negotiate for project flexibility. From our findings, we identify that the client's trust in the designers often derives from past experiences. When a designer could refer to a portfolio with past successful projects, it's easier for clients to allow for more flexibility and less predictability in the projects. Trust plays a role in the decision making of 36 how much freedom a designer will have and allowing agility in the development process. This will offer more room for exploring and doing solution discovery later in the project. Vice versa, if clients have bad past experiences, the likelihood of giving the designers room for exploration will be reduced. A participant stated, “if they have had some past experience due to incidents then they will be much more careful the next time and they want everything to be planned out nicely and they don't want to give too much freedom to the development entity”. In cases such as this, the designers often experience having less freedom regarding the project throughout the whole process, which affects the solution discovery phase. Interestingly, one of our participants explains that in projects with too little flexibility, and accordingly a narrow space for solution discovery, projects may end up developing the wrong solution. This, in turn, will result in less trust for new phases of the project, and accordingly less flexibility. The IT consultancies may hence end up in a vicious cycle of degrading trust, flexibility, and room of solution discovery. A second aspect is how the client perceives the IT consultancy’s domain knowledge. A participant explains that their organization differed from other client organizations because all of their senior consultancies are from a hybrid medical public health and informatics background. Regarding trust in this case, this health background can make the clients have more trust in the designers as they put more faith in their understanding of the domain. Vice versa, a lack of domain knowledge may reduce the trust, and hence gravitate the project towards more predictability than flexibility. 4.2. The procurement line separating solution discovery from creation Whereas some projects suffer from finding the right balance between flexibility and predictability, in others, the IT consultancies enter after the phase of solution discovery. Most IT projects start with a procurement phase, and in some projects, all solution discovery-related activities are completed before the IT consultancies are procured. A procurement line i.e., a temporal boundary may thus separate the phases of solution discovery and solution creation. The procurement line could be problematic because the designers contracted to create the solution do not have the opportunity to influence the overall conceptual idea surrounding the solution. Requirements and user needs tend to change in socio-technical design processes. According to one of the IT consultants, some design projects tend to be plan driven, but the goal is to work iteratively. Solution discovery should take place as a recurring step in an iterative process, but the procurement line works as a barrier. When the procurement is done, it could be difficult to change the overall conceptual plan for the solution and organize for more discovery. This makes it challenging to adapt to the new requirements. 4.3. Client knowledge gap Finally, a client knowledge gap related to how to organize design processes could be a challenge throughout the project because they see limited value in investing time and resources in activities related to solution discovery. A participant stated, “How aware the stakeholders are about this design process…. what they perceive as design is like a set of requirements and the vendor is supposed to design as per whatever they are saying and they are not open to discussions or negotiations then design doesn't really mean much.”. Accordingly, when negotiating the project, a lack of understanding between the designers and members of the client organization often occurs. As a result, when not provided with enough resources, the discovery phase gets deprioritized. The result can be seen as a design debt i.e., additional design work that must be done at a later point as one has invested in creating the wrong solution. In other words, it is the result of leaving out the design process that facilitates solution discovery which can give a solution that is built “correctly”, but still fails to address stakeholder needs and goals. 4.3.1. Ambitious expectations 37 Further, a common challenge in the start phase is that the client has very ambitious expectations for the final solution. A participant stated, “So they [the customers] usually tend to have been, the first time they approach us, usually all of them have a very rosy picture of the technology and the software, and software that they're expecting is very sophisticated”. Many clients have a limited understanding and experience of coping with technology, and as a result, ambitious expectations arise. With the clients' ambitious expectations combined with their low budget, the designers must often work to lower the clients’ expectations to a realistic one. 4.3.2. Time scarcity Several IT projects are affected by a limit of resources such as time, budget, knowledge, tools or infrastructure. Limited time is a consequence of the clients not arranging for it, and therefore affects the discovery phase to a great extent. A participant stated, “So if your project is not having the time resource as freely as we require, then of course there's very limited scope for innovation in design processes”. Because the project is under a strict time limit, decisions tend to be made out of intuition and already known insight. This might not be a problem itself, but it reduces the time spent on discovery. Not being able to spend time on the initial discovery phase may lead to a design debt later in the process, as the design process has not been as extensive as it should have been. To clarify how time affects the design process, a participant explained that the implementation group can receive requirements on Friday and is expected to be finished by Monday. This puts pressure on the designers to skip a proper discovery phase and to deliver a solution that might not be the “right” solution. 4.4. Summing up the challenges Table 2 briefly explains the challenges we found in our ongoing case study. Table 2 Challenges with discovery in socio-technical design projects Challenge Description Flexibility versus predictability Finding a balance between flexibility to adapt requirements along the project and predictability regarding the outcome of the project is a key challenge for doing solution discovery. Procurement line The procurement line represents a boundary for engaging in solution discovery. Client knowledge gap Clients may have limited knowledge on how to organize design processes. Clients might think of a more plan-driven project structure, while solution discovery requires an iterative approach. Further, this may result in unrealistic timelines and resource allocation. 5. Discussion and conclusion In this paper, we set out to explore the research question: What are challenges with organizing for solution discovery in socio-technical design projects? We extend existing knowledge and find that challenges with organizing for solution discovery in socio-technical design projects include 1) balancing flexibility versus predictability 2) the procurement line line as a barrier and 3) client knowledge gap. 38 Our work-in-progress paper directly extends on Brehl et al. [5], where we have identified a set of challenges specifically tied to organizing for solution discovery. In general, the existing literature reports a limited uptake of socio-technical design in real-life digitalization projects [8, 17]. Our preliminary findings is a further exploration of this concern. Concretely, we extend the body of work on boundary conditions for socio-technical design processes by identifying three key challenges tied specifically to solution discovery [13, 11]. In line with findings from as e.g., Martin et al. [14] proposes, our findings highlight the importance of “the contract” in sociotechnical design projects. During our research, we saw the “procurement line” as an interesting contractual challenge which is related to how projects are organized in regard to procurement. This has a great impact on the potential whether the design project obtains a more flexible or predictable structure. Further, Baxter and Sommerville [8] point out that many managers realize that socio-technical issues are important, but still socio-technical design methods are rarely used due to difficulties in using the methods. Our findings add to our knowledge on why design methods are difficult to use, particularly related to solution discovery. For instance, the client knowledge gap constrains the design processes as the clients do not provide sufficient resources for being able to practice socio-technical design methods. We find that the clients often have unrealistic expectations regarding the financial resources and time to spend on the design processes, which can make the design methods deprioritized. The lack of knowledge on design processes is also a substantial reason why socio-technical design methods are rarely used. 5.1. Avenues for further research In this work-in-progress paper we have identified three key challenges to organizing for solution discovery in design projects. Although the work and findings presented in this paper is preliminary, we argue that our findings point to some relevant avenues for further research: First, we do see that elements from the challenges are related to each other. A further investigation would possibly seek to understand how the dynamics of the different factors would have an effect on each other. By investigating the challenges further, we might be able to answer questions such as; How to best arrange the boundary conditions so that they enable rather than constraining solution discovery? Second, we find the issue of the procurement line as a particularly interesting problem. Questions still remaining to be answered are how to deal with the procurement line. Several researchers consider procurement as an activity in a socio-technical life cycle, but little research is done on how the procurement line actually limits socio-technical design processes, or how projects should be organized around it [8, 11]. 6. References [1] Mumford, E. (2000). A Socio-Technical Approach to Systems Design. Requirements Eng, 125– 133. [2] Mumford, E. (2006). The story of socio-technical design: reflections on its successes, failures and potential. Information Systems Journal 16, 317–342. [3] Whitworth, B., & Ahmad, A. (2013). The social design of technical systems: Building technologies for communities. The Interaction Design Foundation. [4] Scacchi, W. (2004). Socio-Technical Design.The encyclopedia of human-computer interaction, 1, 656-659. [5] Brhel, M., Meth, H., Maedche, A., & Werder, K. (2015). Exploring principles of user- centered agile software development: A literature review. Information and software technology, 61, 163-181. [6] Brown, T. (2008). Design thinking. Harvard business review, 86(6), 84. [7] Sommerville, I., Cliff, D., Calinescu, R., Keen, J., Kelly, T., Kwiatkowska, M., McDermid J., & Paige, R. (2012). Large-scale complex IT systems. Communications of the ACM, 55(7), 71-77. 39 [8] Baxter, G. & Sommerville, I. (2011) Socio-technical systems: From design methods to systems engineering. Interacting with Computers 23, 4–17. [9] Jurca, G., Hellmann, T. D., & Maurer, F. (2014, July). Integrating agile and user-centered design: A systematic mapping and review of evaluation and validation studies of agile-UX. In 2014 Agile conference (pp. 24-32). IEEE. doi: 10.1109/AGILE.2014.17. [10] Stompff, G., Henze, L., Jong, F. D., Vliembergen, E. V., Stappers, P. J., Smulders, F., & Buijs, J. (2011). User centered design in the wild. In DS 68-1: Proceedings of the 18th International Conference on Engineering Design (ICED 11), Impacting Society through Engineering Design, Vol. 1: Design Processes, Lyngby/Copenhagen, Denmark, 15.-19.08. 2011. [11] Zahlsen, Ø. K., Svanæs, D., Faxvaag, A., & Dahl, Y. (2020, October). Understanding the Impact of Boundary Conditions on Participatory Activities. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society (pp. 1- 11). doi:10.1145/3419249.3420129. [12] Li, M. (2021). Generic Enterprise Software implementation as Context for User-Oriented Design: Three Conditions and their Implications for Vendors. In 12th Scandinavian Conference on Information Systems (SCIS), Orkanger, Norway. [13] Svanæs, D., & Gulliksen, J. (2008, October). Understanding the context of design: towards tactical user centered design. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges (pp. 353-362). doi:10.1145/1463160.1463199. [14] Martin, D., Procter, R., Mariani, J., & Rouncefield, M. (2007, November). Working the contract. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces (pp. 241-248). doi:10.1145/1324892.1324945. [15] Li, M. & Nielsen, P. (2019). Making Usable Generic Software. A Matter of Global or Local Design?. [16] Ellingsen, G., Hertzum, M., & Melby, L. (2022). The Tension between National and Local Concerns in Preparing for Large-Scale Generic Systems in Healthcare. Computer Supported Cooperative Work (CSCW), 1-31. doi:10.1007/s10606-022-09424-9. [17] Larusdottir, M., Gulliksen, J. & Cajander, Å. (2016). A license to kill – Improving UCSD in Agile development. Journal of Systems and Software 123, 214–222. [18] Myers, M. D. (1997). Critical ethnography in information systems. In Information systems and qualitative research (pp. 276-300). Springer, Boston, MA. [19] Walsham, G. (2006). Doing interpretive research. European journal of information systems, 15(3), 320-330. [20] Nicholson, B., Nielsen, P., Sahay, S. & Sæbø, J. I. (2021). Digital Public Goods Platforms for Development: The Challenge of Scaling. arXiv preprint arXiv:2108.09718. 40