Verification Validation and Testing: Passion and Deployment challenges in the Italian Eco-System Sara Ricciardi Carlo Leardi IASF Bologna Tetra Pak Packaging Solution INAF Modena Italy Bologna, Italy ricciardi@iasfbo.inaf Luca Stringhetti SKA HQ SKA Organization Macclesfield, Great Britain Copyright © held by the authors. Abstract— Having a solid verification and validation Plan On the base of this fact, the Italian Validation, and a clear strategy to implement it plays a crucial role in Verification, and Test (VVT) group formed in late 2014. The successfully delivering any project e.g. governative, commercial group is one of the working group of AISE, Associazione or scientific. A specific tailoring is required but at the same ti me Italiana di System Engineering, the Italian chapter of it is fundamental to share needs, experiences and methods INCOSE. The group is counting a very heterogeneous group spreading the best practices in the Italian Eco-S ystem and of people that belong to different environments. In the group it highlighting the commonalities. In this framework the VVT is easy to find experts and professionals in VVT processes working group of AIS E hosted a 1-day workshop held in Bologna from the academic world, from the research business, and at the Area della Ricerca the 26th of May 2016 “Verification Validation and Testing: Passion and Deployment challenges”. from industry. Its main aim is to share different experiences in This workshop coordinated by AIS E and organized by a different contexts, and to foster members to work together to Research Institute (INAF) and an Industrial partner (Tetra Pak improve the application of VVT processes. During its first Packaging S olutions) gathered together practitioners for meeting in 2014, the VVT group agreed to work on a list of different background (academia, research, industry, software six objectives. The first two points are: 1) to contribute to the vendors) for a full day discussion and collaborative workshop. development of a leaner application of VVT processes, 2) to This paper presents the outcomes and foresees future steps contribute to the dissemination of VVT best practices between the industrial world and the academic world. In order to fulfil Keywords—system engineering; verification and validation; these two objectives the VVT group organized in May 2016 best practices. the first Italian Workshop on VVT. Different kind of workshops on general System I. INT RODUCTION Engineering topics are common abroad and in Italy, and most There is a general rule in software engineer that states that of them are dedicated to requirements processes or to MBSE to fix a problem during requirement analysis it costs 1$, to fix (Model Based System Engineering). There are a few examples the same problem during coding it costs 10$, but the cost to of workshops fully devote to Verification and Test, but fix it during the testing phase is 100$ [1]. In literature there are normally they are dealing with specific environment; one many studies to measure the cost of the verification and test example is the “5th international workshop on verification and processes and to measure the ROI on the life cycle of a test of space systems” that was organized in May 2016 by the project. Indeed, Nagano [2] presents a thorough analysis of the European Space Agency (ESA). In the VVT group we thought verification process through space programs , and he shows the that a more general workshop could stem interesting importance of controlled processes. In [3] it is stated that discussion and could foster a direct sharing of knowled ge on debugging, testing, and verification activities in software VVT processes. projects can easily range from 50 to 75 percent of the total development cost. The verification, validation, and test II. WORKSHOP ORGANIZATION AND MODUS OPERANDI processes are very expensive in any project, but a proper process execution can minimize risks of mission un-success In this framework we, as members of the VVT group, [2]. decided to focus the Italian professionals interested to VVT around an event and we asked ourselves how to do it. We immediately agreed the gathering should be informal, open and it should produce an outcome that in principle could be the incipit for a deeper engagement of the community. We agreed the one-day workshop formula could work for our needs. Organizing the workshop we asked ourselves “small” questions that immediately translated into big ones; e.g. we were wondering which kind of professionals could be interested in such a workshop, and which channel of communication will be more suitable to promote it. In the process, we ended-up wandering about the composition of our community (AISE members), about the Italian SE eco- system, about the VVT professionals already in the market and the companies/ institutions where they belong; we asked ourselves if such professionals and their companies are aware Fig. 1. Number of European Universities offering a SE degree. of AISE activities and if and how this workshop could be ( https://en.wikipedia.org/wiki/List_of_systems_engineering_universities ) appealing for them. Doing that definitely, our point of view was not completely B. Research neuter and our vision was driven and somehow biased by o ur In this macro category we grouped all the stakeholders that personal work environment and previous experiences. apply SE and VVT practices as a tool in the research Luckily, the LOC members belong to different background environment. and experiences (SR researcher at INAF - Istituto Nazionale di Doing science in the 21th century means to be able to design, Astrofisica, CL quantitative Systems Engineer at Tetra Pak, develop, build, and test very large and expensive equipment, LS head project engineer at SKA - SQUARE KILOMETRE before to be able to exploit them scientifically. The project ARRAY Organization). It would be great for the next time to could be a satellite, a large ground facility, an accelerator of have an even more extended and diverse board of particles, but for each of these examples a system approach is organizer/mentor in order to have a more broader view of our mandatory to meet the requirements in term of budget, time community and to target in a more efficient way our next and quality. The vast majority of representatives of this group event. registered to the workshop belonged to INAF. Naturally this At the best of our knowledge we delineated the profiles of the branch of research had more space in the workshop. interested VVT professionals and we tailored the workshop on C. Industry such pictures. During this process we identified four macro classes of possible stakeholders defining the work In this macro category we grouped all the stakeholders environment where they belong: Academia, Research, belonging to the industry. This was probably the slice of Industry, Software Houses. stakeholder more difficult to reach because besides the companies already known for their involvement in AISE it A. Academia was very difficult to reach other companies potentially interested. We attempted to reach other potential companies In this macro category we grouped the professionals and partnering with CRIT, a technology scouting society based on practitioners (University Professors, Researchers, Post-Docs, a consortium of industries based in the Emilia Romagna area, Students) that analys e the System Engineering process es that offers services to private companies regarding themselves and in particular the VVT process es. Since in Italy technological R&D scouting, industrial best practices and currently there is just one University offering a degree in SE innovation support. CRIT also helped us communicating the (University of Rome "Tor Vergata" Postgraduate Master event. degree in Systems Engineering) we invited also Professors from Engineering departments such Department of D. Software Houses Management, Economics, Industrial Engineering, Mechanical A Software Houses definitely is an Industry; because of the Engineering, that developed a strong interest in the System relevance and the role of this particular industry in SE and in point of view. VVT process we decided to separate this kind of companies from other Industries. The software Houses build the SE tools that other Industries will use so somehow they play a different role. SW life cycle has the biggest cost in verification, as stated in the introduction, and a better application of System Engineering process are an effective way to control these costs. The SW discipline has his own dedicated standard on ISO/IEC/IEEE 12207. Once the players were identified, the most important goal of It was important in our point of view to produce an tangible this workshop was to get professionals and practitioners from outcome to direct the efforts of the partecipants and to obtain a these different backgrounds together; to look for product from the workshop. To stimulate this process the commonalities, to highlight good practices , and to depict a LOC, with the contribution of the lecturers, provided a draft of common scenario, the Italian eco-System, where the this map where the conveneers could scatch their ideas. A professionals work. At the same time we were in need to “head” was assigned to each group to explain the preliminary clearly hear the diverse voices from different stakeholders and map and to lead the discussion; each group nominated a get to know different work environments where VVT is “writer” to present the outcome of the discussion to the other applied. For this reason we organized the day in two sessions. groups and then write it down for further refinements and the In the morning we did listen to lecturers for each class of map’s wrap-up. stakeholder giving a general overview of their work environment and focussing on a key statement that can III. PART ECIPANTS somehow summarize their point of view of VVT practices. In this section we will briefly resume some stats about the In the afternoon the workshop itself took place. To get things participation to the workshop. We got about 60 people that really going we needed diverse people talking each other registered for the workshop and about 30 that showed up. The composition of the participant is resumed in Fig. 3 . It is clear about their VVT experiences in an informal way; we needed that half of the participants belong to the research sector and to keep track of the informal discussion; we needed a common within this sector astrophysics play a crucial role; almost 90% framework to highlight natually the commonalities. of the researchers belong to INAF (Istituto Nazionale di Therefore we divided the partecipants in 7 groups assuring Astrofisica) or they have worked there. In the Industry sector, about 35%, we see two large groups composed from some variety in the composition. We asked the partecipants to professionals affiliated to Tetra Pak and to Datalogic. work together to identify the commonalities between different environments either positive (e.g. best practices) or negatives The Software Houses and the Academia sector provide both (e.g. not adequate expertise/education, “walls” to overcome). about 7% of the participants. The professional in the first The basic idea was to create a visual map showing the sector belong to IBM, Blizzard, ESTECO; the ones from connections between the four sectors (academia, research, Academia belong to Politecnico di Milano, Università di industry and software houses) in term of commonalities, walls Modena e Reggio Emilia, DIMES Università della Calabria. and quicksands relatives to VVT practices. SE for complexity management Mith or real opportunity? Know-how Funding opportunities Citation index confoundings Academia Research feedbacks Demonstrative solutions Needs Spin offs Specific fields knowledge Competences/Skills Opportunities feedbacks Competences/Skills Opportunities Competences/Skills Multidisciplinary Design Optimization by Simulation Business Specific fields knowledge Industry SW Vendors COTS/tailored Intellectual Solutions Property Know-how Normative Specialized Networking “Lean” as the key to success? Agile methodology applied to SW development framework Fig. 3. Attendees by their sector. The total number of attendees was 27. Fig. 2. T he visual map proposed to partecipants. The level of participation was pretty good for this kind of event especially thinking that the area of interest of this workshop is very narrow (VVT practices) and the typical number of participants for such events. (E.g. the SESE 2016 tour http://aise-incose-italia.it/events/southern-europe- systems-engineering-tour-sese/). The fact that we record such large participation for INAF and Tetra Pak shows that the promotion of an event plays a crucial role. Another interesting fact is that recently INAF itself organized a workshop on SE so probably this community or at least some members are already adopting a system approach and VVT techniques and were ready to jump in. There is also an historical reason. SE just a myth or a real opportunity? showed a possible and was born with big projects like the ICBM (Inter Continental credible solution of the problem based on MBSE and virtual Ballistic Missile) in the late 50s [4]. The rocket used for the commissioning simulation processes. ICBM, the big Atlas rocket, was used by the just formed Sergio Terzi – Politecnico di Milano – Department of NASA (1958) for its most important dream; to bring a man in Management, Economics and Industrial Engineering: In this an orbital flight. This was the Mercury program. The new born talk the observatory GeCo (Collaborative Process space agency started to configure all the engineering processes Management of Design) was presented. The study presented is that worked so well for the pioneering programs, and with based on a series of surveys among more than 400 small, time they become standards. This effort was concluded in the medium and large Italian manufacturing companies from four publication of the NASA System Engineering Handbook in industrial macro-areas. Those survey are fundamental to 1995. In more recent times (1996) the new European bridge the gap between the literature theory and the industrial Cooperation for Space Standardization (ECSS), established in practice about New Product Development (NPD) process and 1993, released the first set of standards; any contractor of the Innovation. The key statement reported by Prof. Terzi is: “In European Space Agency (ESA) must adhere to these set of their design process companies have still to understand the standards. This means that SE is well known and used in the potentialities of system oriented approaches. Test and space business since a few decades. Even if the main reference experimentations are still made too late. Some excellences for Ground astronomy the European Southern Observatory exist and they can be highlighted as good examples to the was established in 1962, fifteen years before ESA was created, wider manufacturing and engineering context” the standardization in ESO and the application of configured Alfredo Garro, Ph.D. associated professor at the university processes in system engineer came much later. ESO simply of Calabria and visiting professor at NASA proposed , was the first organization of its kind and it couldn’t benefit of connected from Houston, an innovative approach to one of th e the heritage of NASA. This difference is still visible in the pillars of VVT strategies: “Formal Requirements Modelling system culture of the engineers, even if ESO and many of for Simulation-Based Verification”. Writing good ground projects have started to following tailored version of requirements is not sufficient for delivering systems value. ECSS standards or simply adopting other standards (e.g. ISO Efficiently validating and verifying them is the key to success. standards) or handbook as the INCOSE Handbook. Modelling requirements according to a standard and enabling The workshop occasion encouraged many of Tetra Pak all the tasks and the testing strategies in the SE processes by Systems Engineers and VVT practitioners to openly and the model by associating requirements and architecture with actively participate and share during the workshop their behavioural models is what Prof. Garro proposed . The “W” viewpoint with the research community. aspect of systems life-cycles, explicitly including the usage, The workshop got an occasion to engage the community being maintenance and disposal phases is considered. a starting point for following activities. IV. THE WORKSHOP In this paragraph we discuss the key statement that each lecturer provided, we resume the outcome of the workshop and we present the updated conceptual map A. Key statement analysis The academia point of view Cesare Fantuzzi Director of the Doctoral School in Engineering of Industrial Innovation, Department of Examples and live demos of requirements validation were Engineering Sciences and Methods. The development of proposed. mechatronic systems involves the use of different disciplines.. Traditionally, every discipline is developed independently and The research point of view then integrated to generate the final system. However, increasing complexity of automatic machine design makes The research brought two different examples as case studies. unavoidable to integrate simultaneously these disciplines. The The first one related to a scientific instrumentation based on resulting integrated approach carries an intrinsic complexity ground (i.e. the optical telescope VLT) and the second one a into system design process. To achieve this goal, the deep space instrument (i.e. the sorption cooler on the Plank methodology treats the problem of linking the conceptual with satellite). The distinction is important, because of the so executable models to enable the validation by simulation. The different operation phases, the validation and verification key statement presented by Prof. Fantuzzi “System processes are quite different in the two cases. Moreover, the Engineering as a solution to manage complexity in design Plank Satellite followed a very mature set of ECSS standards. processes applied to complex industrial automatic machines: Davide Fierro – INAF – Chief Engineer Office INAF. In his Planck cooler for 7 years (1998-2005) and then he moved talk he reported his personal experience during the complex back to Europe to conduct the ESA test campaign and to integration, verification, and test of the VST telescope [5]. The deliver the instrument. Planck was the first European mission VST is the largest telescope in the world designed for dedicated to map, with unprecedented sensitivity, the surveying the sky in visible light. This state-of-the-art 2.6-m microwave cosmic background radiation (CMB), that is the telescope is on Cerro Paranal, a perfect location for ground- relic of the Big Bang and to investigate the Universe evolution based astronomical observations. The key statement reads that (birth, expansion, future, content). The scientific goals a correct configuration control is a key process for VVT. The required a thermal architecture based on a passive and active configuration control management is part of the technical systems combination and results in the most complex management process, but for complex projects it assumes a cryogenic mission to date. Overall the cryo-chain strategic importance. Projects as the VST, or in general performances are among the biggest technological scientific instruments projects, are a joint venture of different achievement of the Planck mission therefore Planck leaves a and distributed players. These players , eventually, come in the large heritage and a wealth of lessons learned for future cryo same place during the system integration and verification phases. Modern observatories are placed in very remote missions. The specifications required were demanding in places: therefore, the time spent on site must be minimized. terms of operating temperature, cooling power and vibration Documentation plays a strategic role to conduct the interfaces levels and they should be satisfied within the mass and power integration, that normally belong to different systems owned allocations of an ESA medium size mission. No existing by different players, and to test (i.e. verify) them in the correct system was fulfilling these requirements. The ESA approach way and in the most efficient way. at the time was typically very conservative, meaning to relax the requirements and to use whatever technology was already available. The NASA plan to find a solution instead was very ambitious and obviously more risky. The approach implied to come up with a creative and innovative solution: to ass emble a team of people with different expertise able to design, build and verify such a system following an approach similar, under some aspects, to the Faster, Cheaper, Better philosophy. The FCB approach is now controversial due to failures suffered by some projects but it was also able to lead important missions, such as Mars Pathfinder, to a resounding success. The US team had to demonstrate to ESA and to themselves that the undertaken endeavour was solid and promising so they built a prototype for a balloon experiment to demonstrate the TRL. This attempt was unsuccessful but the lessons learned from that first attempt were the inception of a complete new concept for the Planck cryo-coolers and their VVT. Planck Sorption Cooler testing process was complex as for any space instrumentation. The verification of the system took a long time and many resources as depicted in Fig. 5. Fig. 4. VLST Survey T elescope during tests In summary the Planck mission lessons learned in terms of system VVT can be condensed in few bullets: The configuration control is extended to items that deserve  System Engineering and good VVT practices are specific attention. Each of the 84 actuators that composes the fundamental for a mission success specially to avoid active source has a story that must be under configuration over-designing and over-testing control. This story, from the component construction to the  Testing is critical. Test as much and as long as possible: assembly of the single actuator, is extremely important for the anticipating tests at system level even with earlier models fine characterization that is needed for such instrument. (QM’s, EBB’s, etc.) is really important to highlight Without these information, made available in a proper possible issue that don’t show up at subsystem level. configuration control system, would be nearly impossible to  Researchers/scientists need to build a sound knowledge calibrate this instrument at the accuracy requested by the of System Engineering approaches and VVT practices. science cases. This is still a weak spot in the expertise of scientific team Gianluca Morgante – INAF – Head Thermal Engineer for members in space projects. In particular, SE and VVT several ESA space missions had the unusual chance to look at should become part of education programs in the the same problem from two different point of view. The Universities also for space science courses. “problem” was to design, build and test the Planck Hydrogen  When starting new projects, it is critical to involve Sorption Cooler and the two points of view were the NASA competent personnel and build on previous missions and ESA different perspectives. Morgante was member of the lessons learned, avoiding to re-invent the wheel. US team at the Jet Propulsion Laboratory that developed the To demonstate TRL To demonstrate practices are expected to be applied at all level and during all Functionality the phases; the team has a demonstrate experience in the field Cold End of integration and test for spatial mission and for complex Single Component Elegant BreadBoard Qualification Model Testing (EBB) (QM) scientific instrumentation. 2002 - 2006 Flight Model 1 Flight Model 2 The industrial point of view Gaetano Cutrona – Andrea Margini – The key statement Complete functional and performance test campaign on proposed by Dr. Ing. Cutrona Gaetano and Dr. Ing. Margini 2006 - 2008 System Tests ground Andrea from Tetra Pak Packaging Solutions concerned the potentialities of the Agile methodologies to facilitate the identification and introducing new VVT methodologies from 2009 - 2013 Operation the academic to the industrial work. The deployment of SE methodologies in the industry and in the service market in the last decade, focused on the “Lean 2013 End of Life SE” and on the Systems Product Line Engineering. Combining these two aspects an interesting issue for the V&V practitioners is the evaluation of a value-related, sufficient but Fig. 5. Planck/Sorption Cooler Life Cycle not redundant effort during the usage phase of the system life- cycle. In the end it is still hard to find an answer to the starting question: “Test better to test less?”. Is there a definitive The intervention proposed during the workshop, highlighted answer? For sure, VVT practices applied and declined case by the human factors and the varied system responsiveness. New case are one of the possible answer. individuals and organizations, which were only typically marginally informed and consulted during the development phase, are involved in the usage and maintenance phase. The Fabrizio Villa – INAF – Senior Researcher, Head of final user, the customer and the maintenance are typical but Cryowaves Lab. Cryowaves is a made-up word from the not the only examples. The expected net present value fusion of CRYOgenics and microWAVES with the aim to (ENPV) estimations find their confirmation or contradiction. identify a Science and Technology branch mainly focused to Finally, the “intended environment” confirms its rapidly develop technologies and facilities to build and operate changing behaviour. wideband high performance radio, microwave, and mm-wave receivers and antennas operated in cryogenic environment. The SCRUM framework [6] has been used firstly by the Modern microwave and mm-wave astrophysics uses detectors software industry and it is now extending the service one. A that needs to be cooled at cryogenic temperatures to minimize different attitude to developments projects and teams the detector noise, then reaching the impressive sensitivity to organizations is required to support “sprint”, daily or weekly, observe celestial sources as required by science needs. backlogs, design tasks and related verifications for Examples of this kind of technology is the ESA Planck incremental system evolution. Satellite (up to now the most advance cryogenic system fo r A liquid food packaging case study has been proposed. The space) that measured the Big-Bang sound in intensity and V&V activity is strongly driven by effective V&V plans and polarization at microK level and ALMA Observatory Test Reports. The sprint backlogs converge to minimization of (www.almaobservatory.org) that, for instance, recently the requirements. permitted to take an image of a proto-planetary disk rotating around a distant star. Thanks to the iALMA premiale and ESO ALMA upgrades plan Dot. Villa has been engaged in the development of a new prototype receiver for ALMA. The technology legacy of Planck permitted to create a laboratory at INAF/IASF- Bologna (Cryowaves laboratory) to exploit cryowaves at their best. The idea was to setup a state-of-the-art laboratory to integrate, characterize, calibrate complex scientific instrumentation at cryogenic temperatures. In this respect the laboratory and related activities (design and simulations) are well focused on the VTT and more general SE disciplines. Fig. 6. Backlogs requirements convergence to zero by backlogs sprints The final goal is to set up a laboratory and a dedicated workgroup of people where facilities, technologies Development and verification teams integration and knowledges and skills are in place and ready to tackle consistency among cycling artefacts are highlighted as the challenges related to cryogenic, integration, test, verification keys to success for the application of Agile V&V for space technology and beyond. The SE and VVT best methodologies within SCRUM framework. The basic question arose during the workgroups discussion regards the identification of a minimal set of VVT tools and methodologies, part of the ones applied during the development phases, suitable to efficiently enable the usage and maintenance phase of the system. The software house point of view Carlo Poloni – ESTECO – Prof. Carlo Poloni addressed during his intervention the following question: “What is the VVT potential role of the virtual prototyping?” Collaborative Multidisciplinary Optimization generates relevant info while simulations are strictly connected with physical experiments during verification and validation processes. Fig. 8. Data Integration, optimization and then analysis The general observation is that isolated simulation “pearls” are implemented and reported by experts on sub -systems, Looking at another dimension a models’ hierarchy helps to components and sometimes also on systems. They are maintain the coherence between the “VEE” model vertical, i.e. although rarely integrated into a consistent V&V strategy. The system maturity, and the horizontal system evolution models are developed, as necessary, and their results remain dimensions. typically isolated in the overall development overview. Simulations model often anticipate physical testing when Moreover, the required skills to design, run and document the prototypes maturity is not sufficient or not available at all. applications are in the hands of highly skilled experts whom Their development cost is not irrelevant and the re-usability do not relate continuously with the rest of the development must be included into the V&V strategy. team. The coherency among the models and the systems maturity is Prof. Poloni highlighted the importance of a network of one of the key factors to optimize the info/effort ratio. information which sustains the system conceptual and architecture phase with the simulations performed within the Examples and success stories with different multi-domain VVT strategy during the design and verification phase. integrated tools, analytics and graphics were proposed to the attendees. Fioravanti – Blizzard – Dr. Ing. Fioravanti, representing Blizzard srl, proposed the “Simply … Agile” alias methodologies and tools to implement Agile methods in SW development. Basically the actually available Agile development processes are still too complex in order to reach their basic objectives. The main causes of failed SW developed projects are proposed and analysed. Among them: insufficient tests in the intermediate phase. The renovated Agile mantra lies on a wise step-by-step approach where flexibility allows managing the growing complexity with the right VVT effort. Working in small but complete and closed steps allows proposing to the final user, by fast prototyping, allows testing and usage by the final users of the implemented functions . The discussion with the workshop participants regarded the reasons of un-successful Fig. 7. Simulations integrated by a network of information. and successful projects. Raw Data, fundamental to the simulation process require a V. W ORKSHOP OUT COMES structured, coherent and visible effort of integration in order to feed the Optimization and analysis process. The resumed outputs of the groups discussion, proposed to the attention of the overall attendees can be shown in the updated conceptual map Fig. 9 and resumed as in the following paragraphs. feedbacks Test Better to Test Less Gr5 Also a failed test brings useful info Know-how (G. Morgante) Funding opportunities Gr7 Group 5 “Registro” VVT and SE not teached at the university Citation index è Group 7 “Pragmatico” Needs Lack of systematic oriented recognized figures in INAF Not applied in the research Approach properly the issues ```` Specific fields knowledge Lean approach in the great projects Different mindset Academia Focus on the training aspects Bureaucracy “quicksand” in the projects Gr4 Research PhD Thesis and in the organizations (Andrea Margini) Resistance to change and diffuse Negative events which can facilitate the alignment Group 4 “Valore” Job rotations among the two worlds Institutional reflection pause Protect the personal know-how Being informed about WIP (G. Cutrona, minutes to be recovered) Reciprocal info exchange Come out of your’s shell SW development integrated Engineering Projects confoundings Different objectives Pay attention to value return simply not managed Easy simulations Rationalize results (C. Poloni) Rework due to Reqs. Change in the loop Engineers overwhelmed Trade-off between standardization and tailoring Human expertise Use methodologies! Mech-HW-SW integration and plan Gr6 Test Imediately and always. Demonstrative solutions confoundings Document, document, document! Spin offs Group 6 “Obiettivo” Engineers not trained in SE Competences/Skills Gr8 Industry and research: exchange of skills and competences The research produces unique prototypes Avoid scope creeping Opportunities Group 8 “Provare” Opportunities Facilities re-use among research and industry Competences/Skills The industry does not know the frontier applications Entering in the SE can be long an exhausting process Research must acquire systematic and process thinking Competences/Skills (Cremonini) Share resources: information laboratories, skills More a conveyor than a catapult Needs Bureaucracy Specific fields knowledge Gr2 feedbacks Group 2 “Efficace” (Mancin translates) Gr3 Gr1 Group 1 “Vero” Group 1 “Lode” Great project veiculated Different national and international differences The SE practice (Leardi Carlo) Research Spin-offs Analogies and anbiguities in Requirements management Documentation is not time-loosing (Malavasi translates) COTS/tailored Solutions Industry SW Vendors Intellectual Business Property Specific fields knowledge Integrate simulations Know-how Normative In SE processes Specialized Networking “Lean” as the key to success? Symply agile! SE for complexity management Agile methodology applied to Mith or real opportunity? SW development framework Fig. 9. Conclusive conceptual map More similarities than differences among the represented growth can bypass this attitude. E.g. the VVT expert is not viewpoints. Many of the practitioners belong, or belonged, to any more an isolated, although fundamental passage but he is more than one environment at the same time for limited or one of the main wheels that convey, together with the other longer periods. This aspect is remarked as one of the more persons and specialists, to the success. Prejudices do no t h elp powerful catapults to speed-up the innovation deployment. to bypass the resistance to learn and change. Reciprocal Spin-offs, PhD theses, temporary assignments in the knowledge and real opportunities to cooperate are needed. industries, European research funded occasions can furnish the fertile ground for growing-up commonalities. Furthermore, a The fundamentals and the practice of the systems engineering SW tool developer and vendor, a research centre, an are not diffused enough in all involved environments. university, a PMI or a big industry, despite their different Common understanding and faster deployment of research foundations and organizational characteristics, need organized deliverables passes through training and connecting the development processes following the concepts of the systems researcher and the industrial systems engineers on the SE engineering. fundamentals, e.g. the TRL concept, and on the interactions Tailoring is the link among the different environments around among the Integration, Verification, Validation processes with the same concepts. The exposed failures in VVT experiences the other ones. Monitoring the VVT SE applications in order and the recovering strategies proposed from different to furnish a methodological reference is highlighted as one of panellists during the discussion reinforced similarities more the factors to address the state of the art. E.g. Agile processes than differences. Starting from the requirements interpretation are not yet enough known and applied Even if modern ambiguities, through the “verification war”, and finally to the organization in the Astrophysical business are starting to validation challenges, similar issues were presented and appreciate these methods, there is a big margin for discussed. Tailoring for the VVT processes is one of the key. improvement for a fruitful sharing of knowledge. The PMI Great telescopes, one of a type prototype, mass productions o r must be supported by the research centres and big industries to production lines could require VVT processes tailoring. tailor their own way to Systems Engineering. Bureaucracy and resistance to change characterize all the Focus on the overall V&V strategies. In order to avoid the environments. Only a real understanding of how the SE “margin on margin” impact and to apply the “test better to test processes can leverage the system value of the personal less” approach. Single examples of success cannot enough to sustain the development value. A systematic approach in the V&V strategy is required. Clearness in the application of the Next step of the AISE VVT working group is to diffuse and SE methodologies, tools, and a critical attitude vs. what done leverage on the outcomes and deliverables of the workshop helps to avoid replicating the common mistakes. within the larger Systems Engineers and VVT practitioners Integrate by small steps virtual simulations with traditional community. The formalization of this paper goes in this physical testing can be the way to success. An easy, kiss, direction. start-up which doesn’t require a too big once in a time effort. The second step of this activity is planned for spring 2017. It can be continuously extended in scope and deepened by Enlarging the involved community, prioritizing and identify incremental “natural” growths to tend towards maximum real common opportunities on the themes highlighted during value achievement. the workshop are the ambitions of this second sharing occasion. Contacts are also on-going to diffuse the informatio n Documenting, despite the selected implementation (e.g. with the parallel initiatives in the INCOSE EMEA sector. classical or model based) documentation control and configuration is essential. It can require a quantity of effort, Acknowledgment but it is fundamental not to disregard it due lack of time/funds The authors of this work would like to acknowledge the or perform it approximatively. The addressed direction is the invited speakers at the workshop. They would also one of value-related documentation deliverables that goes acknowledge Tetra Pak Packaging Solutions and Cryowaves from: traditional word check-lists, laboratory logs to virtual laboratory for make the workshop possible. This workshop models deeply integrated in the development cycle (a was partly supported by the Italian Ministero dell'Istruzione, documented model itself). Universita' e Ricerca through the grant Progetti Premiali 2012 Re-use of the experiences developed in the university or by -- iALMA (CUP C52I13000140001). the research, after their main objective, for didactic, exemplifying or for continuous testing in the environments different from the starting ones. References [1] G. Labovitz, et al. “Making Quality Work: A Leadership Guide f o r t h e Continue to meet is felt as one of the main needs , talk and Results-Driven Manager” Harpercollins, 1992 create a “common place” where exchanging past, on -going [2] S. Nagano, “Space Systems Verification Program and Management Process”, System Engineering, Vol 11, Issue 1,2008 and future experiences. This is the key to overcome, by [3] B. Hailpern, P. Sant hanam, “Software debugging, testing, and intellectual curiosity, the personal or organizational verification”, IBM, Systems Journal, Vol. 40, No1. 2002,. restrictions. Furthermore, to create reciprocal knowledge, [4] Rescuing Prometheus: Four Monumental Projects T hat Ch a nge d t h e trust and common opportunities. Modern World by T homas P. Hughes, 2000. [5] “First Images from the VLT Survey T elescope” ESO 1119 – VI. NEXT ACT IVITIES Organisation Release, 8 June 2011 [6] T he SCRUM Guide , 2013 K. Schwaber, J. Sutherland