=Paper=
{{Paper
|id=Vol-1728/paper13
|storemode=property
|title=Verification Validation and Testing: Passion and Deployment Challenges in the Italian Eco-System
|pdfUrl=https://ceur-ws.org/Vol-1728/paper13.pdf
|volume=Vol-1728
|authors=Sara Ricciardi,Carlo Leardi,Luca Stringhetti
|dblpUrl=https://dblp.org/rec/conf/ciise/RicciardiLS16
}}
==Verification Validation and Testing: Passion and Deployment Challenges in the Italian Eco-System==
https://ceur-ws.org/Vol-1728/paper13.pdf
Verification Validation and Testing: Passion and
Deployment challenges in the Italian Eco-System
Sara Ricciardi Carlo Leardi
IASF Bologna Tetra Pak Packaging Solution
INAF Modena Italy
Bologna, Italy
ricciardi@iasfbo.inaf
Luca Stringhetti
SKA HQ
SKA Organization
Macclesfield, Great Britain
Copyright © held by the authors.
Abstract— Having a solid verification and validation Plan On the base of this fact, the Italian Validation,
and a clear strategy to implement it plays a crucial role in Verification, and Test (VVT) group formed in late 2014. The
successfully delivering any project e.g. governative, commercial group is one of the working group of AISE, Associazione
or scientific. A specific tailoring is required but at the same ti me Italiana di System Engineering, the Italian chapter of
it is fundamental to share needs, experiences and methods INCOSE. The group is counting a very heterogeneous group
spreading the best practices in the Italian Eco-S ystem and of people that belong to different environments. In the group it
highlighting the commonalities. In this framework the VVT is easy to find experts and professionals in VVT processes
working group of AIS E hosted a 1-day workshop held in Bologna from the academic world, from the research business, and
at the Area della Ricerca the 26th of May 2016 “Verification
Validation and Testing: Passion and Deployment challenges”.
from industry. Its main aim is to share different experiences in
This workshop coordinated by AIS E and organized by a different contexts, and to foster members to work together to
Research Institute (INAF) and an Industrial partner (Tetra Pak improve the application of VVT processes. During its first
Packaging S olutions) gathered together practitioners for meeting in 2014, the VVT group agreed to work on a list of
different background (academia, research, industry, software six objectives. The first two points are: 1) to contribute to the
vendors) for a full day discussion and collaborative workshop. development of a leaner application of VVT processes, 2) to
This paper presents the outcomes and foresees future steps contribute to the dissemination of VVT best practices between
the industrial world and the academic world. In order to fulfil
Keywords—system engineering; verification and validation; these two objectives the VVT group organized in May 2016
best practices. the first Italian Workshop on VVT.
Different kind of workshops on general System
I. INT RODUCTION Engineering topics are common abroad and in Italy, and most
There is a general rule in software engineer that states that of them are dedicated to requirements processes or to MBSE
to fix a problem during requirement analysis it costs 1$, to fix (Model Based System Engineering). There are a few examples
the same problem during coding it costs 10$, but the cost to of workshops fully devote to Verification and Test, but
fix it during the testing phase is 100$ [1]. In literature there are normally they are dealing with specific environment; one
many studies to measure the cost of the verification and test example is the “5th international workshop on verification and
processes and to measure the ROI on the life cycle of a test of space systems” that was organized in May 2016 by the
project. Indeed, Nagano [2] presents a thorough analysis of the European Space Agency (ESA). In the VVT group we thought
verification process through space programs , and he shows the that a more general workshop could stem interesting
importance of controlled processes. In [3] it is stated that discussion and could foster a direct sharing of knowled ge on
debugging, testing, and verification activities in software VVT processes.
projects can easily range from 50 to 75 percent of the total
development cost. The verification, validation, and test II. WORKSHOP ORGANIZATION AND MODUS OPERANDI
processes are very expensive in any project, but a proper
process execution can minimize risks of mission un-success In this framework we, as members of the VVT group,
[2]. decided to focus the Italian professionals interested to VVT
around an event and we asked ourselves how to do it. We
immediately agreed the gathering should be informal, open
and it should produce an outcome that in principle could be
the incipit for a deeper engagement of the community. We
agreed the one-day workshop formula could work for our
needs.
Organizing the workshop we asked ourselves “small”
questions that immediately translated into big ones; e.g. we
were wondering which kind of professionals could be
interested in such a workshop, and which channel of
communication will be more suitable to promote it. In the
process, we ended-up wandering about the composition of our
community (AISE members), about the Italian SE eco-
system, about the VVT professionals already in the market
and the companies/ institutions where they belong; we asked
ourselves if such professionals and their companies are aware
Fig. 1. Number of European Universities offering a SE degree.
of AISE activities and if and how this workshop could be ( https://en.wikipedia.org/wiki/List_of_systems_engineering_universities )
appealing for them.
Doing that definitely, our point of view was not completely B. Research
neuter and our vision was driven and somehow biased by o ur In this macro category we grouped all the stakeholders that
personal work environment and previous experiences. apply SE and VVT practices as a tool in the research
Luckily, the LOC members belong to different background environment.
and experiences (SR researcher at INAF - Istituto Nazionale di Doing science in the 21th century means to be able to design,
Astrofisica, CL quantitative Systems Engineer at Tetra Pak, develop, build, and test very large and expensive equipment,
LS head project engineer at SKA - SQUARE KILOMETRE before to be able to exploit them scientifically. The project
ARRAY Organization). It would be great for the next time to could be a satellite, a large ground facility, an accelerator of
have an even more extended and diverse board of particles, but for each of these examples a system approach is
organizer/mentor in order to have a more broader view of our mandatory to meet the requirements in term of budget, time
community and to target in a more efficient way our next and quality. The vast majority of representatives of this group
event. registered to the workshop belonged to INAF. Naturally this
At the best of our knowledge we delineated the profiles of the branch of research had more space in the workshop.
interested VVT professionals and we tailored the workshop on C. Industry
such pictures. During this process we identified four macro
classes of possible stakeholders defining the work In this macro category we grouped all the stakeholders
environment where they belong: Academia, Research, belonging to the industry. This was probably the slice of
Industry, Software Houses. stakeholder more difficult to reach because besides the
companies already known for their involvement in AISE it
A. Academia was very difficult to reach other companies potentially
interested. We attempted to reach other potential companies
In this macro category we grouped the professionals and partnering with CRIT, a technology scouting society based on
practitioners (University Professors, Researchers, Post-Docs, a consortium of industries based in the Emilia Romagna area,
Students) that analys e the System Engineering process es that offers services to private companies regarding
themselves and in particular the VVT process es. Since in Italy technological R&D scouting, industrial best practices and
currently there is just one University offering a degree in SE innovation support. CRIT also helped us communicating the
(University of Rome "Tor Vergata" Postgraduate Master event.
degree in Systems Engineering) we invited also Professors
from Engineering departments such Department of D. Software Houses
Management, Economics, Industrial Engineering, Mechanical A Software Houses definitely is an Industry; because of the
Engineering, that developed a strong interest in the System relevance and the role of this particular industry in SE and in
point of view. VVT process we decided to separate this kind of companies
from other Industries. The software Houses build the SE tools
that other Industries will use so somehow they play a different
role. SW life cycle has the biggest cost in verification, as
stated in the introduction, and a better application of System
Engineering process are an effective way to control these
costs. The SW discipline has his own dedicated standard on
ISO/IEC/IEEE 12207.
Once the players were identified, the most important goal of It was important in our point of view to produce an tangible
this workshop was to get professionals and practitioners from outcome to direct the efforts of the partecipants and to obtain a
these different backgrounds together; to look for product from the workshop. To stimulate this process the
commonalities, to highlight good practices , and to depict a LOC, with the contribution of the lecturers, provided a draft of
common scenario, the Italian eco-System, where the this map where the conveneers could scatch their ideas. A
professionals work. At the same time we were in need to “head” was assigned to each group to explain the preliminary
clearly hear the diverse voices from different stakeholders and map and to lead the discussion; each group nominated a
get to know different work environments where VVT is “writer” to present the outcome of the discussion to the other
applied. For this reason we organized the day in two sessions. groups and then write it down for further refinements and the
In the morning we did listen to lecturers for each class of map’s wrap-up.
stakeholder giving a general overview of their work
environment and focussing on a key statement that can III. PART ECIPANTS
somehow summarize their point of view of VVT practices. In this section we will briefly resume some stats about the
In the afternoon the workshop itself took place. To get things participation to the workshop. We got about 60 people that
really going we needed diverse people talking each other registered for the workshop and about 30 that showed up. The
composition of the participant is resumed in Fig. 3 . It is clear
about their VVT experiences in an informal way; we needed
that half of the participants belong to the research sector and
to keep track of the informal discussion; we needed a common
within this sector astrophysics play a crucial role; almost 90%
framework to highlight natually the commonalities. of the researchers belong to INAF (Istituto Nazionale di
Therefore we divided the partecipants in 7 groups assuring Astrofisica) or they have worked there. In the Industry sector,
about 35%, we see two large groups composed from
some variety in the composition. We asked the partecipants to
professionals affiliated to Tetra Pak and to Datalogic.
work together to identify the commonalities between different
environments either positive (e.g. best practices) or negatives The Software Houses and the Academia sector provide both
(e.g. not adequate expertise/education, “walls” to overcome). about 7% of the participants. The professional in the first
The basic idea was to create a visual map showing the sector belong to IBM, Blizzard, ESTECO; the ones from
connections between the four sectors (academia, research, Academia belong to Politecnico di Milano, Università di
industry and software houses) in term of commonalities, walls Modena e Reggio Emilia, DIMES Università della Calabria.
and quicksands relatives to VVT practices.
SE for complexity management
Mith or real opportunity? Know-how
Funding opportunities
Citation index
confoundings
Academia Research
feedbacks
Demonstrative solutions
Needs
Spin offs
Specific fields knowledge
Competences/Skills Opportunities feedbacks
Competences/Skills
Opportunities
Competences/Skills
Multidisciplinary Design Optimization
by Simulation
Business
Specific fields knowledge
Industry SW Vendors
COTS/tailored
Intellectual Solutions
Property Know-how
Normative Specialized Networking
“Lean” as the key to success?
Agile methodology applied to
SW development framework
Fig. 3. Attendees by their sector. The total number of attendees was 27.
Fig. 2. T he visual map proposed to partecipants. The level of participation was pretty good for this kind of
event especially thinking that the area of interest of this
workshop is very narrow (VVT practices) and the typical
number of participants for such events. (E.g. the SESE 2016
tour http://aise-incose-italia.it/events/southern-europe-
systems-engineering-tour-sese/). The fact that we record such
large participation for INAF and Tetra Pak shows that the
promotion of an event plays a crucial role. Another interesting
fact is that recently INAF itself organized a workshop on SE
so probably this community or at least some members are
already adopting a system approach and VVT techniques and
were ready to jump in. There is also an historical reason. SE just a myth or a real opportunity? showed a possible and
was born with big projects like the ICBM (Inter Continental credible solution of the problem based on MBSE and virtual
Ballistic Missile) in the late 50s [4]. The rocket used for the commissioning simulation processes.
ICBM, the big Atlas rocket, was used by the just formed
Sergio Terzi – Politecnico di Milano – Department of
NASA (1958) for its most important dream; to bring a man in
Management, Economics and Industrial Engineering: In this
an orbital flight. This was the Mercury program. The new born talk the observatory GeCo (Collaborative Process
space agency started to configure all the engineering processes Management of Design) was presented. The study presented is
that worked so well for the pioneering programs, and with based on a series of surveys among more than 400 small,
time they become standards. This effort was concluded in the medium and large Italian manufacturing companies from four
publication of the NASA System Engineering Handbook in industrial macro-areas. Those survey are fundamental to
1995. In more recent times (1996) the new European bridge the gap between the literature theory and the industrial
Cooperation for Space Standardization (ECSS), established in practice about New Product Development (NPD) process and
1993, released the first set of standards; any contractor of the Innovation. The key statement reported by Prof. Terzi is: “In
European Space Agency (ESA) must adhere to these set of their design process companies have still to understand the
standards. This means that SE is well known and used in the potentialities of system oriented approaches. Test and
space business since a few decades. Even if the main reference experimentations are still made too late. Some excellences
for Ground astronomy the European Southern Observatory exist and they can be highlighted as good examples to the
was established in 1962, fifteen years before ESA was created, wider manufacturing and engineering context”
the standardization in ESO and the application of configured Alfredo Garro, Ph.D. associated professor at the university
processes in system engineer came much later. ESO simply of Calabria and visiting professor at NASA proposed ,
was the first organization of its kind and it couldn’t benefit of connected from Houston, an innovative approach to one of th e
the heritage of NASA. This difference is still visible in the pillars of VVT strategies: “Formal Requirements Modelling
system culture of the engineers, even if ESO and many of for Simulation-Based Verification”. Writing good
ground projects have started to following tailored version of requirements is not sufficient for delivering systems value.
ECSS standards or simply adopting other standards (e.g. ISO Efficiently validating and verifying them is the key to success.
standards) or handbook as the INCOSE Handbook. Modelling requirements according to a standard and enabling
The workshop occasion encouraged many of Tetra Pak all the tasks and the testing strategies in the SE processes by
Systems Engineers and VVT practitioners to openly and the model by associating requirements and architecture with
actively participate and share during the workshop their behavioural models is what Prof. Garro proposed . The “W”
viewpoint with the research community. aspect of systems life-cycles, explicitly including the usage,
The workshop got an occasion to engage the community being maintenance and disposal phases is considered.
a starting point for following activities.
IV. THE WORKSHOP
In this paragraph we discuss the key statement that each
lecturer provided, we resume the outcome of the workshop
and we present the updated conceptual map
A. Key statement analysis
The academia point of view
Cesare Fantuzzi Director of the Doctoral School in
Engineering of Industrial Innovation, Department of Examples and live demos of requirements validation were
Engineering Sciences and Methods. The development of proposed.
mechatronic systems involves the use of different disciplines..
Traditionally, every discipline is developed independently and The research point of view
then integrated to generate the final system. However,
increasing complexity of automatic machine design makes The research brought two different examples as case studies.
unavoidable to integrate simultaneously these disciplines. The The first one related to a scientific instrumentation based on
resulting integrated approach carries an intrinsic complexity ground (i.e. the optical telescope VLT) and the second one a
into system design process. To achieve this goal, the deep space instrument (i.e. the sorption cooler on the Plank
methodology treats the problem of linking the conceptual with satellite). The distinction is important, because of the so
executable models to enable the validation by simulation. The different operation phases, the validation and verification
key statement presented by Prof. Fantuzzi “System processes are quite different in the two cases. Moreover, the
Engineering as a solution to manage complexity in design Plank Satellite followed a very mature set of ECSS standards.
processes applied to complex industrial automatic machines:
Davide Fierro – INAF – Chief Engineer Office INAF. In his Planck cooler for 7 years (1998-2005) and then he moved
talk he reported his personal experience during the complex back to Europe to conduct the ESA test campaign and to
integration, verification, and test of the VST telescope [5]. The deliver the instrument. Planck was the first European mission
VST is the largest telescope in the world designed for dedicated to map, with unprecedented sensitivity, the
surveying the sky in visible light. This state-of-the-art 2.6-m microwave cosmic background radiation (CMB), that is the
telescope is on Cerro Paranal, a perfect location for ground- relic of the Big Bang and to investigate the Universe evolution
based astronomical observations. The key statement reads that (birth, expansion, future, content). The scientific goals
a correct configuration control is a key process for VVT. The required a thermal architecture based on a passive and active
configuration control management is part of the technical systems combination and results in the most complex
management process, but for complex projects it assumes a cryogenic mission to date. Overall the cryo-chain
strategic importance. Projects as the VST, or in general performances are among the biggest technological
scientific instruments projects, are a joint venture of different
achievement of the Planck mission therefore Planck leaves a
and distributed players. These players , eventually, come in the
large heritage and a wealth of lessons learned for future cryo
same place during the system integration and verification
phases. Modern observatories are placed in very remote missions. The specifications required were demanding in
places: therefore, the time spent on site must be minimized. terms of operating temperature, cooling power and vibration
Documentation plays a strategic role to conduct the interfaces levels and they should be satisfied within the mass and power
integration, that normally belong to different systems owned allocations of an ESA medium size mission. No existing
by different players, and to test (i.e. verify) them in the correct system was fulfilling these requirements. The ESA approach
way and in the most efficient way. at the time was typically very conservative, meaning to relax
the requirements and to use whatever technology was already
available. The NASA plan to find a solution instead was very
ambitious and obviously more risky. The approach implied to
come up with a creative and innovative solution: to ass emble a
team of people with different expertise able to design, build
and verify such a system following an approach similar, under
some aspects, to the Faster, Cheaper, Better philosophy. The
FCB approach is now controversial due to failures suffered by
some projects but it was also able to lead important missions,
such as Mars Pathfinder, to a resounding success. The US
team had to demonstrate to ESA and to themselves that the
undertaken endeavour was solid and promising so they built a
prototype for a balloon experiment to demonstrate the TRL.
This attempt was unsuccessful but the lessons learned from
that first attempt were the inception of a complete new
concept for the Planck cryo-coolers and their VVT.
Planck Sorption Cooler testing process was complex as for
any space instrumentation. The verification of the system took
a long time and many resources as depicted in Fig. 5.
Fig. 4. VLST Survey T elescope during tests In summary the Planck mission lessons learned in terms of
system VVT can be condensed in few bullets:
The configuration control is extended to items that deserve System Engineering and good VVT practices are
specific attention. Each of the 84 actuators that composes the fundamental for a mission success specially to avoid
active source has a story that must be under configuration over-designing and over-testing
control. This story, from the component construction to the Testing is critical. Test as much and as long as possible:
assembly of the single actuator, is extremely important for the anticipating tests at system level even with earlier models
fine characterization that is needed for such instrument. (QM’s, EBB’s, etc.) is really important to highlight
Without these information, made available in a proper possible issue that don’t show up at subsystem level.
configuration control system, would be nearly impossible to Researchers/scientists need to build a sound knowledge
calibrate this instrument at the accuracy requested by the
of System Engineering approaches and VVT practices.
science cases.
This is still a weak spot in the expertise of scientific team
Gianluca Morgante – INAF – Head Thermal Engineer for members in space projects. In particular, SE and VVT
several ESA space missions had the unusual chance to look at should become part of education programs in the
the same problem from two different point of view. The Universities also for space science courses.
“problem” was to design, build and test the Planck Hydrogen When starting new projects, it is critical to involve
Sorption Cooler and the two points of view were the NASA competent personnel and build on previous missions
and ESA different perspectives. Morgante was member of the lessons learned, avoiding to re-invent the wheel.
US team at the Jet Propulsion Laboratory that developed the
To demonstate TRL To demonstrate practices are expected to be applied at all level and during all
Functionality
the phases; the team has a demonstrate experience in the field
Cold End of integration and test for spatial mission and for complex
Single Component Elegant BreadBoard
Qualification Model
Testing (EBB)
(QM)
scientific instrumentation.
2002 - 2006 Flight Model 1 Flight Model 2 The industrial point of view
Gaetano Cutrona – Andrea Margini – The key statement
Complete functional and
performance test campaign on
proposed by Dr. Ing. Cutrona Gaetano and Dr. Ing. Margini
2006 - 2008 System Tests ground
Andrea from Tetra Pak Packaging Solutions concerned the
potentialities of the Agile methodologies to facilitate the
identification and introducing new VVT methodologies from
2009 - 2013 Operation the academic to the industrial work.
The deployment of SE methodologies in the industry and in
the service market in the last decade, focused on the “Lean
2013 End of Life SE” and on the Systems Product Line Engineering. Combining
these two aspects an interesting issue for the V&V
practitioners is the evaluation of a value-related, sufficient but
Fig. 5. Planck/Sorption Cooler Life Cycle not redundant effort during the usage phase of the system life-
cycle.
In the end it is still hard to find an answer to the starting
question: “Test better to test less?”. Is there a definitive The intervention proposed during the workshop, highlighted
answer? For sure, VVT practices applied and declined case by the human factors and the varied system responsiveness. New
case are one of the possible answer. individuals and organizations, which were only typically
marginally informed and consulted during the development
phase, are involved in the usage and maintenance phase. The
Fabrizio Villa – INAF – Senior Researcher, Head of final user, the customer and the maintenance are typical but
Cryowaves Lab. Cryowaves is a made-up word from the not the only examples. The expected net present value
fusion of CRYOgenics and microWAVES with the aim to (ENPV) estimations find their confirmation or contradiction.
identify a Science and Technology branch mainly focused to Finally, the “intended environment” confirms its rapidly
develop technologies and facilities to build and operate changing behaviour.
wideband high performance radio, microwave, and mm-wave
receivers and antennas operated in cryogenic environment. The SCRUM framework [6] has been used firstly by the
Modern microwave and mm-wave astrophysics uses detectors software industry and it is now extending the service one. A
that needs to be cooled at cryogenic temperatures to minimize different attitude to developments projects and teams
the detector noise, then reaching the impressive sensitivity to organizations is required to support “sprint”, daily or weekly,
observe celestial sources as required by science needs. backlogs, design tasks and related verifications for
Examples of this kind of technology is the ESA Planck incremental system evolution.
Satellite (up to now the most advance cryogenic system fo r A liquid food packaging case study has been proposed. The
space) that measured the Big-Bang sound in intensity and V&V activity is strongly driven by effective V&V plans and
polarization at microK level and ALMA Observatory Test Reports. The sprint backlogs converge to minimization of
(www.almaobservatory.org) that, for instance, recently the requirements.
permitted to take an image of a proto-planetary disk rotating
around a distant star.
Thanks to the iALMA premiale and ESO ALMA upgrades
plan Dot. Villa has been engaged in the development of a new
prototype receiver for ALMA. The technology legacy of
Planck permitted to create a laboratory at INAF/IASF-
Bologna (Cryowaves laboratory) to exploit cryowaves at their
best. The idea was to setup a state-of-the-art laboratory to
integrate, characterize, calibrate complex scientific
instrumentation at cryogenic temperatures. In this respect the
laboratory and related activities (design and simulations) are
well focused on the VTT and more general SE disciplines. Fig. 6. Backlogs requirements convergence to zero by backlogs sprints
The final goal is to set up a laboratory and a dedicated
workgroup of people where facilities, technologies Development and verification teams integration and
knowledges and skills are in place and ready to tackle consistency among cycling artefacts are highlighted as the
challenges related to cryogenic, integration, test, verification keys to success for the application of Agile V&V
for space technology and beyond. The SE and VVT best methodologies within SCRUM framework. The basic question
arose during the workgroups discussion regards the
identification of a minimal set of VVT tools and
methodologies, part of the ones applied during the
development phases, suitable to efficiently enable the usage
and maintenance phase of the system.
The software house point of view
Carlo Poloni – ESTECO – Prof. Carlo Poloni addressed
during his intervention the following question: “What is the
VVT potential role of the virtual prototyping?”
Collaborative Multidisciplinary Optimization generates
relevant info while simulations are strictly connected with
physical experiments during verification and validation
processes. Fig. 8. Data Integration, optimization and then analysis
The general observation is that isolated simulation “pearls” are
implemented and reported by experts on sub -systems, Looking at another dimension a models’ hierarchy helps to
components and sometimes also on systems. They are maintain the coherence between the “VEE” model vertical, i.e.
although rarely integrated into a consistent V&V strategy. The system maturity, and the horizontal system evolution
models are developed, as necessary, and their results remain dimensions.
typically isolated in the overall development overview. Simulations model often anticipate physical testing when
Moreover, the required skills to design, run and document the prototypes maturity is not sufficient or not available at all.
applications are in the hands of highly skilled experts whom Their development cost is not irrelevant and the re-usability
do not relate continuously with the rest of the development must be included into the V&V strategy.
team.
The coherency among the models and the systems maturity is
Prof. Poloni highlighted the importance of a network of one of the key factors to optimize the info/effort ratio.
information which sustains the system conceptual and
architecture phase with the simulations performed within the Examples and success stories with different multi-domain
VVT strategy during the design and verification phase. integrated tools, analytics and graphics were proposed to the
attendees.
Fioravanti – Blizzard – Dr. Ing. Fioravanti, representing
Blizzard srl, proposed the “Simply … Agile” alias
methodologies and tools to implement Agile methods in SW
development. Basically the actually available Agile
development processes are still too complex in order to reach
their basic objectives. The main causes of failed SW
developed projects are proposed and analysed. Among them:
insufficient tests in the intermediate phase.
The renovated Agile mantra lies on a wise step-by-step
approach where flexibility allows managing the growing
complexity with the right VVT effort. Working in small but
complete and closed steps allows proposing to the final user,
by fast prototyping, allows testing and usage by the final users
of the implemented functions . The discussion with the
workshop participants regarded the reasons of un-successful
Fig. 7. Simulations integrated by a network of information. and successful projects.
Raw Data, fundamental to the simulation process require a V. W ORKSHOP OUT COMES
structured, coherent and visible effort of integration in order to
feed the Optimization and analysis process. The resumed outputs of the groups discussion, proposed to the
attention of the overall attendees can be shown in the updated
conceptual map Fig. 9 and resumed as in the following
paragraphs.
feedbacks
Test Better to Test Less
Gr5 Also a failed test brings useful info Know-how
(G. Morgante) Funding opportunities
Gr7 Group 5 “Registro”
VVT and SE not teached at the university
Citation index è
Group 7 “Pragmatico”
Needs Lack of systematic oriented recognized figures in INAF
Not applied in the research
Approach properly the issues
````
Specific fields knowledge Lean approach in the great projects Different mindset
Academia Focus on the training aspects Bureaucracy “quicksand” in the projects Gr4 Research
PhD Thesis and in the organizations
(Andrea Margini) Resistance to change and diffuse
Negative events which can facilitate the alignment Group 4 “Valore”
Job rotations among the two worlds Institutional reflection pause
Protect the personal know-how Being informed about WIP
(G. Cutrona, minutes to be recovered) Reciprocal info exchange
Come out of your’s shell
SW development integrated
Engineering Projects confoundings Different objectives
Pay attention to value return
simply not managed Easy simulations Rationalize results
(C. Poloni)
Rework due to Reqs. Change in the loop
Engineers overwhelmed Trade-off between standardization and tailoring Human expertise
Use methodologies! Mech-HW-SW integration and plan
Gr6 Test Imediately and always.
Demonstrative solutions confoundings Document, document, document!
Spin offs Group 6 “Obiettivo”
Engineers not trained in SE
Competences/Skills Gr8 Industry and research: exchange of skills and competences
The research produces unique prototypes
Avoid scope creeping
Opportunities
Group 8 “Provare”
Opportunities Facilities re-use among research and industry Competences/Skills
The industry does not know the frontier applications Entering in the SE can be long an exhausting process
Research must acquire systematic and process thinking Competences/Skills (Cremonini)
Share resources: information laboratories, skills
More a conveyor than a catapult
Needs Bureaucracy
Specific fields knowledge Gr2
feedbacks
Group 2 “Efficace”
(Mancin translates)
Gr3
Gr1
Group 1 “Vero” Group 1 “Lode”
Great project veiculated Different national and international differences
The SE practice (Leardi Carlo)
Research Spin-offs
Analogies and anbiguities in
Requirements management
Documentation is not time-loosing
(Malavasi translates)
COTS/tailored
Solutions
Industry SW Vendors
Intellectual Business
Property Specific fields knowledge Integrate simulations Know-how
Normative In SE processes Specialized Networking
“Lean” as the key to success?
Symply agile!
SE for complexity management
Agile methodology applied to
Mith or real opportunity?
SW development framework
Fig. 9. Conclusive conceptual map
More similarities than differences among the represented growth can bypass this attitude. E.g. the VVT expert is not
viewpoints. Many of the practitioners belong, or belonged, to any more an isolated, although fundamental passage but he is
more than one environment at the same time for limited or one of the main wheels that convey, together with the other
longer periods. This aspect is remarked as one of the more persons and specialists, to the success. Prejudices do no t h elp
powerful catapults to speed-up the innovation deployment. to bypass the resistance to learn and change. Reciprocal
Spin-offs, PhD theses, temporary assignments in the knowledge and real opportunities to cooperate are needed.
industries, European research funded occasions can furnish the
fertile ground for growing-up commonalities. Furthermore, a The fundamentals and the practice of the systems engineering
SW tool developer and vendor, a research centre, an are not diffused enough in all involved environments.
university, a PMI or a big industry, despite their different Common understanding and faster deployment of research
foundations and organizational characteristics, need organized deliverables passes through training and connecting the
development processes following the concepts of the systems researcher and the industrial systems engineers on the SE
engineering. fundamentals, e.g. the TRL concept, and on the interactions
Tailoring is the link among the different environments around among the Integration, Verification, Validation processes with
the same concepts. The exposed failures in VVT experiences the other ones. Monitoring the VVT SE applications in order
and the recovering strategies proposed from different to furnish a methodological reference is highlighted as one of
panellists during the discussion reinforced similarities more the factors to address the state of the art. E.g. Agile processes
than differences. Starting from the requirements interpretation are not yet enough known and applied Even if modern
ambiguities, through the “verification war”, and finally to the organization in the Astrophysical business are starting to
validation challenges, similar issues were presented and appreciate these methods, there is a big margin for
discussed. Tailoring for the VVT processes is one of the key. improvement for a fruitful sharing of knowledge. The PMI
Great telescopes, one of a type prototype, mass productions o r must be supported by the research centres and big industries to
production lines could require VVT processes tailoring. tailor their own way to Systems Engineering.
Bureaucracy and resistance to change characterize all the Focus on the overall V&V strategies. In order to avoid the
environments. Only a real understanding of how the SE “margin on margin” impact and to apply the “test better to test
processes can leverage the system value of the personal less” approach. Single examples of success cannot enough to
sustain the development value. A systematic approach in the
V&V strategy is required. Clearness in the application of the Next step of the AISE VVT working group is to diffuse and
SE methodologies, tools, and a critical attitude vs. what done leverage on the outcomes and deliverables of the workshop
helps to avoid replicating the common mistakes. within the larger Systems Engineers and VVT practitioners
Integrate by small steps virtual simulations with traditional community. The formalization of this paper goes in this
physical testing can be the way to success. An easy, kiss, direction.
start-up which doesn’t require a too big once in a time effort. The second step of this activity is planned for spring 2017.
It can be continuously extended in scope and deepened by Enlarging the involved community, prioritizing and identify
incremental “natural” growths to tend towards maximum real common opportunities on the themes highlighted during
value achievement. the workshop are the ambitions of this second sharing
occasion. Contacts are also on-going to diffuse the informatio n
Documenting, despite the selected implementation (e.g. with the parallel initiatives in the INCOSE EMEA sector.
classical or model based) documentation control and
configuration is essential. It can require a quantity of effort, Acknowledgment
but it is fundamental not to disregard it due lack of time/funds The authors of this work would like to acknowledge the
or perform it approximatively. The addressed direction is the invited speakers at the workshop. They would also
one of value-related documentation deliverables that goes acknowledge Tetra Pak Packaging Solutions and Cryowaves
from: traditional word check-lists, laboratory logs to virtual laboratory for make the workshop possible. This workshop
models deeply integrated in the development cycle (a was partly supported by the Italian Ministero dell'Istruzione,
documented model itself). Universita' e Ricerca through the grant Progetti Premiali 2012
Re-use of the experiences developed in the university or by -- iALMA (CUP C52I13000140001).
the research, after their main objective, for didactic,
exemplifying or for continuous testing in the environments
different from the starting ones. References
[1] G. Labovitz, et al. “Making Quality Work: A Leadership Guide f o r t h e
Continue to meet is felt as one of the main needs , talk and Results-Driven Manager” Harpercollins, 1992
create a “common place” where exchanging past, on -going [2] S. Nagano, “Space Systems Verification Program and Management
Process”, System Engineering, Vol 11, Issue 1,2008
and future experiences. This is the key to overcome, by
[3] B. Hailpern, P. Sant hanam, “Software debugging, testing, and
intellectual curiosity, the personal or organizational verification”, IBM, Systems Journal, Vol. 40, No1. 2002,.
restrictions. Furthermore, to create reciprocal knowledge, [4] Rescuing Prometheus: Four Monumental Projects T hat Ch a nge d t h e
trust and common opportunities. Modern World by T homas P. Hughes, 2000.
[5] “First Images from the VLT Survey T elescope” ESO 1119 –
VI. NEXT ACT IVITIES Organisation Release, 8 June 2011
[6] T he SCRUM Guide , 2013 K. Schwaber, J. Sutherland