<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Elsa Marcelino-Jesus</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Artem A. Nazarenko</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>João Sarraipa</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>João Martins</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ricardo</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jardim-Goncalves</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Caparica</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Portugal</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>CTS, UNINOVA, DEE, Faculty of Sciences and Technology, Nova University of Lisbon</institution>
          ,
          <addr-line>2829-516, Monte</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>Technological development in the recent years has significantly affected the business environment. Nowadays enterprises, to remain competitive and to discover the new business opportunities are forced to find the new ways and forms of collaboration. One possible solution is establishment of business clusters that include not only enterprises, but also the academic units introducing new technologies and business models. However, one of the main concerns related to the establishment of manufacturing clusters targets aspects, such as partners selection, business opportunities identification, etc. In this paper we are aiming at representing a framework intended to provide the necessary basis for establishment of successful clusters with the support of an evaluation initiative. Business model, digital manufacturing cluster evaluation, business ecosystem</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        In the recent years the new forms of clusters have emerged. Clusters are geographic concentrations
of interconnected companies and institutions in a particular field [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Beside “classical” clusters that are
limited to the geographical location and common industrial domain there are the “meta clusters”, which
can be regional, unifying companies from different sectors, but from the same regions or cross-cluster
partnerships [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Thus, the notion of clusters or business clusters comes closer to the notion of business
ecosystem, which is according to [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]: “an economic community supported by a foundation of interacting
organizations and individuals - the organisms of the business world. This economic community
produces goods and services of value to customers, who themselves are members of the ecosystem”.
Such ideas may come from the term business cluster, also known as an industry cluster, competitive
cluster, or Porterian cluster that was introduced in [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. In [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] authors discuss the key differences of
industrial clusters and business ecosystem, identifying main characteristics of business ecosystem: (i)
symbiosis, (ii) platform and (iii) co-evolution. First characteristic identifies that every member of the
cluster is affected by the processes in other member organisations. Second characteristic stands for
platforms and tools which are delivered by some members of the cluster to enhance the productivity of
the whole cluster. And the third one identifies how the members of the cluster can benefit from
collaboration allowing them to be
      </p>
      <p>more competitive through variety of competences. These
characteristics can be also applied for cross-cluster partnerships, as well as for meta-clusters. Thereby,
the main aim of this paper is to propose an approach for assessment of the industrial cluster being
developed within the framework of ongoing European digital manufacturing projects ZDMP, EFPF,
Qu4Lity.</p>
      <p>2020 Copyright for this paper by its authors.</p>
    </sec>
    <sec id="sec-2">
      <title>2. GQM (Goal Question Metrics) approach</title>
      <p>
        The lifecycle of the cluster can be separated on three main stages [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]: (i) establishing (ii) operation
and (iii) dissolution. The key factors which can potentially affect the cluster establishment and criteria
to assess potential benefits of collaboration should be defined in the early stages [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. In line to this,
performance indicators may be used for assessing the functioning of a company according to their
strategic and operational goals [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>This paper mainly covers the establishing phase of a cluster supported by an evaluation method to
work as catalyst to reach the operation mode, where real products may be produced or exploited. Such
evaluation method analyses if the goals stated are achieved.</p>
      <p>
        The evaluation process is a key phase of every project, as well as clustering initiative. As a basis for
evaluation the Goal, Question, Metric approach has been chosen. It is based on three levels [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]: (i)
Conceptual, on which the goals are identified, (ii) Operational to formulate the questions characterising
the measured object and (iii) Quantitative, where the metrics are introduced intended to answer the
questions from the previous phase. Moreover, the proposed approach is extended with the importance
or weight metric, which goal is to reflect the importance of each topic to each of the collaborating
partner. Let’s assume, if the goal is completely achieved, the assessment is equal to 1 and if completely
failed, the assessment is equal to 0. In the same time, the final result is directly affected by the
importance coefficient, which is identified by each partner for every goal also differentiating from 0 to
1. Thus, the result of the evaluation can be calculated as following:
 = ∑&amp;   ×   ,
(1)
where r is the result, n is the number of goals/topics identified, an is the assessment of particular goal
and wn is the perception of importance by the collaborative partner.
      </p>
      <p>Thus, the best evaluation result, which can be achieved for the current assessment is equal to eight,
i.e., to the sum of all evaluated characteristics, if all the assessments and importance coefficients are
equal to 1. And the lowest result, respectively, is equal to 0, if the evaluating partner either considers
the goal as completely failed. To every importance coefficient a textual description is assigned, i.e. 1
stands for “Critical” importance and “Insignificant” stands for 0. Thus, utilization of importance
coefficients serves the goal to indicate the focus points and assess if the set goals are achieved.</p>
      <p>
        All of these considerations and guidance represent what the paper authors define as the Cluster
Establishment Framework (CEF). This framework represents a set of guidelines, considerations and
approaches that focus all the work around evaluation and considers it as a catalyst to facilitate an
effective cluster creation. This was inspired in the framework for technological research results
assessment presented in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. The following case study has the objective to show an example of a cluster
creation where evaluation works to push the success of it.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. ZDMP Case Study</title>
      <p>Identification of funding sources and other support, and thanks to individuals and groups that
assisted in the research and the preparation of the work should be included in an acknowledgment
section, which is placed just before the reference section in your document.</p>
      <p>A case study to exemplify the CEF application consists of a set of European projects which are
combined under umbrella of the Zero Defects Manufacturing Platform (ZDMP)2 initiative. The cluster
member projects within the joint imitative are: European Connected Factory Platform for Agile
Manufacturing (EFPF)3, Digital Reality in Zero Defect Manufacturing and Aligning Reference
Architectures (QU4LITY)4 and Open Platforms and Large-Scale Pilots in Digitising European Industry
(CSA OPEN DEI). All of the projects are focused on designing, development and implementation of
2https://www.zdmp.eu/
3https://www.efpf.org/home
4https://qu4lity-project.eu/
Industry 4.0 solutions. The main aim of the ZDMP initiative is to create an ecosystem enabling the core
services used to build the application that are pushed to the marketplace.</p>
      <p>
        Important part of the presented framework is focused on assistance provision in evaluating the
outcome, but also on identification of promising activities direction, whereas the objective is to turn the
results of clustering initiative into the new business. On the current stage the framework is still in its
infancy, which might require more improvement iterations and permanent feedback after being applied
to the cluster. Another difficulty is that the framework is applied to a real cluster, thus in the course new
aspects might appear that need to be properly addressed by the framework. However, as a part of initial
stage of GQM approach, following topics or goals according to GQM approach were identified [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]:
 T.1 – Standardization stands for clustering activities facilitating the compliance of the cluster
results with existing standards and contribution to new standardization activities where possible.
 T.2 – Dissemination on events stands for joint dissemination actions to communicate and
promote the cluster results to technology and service providers as well as other business users and/or
stakeholders.
 T.3 – Joint research activity stands for publishing activities of innovative results in leading
journals and conferences.
 T.4 – Performance management and KPIs stands for an assessment framework containing core
indicators to assess overall performance, including circular economy aspects, will be addressed in a
common way.
 T.5 – Market Analysis and Business Models stands for a set of actions on turning the outcomes
and innovation/technological developments of the cluster projects into value-creating products and
services.
 T.6 – Open Calls stands for joint work to make external stakeholders aware of the available
resources, to potentiate the open calls participation and to combine efforts on their evaluation.
 T.7 – Platforms stands for joint activities to exploit synergies between technology-based
platforms addressing issues such as architecture, interoperability and standards approaches.
 T.8 – Pilots stands for pilot activities addressed by the three projects to increase knowledge on
pilots’ description, conducting, and assessment.
      </p>
      <p>On the next stage, after the goals are set, questions, which help in reaching the goal and a set of
metrics to answer the questions are identified. Below questions and metrics identified are presented in
relation to Market Analysis and Business Models.</p>
    </sec>
    <sec id="sec-4">
      <title>4. ZDMP Case Study</title>
      <p>
        For the purpose of creating the new product or service which can be pushed into the real business,
market analysis and the proper business model are of significant importance. The market analysis allows
companies identifying and assessing the opportunities and risks of a market in order to make informed
decisions regarding manufacturing investments and defining concrete marketing strategies to
implement business ideas [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
      </p>
      <p>
        A business model describes how an organization creates, delivers and control value and how money
is earned in a company [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. It may identify a useful framework to link ideas and technologies to
economic outcomes [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. It also reflects the market research, “the chosen system of inputs, business
activities, outputs and outcomes that aims to create value over the short, medium and long term [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]”.
      </p>
      <p>The cluster vision is to bring the project’s outcomes to market, thus creating jobs and enhancing
Europe’s economic development. Cluster activities will place strong emphasis on turning the outcomes
and innovation/technological developments of the cluster projects into value-creating products and
services. The main objective is to establish a joint 4DMP Market Analysis and portfolio of Business
Models for DMP.</p>
      <p>Below is an example for the topic on market analysis and business models (see Table 1):</p>
      <sec id="sec-4-1">
        <title>Market Analysis and Business Models</title>
        <p>
          How the Market analysis helps to identify the business
opportunity?
Market density represents an estimated number of potential
customers in particular area/region. This metric can be very
important for small and medium enterprises regionally oriented.
Competitive density [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] represents the number of competitors in
particular domain offering similar products/services.
        </p>
        <p>Potential market volume represents the size of the market at
specific time stamp. It can be represented through the
timedependent function.</p>
        <p>Time-to-market represents the amount of time needed for the
product to reach the market. The time includes the whole period
from the idea to the ready-to-sale product/service (all phases from
idea, design and development, entering the market).</p>
        <p>Required resources to reach the market represents estimated
amount of human and financial resources needed to reach the
market with product/service.</p>
      </sec>
      <sec id="sec-4-2">
        <title>What is the impact of developed business models?</title>
        <p>Innovation to product/service conversion rate represents the
number of products/services developed, divided through number of
innovative ideas formulated.</p>
        <p>Applicability of business models developed.</p>
        <p>General Acceptability – subjective assessment of the business
model implementation by the partners involved in the pilots.</p>
        <p>Success rate – e.g. numbers of jobs created, added value (product
price - cost of producing), reduction of the ecological pressure, etc.</p>
        <p>The metrics identified in this paper are aimed at improving the common vision and understanding
of the potential of the idea that need to be turned into the real product or service. Thus, the first stage is
to collaboratively generate the idea of a product or service and afterwards assess the potential market
and corresponding business model for its implementation. The next phase is to answer the following
questions and establish the importance weights for the metrics.</p>
        <p>Moreover, a foreseen joint workshop meeting between the cluster projects (e.g. ZDMP; EFPF) will
provide important information sharing in order to mutually help the projects, mapping of business and
technological services. Next actions should be agreed after the permit to share non confidential
information about business models approaches for the three cluster projects as already individual
projects work has progressed. Similar approaches are expected to address the marketplace issue, based
on previous research initiatives. For instance, it can make sense to have a single marketplace, or to share
some of the apps. In conjunction with the importance weight for each considered topic, as well as for
each single metric this should allow for the cluster projects to identify the most promising ideas and
products/services for joint implementation. Thus, each idea is checked against the mentioned metrics
in the previous table. As an example, if the market density is low for a particular concept, e.g. there are
not many potential clients for the product/service, there is no reason to choose this concept for further
development. And if several concepts have comparable score for the market density metric, further
metrics can support the decision making. The decision making is a complex procedure taking into
consideration the importance weight, so the concept with worst time-to-market metric can have an
advantage over another concept with better time-to-market metric but with better market volume metric
score compensating the longer R&amp;D process. The importance weight of all the metrics is supposed to
be discussed and identified with all cluster partners on the next stages of the work.</p>
        <p>Novel technological developments and fast changing business environments force enterprises to find
suitable forms for future collaboration. In this paper, we introduce the framework for the assessment
and evaluation of the results of industrial ecosystems to be developed through ongoing European digital
manufacturing projects, e.g. ZDMP, EFPF, Qu4Lity. The GQM approach is put in the center of the
presented framework allowing for both qualitative and quantitative aspects of the evaluation process to
be efficiently addressed. Our future work on the framework is to practically evaluate the projects’ results
throughout their lifecycle in order to guarantee an effective establishment of manufacturing ecosystems
and clusters.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>6. Acknowledgements</title>
      <p>The research leading to these results received funding from the European Union H2020 Program
Under grant agreement No. 825631 “Zero Defect Manufacturing Platform (ZDMP)”.</p>
    </sec>
    <sec id="sec-6">
      <title>7. References</title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>M.E.</given-names>
            <surname>Porter</surname>
          </string-name>
          ,
          <source>Clusters and the New Economics of Competition. Harvard Business Review</source>
          ,
          <year>1998</year>
          . URL: https://hbr.org/
          <year>1998</year>
          /11/clusters-and
          <article-title>-the-new-economics-of-competition</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>T.</given-names>
            <surname>Lämmer-Gamp</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>M. zu</article-title>
          <string-name>
            <surname>Köcker</surname>
          </string-name>
          , M. Nerger, Cluster Collaboration and Business Support Tools to Facilitate Entrepreneurship,
          <source>Crosssectoral Collaboration and Growth</source>
          ,
          <year>2014</year>
          . URL: https://ec.europa.eu/docsroom/documents/9972/attachments/1/translations/en/renditions/pdf
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.F.</given-names>
            <surname>Moore</surname>
          </string-name>
          ,
          <article-title>The death of competition: leadership and strategy in the age of business ecosystems</article-title>
          . Harper Collins Publishers,
          <year>1996</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M. E.</given-names>
            <surname>Porter</surname>
          </string-name>
          , The Competitive Advantage of Nations. New York: Free Press,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Y. R.</given-names>
            <surname>Li</surname>
          </string-name>
          and
          <string-name>
            <given-names>W. Z.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <article-title>"Industry clusters and business ecosystems - The smart mobile industry in Taiwan,"</article-title>
          2013 IEEE International Conference on Industrial Engineering and Engineering Management, Bangkok, pp.
          <fpage>537</fpage>
          -
          <lpage>540</lpage>
          ,
          <year>2013</year>
          , doi: 10.1109/IEEM.
          <year>2013</year>
          .
          <volume>6962469</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Camarinha-Matos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Afsarmanesh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Galeano</surname>
          </string-name>
          &amp;
          <string-name>
            <surname>A. Molina</surname>
          </string-name>
          ,
          <article-title>Collaborative networked organizations - Concepts and practice in manufacturing enterprises</article-title>
          .
          <source>Computers &amp; Industrial Engineering</source>
          ,
          <volume>57</volume>
          (
          <issue>1</issue>
          ), pp.
          <fpage>46</fpage>
          -
          <lpage>60</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>E.</given-names>
            <surname>Marcelino-Jesus</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Sarraipa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Beça</surname>
          </string-name>
          , &amp; R.
          <string-name>
            <surname>Jardim-Goncalves</surname>
          </string-name>
          ,
          <article-title>A framework for technological research results assessment</article-title>
          .
          <source>International Journal of Computer Integrated Manufacturing</source>
          , pp.
          <fpage>44</fpage>
          -
          <lpage>62</lpage>
          ,
          <year>2016</year>
          . https://doi.org/10.1080/0951192X.
          <year>2016</year>
          .1145806
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>P.</given-names>
            <surname>Graça &amp; L. M. Camarinha-Matos</surname>
          </string-name>
          ,
          <article-title>Performance indicators for collaborative business ecosystems - Literature review and trends</article-title>
          .
          <source>Technological Forecasting and Social Change</source>
          ,
          <volume>116</volume>
          ,
          <fpage>237</fpage>
          -
          <lpage>255</lpage>
          ,
          <year>2017</year>
          . https://doi.org/10.1016/j.techfore.
          <year>2016</year>
          .
          <volume>10</volume>
          .012
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>V.</given-names>
            <surname>Basili</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Caldeira and H.D. Rombach</surname>
          </string-name>
          ,
          <article-title>The Experience Factory</article-title>
          . In J. Marciniak (ed.),
          <source>Encyclopedia of Software Engineering</source>
          , Wiley,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J.</given-names>
            <surname>Martins</surname>
          </string-name>
          ,
          <source>ZDMP: Zero Defects Manufacturing Platform Clustering Actions. DX2: 4DMP Clustering Plan and Actions - Vs: 0.6</source>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>IONOS</given-names>
            <surname>Inc</surname>
          </string-name>
          .,
          <source>Market analysis - the definition</source>
          ,
          <year>2018</year>
          . URL: https://www.ionos.com/startupguide/grow-your
          <article-title>-business/market-analysis-definition/</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Osterwalder</surname>
          </string-name>
          &amp; Y. Pigneur, Business Model Generation.
          <source>Self Published</source>
          ,
          <year>2009</year>
          . URL: https://www.academia.edu/23846689/BUSINESS_MODEL_GENERATION_OSTERWALDER
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>H.</given-names>
            <surname>Chesbrough</surname>
          </string-name>
          , Open Business Models:
          <article-title>How to Thrive in the New Innovation Landscape</article-title>
          . Boston: Harvard Business School Press,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>International</given-names>
            <surname>Integrated Reporting Council</surname>
          </string-name>
          ,
          <article-title>Business Model Background Paper For &lt;IR&gt;</article-title>
          . ISSN:
          <fpage>2052</fpage>
          -
          <lpage>1723</lpage>
          ,
          <year>2013</year>
          . URL: https://integratedreporting.org/wpcontent/uploads/2013/03/Business_Model.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>G.B.</given-names>
            <surname>Voss</surname>
          </string-name>
          and
          <string-name>
            <given-names>Z.G.</given-names>
            <surname>Voss</surname>
          </string-name>
          ,
          <article-title>Competitive Density and the Customer Acquisition - Retention TradeOff</article-title>
          .
          <source>Journal of Marketing</source>
          , vol.
          <volume>72</volume>
          , no.
          <issue>6</issue>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>18</lpage>
          . JSTOR,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>