<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>EPICS: Pursuing the Quest for Smart Procurement with Artificial Intelligence</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Angelo Impedovo⋆</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Emanuele Pio Barracchia⋆</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Giuseppe Rizzo⋆</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andrea Caprera</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Elena Landor´</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Niuma s.r.l.</institution>
          ,
          <addr-line>Via Giacomo Peroni, 400, 00131 Rome</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2014</year>
      </pub-date>
      <abstract>
        <p>E-procurement involves the set of activities, performed in the manufacturing business, to requisite, order, and purchase goods and services online, usually on a periodic and large-scale basis from a supply chain. Every activity is a decision-making process that rapidly becomes infeasible by human procurement experts, often due to a large amount of information to be taken into account, coming from diferent sources. To overcome this limitation, smart procurement based on artificial intelligence has been proposed. However, the majority of the literature only focuses on specific procurement-related problems, only from a supply chain perspective, without providing a holistic and encompassing solution. In this paper, we identify the 4 pillars of smart procurement based on artificial intelligence and discuss how to accommodate implementations into the preliminary architecture of the EPICS platform.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>E-procurement is the digitization process of the procurement of goods and
services undertaken by a company, from the call for tenders publication to the
payment, through the use of information technology facilities like the Internet,
online bidding, and auctions, and so on. The benefits of e-procurement are
manifold, including i) improved interaction eficiency between procurement actors, ii)
enhanced productivity, by spending the time saved, thanks to process
automation, on strategically meaningful functions and tasks, and iii) increased awareness
of spending resulting in the reduction of costs.</p>
      <p>In parallel with the rising importance and adoption of e-procurement, the
set of data collected by companies that use it has grown increasingly large and
very complex, making it dificult and time-consuming to analyze and extract
useful information from it. To deal with this problem, recently much focus has
been placed on smart procurement, that is the set of technologies used to
support human experts in analyzing the huge amount of available data and taking
strategic decisions related to, for example, the management of supplier
relationships, or the denfiition of product specifications and quantities needed. Due to
its characteristics, a good application of smart procurement would lead to the
achievement of competitive advantage for the company.</p>
      <p>Artificial intelligence has been proposed as a key technology behind smart
procurement since it can be used to formulate hypotheses on data, generate
suggestions on a particular decision, and identify patterns in a large amount of data.
However, the vast majority of the literature only discusses artificial intelligence
applications to very specific procurement problems that, in their turn, remain
somewhat disconnected from each other, thus failing in delivering a complete
and sound view on the procurement to the users. In this paper, we identify
the 4 pillars of smart procurement and discuss how to accommodate intertwined
implementations into the preliminary architecture of the EPICS platform,
developed in the scope of the EPICS research project conducted with both industrial
and academic partners.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>
        Many works have been proposed to analyze the problems and challenges
characterizing smart procurement, proposing new approaches and techniques to face
them. In particular, a considerable body of research is concerned about two
perspectives: i) monitoring the performances of, potentially unknown, suppliers and
selecting the optimal ones, and ii) monitoring and forecasting adverse
procurement events involving materials (goods or services) and suppliers.
Methodologically, the proposed solutions exploit either operational research – such as
multicriteria and multi-attribute decision making methods (MCDM and MADM),
multi-objective programming (MOP), mixed integer programming (MIP) and
dynamic programming [
        <xref ref-type="bibr" rid="ref10 ref12 ref5">15,18,12,17,5,10</xref>
        ]– or machine learning methods.
      </p>
      <p>
        The tasks of supplier selection and performance evaluation have been tackled
with both supervised and unsupervised methods. Among the supervised ones,
neural networks [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], support vector regression (SVR), support vector machines
(SVM) [
        <xref ref-type="bibr" rid="ref13">13,19</xref>
        ], logistic regression [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], rule learning, k-nearest neighbors, linear
regression, nıav¨ e bayes have been used [
        <xref ref-type="bibr" rid="ref1 ref3 ref4">3,1,16,4</xref>
        ]. However, in the context of
smart procurement, they seem to be at an early stage, due to the lack of publicly
available datasets of labeled suppliers, a requirement not always easy to satisfy
due to the subjective nature of the evaluation. As concerns unsupervised model,
process mining, fuzzy association rule mining, and Markov decision process have
been used to solve the aforementioned problems [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Clustering methods and
neural networks have been also proposed for discovering homogeneous groups of
suppliers (segmentation) [
        <xref ref-type="bibr" rid="ref1 ref2">1,2</xref>
        ]. Furthermore, unsupervised approaches have been
also used for supplier classification [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>The tasks of adverse procurement event detection (e.g. fraud or failure
detection) cast anomaly detection problems, whereas the tasks of adverse procurement
Procurement</p>
      <p>Data</p>
      <p>Strategic</p>
      <p>Sourcing
Risk
Management</p>
      <p>
        Spend Analysis
event forecasting (e.g. failure or demand forecasting) cast forecasting problems.
Specifically, in the case of event detection, process mining techniques have been
exploited to detect anomalous events occurring during procurement process
executions [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. Alternatively, some works tried to solve the problem using a
supervised setting through bayesian networks, nıav¨ e bayes and decision tree learning
algorithms [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. In the case of event forecasting, instead, unsupervised learning
methods, e.g. association rule mining and clustering, have been used for solving
failure and demand forecasting problems [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Case study: the EPICS platform</title>
      <p>In this section, we present the EPICS (E-Procurement Innovation For
Challenging Scenarios ) platform: a proprietary e-procurement solution, designed and
developed by NIUMA as a research collaboration with public Italian
universities, that aims at empowering the performances of procurement ofices in buyer
companies with the help of articfiial intelligence.</p>
      <p>The EPICS platform is a multi-tenant, multi-cloud, and microservices
oriented architecture implementing a full-fledged solution guiding the procurement
ofices in initiating, managing, and monitoring the execution of
procurementrelated processes, such as requesting ofers and quotations, inviting suppliers to
auctions, managing their bids, thus ensuring the visibility and the governance of
the whole procurement.</p>
      <p>To accommodate smart procurement capabilities into the EPICS platform,
ifrstly, we have extensively studied the relevant literature on the four pillars of
smart procurement (namely, procurement data, risk management, spend analysis
and strategic sourcing - as depicted in Figure 1) and spotted interesting
problems. Then, we have deployed solutions to these problems into microservices,
mostly serving machine learning models to the production environment.
3.1</p>
      <sec id="sec-3-1">
        <title>Procurement Data</title>
        <p>Procurement data, lying at the EPICS platform core, consists of quantitative
and qualitative attributes of entities, mostly materials (goods or services) and
suppliers, as perceived from the buyer company and involved in the
procurement processes. Although seemingly similar, procurement data may profoundly
difer from supply chain data. The diference lies in their scope of observation:
while supply chain data depicts an often incomplete view of how companies are
entangled around the globe in buyer-supplier relationships, procurement data
depicts the complete scenario of the supplying relationships engaged by the buyer
company concerning the immediate, well known, qualified suppliers that are
continuously monitored over time.</p>
        <p>It is evident that procurement and supply chain data only partially overlaps:
procurement data neglects information related to suppliers of the immediate
suppliers, while supply chain data does not account for details of transactions
between a buyer company and his suppliers, which are often secretly kept. As
a consequence, for smart procurement to be efective, there is a need to collect
and store procurement data coming from diferent, both internal/external and
structured/unstructured, data sources such as the procurement platform itself,
the ERP systems, and the information providers available online.</p>
        <p>Available data necessitates continuous aggregation and cleaning to stay
upto-date. This is particularly relevant, for instance, when qualifying both materials
and suppliers by matching their publicly known characteristics against what is
self-declared on written documents and certifications that have been requested
by the buyer during the qualification process. Such processing activity is usually
human-made and, therefore, time-consuming and error-prone. To mitigate this
problem, artificial intelligence and machine learning techniques may represent an
added value for EPICS because they should help, in principle, to i) automatize
the tasks of data collection, integration, and evaluation (with process mining
and robotic process automation techniques) and ii) increase the overall accuracy
of the results (with machine learning techniques).</p>
        <p>Currently, EPICS integrate diferent machine learning models and techniques
as follows: given a set of documents envelopes – each of them containing
documents such as DURC, ISO, etc. – they are firstly acquired by using OCR systems
(intelligent document recognition) and converted into a set of electronic
documents that are firstly classified into a taxonomy of known document types, and
pruned away in case of anomalous ones; and, secondly, they are further processed
via natural language process techniques. In particular, named entity recognition
methods are employed to recognize the diferent parts of an envelope to
convert it into a structured record containing the relevant fields extracted from the
original document.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Risk Management</title>
        <p>Managing risks in the e-procurement means solving three problems: i)
identifying the exposure of suppliers and materials to one or more risk categories,
ii) analyzing the association between risky suppliers/materials and the negative
events recorded while executing business processes within the buyer
organization, and iii) evaluating the likelihood and the impact of risky elements on the
overall business.</p>
        <p>Risk-related data is often implicitly contained in procurement data and needs
to be exposed in a risk warehouse. Here, risky elements (materials and suppliers)
are qualified by assigning them one or more risk categories to which they are
exposed. Such an activity, referred to as risk identification , qualifies materials and
suppliers based on their quantitative features, eventually computing indirect risk
KPIs. However, identifying risky elements is not always straightforwardly done
because i) KPIs interpretation is often subjective, and ii) the risk predicate is not
known in advance or it is hard to be made explicit. Therefore, in EPICS, machine
learning models significantly contribute to inferring the risk predicates from
data in the risk warehouse. Then, risky elements are associated with event logs
generated during business processes executions seeking correlations or frequent
patterns denoting the root causes of the negative events.
3.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>Spend Analysis</title>
        <p>Managing the spend means keeping track overtime of how the budget is deployed
to the purchase of diferent materials, and, therefore, to the diferent suppliers in
charge of supplying them. Controlling the spend over time allows to the buyer to
spot saving opportunities and eficiency pitfalls. For an accurate spend analysis,
three conditions should be met: i) spend data needs to be continuously cleaned
to increase the spend visibility, ii) cleaned data should be stored in one or more
data warehouses and referred to as spend cubes, and iii) stored data should be
processed with advanced techniques in order to find relevant trends and patterns.</p>
        <p>In particular, the spend cleaning aims at standardizing and aggregating
every relevant attribute, concerning materials and suppliers involved in the spend
records, that could potentially fragment the observed spend if left untreated.
For example, the cleaning could collapse two product codes denoting the same
material into a single one, thus letting the user noticing that it is currently
purchasing the same good (or service) with diferent prices by requesting two
distinct product codes to diferent suppliers. Such an activity of material/supplier
segmentation can be immediately solved by clustering algorithms or by entity
linking (also known as deduplication) methods aiming at discovering the names
referring to the same entities (materials or suppliers). Furthermore, machine
learning models are potentially useful also during the spend analysis step, where
relational learning algorithms designed for multidimensional data cubes manage
to discover more complex and expressive spend patterns and trends.
3.4</p>
      </sec>
      <sec id="sec-3-4">
        <title>Strategic Sourcing</title>
        <p>Strategic sourcing refers to the process of developing an optimal procurement
plan by establishing durable supply relationships with reliable suppliers. Such a
plan is built by i) selecting the right materials according to the buyer’s needs,
ii) continuously evaluating the, potentially unknown, suppliers for the selected
materials, and iii) selecting the suppliers by maximizing diferent, often
contrasting, criteria considering all the possible alternatives and outcomes. Due to the
nature of this task, the strategic sourcing cannot be performed independently
of the other pillars of smart procurement: it needs to elaborate and gain
insights from procurement data, consider the risks highlighted by the risk analysis
(e.g., the sole-sourcing risk, whereby a material is supplied by only one supplier),
and, lastly, it should be guided by the valuable information extracted during the
spend analysis.</p>
        <p>Strategic sourcing is one of the most important components of the supply
chain because, thanks to it, companies can establish and maintain long-term
relationships with the suppliers, resulting in timely deliveries with consistent
quality, lower inventory costs, and the possibility of product improvement. To
achieve this goal, the tasks of supplier recommendation and selection acquire
remarkable value. Both tasks can be carried out taking into account diferent
criteria, either subjective or objective, elaborated on the basis of the available
procurement data or, more generally, supply chain management data. Often, the
analyzed criteria used to evaluate suppliers include, but are not limited to, their
economic power, financial data, or the quality of the services or the products
supplied. Machine learning could play a key role in the recommendation of new
suppliers through the use of recommender systems, software able to provide
suggestions on the basis of the user preferences. At the time of writing, the
EPICS platform leverages a recommender system that provides suggestions on
the basis of the interactions between buyers and suppliers. Such suggestions are
shown to the users as a ranking sorted by a score of relevance, that is, the
recommendation score.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Conclusions and Future Work</title>
      <p>In this work we have discussed how artificial intelligence supports the four main
pillars in the smart procurement process, namely: procurement data, risk
management, spend analysis, and strategic sourcing. In particular, based on evidence
from the scientific literature, we discussed a framework, drawn from the blueprint
of the EPICS platform, in which to accommodate actual machine learning
solutions to the aforementioned procurement problems. Diferent from the existing
approaches, in which procurement problems are separately solved, our attempt
paves the way towards the actual adoption of intelligent solutions encompassing
the whole spectrum of the procurement processes.</p>
      <p>Since every problem can be solved by computational approaches drawn from
a wide range of algorithms, there is room for a large number of empirical
validations. For this reason, as future research directions, we aim at independently
gaining insight into the performances of every proposed solution, perhaps by
quantitatively evaluating the performances (e.g.: accuracy, eficiency, etc.) in a
comparative setting against existing solutions.</p>
      <p>Acknowledgements The EPICS (E-Procurement Innovation For Challenging
Scenarios) project has been co-funded by Programma del Regolamento regionale
della Puglia per gli aiuti in esenzione n. 17 del 30/09/2014 (BURP n. 139 suppl.
In: 2016 International Conference on Information Communication Technology and
Systems (ICTS). pp. 249–254 (2016)
15. Rao, C., Xiao, X., Goh, M., Zheng, J., Wen, J.: Compound mechanism design of
supplier selection based on multi-attribute auction and risk management of supply
chain. Computers &amp; Industrial Engineering 105, 63–75 (2017)
16. Roberts, P.J., Mitchell, R.J., Ruiz, V.F., Bishop, J.M.: Classification in
eprocurement. International Journal of Applied Pattern Recognition 1(3), 298–314
(2014)
17. Sari, K.: A novel multi-criteria decision framework for evaluating green supply
chain management practices. Computers &amp; Industrial Engineering 105, 338–347
(2017)
18. Sinha, A.K., Anand, A.: Towards fuzzy preference relationship based on decision
making approach to access the performance of suppliers in environmental conscious
manufacturing domain. Computers &amp; Industrial Engineering 105, 39–54 (2017)
19. Vahdani, B., Mousavi, S.M., Tavakkoli-Moghaddam, R., Hashemi, H.: A new
enhanced support vector model based on general variable neighborhood search
algorithm for supplier performance evaluation: A case study. International Journal of
Computational Intelligence Systems 10(1), 293–311 (2017)</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Abdollahnejadbarough</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mupparaju</surname>
            ,
            <given-names>K.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shah</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golding</surname>
            ,
            <given-names>C.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leites</surname>
            ,
            <given-names>A.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Popp</surname>
          </string-name>
          , T.D.,
          <string-name>
            <surname>Shroyer</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golany</surname>
            ,
            <given-names>Y.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Robinson</surname>
            ,
            <given-names>A.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Akgun</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Verizon uses advanced analytics to rationalize its tail spend suppliers</article-title>
          .
          <source>INFORMS Journal on Applied Analytics</source>
          <volume>50</volume>
          (
          <issue>3</issue>
          ),
          <fpage>197</fpage>
          -
          <lpage>211</lpage>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Aghdaie</surname>
            ,
            <given-names>M.H.</given-names>
          </string-name>
          :
          <article-title>Data mining group decision-making with fahp: An application in supplier evaluation and segmentation</article-title>
          . Fuzzy analytic hierarchy process pp.
          <fpage>99</fpage>
          -
          <lpage>124</lpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Barraclough</surname>
            ,
            <given-names>R.W.</given-names>
          </string-name>
          , PLC,
          <string-name>
            <given-names>U.</given-names>
            ,
            <surname>House</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Park</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Bishop</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.M.</given-names>
            ,
            <surname>Danicic</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Nasuto</surname>
          </string-name>
          , S.J., Mitchell, R.J.:
          <article-title>Spendinsight: some remarks on deploying an intelligent spend-analysis system</article-title>
          .
          <source>In: Proceedings of the 1st Symposium on Nature Inspired Computing and Applications</source>
          (NICA) AISB/IACAP World Congress (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Caspersen</surname>
            ,
            <given-names>K.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Madsen</surname>
            ,
            <given-names>M.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eriksen</surname>
            ,
            <given-names>A.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thiesson</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>A hierarchical tree distance measure for classification</article-title>
          .
          <source>In: ICPRAM</source>
          . pp.
          <fpage>502</fpage>
          -
          <lpage>509</lpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Chai</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ngai</surname>
            ,
            <given-names>E.W.</given-names>
          </string-name>
          :
          <article-title>Decision model for complex group argumentation</article-title>
          .
          <source>Expert Systems with Applications</source>
          <volume>45</volume>
          ,
          <fpage>223</fpage>
          -
          <lpage>233</lpage>
          (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Du</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guo</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guo</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>A pareto supplier selection algorithm for minimum the life cycle cost of complex product system</article-title>
          .
          <source>Expert Systems with Applications</source>
          <volume>42</volume>
          (
          <issue>9</issue>
          ),
          <fpage>4253</fpage>
          -
          <lpage>4264</lpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Ghadimi</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dargi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Heavey</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Sustainable supplier performance scoring using audition check-list based fuzzy inference system: A case application in automotive spare part industry</article-title>
          .
          <source>Computers &amp; Industrial Engineering</source>
          <volume>105</volume>
          ,
          <fpage>12</fpage>
          -
          <lpage>27</lpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Islek</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          , O¨ug¨duu¨c¨, S.G.:
          <article-title>A retail demand forecasting model based on data mining techniques</article-title>
          .
          <source>In: 24th IEEE International Symposium on Industrial Electronics, ISIE</source>
          <year>2015</year>
          , Rio de Janeiro, Brazil, June 3-5,
          <year>2015</year>
          . pp.
          <fpage>55</fpage>
          -
          <lpage>60</lpage>
          . IEEE (
          <year>2015</year>
          ), https: //doi.org/10.1109/ISIE.
          <year>2015</year>
          .7281443
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Singh</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yadav</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mishra</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Using data mining synergies for evaluating criteria at pre-qualification stage of supplier selection</article-title>
          .
          <source>Journal of Intelligent Manufacturing</source>
          <volume>25</volume>
          (
          <issue>1</issue>
          ),
          <fpage>165</fpage>
          -
          <lpage>175</lpage>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Jouida</surname>
            ,
            <given-names>S.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Krichen</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>A genetic algorithm for supplier selection problem under collaboration opportunities</article-title>
          .
          <source>Journal of Experimental &amp; Theoretical Artificial Intelligence</source>
          <volume>0</volume>
          (
          <issue>0</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>27</lpage>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Khoraskani</surname>
            ,
            <given-names>M.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kheradmand</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Khamseh</surname>
            ,
            <given-names>A.A.</given-names>
          </string-name>
          :
          <article-title>Application and comparison of neural network, c5.0, and classification and regression trees algorithms in the credit risk evaluation problem case study: A standard german credit dataset</article-title>
          .
          <source>International Journal of Knowledge Engineering and Data Mining</source>
          <volume>4</volume>
          (
          <issue>3-4</issue>
          ),
          <fpage>259</fpage>
          -
          <lpage>276</lpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Kundu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chandra</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>A journey from normative to behavioral operations in supply chain management: A review using latent semantic analysis</article-title>
          .
          <source>Expert Systems with Applications</source>
          <volume>42</volume>
          (
          <issue>2</issue>
          ),
          <fpage>796</fpage>
          -
          <lpage>809</lpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Moghaddam</surname>
            ,
            <given-names>K.S.</given-names>
          </string-name>
          :
          <article-title>Fuzzy multi-objective model for supplier selection and order allocation in reverse logistics systems under supply and demand uncertainty</article-title>
          .
          <source>Expert Systems with Applications</source>
          <volume>42</volume>
          (
          <fpage>15</fpage>
          -
          <lpage>16</lpage>
          ),
          <fpage>6237</fpage>
          -
          <lpage>6254</lpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Rahmawati</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ainul</surname>
            <given-names>Yaqin</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Sarno</surname>
          </string-name>
          , R.:
          <article-title>Fraud detection on event logs of goods and services procurement business process using heuristics miner algorithm</article-title>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>