=Paper=
{{Paper
|id=Vol-2588/paper32
|storemode=property
|title=The Method for Determining the Readiness Level of Technologies for the Safety Transfer
|pdfUrl=https://ceur-ws.org/Vol-2588/paper32.pdf
|volume=Vol-2588
|authors=Nataliya Shakhovska,Nataliya Chukhray,Roman Hasko,Natalia Lotoshynska,Marta Voronovska
|dblpUrl=https://dblp.org/rec/conf/cmigin/ShakhovskaCHLV19
}}
==The Method for Determining the Readiness Level of Technologies for the Safety Transfer==
https://ceur-ws.org/Vol-2588/paper32.pdf
The Method for Determining the Readiness Level of
Technologies for the Safety Transfer
Nataliya Shakhovska*[0000-0002-6875-8534], Nataliya Chukhray[0000-0001-8591-2487],
Roman Hasko[0000-0002-0567-2995], Natalia Lotoshynska[0000-0002-6618-0070],
Marta Voronovska[0000-0002-3539-0171]
Lviv Polytechnic National University, Lviv 79013, Ukraine
Natalia.i.chuhraj@lp.edu.ua, Nataliya.b.shakhovska@lpnu.ua,
r.hasko@gmail.com, natlot@ukr.net, marta_tomych@ukr.net
Abstract. The ensemble of methods for the readiness level of technology pre-
diction was made. The main features for prediction were selected. Different
clustering methods show different results, that’s mean more complex analysis
needs. The paper describes technology to establish the readiness level of scien-
tific and technological development for commercialization, on the basis of
which an integral indicator is determined, which includes the values of the pa-
rameters of technological, analytical, patent, market levels of readiness of scien-
tific and technological development for commercialization and the level of its
social impact.
Keywords: Readiness Level, Clustering, Dimension Reduction, Project Man-
agement, Safety Transfer.
1 Introduction
Among the modern factors of development of the world economy one of the leading
is considered knowledge, which is transformed into skills and implemented in innova-
tive technologies. Nowadays knowledge is the core for increasing the effectiveness of
interaction between participants in innovative infrastructures of countries. An im-
portant tool for such interaction is technology transfer. The efficiency of technology
transfer depends on the reduction of time between the projection, development and
generation of market effects on innovation, and, accordingly, the development of a
competitive economy in the region. That is why it is important to analyze influence of
different features to technology’s success on the market. One of the main components
of successful and safety technology transfer implementation is the assessment of
technology readiness for transfer.
The paper presents development of technology to establish the readiness level of
scientific and technological development for commercialization, on the basis of which
an integral indicator is determined, which includes the values of the parameters of
technological, analytical, patent, market levels of readiness of scientific and techno-
logical development for commercialization and the level of its social impact. This
technology is characterized the integral indicator of the readiness level of the technol-
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib-
ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in
Global Information Networks.
ogy. This integral indicator is determined by an ensemble of methods of computation-
al intelligence, namely hierarchical clustering, k-means, DBSCAN and phase-
clustering methods, followed by the use of a fully connected neural network with a
defined experimental architecture.
The ensemble of methods for the readiness level of technology prediction allows us
not only to predict the level of readiness, but also to find the most important parame-
ters for such prediction.
2 State of art
The issue of assessing technology readiness for transfer has not been given due im-
portance by the global community of scientists and practitioners. For the most part,
experts focus on the development of technology-based principles for commercializa-
tion, taking into account the specificities of countries.
The primary purpose of using technology readiness levels is to assist management
in making decisions about their development and transfer. In the traditional method,
NASA uses three groups of indicators for this purpose: the level of technological
readiness of technology, the level of market readiness of technology, the level of pa-
tent readiness of technology. However, this technique provides a sufficiently general-
ized assessment of the technology's market readiness for transfer, does not differenti-
ate the external and internal marketing characteristics of the technology, which leads
to judgmental judgment. In addition, the method does not provide regulatory limits
for the assessment of established steps.
NASA technological readiness assessment model based on evaluating technology
from the developer's baseline and formulation process to fully proven, approved, and
commissioned production for which the product has a competitive [1]. This model
includes several successive steps, each of which can be started only after the previous
one. The process is long-lasting, the quality and reliability of determining the level of
readiness of the technology for commercialization is low, since the model does not
reflect other aspects of the development of scientific and technological development,
in particular - market, analytical, patent and social impact.
Known author evaluation model [2] and [3] used Technology Readiness Levels
(TRLs) developed by NASA as a common metric for technology advancement. In
order to investigate the adequacy of this tool, the first study searched for academic
and applied research on the military and civilian organizations of Turkey. The TRL
Awareness and TRL Calculator applied to defense firms in Ankara, and interviews
were conducted with technology developers, firm speakers, and defense agencies. For
both the first approach and the second, the calculations were performed based on the
expertly determined critical levels of each indicator, so the prediction accuracy de-
pends on the expert's level of knowledge. Therefore, it does not allow substantiating
the level of readiness of scientific and technological development for commercializa-
tion with high accuracy and reliability [4]
The authors proposed own TAPDS methodology. This methodology is given in [5]
and [6]. In contrast to the existing methodological approaches, this methodology ena-
bles the evaluators more thoroughly to take into account the market component and
readiness of consumers to acquire the researched R&D products. The complexity of
the assessment is ensured by a method of hierarchy analysis. The formalized toolkit
includes the evaluation of technology at the analytical, technological and patent levels
of its readiness, as well as the level of readiness of demand for technology and the
impact of society on its development.
The paper [7] proposed to use extended operations; also, definition and sufficiency
of TRL are given. However, the specificity of different technologies, particularly IT,
does not allow us to use these extended operations.
The papers [8 – 10] analyze the importance of technology transfer, but method of
technology’s evaluation is not presented.
The purpose of the paper is to develop the method for the readiness level of tech-
nology prediction using technics of computation intelligence.
3 Methods and materials
Technology readiness for transfer is defined as a set of capabilities for planning, cata-
lyzing, supporting and monitoring, transfer reporting and technology development
under specific conditions of use [1].
The basis of the invention is to create a technology for establishing the level of
readiness of scientific and technological development for commercialization, based
on the parameters of technological, analytical, patent, market levels of readiness of
scientific and technological development for commercialization and the level of its
social impact. The next step is to determine the integral indicator using the ensemble
of methods of computational intelligence (sequential use of an ensemble of clustering
methods and a fully connected neural network) that enables to achieve a high degree
of accuracy and reliability. In addition, it is very important to define the influents of
parameters. The technology of establishing the level of readiness of scientific and
technological development for commercialization substantiates the parameters of its
readiness, in accordance with the invention. This approach is based on the values of
the parameters of technological, analytical, patent, market levels of readiness of scien-
tific and technological development for commercialization social impact, and the
integral indicator of technology readiness is determined by benchmarking, fuzzy clus-
tering and PCA signs depend on each other not so strictly and not so explicitly.
The list of parameters of readiness level looks like this:
1. The level of analytical readiness,
2. The patent level,
3. The demand readiness level,
4. The society impact level,
5. Age of developers,
6. Influence level,
7. Wide usage level,
8. Technological complexity
9. Area of usage,
10. Part of market,
11. Novelty,
12. Education level,
13. Scientific level,
14. Level of new knowledge for market,
15. Type of scientific research,
16. Social group,
17. Direction of technology for the consumer,
18. Direction of action,
19. Value,
20. Innovative level.
The dataset consists of 27 technologies. Therefore, amount of parameters (dimen-
sions) is not too less than the size of dataset.
Knowing the dependencies and their strength, we can express several features
through one, merge them, so to work with a simpler model. Of course, it is most like-
ly that it will not be possible to avoid information loss, but just the PCA method will
help us minimize it.
Expressed more strictly, this method approximates an n-dimensional cloud of ob-
servations to an ellipsoid (also n-dimensional), whose semi axes will be the future
main components. In addition, when projecting onto such axes (dimensionality reduc-
tion), the largest amount of information is stored.
The results of PCA are given on Fig.1 and Fig. 2.
Fig. 1. The PCA results representation
Importance of components:
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8
Standard deviation 2.316 1.939 1.7891 1.440 1.185 1.109 0.923 0.895
Proportion of Variance 0.255 0.179 0.1524 0.099 0.068 0.058 0.041 0.038
Cumulative Proportion 0.255 0.435 0.5871 0.686 0.753 0.811 0.852 0.890
PC9 PC10 PC11 PC12 PC13 PC14 PC15 PC16
Standard deviation 0.831 0.659 0.553 0.500 0.450 0.393 0.311 0.273
Proportion of Variance0.034 0.027 0.015 0.012 0.009 0.007 0.005 0.004
Cumulative Proportion 0.923 0.944 0.958 0.970 0.979 0.987 0.921 0.995
PC17 PC18 PC19 PC20 PC21
Standard deviation 0.21563 0.15734 0.14160 0.05980 0.02414
Proportion of Variance 0.00221 0.00118 0.00095 0.00017 0.00003
Cumulative Proportion 0.99767 0.99885 0.99980 0.99997 1.00000
Fig. 2. PCA matrix
So, the most important features determined using PCA are: the level of analytical
readiness, the patent level, the demand readiness level, the society impact level, age of
developers, influence level.
The next step is the clustering. First, we found optimal number of clusters using
gap statistic. The gap statistic can be applied to any clustering method. It implies mul-
tiple cyclic execution of the algorithm with an increase in the number of selectable
clusters, as well as subsequent postponement of the clustering score on the graph,
calculated as a function of the number of clusters. (Fig. 3):
Fig. 3. The optimal number of clusters
The silhouette coefficient is calculated using the average intracluster distance (a) and
the average distance to the nearest cluster (b) for each sample. The silhouette is calcu-
lated as (b - a) / max (a, b). The k-means method is used with 5 clusters. The visuali-
zation of results is given on Fig. 4.
Fig. 4. K-means results
As result, the intersection of clusters 1 and 3 is presented. DBSCAN (Density-based
spatial clustering of applications with noise) method operates with data density. We
analyze the radius of the neighborhood and the number of neighbors. The visualiza-
tion of data density is given on Fig. 5. So, dataset consists of technologies without
similarity between each other.
Fig. 5. The DBSCAN clustering results
The next group of method is used for prediction of the readiness level. The predictive
model must be built.
Support Vector machine solves the problems of classification and regression by
constructing a nonlinear plane separating the solutions. Due to the nature of the fea-
ture space in which the boundaries of the solution are constructed, the support vector
method has a high degree of flexibility in solving regression and classification prob-
lems of various difficulty levels. There are various types of SVM models: linear, pol-
ynomial, RBF (radial basis functions), and sigmoid. In regression SVM, we must
evaluate the functional dependence of the dependent variable y on the set of inde-
pendent variables x. In our case, the dependence variable is the readiness level. The
rest variables belong to group x.
This suggests that, as in other regression problems, the relations between inde-
pendent and dependent variables are determined by the deterministic function f and
the addition of some additive noise:
y = f (x) + noise. (1)
The challenge is to find a functional form for f that can correctly predict new val-
ues.
Functional dependence is sought by training the SVM model on a sample popula-
tion, i.e. learning set; this process includes both classification (see above) and sequen-
tial optimization of the error function. For our SVM model, the error function is de-
termined by the formula:
𝑁
1 𝑇 1
𝑤 𝑤 − 𝐶 (𝑣𝜀 + ∑(𝜉𝑖 + 𝜉𝑖` )).
2 𝑁
𝑖=1
The function is minimized provided:
The accuracy of SVM is:
Svm
0.01031017
So, the accuracy of SVM allow us to use this model for future prediction.
The k-Nearest Neighbors method is a memory-based method and, unlike other sta-
tistical methods, does not require prior training (i.e., does not fit models). The method
is based on the intuitive assumption that nearby objects most likely belong to the
same category. Thus, forecasts are made based on a set of prototype samples that
predict new (that is, not yet observable) values, using the "majority vote" principle for
classification and the averaging principle for regression tasks for the closest samples
(hence the name of the method). The accuracy of knn is given below:
Knn
0.01021034
The Root Mean Square Error analysis shows us again the difference between tech-
nologies presented in dataset (Fig. 6).
Fig. 6. The relation between count of neighbors and RMSE
Random Forest is a collection of decision trees. The input vector goes through several
decision trees. For regression, the output value of all trees is averaged. The structure
of regression tree is given on Fig. 7.
Fig. 7. The regression tree structure
The accuracy of regression tree and random forest is 0.05021034.
The last stage is neural network development and training. To achieve high per-
formance, neural networks require a huge amount of data, and as a result, as a rule,
neural networks are inferior to other ML algorithms in cases where there is little data.
The structure of fully-connected neural network (NN) with back propagation is given
on Fig. 7. The gradient descent is used in this NN.
Fig. 8. The structure of neural network
The accuracy of neural network is 0.01021034. After that, we try to rebuild the
neural network using only important parameters as result of PCA. The accuracy is
0.00102.
4 Results
We used RStudio for data analysis and prediction. When building a model, you need
to check its accuracy. Therefore, we divide our data into two parts: training (80%) and
testing (20%).
set.seed (9)
index <- sample(1:nrow(x),round(0.8*nrow(x)))
train <- x[index,]
test <- x[-index,]
When evaluating the parameter y, the model calculates the probability of reading the
letter, and not the specific value 0 or 1. That is, we get the value from the interval [0,
1]. It is necessary to determine at what threshold of probability, we will assign the
user to group 0 or 1. Now, as a threshold value, take the threshold parameter equal to
0.09. We proceed as follows:
if ŷ ≤ threshold, then Response = 0,
if ŷ> threshold, then Response = 1.
We compare the results of the forecast model with real data.
(testResult <- t (table (Act = test $ Response, Predic-
tion = glmpredRound)))
The contingency table of the actual and predicted response values is given in Table 1.
Table 1. Confusion matrix
Fact/ Forecast 0 1
0 12 0
1 1 16
Therefore, prediction error is not huge.
5 Conclusions
The method for the readiness level of technology prediction using technics of compu-
tation intelligence is developed. The presence of acceptable correlation coefficients
between clearly dependent parameters indicates that the methodology and the re-
spondents' answers are true. On the other hand, there are a number of parameters that
are not clearly affected. Therefore, the proposed method can be used for readiness
level prediction, but features of the model must be analyzed more detail.
6 References
1. NASA (2017). Technology Readiness Level, available at:
https://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html
2. ALTUNOK, Taner; CAKMAK, Tanyel. A technology readiness levels (TRLs) calculator
software for systems engineering and technology management tool. Advances in Engineer-
ing Software, 2010; 41.5: 769-778
3. HICKS, Ben, et al. A methodology for evaluating technology readiness during product de-
velopment. In: International Conference on Engineering Design: 24/08 / 2009-27 /
08/2009. Design Research Society, 2009. p. 157-168
4. Munari, F., Rasmussen, E., Toschi, L., & Villani, E. (2016). Determinants of the univer-
sity technology transfer policy-mix: a cross-national analysis of gap-funding instru-
ments. The Journal of Technology Transfer, 41(6), 1377–1405.
https://doi.org/10.1007/s10961-015-9448-1
5. Chukhray, N., Shakhovska, N., Mrykhina, O., Bublyk M., Lisovska, L. Consumer aspects
in assessing the suitability of technologies for the transfer. In: Proc of International scien-
tific conferece “Computer sciences and information technologies” (CSIT-2019), IEEE, vol.
3 pp. 142-147
6. Chukhray, N., Shakhovska, N., Mrykhina, O., Bublyk, M., & Lisovska, L. (2019, Septem-
ber). Methodical Approach to Assessing the Readiness Level of Technologies for the
Transfer. In International Conference on Computer Science and Information Technology
(pp. 259-282). Springer, Cham.
7. Straub, J. (2015). In search of technology readiness level (TRL) 10. Aerospace Science
and Technology, 46, 312-320.
8. De Noni, I., Orsi, L., & Belussi, F. (2018). The role of collaborative networks in support-
ing the innovation performances of lagging-behind European regions. Research Policy,
47(1), 1-13.
9. Chaffey, D., Hemphill, T., & Edmundson-Bird, D. (2015). Digital business and e-
commerce management. Pearson UK.
10. Avelino, F., Wittmayer, J., Kemp, R., & Haxeltine, A. (2017). Game-changers and trans-
formative social innovation. Ecology and Society: a journal of integrative science for resil-
ience and sustainability, 22(4)