The Method for Determining the Readiness Level of Technologies for the Safety Transfer Nataliya Shakhovska*[0000-0002-6875-8534], Nataliya Chukhray[0000-0001-8591-2487], Roman Hasko[0000-0002-0567-2995], Natalia Lotoshynska[0000-0002-6618-0070], Marta Voronovska[0000-0002-3539-0171] Lviv Polytechnic National University, Lviv 79013, Ukraine Natalia.i.chuhraj@lp.edu.ua, Nataliya.b.shakhovska@lpnu.ua, r.hasko@gmail.com, natlot@ukr.net, marta_tomych@ukr.net Abstract. The ensemble of methods for the readiness level of technology pre- diction was made. The main features for prediction were selected. Different clustering methods show different results, that’s mean more complex analysis needs. The paper describes technology to establish the readiness level of scien- tific and technological development for commercialization, on the basis of which an integral indicator is determined, which includes the values of the pa- rameters of technological, analytical, patent, market levels of readiness of scien- tific and technological development for commercialization and the level of its social impact. Keywords: Readiness Level, Clustering, Dimension Reduction, Project Man- agement, Safety Transfer. 1 Introduction Among the modern factors of development of the world economy one of the leading is considered knowledge, which is transformed into skills and implemented in innova- tive technologies. Nowadays knowledge is the core for increasing the effectiveness of interaction between participants in innovative infrastructures of countries. An im- portant tool for such interaction is technology transfer. The efficiency of technology transfer depends on the reduction of time between the projection, development and generation of market effects on innovation, and, accordingly, the development of a competitive economy in the region. That is why it is important to analyze influence of different features to technology’s success on the market. One of the main components of successful and safety technology transfer implementation is the assessment of technology readiness for transfer. The paper presents development of technology to establish the readiness level of scientific and technological development for commercialization, on the basis of which an integral indicator is determined, which includes the values of the parameters of technological, analytical, patent, market levels of readiness of scientific and techno- logical development for commercialization and the level of its social impact. This technology is characterized the integral indicator of the readiness level of the technol- Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib- ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in Global Information Networks. ogy. This integral indicator is determined by an ensemble of methods of computation- al intelligence, namely hierarchical clustering, k-means, DBSCAN and phase- clustering methods, followed by the use of a fully connected neural network with a defined experimental architecture. The ensemble of methods for the readiness level of technology prediction allows us not only to predict the level of readiness, but also to find the most important parame- ters for such prediction. 2 State of art The issue of assessing technology readiness for transfer has not been given due im- portance by the global community of scientists and practitioners. For the most part, experts focus on the development of technology-based principles for commercializa- tion, taking into account the specificities of countries. The primary purpose of using technology readiness levels is to assist management in making decisions about their development and transfer. In the traditional method, NASA uses three groups of indicators for this purpose: the level of technological readiness of technology, the level of market readiness of technology, the level of pa- tent readiness of technology. However, this technique provides a sufficiently general- ized assessment of the technology's market readiness for transfer, does not differenti- ate the external and internal marketing characteristics of the technology, which leads to judgmental judgment. In addition, the method does not provide regulatory limits for the assessment of established steps. NASA technological readiness assessment model based on evaluating technology from the developer's baseline and formulation process to fully proven, approved, and commissioned production for which the product has a competitive [1]. This model includes several successive steps, each of which can be started only after the previous one. The process is long-lasting, the quality and reliability of determining the level of readiness of the technology for commercialization is low, since the model does not reflect other aspects of the development of scientific and technological development, in particular - market, analytical, patent and social impact. Known author evaluation model [2] and [3] used Technology Readiness Levels (TRLs) developed by NASA as a common metric for technology advancement. In order to investigate the adequacy of this tool, the first study searched for academic and applied research on the military and civilian organizations of Turkey. The TRL Awareness and TRL Calculator applied to defense firms in Ankara, and interviews were conducted with technology developers, firm speakers, and defense agencies. For both the first approach and the second, the calculations were performed based on the expertly determined critical levels of each indicator, so the prediction accuracy de- pends on the expert's level of knowledge. Therefore, it does not allow substantiating the level of readiness of scientific and technological development for commercializa- tion with high accuracy and reliability [4] The authors proposed own TAPDS methodology. This methodology is given in [5] and [6]. In contrast to the existing methodological approaches, this methodology ena- bles the evaluators more thoroughly to take into account the market component and readiness of consumers to acquire the researched R&D products. The complexity of the assessment is ensured by a method of hierarchy analysis. The formalized toolkit includes the evaluation of technology at the analytical, technological and patent levels of its readiness, as well as the level of readiness of demand for technology and the impact of society on its development. The paper [7] proposed to use extended operations; also, definition and sufficiency of TRL are given. However, the specificity of different technologies, particularly IT, does not allow us to use these extended operations. The papers [8 – 10] analyze the importance of technology transfer, but method of technology’s evaluation is not presented. The purpose of the paper is to develop the method for the readiness level of tech- nology prediction using technics of computation intelligence. 3 Methods and materials Technology readiness for transfer is defined as a set of capabilities for planning, cata- lyzing, supporting and monitoring, transfer reporting and technology development under specific conditions of use [1]. The basis of the invention is to create a technology for establishing the level of readiness of scientific and technological development for commercialization, based on the parameters of technological, analytical, patent, market levels of readiness of scientific and technological development for commercialization and the level of its social impact. The next step is to determine the integral indicator using the ensemble of methods of computational intelligence (sequential use of an ensemble of clustering methods and a fully connected neural network) that enables to achieve a high degree of accuracy and reliability. In addition, it is very important to define the influents of parameters. The technology of establishing the level of readiness of scientific and technological development for commercialization substantiates the parameters of its readiness, in accordance with the invention. This approach is based on the values of the parameters of technological, analytical, patent, market levels of readiness of scien- tific and technological development for commercialization social impact, and the integral indicator of technology readiness is determined by benchmarking, fuzzy clus- tering and PCA signs depend on each other not so strictly and not so explicitly. The list of parameters of readiness level looks like this: 1. The level of analytical readiness, 2. The patent level, 3. The demand readiness level, 4. The society impact level, 5. Age of developers, 6. Influence level, 7. Wide usage level, 8. Technological complexity 9. Area of usage, 10. Part of market, 11. Novelty, 12. Education level, 13. Scientific level, 14. Level of new knowledge for market, 15. Type of scientific research, 16. Social group, 17. Direction of technology for the consumer, 18. Direction of action, 19. Value, 20. Innovative level. The dataset consists of 27 technologies. Therefore, amount of parameters (dimen- sions) is not too less than the size of dataset. Knowing the dependencies and their strength, we can express several features through one, merge them, so to work with a simpler model. Of course, it is most like- ly that it will not be possible to avoid information loss, but just the PCA method will help us minimize it. Expressed more strictly, this method approximates an n-dimensional cloud of ob- servations to an ellipsoid (also n-dimensional), whose semi axes will be the future main components. In addition, when projecting onto such axes (dimensionality reduc- tion), the largest amount of information is stored. The results of PCA are given on Fig.1 and Fig. 2. Fig. 1. The PCA results representation Importance of components: PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 Standard deviation 2.316 1.939 1.7891 1.440 1.185 1.109 0.923 0.895 Proportion of Variance 0.255 0.179 0.1524 0.099 0.068 0.058 0.041 0.038 Cumulative Proportion 0.255 0.435 0.5871 0.686 0.753 0.811 0.852 0.890 PC9 PC10 PC11 PC12 PC13 PC14 PC15 PC16 Standard deviation 0.831 0.659 0.553 0.500 0.450 0.393 0.311 0.273 Proportion of Variance0.034 0.027 0.015 0.012 0.009 0.007 0.005 0.004 Cumulative Proportion 0.923 0.944 0.958 0.970 0.979 0.987 0.921 0.995 PC17 PC18 PC19 PC20 PC21 Standard deviation 0.21563 0.15734 0.14160 0.05980 0.02414 Proportion of Variance 0.00221 0.00118 0.00095 0.00017 0.00003 Cumulative Proportion 0.99767 0.99885 0.99980 0.99997 1.00000 Fig. 2. PCA matrix So, the most important features determined using PCA are: the level of analytical readiness, the patent level, the demand readiness level, the society impact level, age of developers, influence level. The next step is the clustering. First, we found optimal number of clusters using gap statistic. The gap statistic can be applied to any clustering method. It implies mul- tiple cyclic execution of the algorithm with an increase in the number of selectable clusters, as well as subsequent postponement of the clustering score on the graph, calculated as a function of the number of clusters. (Fig. 3): Fig. 3. The optimal number of clusters The silhouette coefficient is calculated using the average intracluster distance (a) and the average distance to the nearest cluster (b) for each sample. The silhouette is calcu- lated as (b - a) / max (a, b). The k-means method is used with 5 clusters. The visuali- zation of results is given on Fig. 4. Fig. 4. K-means results As result, the intersection of clusters 1 and 3 is presented. DBSCAN (Density-based spatial clustering of applications with noise) method operates with data density. We analyze the radius of the neighborhood and the number of neighbors. The visualiza- tion of data density is given on Fig. 5. So, dataset consists of technologies without similarity between each other. Fig. 5. The DBSCAN clustering results The next group of method is used for prediction of the readiness level. The predictive model must be built. Support Vector machine solves the problems of classification and regression by constructing a nonlinear plane separating the solutions. Due to the nature of the fea- ture space in which the boundaries of the solution are constructed, the support vector method has a high degree of flexibility in solving regression and classification prob- lems of various difficulty levels. There are various types of SVM models: linear, pol- ynomial, RBF (radial basis functions), and sigmoid. In regression SVM, we must evaluate the functional dependence of the dependent variable y on the set of inde- pendent variables x. In our case, the dependence variable is the readiness level. The rest variables belong to group x. This suggests that, as in other regression problems, the relations between inde- pendent and dependent variables are determined by the deterministic function f and the addition of some additive noise: y = f (x) + noise. (1) The challenge is to find a functional form for f that can correctly predict new val- ues. Functional dependence is sought by training the SVM model on a sample popula- tion, i.e. learning set; this process includes both classification (see above) and sequen- tial optimization of the error function. For our SVM model, the error function is de- termined by the formula: 𝑁 1 𝑇 1 𝑤 𝑤 − 𝐶 (𝑣𝜀 + ∑(𝜉𝑖 + 𝜉𝑖` )). 2 𝑁 𝑖=1 The function is minimized provided: The accuracy of SVM is: Svm 0.01031017 So, the accuracy of SVM allow us to use this model for future prediction. The k-Nearest Neighbors method is a memory-based method and, unlike other sta- tistical methods, does not require prior training (i.e., does not fit models). The method is based on the intuitive assumption that nearby objects most likely belong to the same category. Thus, forecasts are made based on a set of prototype samples that predict new (that is, not yet observable) values, using the "majority vote" principle for classification and the averaging principle for regression tasks for the closest samples (hence the name of the method). The accuracy of knn is given below: Knn 0.01021034 The Root Mean Square Error analysis shows us again the difference between tech- nologies presented in dataset (Fig. 6). Fig. 6. The relation between count of neighbors and RMSE Random Forest is a collection of decision trees. The input vector goes through several decision trees. For regression, the output value of all trees is averaged. The structure of regression tree is given on Fig. 7. Fig. 7. The regression tree structure The accuracy of regression tree and random forest is 0.05021034. The last stage is neural network development and training. To achieve high per- formance, neural networks require a huge amount of data, and as a result, as a rule, neural networks are inferior to other ML algorithms in cases where there is little data. The structure of fully-connected neural network (NN) with back propagation is given on Fig. 7. The gradient descent is used in this NN. Fig. 8. The structure of neural network The accuracy of neural network is 0.01021034. After that, we try to rebuild the neural network using only important parameters as result of PCA. The accuracy is 0.00102. 4 Results We used RStudio for data analysis and prediction. When building a model, you need to check its accuracy. Therefore, we divide our data into two parts: training (80%) and testing (20%). set.seed (9) index <- sample(1:nrow(x),round(0.8*nrow(x))) train <- x[index,] test <- x[-index,] When evaluating the parameter y, the model calculates the probability of reading the letter, and not the specific value 0 or 1. That is, we get the value from the interval [0, 1]. It is necessary to determine at what threshold of probability, we will assign the user to group 0 or 1. Now, as a threshold value, take the threshold parameter equal to 0.09. We proceed as follows: if ŷ ≤ threshold, then Response = 0, if ŷ> threshold, then Response = 1. We compare the results of the forecast model with real data. (testResult <- t (table (Act = test $ Response, Predic- tion = glmpredRound))) The contingency table of the actual and predicted response values is given in Table 1. Table 1. Confusion matrix Fact/ Forecast 0 1 0 12 0 1 1 16 Therefore, prediction error is not huge. 5 Conclusions The method for the readiness level of technology prediction using technics of compu- tation intelligence is developed. The presence of acceptable correlation coefficients between clearly dependent parameters indicates that the methodology and the re- spondents' answers are true. On the other hand, there are a number of parameters that are not clearly affected. Therefore, the proposed method can be used for readiness level prediction, but features of the model must be analyzed more detail. 6 References 1. NASA (2017). Technology Readiness Level, available at: https://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html 2. ALTUNOK, Taner; CAKMAK, Tanyel. A technology readiness levels (TRLs) calculator software for systems engineering and technology management tool. Advances in Engineer- ing Software, 2010; 41.5: 769-778 3. HICKS, Ben, et al. A methodology for evaluating technology readiness during product de- velopment. In: International Conference on Engineering Design: 24/08 / 2009-27 / 08/2009. Design Research Society, 2009. p. 157-168 4. Munari, F., Rasmussen, E., Toschi, L., & Villani, E. (2016). Determinants of the univer- sity technology transfer policy-mix: a cross-national analysis of gap-funding instru- ments. The Journal of Technology Transfer, 41(6), 1377–1405. https://doi.org/10.1007/s10961-015-9448-1 5. Chukhray, N., Shakhovska, N., Mrykhina, O., Bublyk M., Lisovska, L. Consumer aspects in assessing the suitability of technologies for the transfer. In: Proc of International scien- tific conferece “Computer sciences and information technologies” (CSIT-2019), IEEE, vol. 3 pp. 142-147 6. Chukhray, N., Shakhovska, N., Mrykhina, O., Bublyk, M., & Lisovska, L. (2019, Septem- ber). Methodical Approach to Assessing the Readiness Level of Technologies for the Transfer. In International Conference on Computer Science and Information Technology (pp. 259-282). Springer, Cham. 7. Straub, J. (2015). In search of technology readiness level (TRL) 10. Aerospace Science and Technology, 46, 312-320. 8. De Noni, I., Orsi, L., & Belussi, F. (2018). The role of collaborative networks in support- ing the innovation performances of lagging-behind European regions. Research Policy, 47(1), 1-13. 9. Chaffey, D., Hemphill, T., & Edmundson-Bird, D. (2015). Digital business and e- commerce management. Pearson UK. 10. Avelino, F., Wittmayer, J., Kemp, R., & Haxeltine, A. (2017). Game-changers and trans- formative social innovation. Ecology and Society: a journal of integrative science for resil- ience and sustainability, 22(4)