Choosing perfect work path is not an easy task especially in IT sector. Lately we can notice that data science and jobs connected to this field are getting more and more popular. To reduce time consumed on finding perfect work position in data science, authors have presented solution, which selects best job based on factors introduced by user. Final job title is a result of combining soft set algorithm with analyzed accuracies of k-nearest neighbours algorithms classified with diferent k parameters and on various collections.
rithm. The soft set table consists of columns that are a
specific factor on which we focus, and the rows are the
Nowadays, IT systems [
We created a program that allows you to choose a ca- closest to the new element and assigning this element reer path based on specific factors. The program will to the group to which most of its neighbors belong. make it possible to select the optimal result using the Several metrics are used to determine the similarity, this k nearest neighbors algorithm and using soft sets. We program uses two: Manhattan (Taxi Cab) and Minkowski. create a table for soft sets with the accuracy of various types of distance calculation methods in the KNN algo- Manhattan metric (x, y) = ∑︁ | − | =1
(1) x – value of a sample, y – value of a classified element, n – amount of elements in the sample Minkowski metric – a modified Euclidean metric (x, y) = ︃( ∑︁ | − | =1 )︃1/ .
(2)
d – distance, x – value of a sample, y – value of a classified element, n – amount of elements in the sample m – any small integer,
After calculating the distance, the data is clustered: ifrst sorted in ascending order, then voting is done on the basis of a 1:1 matching of the sample attribute to the test set attribute - the same elements are added to the common set. Then, the percentage share of the searched elements in relation to the entire data set is calculated: = ℎℎℎℎ × 100% The variable k largely determines the behavior of the classifier. Determines the number of the closest neighbors that decide on the classification of the element. It is a natural number. This parameter is arbitrary, but if we want our classifier to work eficiently, we must make a few assumptions: • K must be greater than the square root of the number of all classified elements ≥
√, n - number of classified elements • If the number of groups is even, k must be odd.
Otherwise, k must be even.
= {︃2 + 1, |2 2, otherwise (3) c – number of groups, ∈ • K must be greater than the number of groups >
Let be the initial infinite set and the set of parameters or attributes relative to . Let ( ) denote the power set i ⊆ . The (, ) pair is called 41–48 (4) the soft set above , where is the mapping given by : → ( ). Others in words, the soft set (, ) over U is a parameterized family of the subset . For ∈ , () can be considered a set of e-elements or e-approximate elements of soft sets (, ). Thus, (, ) is defined as: (, ) = { () ∈ ( ) : ∈ , () = ∅ , if ∈/ } ∑︁ · =1 • – element of the sample • – weight • – length of the sample
sion statistical sample for subsets, and then conducting analyzes of the training set, while the test set is used to confirm the plausibility of its results.
ful in the study.
Min-max normalization using a linear function, it reduces the data to the interval specified by the user (newmin, newmax). At the same time, we should know the range that the data can achieve. If we do not know it, we can use the highest and the smallest value in the analyzed set.
− ′ = − · − +
algorithms based on accuracy which we get from analyzing diferent options. We implemented two types of KNN algorithms, one based on distances between val- Figure 1: Histogram presenting values of annual salary ues of sample and dataset tried to give best job position sorting by distances and summing appearance of various job titles. Second algorithm also calculated distances but ifrstly it focused on getting a specific category of work and then from this limited collection of data it returned nearest neighbours for job positions. Both of these algorithms were closely analyzed and results showed that classic KNN algorithm without any categorization gives best accuracy. the classified data. Also salaries in diferent jobs positions overlap in ranges, which may make it dificult to distinguish positions based on the amount of salary.
Plots Fig. 4 and Fig. 5 presenting the connection between the location or the nationality of employee and the amount of salary shows that the research was conducted mainly on the American market, also the scope of salaries of employees of diferent nationalities and companies Data: Input , , from other countries rather coincides, i.e. the amount Result: of the salary does not depend on the citizenship of the := []; employee or the country in which he works. Therefore it := []; can be concluded that there are certain salary scales that while < ( ) do are ofered in IT positions in data analysis regardless of Calculate distance between sample and record location or nationality.
in , save it to ; Fig. 6 shows the connection between the employee’s end level of experience and his salary. The highest rate was Add as new column to ; ofered to the person with the greatest responsibility, Sort by column ; i.e. working in an executive position, for example the for S ainverannugme(b0e,kr)odfodiferent job title’s position of director, leader or project manager. Then occurrences for first records in to the seniors have the highest stake. The lowest stake is accumulated in the junior experience group. There are ; also single outliers in each group. end Fig. 7 checks if there is any connection between the return that appeared most frequently amount of the salary and company’s size. We may notice in ; the lack of huge diferences in stakes for employees from Algorithm 1: Algorithm of our implementation of various companies.
KNN Pie charts Fig. 8 and Fig. 9 were generated to verify the percentage of various work modes and the types of employments. It shows that remote or semi-remote work is 7. Analyzing dataset provided in almost 85 percent of positions, while full-time employment predominates in the type of employment.
The histogram Fig. 1 and plot Fig. 3 show the perfor- Summing up, the data available does not stand out for mance of the earnings in the field of datascience. It in- a specific group of job positions or, for example, for a forms us that there are over 160 people earning between certain location of the company, which may result in the 0 to 10000 USD per year. We note that earnings cumulate dificulty of their classification and lower accuracy. The in the range of approximately 50000 to 200000 USD. The lack of visible boundaries in the rates due to the size of the remaining values are sporadic and we look at them as company shown in Fig. 7 or the small number of records outliers. for certain positions Fig. 2 will be factors that make
From the chart Fig. 2, we obtain information about the classification dificult. Also, the predominance of the earnings for a specific position. We also note the number location of companies and the citizenship of employees of records that will define a given job. Positions such from the United States makes the data reflect the reality as Data Scientist or Data Engineer have more records rather for developed countries. than, for example Data Specialist, which appears only once in the database. Not having the same number of records for diferent positions will afect the accuracy of
result, we get low accuracy of the algorithm’s operation.
Therefore, in further action, despite the re-verification Presented KNN algorithms have achieved an accuracy be- of the operation on normalized values, we gave up using tween 37 to 89% for classification based on job title. Data this normalized data due to the very low accuracy. were divided in proportions adequately 30% testing and We may notice on Fig. 12 that normalizing only salary 70% training part. Results were analyzed to determine column itself, which initially takes values in thousandths, perfect combination of dataset, k parameter and variety allows to increase the accuracy of the classification with of distance metrics used in KNN algorithm. We focused the use of job type categorization. One more time, the on two types of distance metrics Minkowski and Manhat- taxicab metric is a better method of calculating distances. tan. Comparison test consist on checking performance Graph and table on Fig. 13 show the accuracies for of KNN algorithm on normalized dataset, not normalized diferent k using the classic KNN algorithm without addidataset and normalized data but only in salary column. tional categorization. The accuracy values are practically
Graphs presented in Fig. 10 show the influence of k the same with minimal variation depending on the dison the accuracy of the algorithm for k nearest neighbors tance metric used. using an additional column of job categories. We can Working on completely normalized data in each of see that for k equal to 8 there is a sudden decrease in the columns turns out to be pointless due to the very accuracy for both the Minkowski method of distance low accuracy that we obtain regardless of the parameter calculation and the taxi method. Then the values from k k Fig. 14. Therefore, in the created table for the soft equal to 9 decrease. Better accuracy is obtained by using set algorithm, we do not take into account the accuracy the Manhattan distance metrics. obtained when working on this type of data sets.
The impact of k on accuracy shown in Fig. 11, informs In the presented graphs Fig. 15, we may notice that us that normalizing all columns with little variation in the parameter k afects the determination of the accuracy. data does not allow algorithm to classify properly. As a
decreases. On the other hand, when using the Manhattan metric, values decrease from the intial k.
Table Fig. 16 presents the obtained table for the operation of the soft set algorithm. This table contains accuracies for the following KNN algorithms from the lines, using the given parameter k as well as a specific data set. We obtain this soft set table after analyzing for which parameters k gives the best accuracy. 10. Conclusion
perfect job position based on factors, which he or she focuses on. Because of in-depth reporting of data set we could distinguish best combinations of KNN algorithm in terms of k parameter, distance metric and data set itself. Thanks to creating soft set table of accuracies of diferent KNN solutions we get best algorithm, which also gives factors we focus on the most the utmost importance.