=Paper=
{{Paper
|id=Vol-2871/paper9
|storemode=property
|title=Text Mining on Job Advertisement Data: Systematic Process for Detecting Artificial Intelligence Related Jobs
|pdfUrl=https://ceur-ws.org/Vol-2871/paper9.pdf
|volume=Vol-2871
|authors=Asta Back,Arash Hajikhani,Arho Suominen
|dblpUrl=https://dblp.org/rec/conf/iconference/BackHS21
}}
==Text Mining on Job Advertisement Data: Systematic Process for Detecting Artificial Intelligence Related Jobs==
1st Workshop on AI + Informetrics - AII2021 Text Mining on Job Advertisement Data: Systematic Process for Detecting Artificial Intelligence Related Jobs Asta Bäck1, Arash Hajikhani1[0000-0003-2032-9180] and Arho Suominen1,2[0000-0001-9844-7799] 1 Quantitative Science and Technology Studies, VTT Technical Research Centre of Finland, Tekniikantie 21, 02044 Espoo, Finland 2 Tampere University, P.O. Box 541, Tampere FI-33014, Finland Abstract. The use of online job advertisement has made them an important source of quantitative information about the innovation system. This data offers significant opportunities to study trends, transitions in the job markets and skill demands. In this study, we have utilized the job ads data of a major Finnish job market platform to investigate the emergence of AI-related jobs. More than 480 000 job advertisements during 2013-2020 was used to create insight on skills transitions, particularly focusing on artificial intelligence related skills. A glos- sary of AI-related skills was created and applied to the job data to identify the relatedness spectrum of ads to AI using a three-tier system. By incorporating sectoral firm-level information, we explored the variation in AI-related skills demand over time and sectors. Our study presents a systematic way to utilize job advertisement data for detecting demand trends for specific skills. Keywords: 1 Introduction With the increase in data, and with more accessible data, novel avenues of informet- rics are emerging. We know that approximately 90 % of data is unstructured and needs restructuring and cleaning prior to being used for existing machine learning methods [1]. However, with tools such as natural language processing and artificial intelligenceµ, we are able to create new possibilities for discovering new relationships and inference on a multitude of problems. One exciting area is to understand the changes in skills adoption and industrial structures through novel datasets. Approaches to measuring technological and industrial change have been reliant on innovation output measures such as patenting as a proxy for innovation outcomes. For example, existing frameworks for measuring productivity, such as the Crépon, Duguet and Mairesse (CDM) model, use patents as an innovation output measure, albeit this includes significant caveats. In the case of productivity measuring, current debate highlights the possibility that existing measuring creates mismeasurement [2], [3]. In this, Byrne et al. [3] highlight two issues. 1) The “mismeasurement of information technology hardware is significant preceding the slowdown” and that 2) “tremendous consumer benefits from the “new” economy such as smartphones, Google searches, Copyright 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 and Facebook are, conceptually, nonmarket”. These explanations point towards the impact of for example, digitalization and artificial intelligence. While there is critique on the explanation being solely from digitalization and artificial intelligence, as for example [4] looking towards industrial dynamics change and stalling diffusion, and [5] who looks towards lags in impacts, we can argue that by better understanding the adoption of these new techno-logical capabilities we would be better informed on their implications. This study aims to show an approach of deriving novel metrics from raw data to in- form the impact digital and artificial intelligence skills have on the economy. We use a dataset of job advertisements from Finland to understand society's needs for skills and knowledge, both in public sector and private companies. Analyzing the unstruc- tured natural language and available metadata provided by the data source, and the additional metadata from Orbis, we focus on creating an approach to identify how digital and artificial intelligence-related skills are adopted in the workforce. Our study broadens the vantagepoint offered by existing measures and opens ave- nues for future research. We discuss the cross-disciplinarily of the novel metrics providing an avenue to apply the measures into understanding knowledge diffusion to society and implications to econometrics, such as measuring productivity. Subsequent paragraphs, however, are indented. 2 Background 2.1 Text mining job advertisement Text mining job advertisements have recently drawn interest among scholars [6]– [8]. Studies have used the jobs data in different ways creating novel measures to un- derstand, for example discrimination [7], [9], skills needs in an industry [6], [10], [11] professionals in particular fields [12], creating recommendation system for jobs [13] and to understand knowledge needs in a broader techno-logical domain [8]. Pejic-Bach et al. [8] classify these studies into three groups. First, studies use text mining in analyzing job advertisements to create novel classification schemes for job advertisements [6], [14]. Second, text mining has been used to improve matching candidates for specific jobs. [13], [15] and third, analyzing the performance of em- ployees [16], [17]. Beyond this broad classification, text mining of job advertisements has been addressed in different studies, loosely adhering to the clustering by Pejic- Bach et al. [8]. As an example, studies look at employee selection, but focuses are different. Geor- giou et al. [18] focus on gamification in employee selection, focusing on if game ele- ments can be used in testing candidate aptitude. We also see practical applications of matching candidate profiles to job advertisements [19]. Tavakoli et al. [13] also focus on recommendations and focus on supporting labor force to know and acquire skills demanded in the market. One of the main lines of research, and application of job advertisements, focus on profiling skills. Studies have looked at what is needed to become a data scientist [20]. The impact of technological changes, such as Industry 4.0., on skills has also attracted research [8], [21]. Verma et al. [22] focus on artificial 3 intelligence and machine learning and what type of skills transition the adoption of the technologies will have, linking development to curriculum development. Similar- ly, Rampasso et al. [23] use job advertisements to inform on undergraduate graduates' needs. Less attention has been put to understand macro-level changes in the economy, while research show job advertisements to be a practical vantage point. Thurgood et al. [24] focus on the United Kingdom labor market creating a segmentation of the job market using 15 million job advertisements. The ‘bottom-up’ segmentation of the labor market cuts across wage, sector, and occupation. Our segmentation is based upon applying text mining and concept creation techniques to aggregate and capture the job ads' demand. Similarly, Faryana [25] focus on wage dynamics and the overall labor market condition. This, however, remains an emerging stream of research. 2.2 Artificial Intelligence related jobs – case Finland Digital transformation and the application of artificial intelligence is expected to have significant positive impacts on productivity [26]. While there is a discus-sion on which extent AI and digitalization promise is materializing [27], studies have looked at the process industry areas [28] to find significant productivity increases. This para- dox may relate to the fact that the practical applications of AI and digitalization in the industry focus on automation, and we are missing positive impact to the economy and social outcomes [29]. The interplay between the great promise of AI and digitalization to improve productivity and the stagnating productivity begs the question of what type of task is emerging into the job market relating to AI and digitalization. In 2017, Finland became the first country to develop an AI strategy [30]. Simulta- neously, the national innovation funding agency, Business Finland, launched a large- scale research programme on AI and platform economy. This was a response to the much-discussed predictions of the broad and deep impact AI and digitalization would have on the economy, but also to the society more broadly. The Ministry of Economic Affairs and Employment in Finland predicted that that by applying AI, annual GDP growth could increase from 0.8% to 3% by 2030 [30]. The ministry's expectation was echoed with consultancy companies predicting that labor productivity could increase by 36% in Finland in 2035 if AI is applied successfully [31]. From a macro view, Finland could expect good things from AI. Finland has in its history invested significantly in ICT related skills. The application of ICT has in- creased the productivity of work in Finland and the gross domestic product calculated per person through it. In the Finnish case, investments in ICT account for less than one-fifth of all investments, but they have increased labor productivity more than all other investments combined [32]. However, looking towards comparative economies Finland’s development has been challenging. In 2018, Finland’s labor productivity difference to countries such as Sweden and Germany was 10%. It seems that Finland, unlike its competitor countries, has not recently benefited from techno-logical ad- vances. To better understand the implications of the adoption of AI there is a need for firm-level information. While there is a significant amount of research on the use of 4 AI and digitalization, there is less publicly available data on the utilization or adop- tion of AI at the micro level.[33] There is some ambiguity if, and to what extent, AI in particular will destroy jobs or will there be job creation [34]. Even if AI and digitali- zation will be used for automation, there are questions if that will lead to “job elimi- nation”[35]. What is clear is that there is a change in skills required due to the adop- tion of AI and digitalization[36]. Trajtenberg [37] highlights that there will be a per- sistent and increasing demand for analytical and creative thinking, communication skills and emotional control. While these skills of tomorrow are seen as more broad changes in the economy, a better understanding of the tasks where humans now inter- act with AI and digitalization can serve as a vantage point to the tasks of the future. 3 Data and method 3.1 Data Job advertisements open a view on the skills required by the labor market at any given time. Historically, we have seen job advertisements communicated via print media, but more recently job advertisements have moved to specialized sites and social me- dia platforms. In our work, we accessed data from a specialized job advertisement site. We analyzed the job advertisements of Oikotie Oy’s Job Advertisement service, one of the two leading commercial job advertisement services in Finland. In total, our data extends from 2013 to 2020 and contains 480,000 job advertise- ments. This period and particularly its second half were a period of growth in the number of job vacancies. The largest number of job advertisements in the dataset, slightly over 90,000, was in 2019. The most significant growth in the yearly number of jobs, over 20,000 jobs, occurred in 2017. The brisk growth also continued in 2018. For GDP, 2017 was also the period of rapid growth during the period considered; since then, annual growth has been lower. The access to the data provides full details of the job advertisement, created by the job poster. The dataset included the job titles, job descriptions and information of which company had posted the ad. The data is self-reported by the job posting com- pany or agency used in the process. This will include the usual caveats of self- reported data. However, the most significant challenge with the data is the use of anonymous recruitment. In this, the advertisement will refer to a recruitment agency masking the employer. Another consideration is that the data multilingual. As the service is run in Finland, most of the ads were in Finnish. However, a significant por- tion of the advertisements are in English so that both these languages need to be taken into account. The skills requirements were expressed in two main ways. Often, there is a list of skills and educational requirements, but the requirements may also be expressed im- plicitly by describing what kind of tasks the job consists of. 5 3.2 Artificial Intelligence glossary In this study, the focus is on understanding job market transitions specifically regard- ing the needs of Artificial intelligence skills. Towards this objective, a central issue is to create a method to identify the AI related job ads and through them to under-stand the job market transition for acquiring AI skills. To achieve this objective, the imme- diate first step was to comprehend which jobs advertisements refer to AI skills. We approached this issue by creating a glossary of words and concepts that refer to AI in various levels such as methods, tools, and technology. The AI glossary was based on a reviewing the publications in which the AI termi- nology and taxonomy were explained. We have adopted the definitions from [38] and [39]. Additional information to finalize the glossary, was taken from Stack Overflow survey results for the year 2020 and taken from the Wikipedia AI glossary. The sources we covered create a comprehensive view on AI. The glossary is also built to take into consideration the different levels of abstraction, from very general to tech- nologies and supportive solutions. The three separate tiers of AI relatedness built from the reviewed sources are: • Tier 1: Main generic terms referring to AI (i.e., artificial intelligence, machine learning). • Tier 2: Core technologies associated to AI (i.e., NLTK, Decision tree) • Tier 3: Technologies that support or enhance AI solutions but not direct AI core technologies (i.e., Cloud, Database, Matlab) Each of the Tiers is represented by a vocabulary of terms, implemented in two lan- guages (Finnish and English). Some of the terms do not have a Finnish language translation, and these professional terms were only implemented in English. The terms in the glossary were there after searched if included in the job advertisements description or titles. The process used a hierarchical approach where each job adver- tisement was linked to one tier only starting from Tier 1. In other words, if an ad in- cluded Tier 1 terms, it was included only in Tier 1 even though it may have had terms from Tier 2 or 3. As a quality check, we assessed the job titles of the resulting dataset to check the relevance of the result set and found some job titles that were not in the scope, and we removed them before the analysis. 3.3 Enriching company data To include more detail on the job market changes, additional data from the compa- nies were merged to the job advertisement data. We used Orbis data with information on more than 400 million companies globally. The dataset included the in-formation of which company had posted the advertisement, and this let us link the companies to Orbis data that allows for in depth analysis of company financials, and other identifi- ers such as legal status, industrial classification, management. In case a recruitment company was involved, the ad is linked to the recruitment company’s sector, which is 6 in most cases N: Administrative services. The full process from raw data to final set for analytics is drafted in Fig 1. Raw data Unique Job ads in Dataset during 2013-2020 (n=467,715 ) Search ads using three sets (tiers) of search terms. All (AI) related ads identified based on Selection (AI_Glossary) (n=51,621) Remove irrelevant jobs based on the job title Ads thematic Remaining ads categorization I (n=45,556) Remove duplicates (Ad only in one tier; n = 36,978) Ads thematic AI ads tier_1 AI ads tier_2 AI ads tier_3 categorization II (n=2,098) (n=13,560) (n=21,320) Fetch company business id 36,760 ads linked to Final Set 3094 companies 10,428 ads linked to 190 26,332 ads linked to companies in sector N (~ 2,905 companies requitement) excluding sector N Fig. 1. Data analytics process illustration 7 4 Results The data gives us the opportunity to analyse both at the changes in the AI skills de- mand through the different tiers, and in the different sectors. Fig 2 shows the volume of job ads in the different tiers and the share of these ads of the number of total yearly job ads. We can see that the absolute number of AI related jobs in-creased until 2019 but lagged in growth when compared to the total increase of jobs in the dataset. We can also see that most of the job ads belong to the more general Tiers 2 and 3. There share of the more specialized skills has increased: the share of Tier 1 jobs has increased from 2.9% in 2013 to 6.5% in 2020, the share of Tier 2 from 29.4% in 2013 to 41.9% in 2020, while the share of Tier 3 jobs has dropped from 68.1% to 51.6%. This indicates a shift to increasing adoption of AI in organizations. Fig. 2. Number of jobs for the different sectors from 2013 to 2020 (left scale) and share of all jobs (right scale). To compare the development of AI demand in the different industrial sectors, we looked at the how the share of the different tiers was distributed to the ten most ac- tively recruiting sectors. The biggest changes in sectors can be seen in the most spe- cific Tier 1, seen in Fig 3, where only four sectors were hiring in 2013 but all ten were hiring in 2020. Financial & Insurance sector has increasingly looked for AI skills during this period. All ten sectors were recruiting Tier 2 skills already in 2013, as seen in Fig 4. Wholesale & Retail, Financial & Insurance, and Public sector had in- creased their share most. Manufacturing sector’s share has dropped from 7.0% in 2013 to 3.8% in 2020. The share of Tier 3, as seen in Fig. 5, jobs has dropped from 68.1% in 2013 to 51.6% in 2020. The share of these jobs has decreased also within all sectors except Public sector and Electricity & Energy. 8 Fig. 3. The share of jobs in the different tiers and in the different industrial sectors in Tier 1 Fig. 4. The share of jobs in the different tiers and in the different industrial sectors in Tier 2 9 Fig. 5. The share of jobs in the different tiers and in the different industrial sectors in Tier 3 Another way to look at the results is to look at the trends in the numbers of compa- nies looking for AI skills, and how actively they are recruiting. These indicators tell about the spread and intensity of technology adoption. Fig 6 shows the numbers of different companies looking for Tier 1 or Tier 2 AI skills each year, and the average number of advertisements per company for three sectors: Manufacturing, Finance and Insurance, and Wholesale and Retail. The absolute number of companies looking for AI skills in highest for Manufacturing, and they also had highest average number of advertisements per company except for the last two years when Finance and Insur- ance had the highest average. We can see that the increase in demand for AI skills in the financial sector during the last three years comes from the increased activity of companies and not from a higher number of companies recruiting AI skills. Comparing these sectors activities in 2019 and 2020, we can see that the manufac- turing sector clearly decreased their recruitment activity in 2020, most likely because of the Covid-19 pandemic, whereas Wholesale and Retail remained at the same level, and Finance and Insurance increased their recruitments. 10 Fig. 6. The yearly numbers of different companies looking for AI skills in Tier 1 and 2 (col- umns, left scale) and average number of advertisements per company in each sector and year (lines, right scale). 5 Summary and conclusions This paper aimed to evaluate the usefulness of job advertisements in monitoring tech- nology adoption in companies and the public sector. The focus technology was Artifi- cial Intelligence, and our dataset consisted of job ads published in one of the leading job ad services in Finland from 2013 to 2020. The ads were analyzed using natural language processing and the data was enriched with sector information classifications retrieved from the Orbis service. We developed a three-tiered vocabulary to identify AI related ads. Defining the terms to match one specific area and result in a dataset that includes all relevant jobs is hard and defining three tiers of terms ad-dressed this issue. Tier 1 included the generic terms directly linked to AI, Tiers 2 technologies related to AI applications, and Tier 3 covered terms that are often linked to AI but are more general and can also be used in tasks not directly linked to AI. The tier-based approach allowed us to get an expansive view of the technology adoption, and by looking at companies recruitments from year to year we can see how many of the ads belong to each of these tiers. When a company has ads in all three tiers, it is a clear indication of serious AI adoption. Based on the data, we can see an apparent increase in demand for core AI skills, but the absolute numbers of AI ads are relatively low. When looking at the shares of the three tiers over the years, we can see that the shares of Tier 1 and 2 linked ads have increased and the share of Tier 3 decreased. That could be interpreted to indicate that companies are shifting from setting up the infrastructure to actual AI applications. The results also reveal significant differences between sectors. Financial & Insurance is a sector that has increasingly looked for AI skilled employees, whereas the im- 11 portant sector of manufacturing does not show any clear tendency to increase the recruitments of AI skills. We demonstrate a use case for job ads data by showing the trend within AI related job in Finland. The data has the capability of being structed, timely and comprehen- sive and therefore important to consider for many future research endeavors. Compa- nies recruitment behavior is an early indication of capability building and therefore a good source to compile proxies to describe research and development prospects and technology adoption within companies. The extensive textual content in job ads makes it possible to have an accurate view on companies needs and challenges. While the boundaries around technologies are not definite, it is important to give each tier a clear focus when developing the vocabularies for the different tiers. That helps in analyzing the results and understanding the development trends. Acknowledgements This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 870822 References 1. G. H. Kim, S. Trimi, and J. H. Chung, “Big-data applications in the govern- ment sector,” Communications of the ACM, vol. 57, no. 3, pp. 78–85, Mar. 2014, doi: 10.1145/2500873. 2. C. Syverson, “Challenges to mismeasurement explanations for the US produc- tivity slowdown,” in Journal of Economic Perspectives, Mar. 2017, vol. 31, no. 2, pp. 165–186, doi: 10.1257/jep.31.2.165. 3. D. M. Byrne, J. G. Fernald, and M. B. Reinsdorf, “Does the united states have a productivity slowdown or a measurement problem?,” Brookings Papers on Economic Activity, vol. 2016, no. SPRING, pp. 109–182, Mar. 2016, doi: 10.1353/eca.2016.0014. 4. D. Andrews, C. Criscuolo, and P. Gal, “The Global Productivity Slowdown, Technology Divergence and Public Policy: A Firm Level Perspective,” The Future of Productivity: Main Background Papers, no. September, pp. 1–50, 2016, Accessed: Feb. 16, 2021. Online.. Available: https://www.brookings.edu/wp-content/uploads/2016/09/wp24_andrews-et- al_final.pdf. 5. N. Crafts, “The productivity slowdown: Is it the ‘new normal’?,” Oxford Re- view of Economic Policy, vol. 34, no. 3, pp. 443–460, Jul. 2018, doi: 10.1093/oxrep/gry001. 6. I. Karakatsanis et al., “Data mining approach to monitoring the requirements of the job market: A case study,” Information Systems, vol. 65, pp. 1–6, Apr. 2017, doi: 10.1016/j.is.2016.10.009. 7. P. K. Ningrum, T. Pansombut, and A. Ueranantasun, “Text mining of online job advertisements to identify direct discrimination during job hunting pro- 12 cess: A case study in Indonesia,” PLOS ONE, vol. 15, no. 6, p. e0233746, Jun. 2020, doi: 10.1371/journal.pone.0233746. 8. M. Pejic-Bach, T. Bertoncel, M. Meško, and Ž. Krstić, “Text mining of indus- try 4.0 job advertisements,” International Journal of Information Manage- ment, vol. 50, pp. 416–431, Feb. 2020, doi: 10.1016/j.ijinfomgt.2019.07.014. 9. P. Kuhn, K. Shen, and S. Zhang, “Gender-targeted job ads in the recruitment process: Facts from a Chinese job board,” Journal of Development Econom- ics, vol. 147, p. 102531, Nov. 2020, doi: 10.1016/j.jdeveco.2020.102531. 10. P. A. Todd, J. D. McKeen, and R. B. Gallupe, “The evolution of IS job skills: A content analysis of IS job advertisements from 1970 to 1990,” MIS Quar- terly: Management Information Systems, vol. 19, no. 1, pp. 1–23, 1995, doi: 10.2307/249709. 11. I. Khaouja, I. Rahhal, M. Elouali, G. Mezzour, I. Kassou, and K. M. Carley, “Analyzing the needs of the offshore sector in Morocco by mining job ads,” in IEEE Global Engineering Education Conference, EDUCON, May 2018, vol. 2018-April, pp. 1380–1388, doi: 10.1109/EDUCON.2018.8363390. 12. M. W. Barbosa and V. M. de Oliveira, “The Corporate Social Responsibility professional: A content analysis of job advertisements,” Journal of Cleaner Production, vol. 279, p. 123665, Jan. 2021, doi: 10.1016/j.jclepro.2020.123665. 13. M. Tavakoli, S. T. Mol, and G. Kismihók, “Labour Market Information Driv- en, Personalized, OER Recommendation System for Lifelong Learners,” CSEDU 2020 - Proceedings of the 12th International Conference on Comput- er Supported Education, vol. 2, pp. 96–104, May 2020, doi: 10.5220/0009420300960104. 14. M. Mezzanzanica, “Italian Web Job Vacancies for Marketing-Related Profes- sions,” Symphonya. Emerging Issues in Management, vol. 0, no. 3, p. 110, Oct. 2015, doi: 10.4468/2015.3.14mezzanzanica. 15. Y. Kino, H. Kuroki, T. Machida, N. Furuya, and K. Takano, “Text Analysis for Job Matching Quality Improvement,” in Procedia Computer Science, Jan. 2017, vol. 112, pp. 1523–1530, doi: 10.1016/j.procs.2017.08.054. 16. T. Chamorro-Premuzic, R. Akhtar, D. Winsborough, and R. A. Sherman, “The datafication of talent: how technology is advancing the science of human po- tential at work,” Current Opinion in Behavioral Sciences, vol. 18. Elsevier Ltd, pp. 13–16, Dec. 01, 2017, doi: 10.1016/j.cobeha.2017.04.007. 17. T. Chamorro-Premuzic, D. Winsborough, R. A. Sherman, and R. Hogan, “New talent signals: Shiny new objects or a brave new world?,” Industrial and Organizational Psychology, vol. 9, no. 3, pp. 621–640, Sep. 2016, doi: 10.1017/iop.2016.6. 18. K. Georgiou, A. Gouras, and I. Nikolaou, “Gamification in employee selec- tion: The development of a gamified assessment,” International Journal of Selection and Assessment, vol. 27, no. 2, pp. 91–103, Jun. 2019, doi: 10.1111/ijsa.12240. 13 19. P. Duggan, “Methods of matching job profiles and candidate profiles,” Dec. 2008. Accessed: Feb. 16, 2021. Online at https://patents.google.com/patent/US20100153290A1/en 20. F. Baumeister, M. W. Barbosa, R. R. Gomes, F. Baumeister, M. W. Barbosa, and R. R. Gomes, “What Is Required to Be a Data Scientist?: Analyzing Job Descriptions With Centering Resonance Analysis,” International Journal of Human Capital and Information Technology Professionals (IJHCITP), vol. 11, no. 4, pp. 21–40, 2020, Accessed: Feb. 16, 2021. Online.. Available: https://EconPapers.repec.org/RePEc:igg:jhcitp:v:11:y:2020:i:4:p:21-40. 21. S. Fareri, G. Fantoni, F. Chiarello, E. Coli, and A. Binda, “Estimating Industry 4.0 impact on job profiles and skills using text mining,” Computers in Indus- try, vol. 118, p. 103222, Jun. 2020, doi: 10.1016/j.compind.2020.103222. 22. A. Verma, K. Lamsal, and P. Verma, “An investigation of skill requirements in artificial intelligence and machine learning job advertisements,” Industry and Higher Education, p. 095042222199099, Feb. 2021, doi: 10.1177/0950422221990990. 23. I. S. Rampasso et al., “An investigation of research gaps in reported skills required for Industry 4.0 readiness of Brazilian undergraduate students,” Higher Education, Skills and Work-based Learning. Emerald Group Publish- ing Ltd., Feb. 19, 2020, doi: 10.1108/HESWBL-10-2019-0131. 24. J. Thurgood, A. Turrell, D. Copple, J. Djumalieva, and B. Speigner, “Using Online Job Vacancies to Understand the UK Labour Market from the Bottom- Up,” SSRN Electronic Journal, Aug. 2018, doi: 10.2139/ssrn.3222698. 25. O. Faryna, T. Pham, O. Talavera, and A. Tsapin, “Wage Setting and Unem- ployment: Evidence from Online Job Vacancy Data,” 2020. Accessed: Feb. 16, 2021. Online.. Available: https://www.econstor.eu/handle/10419/215479. 26. H. Aly, “Digital transformation, development and productivity in developing countries: is artificial intelligence a curse or a blessing?,” Review of Econom- ics and Political Science, vol. ahead-of-print, no. ahead-of-print, May 2020, doi: 10.1108/reps-11-2019-0145. 27. E. Brynjolfsson, D. Rock, and C. Syverson, Artificial Intelligence and the Modern Productivity Paradox: a Clash of Expectations and Statistics, vol. Publisher, no. October 2017. University of Chicago Press, 2017. 28. F. F. Baesler, M. Moraga, and F. J. Ramis, “Productivity improvement in the wood industry using simulation and artificial intelligence,” in Winter Simula- tion Conference Proceedings, 2002, vol. 2, pp. 1095–1098, doi: 10.1109/wsc.2002.1166362. 29. D. Acemoglu and P. Restrepo, “The wrong kind of AI? Artificial intelligence and the future of labour demand,” Cambridge Journal of Regions, Economy and Society, vol. 13, no. 1, pp. 25–35, May 2020, doi: 10.1093/cjres/rsz022. 30. TEM, “Finland´s Age of Artificial Intelligence,” 2017. Online.. Available: https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/160391/TEMrap_47 _2017_verkkojulkaisu.pdf. 31. M. Purdy and P. Daugherty, “HOW AI INDUSTRY PROFITS AND INNOVATION BOOSTS.” Accessed: Feb. 20, 2021. Online.. Available: 14 https://www.accenture.com/fr- fr/_acnmedia/36DC7F76EAB444CAB6A7F44017CC3997.pdf. 32. M. Pohjola, “Teknologia, investoinnit, rakennemuutos ja tuottavuus – Suomi kansainvälisessä vertailussa,” 2020. Accessed: Feb. 20, 2021. Online.. Avail- able: https://julkaisut.valtioneuvosto.fi/bitstream/handle/10024/162051/TEM_2020 _05.pdf. 33. M. Raj and R. Seamans, Artificial Intelligence, Labor, Productivity, and the Need for Firm-Level Data, vol. Publisher. University of Chicago Press, 2019. 34. J. Bessen, “Artificial Intelligence and Jobs: The Role of Demand,” The Eco- nomics of Artificial Intelligence: An Agenda, vol. 15, no. September, pp. 291– 307, 2019, Accessed: Feb. 16, 2021. Online.. Available: https://www.nber.org/books-and-chapters/economics-artificial-intelligence- agenda/artificial-intelligence-and-jobs-role-demand. 35. O. H. Hamid, N. L. Smith, and A. Barzanji, “Automation, per se, is not job elimination: How artificial intelligence forwards cooperative human-machine coexistence,” in Proceedings - 2017 IEEE 15th International Conference on Industrial Informatics, INDIN 2017, Nov. 2017, pp. 899–904, doi: 10.1109/INDIN.2017.8104891. 36. S. Jagannathan, S. Ra, and R. Maclean, “Dominant recent trends impacting on jobs and labor markets - An Overview,” International Journal of Training Re- search, vol. 17, no. sup1, pp. 1–11, Jul. 2019, doi: 10.1080/14480220.2019.1641292. 37. M. Trajtenberg, “AI as the next GPT: a Political-Economy Perspective,” Cambridge, MA, Jan. 2018. doi: 10.3386/w24245. 38. L. Aristodemou, F. T.-W. P. Information, and undefined 2018, “The state-of- the-art on Intellectual Property Analytics (IPA): A literature review on artifi- cial intelligence, machine learning and deep learning methods for,” Elsevier, Accessed: Feb. 20, 2021. Online.. Available: https://www.sciencedirect.com/science/article/pii/S0172219018300103. 39. R. Flemmer, C. Flemmer, E. S. Brunette, R. C. Flemmer, and C. L. Flemmer, “A review of artificial intelligence,” ieeexplore.ieee.org, 2009, doi: 10.1109/ICARA.2000.4804025.