Adult learners aiming to upskill themselves are often bombarded with overwhelming and contradicting information in terms of what skills are relevant to their occupations at a given time. Companies and governments also face the same concerns as they navigate ways to optimally manage their workforce and citizens' employability. The goal of this paper is to explore the use of knowledge graphs to better understand the complex relationship between occupations and skills, aiming to provide a cohesive answer to the question of how a skill relates to an occupation. We focus on harnessing taxonomies represented as relationships within a temporal knowledge graph. The contribution of this paper is a methodology to construct a comprehensive Jobs and Skills knowledge base that evolves dynamically over time, starting with the Singapore SkillsFuture Skills Framework as a foundational resource. Through the integration of expert knowledge and extracted entity information from labour market data, we employ data analytics techniques to refine and update the knowledge base to accurately reflect the evolving needs of the Singaporean economy. Through this, technologists in organizations working with jobs and skills can seek to build knowledge bases that incorporate expert knowledge and labour market signals so as to answer jobs and skills questions.
The cacophony of signals coming from workforce development specialists, human resource consultants and job market advocates, including adult education oferings, can leave individuals seeking to upskill (or re-skill) themselves confused about what skills are critical for a certain occupation and which are not.
Companies express their needs by hiring individuals with specific skills. Hence, incorporating labour market data from job postings is crucial to get a direct reading of the skills demand for given occupations. Additionally, expert knowledge coming from specialists in various sectors of the economy presents contextualised views of the overall role that specific occupations and skills play. Together, the labour market insights and domain knowledge give a holistic picture of how skills and occupations are related at certain point in time.
In this paper, we will focus on unlocking the power of taxonomies represented as relationships between entities in a temporal knowledge graph to address the question of what skills are required by occupations and how those relationships evolve over time.
With the transition from traditional newspapers to
online job portals, researchers initially [
SkillsFuture Singapore (SSG), the national skills authority
of the Singapore public service, together with employers,
industry associations, institutes of higher learning and unions,
has created its very own occupation and skill knowledge
base, the SSG Skills Framework [
Relying on rich occupational and skill bases, labour
market data analytics and knowledge graph applications have
emerged as pivotal tools for understanding workforce
dynamics and optimizing skills matching in the job market.
[
Meanwhile, [
Our contributions are as follows: (i) presenting a comprehensive Jobs and Skills knowledge base, and (ii) creating an approach for that knowledge base to evolve over time. Using the SSG Skills Framework as a starting point, the Jobs and Skills knowledge base combines expert knowledge with extracted entity information from large amounts of unstructured data such as job postings, course listings and other available datasets. Data analytics in the form of machine learning classifiers and sliding window weighted averages are used to refine the information extracted from unstructured data and integrate it with expert knowledge, enabling the Jobs and Skills knowledge base to evolve to reflect the needs of the ever-changing Singapore economy.
A knowledge graph is a structured model of information,
comprising entities, relationships, and semantic
descriptions [
Within knowledge graphs, there is a sub type called property graphs (example provided in Fig. 1). These graphs represent data using nodes (entities) and relationships (edges) with properties associated with both nodes and relationships. These graphs are flexible and intuitive, making them suitable for representing complex relationships and attributes within data sets. By following the direction of the relationship, it is possible to compose sentences in English that provide the semantic information of the relationship (e.g. Occupation (Instrumentation Engineer) - REQUIRES SKILL→− Skill (electrical, electronic and control engineering)")
We present the Jobs Skills Knowledge Graph (JSKG), a knowledge graph that can be queried based on the proximity of entities within it. By capturing the information in the form of a knowledge graph, relationships between entities such as Occupations and Skills can be easily investigated to determine commonalities and relatedness between entities. Figure 2 provides an example of various types of nodes and their relationships. The following section provides details on how the JSKG was constructed, with a focus on the Occupation-Skill relationships.
The JSKG itself is structured as a property graph and stored in a graph database, which can be queried through an API. A website and a large language model-enabled chatbot make use of this API to consume the results of queries between entities (e.g. two occupations) and present the information in a compelling way to non-expert users.
Details on the implementation and software tools utilised can be found in the Appendix A.
We leverage a comprehensive dataset encompassing three types of entities to examine the evolving landscape of skills demand and occupational structures in Singapore.
Firstly, the primary data source comprises of job postings collected from the internet spanning the years 2018 to 2023, ofering insights into the dynamic nature of skill demand within the job market over this period. We shall refer to this as Labour Market data.
Secondly, complementing the labour market data, we
incorporate expert knowledge from SkillsFuture’s
Singapore Skills Framework (SFw) [
Thirdly, we also include data on courses and training providers validated by SkillsFuture Singapore to enrich the JSKG with information on options for increasing skills supply to meet market demand.
The integration of these diverse entities allows for a nuanced exploration of the intersections between job market dynamics, skill evolution, and occupational structures, contributing to a comprehensive understanding of the Singaporean workforce and continuous education landscape.
The JSKG uses the Singapore Standard Occupation Classification (SSOC) from the year 2020, which ofers a standardized framework for classifying and organizing occupations within the Singaporean context. SSOC uses a 5 digit hierarchical classification, so that every 5 digit code is contained within a 4 digit code and so on for for fewer digits. Table 1 contains an example SSOC 3-digit, 4-digit and 5-digit:
At the highest level of abstraction, major groups of occupations are categorised at the SSOC 1D level given by Table 2.
Throughout the rest of this manuscript, we’ll refer to SSOCs and Occupations interchangeably. Job portals such as "LinkedIn" or "Indeed.com" are widely used by employers to advertise Job Postings for candidates to apply to. Given that the job market evolves continuously, periodically analysing Job Postings provides a way to measure the relative changes in demand for workers, as well as the evolving descriptions of the Job Roles they perform.
By relying on data purchased from a job aggregator company, the authors analysed over 20M job postings between years 2018 and 2023. Each job posting is defined by a title and plain text description. Given the unstructured nature of the data, additional tools were needed to classify a job posting by the occupation it represents and to identify the skills it requires. More details on the classification and extraction tools are presented below in section 3.5.
SkillsFuture Singapore (SSG) has three variants of skills data. In this section we’ll present them in greater detail.
SSG’s Skills Framework (SFw) provides an expert view on the relationships between Occupations and Skills. It is important to note that the SFw defined a taxonomy of skills which were classified into two types: Technical Skills and Competencies (TSCs) and Critical Core Skills (CCS).
TSCs refer to over 11,000 skills that are typically nontransferrable across occupations, such as "Digital Marketing" or "Airport Service Quality Management". CCSs refer to 16 skills typically referred as soft skills, such as "Problem Solving" or "Communication". Every Job Role in the SFw has TSCs and CCSs associated with it. In turn, every Job Role is linked to an occupation found in the SSOC (Occupations are a superset of Job Roles).
We note that the SFw accounts for similar skills belonging to diferent sectors as two diferent skills. For example, "Digital Marketing" in the "Tourism" sector is considered a diferent skill than "Digital Marketing" in the "Financial Services" sector. This is because the SFw is developed and validated by domain experts representing diferent sectors, such as other statutory boards or industry and trade associations.
Additionally, each skill in the SFw is further broken down into proficiency levels. Each proficiency level, represents a set of knowledge and abilities. An example of this is the TSC skill "Financial Modelling" which has a level 3 proficiency associated to the knowledge "Basic accounting theories of recording and reporting financial information" and a level 6 proficiency associated to the knowledge "Specialised models including real options".
As mentioned in the subsection 3.3.1, the SFw labels skills using sector allowing for some of the skill titles to be duplicates or synonyms of each other across diferent sectors. This is also the case for similar skills classified into diferent proficiency levels. In order to obtain a more general view, SSG has created sector- and proficiency-agnostic skills. After de-duplicating and mapping, there are ∼ 2, 000 sector- and proficiency-agnostic skills (compared to ∼ 11, 000 sectorbased skills). We note that this de-duplication was performed based on expert knowledge. An example of the mapping is presented in the Table 3.
During the extraction of skills from a job posting, there might not be any information on what sector the skill belongs to. The sector-agnostic skills are then a suitable choice for automatic extraction of skills from raw text. More details on the extraction will be provided in section 3.5.
The skills considered up until now are generic. For example, the skill programming and coding does not specify which programming language is involved. When looking for courses and job postings, specifics can be very important. For this reason, SSG has created a list of ∼ 1000 Application and Tools.
Applications and Tools are often required to deliver specific tasks of many occupations. These refer to brand specific software applications or tools such as the programming language Python or the 3D creation platform Unreal Engine. As opposed to SFw Skills, Applications and Tools are not related to a specific sector of the economy.
The third data source is courses present on the SkillsFuture Singapore portal. Each of these courses has a description of its learning goals (in plain text), alongside other information such as the fees and duration. A subset of the courses also have specific SFw skills and proficiency levels associated to them with the validation of SkillsFuture Singapore.
In order to link Labour Market and Course data with the SFw, we leveraged two diferent tools to identify the Skills associated with every job post and course listing, and the SSOC associated with every job post analysed. The tools were not developed by the authors, but we will provide an overview of how they work in the following section.
Two key entities are required to analyse job postings and course data - Occupation associated with any job posting, and the skills required by a job or taught by a course. By extracting these entities from any job post or course listing, it is possible to aggregate and create a knowledge graph representation in which the entities are Occupations, Skills and Courses.
Two proprietary tools, developed by other teams within
the Singapore Public Service, were utilised to achieve this
linkage. These two tools rely on Natural Language
Processing (NLP) and Named Entity Recognition (NER) ([
First we will present the tool to extract SSOC: The SSOC
Autocoder. The SSOC Autocoder is a tool that leverages
transformers [
Secondly, we will present the proprietary tool to extract Skills: The Skill Extraction Algorithm (SEA). Given the possibility of highly similar skills existing in diferent SFw sectors, the SEA developers decided to utilise the ∼ 2000 sectorand proficiency-agnostic de-duplicated skills (see section 3), removing the sectoral dependence. Including the curated list of approximately 1000 Applications and Tools, the SEA is able to extract 3000 unique skills, where each skill has a type given by one of the following: ’TSC’, ’CCS’ and ’App/Tools’. The algorithm uses a NER transformer model and cosine similarity to identify skills that most closely match spans extracted from unstructured text found in Labour Market job postings and course descriptions.
We highlight that if several job postings for the same occupation have skills extracted using SEA, it is then possible to count how often each skill is needed for a given occupation. This ability to quantify the presence of a skill for a given occupation will be utilised in the following section to rank the importance of a skill.
The knowledge graph is composed of entities intrinsic to the topic of jobs and skills. An example of such entities and their relationships is depicted in Fig 2. The entity types in the JSKG are comprehensively listed here: 1. SFw Skills: Defined through expert knowledge in the
Skills Framework.
2. Skills: Sector- and proficiency-agnostic skills &
Application and Tools, extracted using the Skills
Extraction Algorithm.
3. Occupations: As defined by the SSOC 2020 [
MySkillsFuture web portal 6. Proficiencies : Specific capabilities within a SFw Skill 7. Job Roles: Specific roles within an SSOC occupation At this point we have presented the data sources and the processing done on them to extract the entities (nodes) and relevant relationships between them. In the next section we will present how the entities are linked in the JSKG.
Job Postings, found on Job Portals such as "LinkedIn", "JobStreet" and "MyCareersFuture", provide a window into the instantaneous demands of the labour market. However, we must highlight that these portals represent a proxy for the total number of jobs in the entire economy. Specific companies/sectors of the economy might rely on alternative ways to attract job seekers and it is important we highlight that a pure reliance on Job Posting portals does not give a complete picture1.
Additionally, the retrieval at scale of Job Posting from such portals can be a source of inaccuracy in itself. The portal’s changes in popularity by employers posting jobs, server downtime and errors in data processing can lead to lfuctuating sampling of Job Postings. The outcome of these variations can be large diferences in the volume of job postings for two diferent time periods that are not solely due to an observed economic shift in the workforce, but rather potentially attributable to duplicated postings, repostings and intermittent accessibility to portals, just to name some causes. For example, the year of 2018 could have a total of ∼ 1.5 million job postings, whereas a diferent time period (the year of 2019) can have up to ∼ 2.3 million job postings 2
This immediately leads us to realize that comparisons on extensive quantities (e.g. job post counts across years) are unreliable. A specific occupation could garner double its job post count from one period to the next, but that could be a data sampling problem rather than a reflection of a stark change in the economy.
To mitigate this problem, we opt to represent and compare quantities derived from Job Postings in terms of intensive quantities (e.g. ranking), which are less sensitive to sampling lfuctuations.
Particularly for the case of quantifying the relationship between an occupation and a skill, we can compare the top most popular skills for a given occupation across the years analysed without having to be as concerned about the absolute number of job postings. The popularity of the skill, for a given occupation, is given by counts of skill presence when extracting job postings associated to a specific occupation. By ranking these counts for a given occupation , we can then compute a ranking (, , ) of skills in each time period . Below, we will present the notation to represent the relationships between occupations and skills.
For each skill and occupation relationship ( − ), we compute a ranking (, , ) in terms of the weight between − for a given time period . The weights are based on the confidence of the SEA extraction, the frequency with which a job posting mentions a skill, and the frequency with which a skill is required for an occupation. Additionally, 1We’d like to note that this study solely focuses on Job Postings listed on Singapore. 2There is no observable macroeconomic indicator that signaled that Singapore experienced a ∼ 53% growth in demand of workers, more likely, this increase in job postings is due to a variation in sampling. we can also rank the − relationships derived from expert knowledge in the SFw, which is represented as (, ).
In section 4.2 we’ll use the 1/(, ) as a score for every − relationship. The same is applied to (, ) for − relationship derived from the SFw. In this way, we can aggregate scores over time periods and incorporate expert knowledge and labour market data. More importantly, it also enables us to deal with the issue of missing − relationships during certain periods. By working uniquely with rank, we would be forced to assign a rank value to a missing skill. Operating with inverse values allows us to assign 1/(, ) = 0 for missing skills. Table 4 depicts an example for the Occupation Data Scientist on selected skills and year.
The temporal aspect of the JSKG requires relationships to be updated as new information comes in. In particular, this afects the relationships between occupations and skills. Our goal is to compute a score that can be updated over time to better reflect the reality of the changes in the job market.
Every time period, a new dataset of newly posted job postings is analysed. Aggregated relationships between skills and occupations are thus computed for that period.
Although we keep relationships labeled by year, having an overall consolidated relationship between occupations and skills is best for analysts that need holistic information, while retaining information of the evolution of the relationship.
Using the SFw as a starting point, we can use each new set of job postings to update or reinforce the original expert knowledge (if any).
There are three considerations at this point: 1. Skills of type App/Tools, which are not present in the SkillsFuture’s Singapore Skills Framework (SFw).
Their starting weight derived from expert knowledge is zero. 2. The SFw had by design 6 skills associated to a given occupation. The SEA is unbounded and able to retrieve any number of skills from a single Job Posting.
This implies that skills in SFw will have a boosted weight, as compared to those that are not in the SFw. 3. Expert knowledge (SFw) on TSCs and CCSs is considered ever-green. The weight contribution from the SFw should not decay over time.
To address these considerations, we make a judgment call to take the SFw as a starting point, from which variations are made based on signals from labour market data. Secondly, we compare App/Tools separately from TSCs and CCSs, to prevent these from consistently having weaker relationships (by virtue of having zero representation in the SFw).
To control the influence of the ever-green SFw, we can introduce parameter 0 < SFw ≤ 1 to regulate how strong the influence of expert knowledge is on the computed score from labour market signals (job postings). The more time passes before the SFw is updated with expert knowledge, the more SFw should approach 0.
We propose the mathematical formula given by Eq 2 to compute an evolving score for the weight of the relationship between an occupation and a skill . LM represents the ranking of the values of a sliding window of length ∆ in which weights over the years are computed using We present the ranking and scores for Occupation Data Scientist and selected skills over the years. We can observe the values of (, , ) and 1/(, , ) for = 2018 and the expert knowledge from the Singapore SkillsFuture Skills Framework (SFw), handling the missing skill as 0 contribution.
Skill Data and Statistical Analysis
Programming and Coding Mathematical Concepts Application missing (SFw)
(2018) 1 2 2 5 38 ( ) 1
While the frequency with which a skill is required by an
occupation is an indicator of its importance for the tasks
the occupation entails, it is also useful to be able to identify
skills that diferentiate one occupation from another,
especially within a cluster of similar occupations. To achieve
this, similar to [
Using the consolidated skills requirement as the relationship between occupations and skills, diferentiator skill scores are calculated using this formula: diferentiator score
= The terms in formula are:
1 LM × ︂( log ︂( 1 + N 1 + n + 1 ︂) (3) • LM is the rank of the Consolidated Skill Requirement detailed in the section above.
the specified cluster • refers to the total number of occupations within • refers to the number of occupations within the cluster requiring the skill in question
This formula represents an adaptation of TF-IDF, in which we substitute the inverse rank of consolidated skills requirement relationships for TF. The inverse rank gives a measure of how important a skill is to a specific occupation. This substitution is necessary because consolidated skills requirement relationships do not have a direct measure of the frequency with which a skill is needed for a given occupation, which is a more direct parallel for TF. diferentiator skills ("DIFFERENTIATOR"). relationships through Large Language
The recent advances in Large Language Models (LLMs) [
Below is the methodology followed:
1. Take top = 50 TSC and CCS skills associated to
an SSOC given by consolidated skills requirement
2. Take the title and paragraph definition of SSOC 5D
definition given by [
The Jobs Skills Analysts evaluated a sample of ∼ SSOC 5D across the total 1002 SSOC 5D in the classifica400 tion. In that sample, they found that 67% of relationships labeled False with high confidence should be dropped, as they deemed that the skills are unrelated to the occupations. For skill-to- SSOC relationships labeled False by GPT-4o with medium and low confidence, the human evaluation indicated that most relationships are not unrelated. For those with medium confidence, < 5% were deemed to be unrelated, while for those with high confidence, 0% was deemed to be unrelated. For relationships labeled True, for those with low confidence,
< 1% were deemed to be unrelated and 0% of those labeled mid, high were found to be unrelated. In favour of leaning towards less but true values, all relationships labelled False with high confidence by GPT-4o were dropped across all SSOC 5D. The downstream implication is that now the top 50 ranked skills might be less than 50 distinct skills on several SSOC 5D. The following section presents the results obtained in applying these methodologies.
By leveraging the entities presented in section 3.6 we can get a full picture of how occupations and skills come together. In Figure 4 we present the example of the Instrumentation Engineer mentioned in Table 5.
Given that the focus of our discussion has been regarding the refinement/updating of the SSOC-Skill relationship, we’d like to present two illustrative use cases of how the JSKG can be used to derive value for two distinct types of users.
In order to provide an evaluation of the methods to extract skills across all SSOC, we consider the SFw Skill-Occupation linkage as ground truth and run a comparison with the skills associated purely through the labour market data as inference between the years 2020 to 2023. Borrowing from machine learning approaches, we take the top skills extracted by those two approaches and compare how these skills overlap using precision and recall metrics in terms of true values (Top SFw TSC+CCS Skills) and inferred values (Top Labour Market TSC+CCS Skills). The evaluation can be made in terms of the confusion matrix presented in Table 6. App/Tools must be excluded from the comparison, as these are not present in the Skills Framework.
Precision and recall are given by the following formulas: precision = + and recall = + . We can contextualise our findings in terms of the domains of SSOC 1D and by the median amount of job postings analysed per SSOC 5D, which is captured as Job Density also on Table 7.
Table 7 also includes the amount of skills-occupation relationships dropped using the approach presented in 4.4, aggregated at the SSOC 1D level. We can observe that high precision is correlated with lower drop rates of occupationskill relationships. Definitions for each of the major groups in SSOC 1D are given by Table 2.
Across all SSOCs found in both the SFw and labour market data, 71% of the top 10 skills are the same. This decreases to 59% for the top 20 skills and 37% for the top 50 skills. Some divergence in the skills signals for the labour market and the SFw is expected due to changes over time, so these ifgures suggest that the skill-occupation relationships in the knowledge graph are most useful when considering the most important skills for each occupation. This is particularly so for jobs that are advertised more commonly in Singapore (high job post count). Possible ways to improve the rest of the skill-occupation relationships further are covered in section 7.1.
Next, we illustrate the use of the JSKG by providing two diferent customer use cases.
Mid-career Switcher. The user is a mid-career marketing manager who is considering their options to exit their current occupation in favour of one that is more rewarding monetarily and that brings some new challenges. The user needs to be able to take stock of their existing Skills and expected pay in their occupation, and use that information to gauge adjacent occupations in terms of Skill similarity and salary increment.
The user is able to tap on a web portal linked to the JSKG to perform a comparison in terms of skill similarity, obtaining the occupations that have the most skills in common, while providing an increase in pay. The user can then retrieve the skill gap through an occupation comparison and
Jobs Density
Job Posts
Drop Rate ifnd optimal courses that will equip them with the required skills for the best cost and least amount of time. In this example, the marketing manager can identify the occupation of Business Development Manager as an occupation with 9 skills in common and a skills gap of 9 skills. The skill gap can then be fulfilled by identifying the optimal course set required to achieve such skills for the minimum cost, duration, and wait time to earliest opportunity for training.
At this point, the user can get a sense of the dificulty involved in making the career switch, but can also quantify the monetary and time commitment required to achieve basic literacy to bridge the skill gap and achieve their goals of switching careers.
The Learning and Development HR analyst. Some of the key tasks for a L&D analyst involve three aspects: (i) analyzing the gaps between the current skills of employees and the skills required for their roles or for future organizational needs; (ii) Designing and developing training programs and learning materials tailored to address identiifed skill gaps;(iii) Implementation of Training Programs to re-skill or up-skill the workforce to achieve organizational needs organically.
The L&D analyst, might be tasked to identify roles for which demand is declining to train individuals to move into adjacent occupations with higher demand, such as training secretaries to become compliance oficers. By comparing occupations, the analyst identifies the usual skill gaps between the roles. In this way, it is possible to determine the optimal course set to obtain the missing diferentiator skills to achieve basic literacy in the new roles. By considering the costs in terms of fees for training, the analyst can also determine the optimal training strategy and the programs to carry out such transformation.
One of the primary challenges encountered was with the Skills Extractor Algorithm (SEA), which at times provided incorrect skill extractions. This issue arose from the need for the tool to operate quickly enough to ensure computational feasibility. Consequently, skills were sometimes omitted or incorrectly extracted based on the phrasing of job descriptions. To curb the incorrectly extracted skills, we found that ranking extracted skills by their popularity typically relegated incorrect extractions to lower tiers. Furthermore, relying on ground truth SkillsFuture (SFw) skills and updating them with labour market data helped to purge incorrect extractions. Additionally, using the GPT-4 Large Language Model (LLM) to prune skills, based on its general knowledge, further reduced the number of incorrect extractions, albeit at the cost of occasionally removing some correct ones.
We also learned that rankings, rather than absolute values, provide more intuitive insights to users. Given the wide variation in the popularity of occupations, absolute numbers often do not convey meaningful information. For instance, in our job posting dataset for 2018, the occupation "Marketing Manager" had 2039 extractions, while "Management Consultant" had only 13. In that same year, the skill "Business Opportunities Development" appeared 314 times for "Marketing Manager," making it the 13th most popular skill for that occupation, whereas for "Management Consultant," the same skill appeared 10 times, ranking as the most popular skill for that occupation.
We faced the dilemma of whether to feature jobs that are not typically ofered in the job market, such as "Legislator," through synthetic methods such as large language models (LLMs). The absence of such jobs in the data is likely due to the nature of hiring for those occupations rather than a lack of data. However, given the definitions of these occupations and their associated tasks, it is feasible to include them in the knowledge graph through synthetic means.
Another challenge was dealing with changes in skill or occupation taxonomy. Should the SEA extraction tool adopt a new list of skills (e.g., new Technical Skills and Competencies (TSCs) included) or if a new version of the SSOC is released, a re-extraction of SSOCs or skills across all historical job postings would be necessary. This re-extraction is required for consistency and to ensure fair comparison.
Finally, Section 4.2 discusses the use of 1/(, ) as a score to compute the aggregated efect within a sliding time window and diferentiator scores. We found that using a succession of 1 provides an over-estimation of the diference between the highest-ranking skills compared to directly comparing rankings (, ). However, the variation in the number of elements across rankings makes the direct use of (, ) challenging, particularly when a skills-occupation pair is absent in certain time periods. This 1 approach helps address the problem of missing rankings, ofering a practical solution.
This research contributes to a deeper understanding of workforce dynamics and supports timely and informed decisionmaking in skills matching and career planning initiatives. The creation of the Jobs Skills Knowledge Graph delineates a dynamic repository of occupational roles, skills, and associated entities within the realm of continuous adult education. Leveraging occupation and skill classifiers serves as foundational mechanisms for constructing a knowledge base of occupations and competencies. Central to this endeavour is the ongoing refinement and updating of relationships between occupations and skills, which serves as cornerstone for deriving ever-green graph analytics insights. The collaboration of LLMs and human experts further refines the end-result. An evaluation of the results of this methodology is presented in terms of precision and recall, after the human-LLM layered drops the more unrelated skill associations. The evaluation results throw light on which occupations major groups (SSOC 1D) are more accurately represented by the methodology proposed.
Through the linkages derived between occupations and skills, it is possible to derive a network purely of skills (or occupations). Such a network allows us to investigate the inter-relationship of skills. This is particularly relevant for the linkage between Technical Skill Competencies (TSCs) and the ever growing list of Applications/Tools that allow the skill to be applied.
Investigating the network using centrality metrics or clustering methods could enable us to identify key clusters of skills that are highly transferable or that are often required together. These can serve as strong recommendations for training providers to adapt their oferings to the ever-changing landscape of the needs of the economy.
Robust evaluation of the results remains only partially addressed. Although the precision and recall results in Table 7 give a quantified figure of how well the occupation categories (SSOC 1D) match expert knowledge, this does not tell us if the diference is due to an evolution in the job definition in Singapore, or due to limitations in the approach. Potential verifications could involve leveraging datasets from other countries (which would need to be contextualised in the taxonomies of occupation and skills used in this paper) or leveraging on another round of costly Labour Market experts to validate the findings with updated validated relationships. Both approaches are highly taxing in terms of efort and cost, however, rounds of human validation of small samples can give indications of the benefits of the method for smaller costs.
Lastly, recent growing interest in applications between
knowledge graphs and large language models, underline the
importance of reliable knowledge bases to power such
models through graph retrieval-augmented generation
(GraphRAG) [
The authors would like to gratefully acknowledge the support of SkillsFuture Singapore in facilitating the research and application of the Jobs and Skills Knowledge Graphs. The authors also extend their sincere thanks to Ong Meng Hwee Victor for his valuable comments and help with the review of the manuscript.
Disclaimer: The content of this paper do not imply that the research outcomes are currently deployed or planned for deployment by SkillsFuture Singapore at any given time. The views and opinions expressed in this paper are solely those of the authors and do not necessarily represent those of SkillsFuture Singapore, Government Technology Agency of Singapore or A-STAR.
and deploy a proof of concept
The graph database utilized was Neo4j version 5.3, selected due to its eficiency and ease in handling relationships within complex datasets. The application programming interface (API) implemented for seamless communication and data exchange was FastAPI.
The web portal presented in Figure 5 was crafted using Streamlit, a user-friendly and interactive framework for data visualization and exploration. To ensure the accessibility and availability of the developed API and web portal, both were deployed on the Heroku platform, ofering a scalable and reliable cloud infrastructure. For the hosting of the Neo4j graph database, the AuraDB service provided by Neo4j was utilized, facilitating a cloud-native, fully managed database-as-a-service solution.