Practices in an Organization Using RDF-star

Abstract

An increasing number of scientists share the source code used or developed during their research (i.e., their research software) in open repositories, in order to support the results described in their publications. Recent best practices have been proposed by the community by aligning the Findable, Accessible, Interoperable and Reusable (FAIR) principles to Research Software. However, there are currently no means to assess the systematic adoption of these practices in a research organization. In this poster, we propose an automated pipeline to transform the software metadata of an organization as a Knowledge Graph in order to assess the current adoption of FAIR Research Software best practices. Our poster shows results from our own GitHub organization, the Ontology Engineering Group.

1. Introduction

Research Software (RS) [ 1] plays a crucial role in reproducing computational experiments, where

it can range from simple visualization scripts or data cleaning libraries to complex computational pipelines that deliver the main findings described in a publication. RS has become key in many application domains, ranging from Astronomy1 to Computational Biology [ 2 ]. Following the

Findable, Accessible, Interoperable and Reusable (FAIR) principles for data [3] the scientific

community has discussed and adapted FAIR to RS [ 1 ], making available guidelines and best practices for researchers and RS engineers [ 4, 5 ]. Unfortunately, while there are tools for helping developers adopt some of these practices [ 6 ], there is little work on assessing their overall adoption within a given organization.

In this poster we propose an automated pipeline that creates a Knowledge Graph (KG) of

RS metadata to quantify the adoption of FAIR best RS practices within an organization. Our

pipeline assesses online code repositories using existing software metadata extraction tools [ 7, 8 ] and combines them with RML-star mappings [ 9 ] to quantify the adoption of an illustrative set of best practices while tracking the provenance of each assertion.

The reminder of the paper first describes the best practices we focus on in Section 2, followed by the data model and mappings used to create our KG in Section 3. We show how we quantify best practices in Section 4, concluding the paper in Section 5.

CEUR Workshop Proceedings

2. Metadata requirements for FAIR RS Best Practices

Table 1 shows a summary of the ten best practices we address in our work, based on existing guidelines for publishing FAIR RS [ 5, 4 ]. The list includes requirements for all principles, emphasizing those that rely on the metadata available in a code repository (e.g., availability of a description, license, keywords, citation, usage documentation, installation instructions). The list is a subset of [ 4 ], as i) our work is scoped towards code repositories and not external metadata registries, and ii) some of the guidelines are still under discussion within the community [ 4 ].

However, we consider this list as a first step towards assessing the adoption of FAIR best practices within a research organization.

While software metadata interchange vocabularies have been proposed by the scientific

community (e.g., Codemeta2), there is no well adopted standard for describing software metadata within code repositories. Instead, diferent practices are followed for specific metadata or programming languages. For example, a common practice is to describe LICENSES is to generate a LICENSE file, although in some cases this information is found in a section of the

README file, in a configuration file (e.g., pom.xml, setup.py) or in source code files. Similarly,

the Citation File Format [ 10 ] has been increasingly adopted in thousands of research projects.

However, it is still common to find references as Bibtex text within README files to indicate the publication associated with a software project. For these reasons, we consider key including the source of each assertion as part of our KG’s metadata.

3. Creating a Research Software Metadata Knowledge Graph

2https://codemeta.github.io/terms/

RML-star mapping

Github organisation

RS code repositories

RS extracted metadata

RS metadata Knowledge Graph extract a structured and integrated metadata record. SOMEF extracts information from more than 40 metadata categories, keeping a clear provenance record of the source file where each metadata property was found (e.g., README, LICENSE.md, setup.py) the technique used in their extraction (e.g., supervised classification, regular expressions, analysis of the documentation headers) and the confidence (numeric value between 0 and 1) associated with the extraction. All metadata categories for a given software repository are included in a JSON file. 3

Finally, we apply an RML-star mapping [9] with the Morph-KGC engine [11], in order to

capture the confidence, techniques and source for each metadata property when building the

Knowledge Graph. We rely on the Software Description Ontology (SDO) [ 12] to represent RS

projects (sd:Software) their diferent releases ( sd:SoftwareVesion s) and respective metadata.

SDO extends Codemeta and Schema.org [13] for the Research Software domain, where both are gaining traction as a lightweight metadata interchange schema.

4. FAIR Research Software Best Practice Assessment

3https://somef.readthedocs.io/en/latest/output/

(b) Short and long descriptions (a) Compliance with Best Practices (c) Citations per source date, we use a named graph to store all triples in a given time stamp. For example, the following query assesses the number of citations in repositories which come from CFF files: P R E F I X s d : < h t t p s : / / w 3 i d . o r g / o k n / o / s d # > P R E F I X p r o v : < h t t p : / / w w w . w 3 . o r g / n s / p r o v # > S E L E C T ( C O U N T ( D I S T I N C T ? s o f t w a r e ) A S ? c o u n t _ s o f t w a r e ) F R O M < h t t p s : / / w 3 i d . o r g / o k n / i / g r a p h / 2 0 2 3 0 6 2 8 > W H E R E { < < ? s o f t w a r e s d : c i t a t i o n ? c i t e > > p r o v : h a d P r i m a r y S o u r c e ? s o u r c e

F I L T E R ( C O N T A I N S ( s t r ( ? s o u r c e ) , ’ . c f f ’ ) ) }

Small edits to the query may be used to assess other metadata properties. All the data,

mappings, KG, SPARQL queries and notebooks to run the pipeline are available online [15].4

5. Conclusions

In this poster we explore the FAIR RS best practices adoption within an organisation using an

automated workflow. Our approach tracks the confidence, technique and provenance used in the extraction, enabling detailed queries and opening up the way towards designing validation mechanisms (e.g., SHACL shapes) and assistants to help researchers follow the FAIR principles.

Our future work aims at simplifying the extraction pipeline by integrating the RML-star mappings in SOMEF, thus expanding the best practice list with additional metadata, and addressing incomplete metadata in the extraction process.

4https://github.com/oeg-upm/oeg-software-graph

Acknowledgments

Ana Iglesias-Molina is supported by the project Knowledge Spaces (Grant PID2020-118274RB

I00 funded by MCIN/AEI/10.13039/501100011033). Daniel Garijo is supported by the Madrid

Government (Comunidad de Madrid - Spain) under the Multiannual Agreement with Universidad

Politécnica de Madrid in the line Support for R&D projects for Beatriz Galindo researchers, in

the context of the VPRICIT, and through the call Research Grants for Young Investigators from

Universidad Politécnica de Madrid.

[12] D. Garijo, M. Osorio, D. Khider, V. Ratnakar, Y. Gil, Okg-soft: An open knowledge graph with machine readable scientific software metadata, in: 2019 15th International Conference on eScience (eScience), IEEE, 2019, pp. 349–358. [13] R. V. Guha, D. Brickley, S. Macbeth, Schema. org: evolution of structured data on the web,

Communications of the ACM 59 (2016) 44–51.

[14] T. Pellissier Tanon, Oxigraph, 2023. URL: https://doi.org/10.5281/zenodo.8034456. doi:1 0 .

5 2 8 1 / z e n o d o . 8 0 3 4 4 5 6 , if you use this software, please cite it as below. [15] A. Iglesias-Molina, D. Garijo, oeg-upm/oeg-software-graph: v1.0.0, 2023. doi: 1 0 . 5 2 8 1 / z e n o d o . 8 1 1 4 6 7 7 .

References

[1] N. P. Chue Hong , D. S. Katz , M. Barker , A.-L. Lamprecht , C. Martinez , F. E. Psomopoulos , J. Harrow , L. J. Castro , M. Gruenpeter , P. A. Martinez , et al., FAIR Principles for Research Software (FAIR4RS Principles) , 2022 . doi: 1 0 . 1 5 4 9 7 / R D A 0 0 0 6 8 .

[2] A. Prlić , H. Lapp , The plos computational biology software section , PLOS Computational Biology 8 ( 2012 ) 1-2. doi:1 0 . 1 3 7 1 / j o u r n a l . p c b i . 1 0 0 2 7 9 9 .

[3] M. D. Wilkinson , M. Dumontier , I. J. Aalbersberg , G. Appleton, M. Axton , A. Baak , N. Blomberg , J.-W. Boiten , L. B. da Silva Santos , P. E. Bourne , et al., The fair guiding principles for scientific data management and stewardship , Scientific data 3 ( 2016 ) 1 - 9 . doi:1 0 . 1 0 3 8 / s d a t a . 2 0 1 6 . 1 8 .

[4] M. Gruenpeter , S. Granger , A. Monteil , N. Chue Hong , E. Breitmoser, M. Antonioletti , D. Garijo , E. González Guardia , A. Gonzalez Beltran , et al., D4 .4 - Guidelines for recommended metadata standard for research software within EOSC , 2023 . doi: 1 0 . 5 2 8 1 / z e n o d o . 8 0 9 7 5 3 7 .

[5] P. A. Martinez , C. Erdmann , N. Simons , R. Otsuji , S. Labou , R. Johnson , G. Castelao, B. V. Boas , A.-L. Lamprecht , C. M. Ortiz , et al., Top 10 fair data & software things , 2019. doi:1 0 . 5 2 8 1 / z e n o d o . 3 4 0 9 9 6 8 .

[6] J. H. Spaaks , S. Verhoeven , E. Tjong Kim Sang , F. Diblen , C. Martinez-Ortiz , E. Etuk , M. Kuzak , B. van Werkhoven , A. Soares Siqueira , S. Saladi , A . Holding, howfairis, 2022 . URL: https://github.com/fair-software/howfairis.

[7] A. Kelley , D. Garijo , A Framework for Creating Knowledge Graphs of Scientific Software Metadata, Quantitative Science Studies ( 2021 ). doi:1 0 . 1 1 6 2 / q s s _ a _ 0 0 1 6 7 .

[8] A. Mao , D. Garijo , S. Fakhraei , Somef: A framework for capturing scientific software metadata from its documentation , in: 2019 IEEE International Conference on Big Data (Big Data) , 2019 , pp. 3032 - 3037 . doi:1 0 . 1 1 0 9 / B i g D a t a 4 7 0 9 0 . 2 0 1 9 . 9 0 0 6 4 4 7 .

[9] T. Delva , J. Arenas-Guerrero , A. Iglesias-Molina , O. Corcho , D. Chaves-Fraga , A. Dimou , RML-star: A Declarative Mapping Language for RDF-star Generation , in: International Semantic Web Conference, P&D, volume 2980 , CEUR Workshop Proceedings, 2021 . URL: http://ceur-ws. org/ Vol- 2980 /paper374.pdf.

[10] S. Druskat , J. H. Spaaks , N. Chue Hong , R. Haines , J. Baker , S. Bliven , E. Willighagen , D. Pérez-Suárez , O. Konovalov , Citation File Format, 2021 . doi:1 0 . 5 2 8 1 / z e n o d o . 5 1 7 1 9 3 7 .

[11] J. Arenas-Guerrero , D. Chaves-Fraga , J. Toledo , M. S. Pérez , O. Corcho , Morph-KGC: Scalable knowledge graph materialization with mapping partitions, Semantic Web ( 2022 ). doi:1 0 . 3 2 3 3 / S W - 2 2 3 1 3 5 .