MLentory, an FDO registry for machine learning models

Abstract

Here we introduce MLentory, an FDO registry for Machine Learning models and their corresponding workflows, from creation to deployment. MLentory relies on FAIR Digital Objects (FDOs) to improve Findability, Accessibility, Interoperability, and Reusability while also improving reproducibility and transparency. MLentory aggregates, harmonizes and FAIRifies data from various ML model and model-related repositories and platforms. Here we present the initial architecture for data extraction, transformation, and loading.

Keywords

• eol>Machine learning models
• FAIR
• FDOs
• registry1

1. Background

2. MLentory

MLentory aims at providing a registry (aka directory, inventory) for ML models and corresponding workflows, from creation to deployment. MLentory relies on FAIR Digital Objects [ 3 ] to improve Findability, Accessibility, Interoperability, and Reusability (FAIR layer in FDOs) while also improving reproducibility and transparency (operations layer in FDOs). It will rely on metadata agreements reached by the aforementioned communities, mapped (whenever needed) to schema.org [ 4 ], a lightweight approach to semantics already considered by the scientific community for datasets and software. Here we introduce MLentory architecture together with an initial proposal for ML models metadata based on schema.org. MLentory relies on data harvesting from third-party platforms, with aggregation and harmonization modules for the final shape of the ML model FDOs. A scheduler is available to keep the inventory continuously updated. The data storage corresponds to an RDF graph with an ElasticSearch module for indexing and communication with the front-end interface and corresponding RESTful services.

3. Conclusions and future work

Here we have outlined the MLentory architecture to collect, aggregate and harmonize reporting of ML models together with the initial consideration for a possible metadata schema. We aim to share our framework to improve ML model metadata, paving the way for more robust and transparent ML practices.

Acknowledgements References

This work has been partially supported by NFDI4DataScience, a consortium funded by the German Research Foundation (DFG), project number 460234259.

References

[1] Mitchell M , et al. Model Cards for Model Reporting . Proceedings of the Conference on Fairness, Accountability, and Transparency . 2019 . doi: 10 .1145/3287560.3287596

[2] Bender EM , Friedman B . Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science . Transactions of the Association for Computational Linguistics . 2018 . doi: 10 .1162/tacl_a_ 00041

[3] De Smedt K , Koureas D , Wittenburg P. FAIR Digital Objects for Science: From Data Pieces to Actionable Knowledge Units . Publications . 2020 ; 8 : 21 . doi: 10 .3390/publications8020021

[4] Guha RV , Brickley D , Macbeth S. Schema . org: evolution of structured data on the web . Communications of the ACM . 2016 ; 59 : 44 - 51 . doi: 10 .1145/2844544