Implementing a Scalable, Redeployable and Multitiered Repository for FAIR and Secure Scientific Data Sharing: The BIG-MAP Archive

2025-12-17

AI Summary

To address scalability, secure access control, and FAIR compliance challenges in cross-organizational data sharing within large scientific consortia, this work designs and implements a cloud-native, multi-tier scientific data archiving system built on InvenioRDM. Methodologically, it introduces a novel, consortium-oriented hybrid RBAC+ABAC permission model with fine-grained authorization and a formalized upload workflow, enabling community-level isolation, sensitive-data protection, and automated publication to open repositories. The system integrates standardized metadata management, automatic format validation, and a publication-ready curation pipeline, ensuring both high customizability and cross-consortium reusability. Evaluated in European AI and materials science initiatives—including BIG-MAP, MaterialsCommons4.eu, and RAISE—the system demonstrably enhances data findability, interoperability, and reuse efficiency while maintaining strict security and compliance requirements.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Application Domains: Humanities & Computational Social Science
• Intelligent Robots: Multi-Robot Systems

Application Category

• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Semantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic data

Abstract

Data sharing in large consortia, such as research collaborations or industry partnerships, requires addressing both organizational and technical challenges. A common platform is essential to promote collaboration, facilitate exchange of findings, and ensure secure access to sensitive data. Key technical challenges include creating a scalable architecture, a user-friendly interface, and robust security and access control. The BIG-MAP Archive is a cloud-based, disciplinary, private repository designed to address these challenges. Built on InvenioRDM, it leverages platform functionalities to meet consortium-specific needs, providing a tailored solution compared to general repositories. Access can be restricted to members of specific communities or open to the entire consortium, such as the BATTERY 2030+, a consortium accelerating advanced battery technologies. Uploaded data and metadata are controlled via fine grained permissions, allowing access to individual project members or the full initiative. The formalized upload process ensures data are formatted and ready for publication in open repositories when needed. This paper reviews the repository's key features, showing how the BIG-MAP Archive enables secure, controlled data sharing within large consortia. It ensures data confidentiality while supporting flexible, permissions-based access and can be easily redeployed for other consortia, including MaterialsCommons4.eu and RAISE (Resource for AI Science in Europe).

Problem

• Designing a scalable, secure repository for scientific data sharing
• Enabling controlled access within large research consortia
• Supporting flexible redeployment for diverse collaborative initiatives

Innovation

• Cloud-based disciplinary private repository for secure data sharing
• Built on InvenioRDM with fine-grained access control permissions
• Redeployable scalable architecture tailored for large consortia