json schema design

In the context of open science, there is an urgent need to implement the FAIR principles (Findable, Accessible, Interoperable, Reusable) in scientific data management, yet systematic guidance on achieving FAIR compliance through big data software reference architectures (SRAs) remains lacking. Method: We conducted a rigorous systematic literature review, screening 323 publications—including those from authoritative databases and expert recommendations—and performed structured data extraction and evaluation aligned with predefined research questions. Contribution/Results: The study identifies seven generic FAIR-compliant SRAs, thirteen scenario-specific FAIR pipelines, and three fully FAIR-compatible SRAs. It uncovers critical bottlenecks in metadata standardization, cross-platform interoperability, and long-term reusability. Furthermore, it establishes the first classification framework and empirical evaluation system for FAIR-oriented big data SRAs, thereby filling a significant research gap and providing a methodological foundation and strategic direction for future SRA design, policy formulation, and tool development.

To address the error-prone and inefficient manual rewriting of data transformation logic upon JSON Schema evolution, this paper proposes a type-directed, top-down program synthesis approach for automatically generating semantics-preserving JSON Schema converters. Our method integrates type inference, semantic constraint modeling, a rewrite system, and intermediate representation (IR)-driven code generation to guarantee lossless data transformation and formal verifiability. It natively supports complex nested schemas and synthesizes correct, efficient, and human-readable Python and JavaScript conversion code. We evaluate our approach on real-world API configuration schemas and healthcare data integration scenarios, demonstrating its safety—via formal guarantees and empirical validation—its practical utility in industrial settings, and its generalizability across diverse schema evolution patterns. Experimental results confirm high accuracy, robustness to structural changes (e.g., field additions, type refinements, nested object restructuring), and scalability to large, deeply nested schemas.

Current structured data modeling and cross-format schema mapping lack accessible, low-threshold tools—particularly hindering non-expert users. This paper proposes a hybrid approach synergizing large language models (LLMs) with deterministic rule-based processing: LLMs interpret natural-language requirements to generate or refine JSON Schema, while a verifiable rule engine performs high-precision, scalable schema mapping across multiple formats (JSON, CSV, XML, YAML). The method is implemented in the open-source tool MetaConfigurator, supporting visual schema modeling and automated code generation. Empirical evaluation in the chemistry domain demonstrates substantial reductions in modeling barriers, significant improvements in schema construction efficiency and mapping accuracy, and—critically—the first end-to-end data schema engineering solution that is natural-language-driven, flexible, and formally reliable.

This work addresses the lack of semantic interoperability in structured data (e.g., JSON, YAML, CSV) within scientific workflows, which hinders consistent interpretation. The authors introduce an RDF authoring view in the MetaConfigurator editor that leverages AI-assisted generation of RML mappings to automatically transform structured data into RDF. The system supports triple editing, SPARQL querying, and knowledge graph visualization. Key innovations include the first integration of large language models for natural language-to-SPARQL translation, bidirectional synchronization between JSON-LD and RDF triples, and ontology-aware IRI auto-completion. Demonstrated on MOF synthesis experiments, the approach successfully converts JSON protocols into semantic knowledge graphs, enabling interactive exploration of relationships between experimental conditions and outcomes, thereby significantly lowering the barrier to adopting Semantic Web technologies.

To address insufficient standardization, poor domain adaptability, and practical challenges in implementing FAIR principles for research data metadata in open science, this paper proposes a knowledge engineering–based metadata templating approach. It formalizes domain-specific metadata standards as reusable, logically inferable knowledge templates; designs lightweight, dual-mode (Web form and spreadsheet) acquisition interfaces with real-time validation; and enables cross-platform intelligent integration via declarative knowledge representation. The method has been adopted by multiple international scientific consortia to establish standardized metadata frameworks and has driven the development of over ten data annotation systems. Empirical outcomes demonstrate significant improvements in metadata syntactic and semantic consistency, domain alignment, and cross-platform interoperability. By embedding domain knowledge into machine-processable templates, the approach delivers a scalable, maintainable, knowledge-driven paradigm for FAIR data infrastructure.

This work proposes a novel dataset discovery framework that leverages citation contexts from scientific papers to better capture the semantic intent behind research queries, addressing the limitations of existing dataset search engines that rely primarily on metadata and keyword matching and consequently suffer from low recall. By treating citation context as the core signal—combined with large-scale context extraction, large language model–guided pattern recognition, and provenance-preserving entity resolution—the approach significantly reduces dependence on incomplete or inconsistent metadata. Evaluated on eight computer science queries, the method achieves an average normalized recall of 47.47% (peaking at 81.82%), substantially outperforming Google Dataset Search and DataCite Commons. The framework’s novelty and practical utility have been affirmed by domain experts across multiple disciplines.

Scientific process descriptions are often embedded in unstructured text, hindering reproducibility, comparison, and automation. To address this challenge, this work presents the first cross-disciplinary, expert-driven repository of structured scientific process schemas, encompassing 16 expert-annotated patterns across five domains. Through a human-in-the-loop workflow, candidate schemas generated by large language models were iteratively refined via domain expert feedback, yielding reusable fields such as inputs, outputs, steps, and parameters. The resulting schemas are formalized in both JSON Schema and SHACL formats and accompanied by an integrated toolchain. The project also releases a comprehensive dataset—including schemas, intermediate artifacts, review records, and analysis scripts—to support knowledge graph construction, semantic publishing, and cross-study comparison.

Although scientific data increasingly adhere to the FAIR principles and employ standardized identifiers, practical interoperability remains hindered by heterogeneity in identifier systems and data models. This work proposes and implements two synergistic tools—Babel and ORION—to bridge this gap. Babel constructs clusters of equivalent identifiers through mapping-based clustering and exposes them via a high-performance quantitative API, while ORION standardizes heterogeneous knowledge bases by aligning them to a community-governed common data model. Together, they systematically address the longstanding disconnect between the FAIR “Interoperable” principle and its real-world implementation. The integration of these tools has enabled the construction of a fully interoperable knowledge base, substantially enhancing cross-resource data integration and query capabilities. The resulting framework is publicly available.

Existing approaches to JSON Schema inclusion checking struggle to balance efficiency and completeness: rule-based methods are efficient but incomplete, while instance-generation techniques are complete yet computationally expensive. This work proposes a refutation normalization technique that synergistically combines the efficiency of rule-based reasoning with the completeness of instance generation, enabling fast and reliable inclusion checking through optimized logical inference paths. Evaluated on both real-world and synthetic datasets, the proposed method significantly outperforms state-of-the-art tools, achieving theoretical completeness while substantially improving verification efficiency. The approach effectively supports complex practical applications and advances the practical applicability boundary of JSON Schema validation technologies.

Production agent frameworks (OpenAI Function Calling, Anthropic Tool Use, MCP) transmit tool schemas as JSON, a format designed for machine parsing, not for interpretation by language models. For small models (4B-14B), this protocol mismatch accounts for the majority of tool-use failure at production catalog sizes. We present TSCG, a deterministic tool-schema compiler that resolves this mismatch at the API boundary, converting JSON schemas into token-efficient structured text without model access, fine-tuning, or runtime search. TSCG combines eight composable operators with a formal compression bound (>=51% on well-formed schemas). On TSCG-Agentic-Bench (about 19,000 calls, 12 models, 5 scenarios), TSCG restores Phi-4 14B from 0% to 84.4% accuracy at 20 tools (90.3% at 50 tools) and achieves 108-181% accuracy-retained ratio across three models on BFCL. Format-versus-compression decomposition (R^2=0.88 -> 0.03) establishes representation change as the dominant mechanism. Per-operator isolation across three frontier models reveals three distinct operator-response profiles: operator-hungry (Opus 4.7), operator-sensitive (GPT-5.2), and operator-robust (Sonnet 4), providing per-model deployment guidance. Scaling experiments show accuracy advantages persisting on heavy production MCP schemas (+5.0 pp at about 10,500 input tokens) despite saturation on light synthetic catalogs, with 52-57% token savings throughout. The synthetic benchmark generalizes to real MCP schemas within 0.1 accuracy points. TSCG ships as a 1,200-line zero-dependency TypeScript package.