Score
Designs and documents label taxonomies and annotation schemas for datasets—mapping concepts to categorical labels, hierarchical classes, and metric definitions across content, product, harm, skills, and other domain taxonomies. Builds governance and versioning for taxonomy management and labeling processes, and analyzes or adapts schemas to topic-specific discourse and to measure their effects on dataset quality and model performance.
To address insufficient semantic description of multidimensional aggregate/summary data, poor adaptability of metadata standards, and cross-source interoperability challenges in big data environments, this paper proposes a multidimensional data source profiling metadata model tailored for data ecosystems. Built upon RDF, the model is the first to support extensible semantic modeling of both aggregate and summary multidimensional data, enabling semantic alignment of dimensions and measures with reference knowledge graphs. It integrates multi-granularity metadata profiles—spanning source-level, attribute-level, and value-distribution characteristics. The model ensures flexible extensibility and cross-source interoperability. Experimental results demonstrate that profile generation time scales linearly with data cardinality, confirming its engineering practicality and predictable performance.
This study addresses the low accuracy of large language models (LLMs) in FAIR-compliance validation of biosample metadata. We propose a structured-knowledge-guided prompting method, integrating the CEDAR template repository, domain-specific data dictionaries, and GPT-4 to construct a metadata standards-conformance verification framework—demonstrated on human lung cancer biosamples. Experimental results show that incorporating structured knowledge significantly improves field-level standards compliance from 79% to 97% (p < 0.01), providing the first empirical evidence that structured knowledge bases can overcome performance bottlenecks inherent to purely text-based LLM prompting in metadata governance. Our approach establishes a novel paradigm for automated, high-accuracy, and interpretable FAIR metadata quality control, enabling scalable, standards-aware curation of biomedical metadata.
Identifying appropriate parent classes for novel concepts in small-scale existing taxonomies (<100 nodes) remains challenging due to sparse structural signals and lack of labeled training data. Method: This paper proposes a label-free, large language model (LLM)-driven taxonomy expansion method. Its core innovation is *code-style prompting*: explicitly encoding hierarchical semantic structure via indentation, nesting, and functional abstraction—programming conventions that enable zero- or few-shot LLM comprehension and relational reasoning over taxonomy hierarchies. The approach integrates taxonomy-aware input representations with structured code prompts, eliminating reliance on large-scale annotated or self-supervised data construction. Results: Evaluated on five cross-domain real-world benchmarks, the method achieves an average 12.6% absolute improvement in parent-class prediction accuracy over state-of-the-art methods. Gains are especially pronounced in extremely small-scale settings (30–80 nodes), demonstrating robustness where conventional supervised or embedding-based approaches falter.
Addressing the challenge of multi-label automatic annotation for large-scale, hierarchical classification systems in software requirements engineering, this study proposes a sentence-level zero-shot classification paradigm to circumvent the high annotation costs associated with supervised training. We introduce the first industrial-scale requirements annotation benchmark comprising 769 taxonomy labels and systematically demonstrate a strong negative correlation between the number of taxonomy leaf nodes and classification recall. We further propose a zero-shot multi-label classification method leveraging SBERT sentence embeddings, achieving significant improvements in recall. Empirical evaluation reveals that hierarchical strategies yield no consistent performance gain across settings. Our work validates the effectiveness and feasibility of zero-shot learning for large-scale requirements classification, offering a scalable, low-human-effort automation solution for requirements tracing. (138 words)
Scientific process descriptions are often embedded in unstructured text, hindering reproducibility, comparison, and automation. To address this challenge, this work presents the first cross-disciplinary, expert-driven repository of structured scientific process schemas, encompassing 16 expert-annotated patterns across five domains. Through a human-in-the-loop workflow, candidate schemas generated by large language models were iteratively refined via domain expert feedback, yielding reusable fields such as inputs, outputs, steps, and parameters. The resulting schemas are formalized in both JSON Schema and SHACL formats and accompanied by an integrated toolchain. The project also releases a comprehensive dataset—including schemas, intermediate artifacts, review records, and analysis scripts—to support knowledge graph construction, semantic publishing, and cross-study comparison.
This work addresses the challenge of efficiently constructing a comprehensive and well-structured taxonomy of artificial intelligence skills and tasks from massive hiring data. To this end, the authors propose TaxonomyBuilder, a framework that integrates systematic data filtering, clustering algorithms, and large language model–enhanced hierarchical label generation to automatically derive domain-specific taxonomies from curated, high-quality data subsets. Experimental results demonstrate that taxonomies built from filtered data exhibit significantly broader coverage and superior structural coherence compared to those generated from raw, unfiltered data using existing methods. The study thus establishes a novel paradigm for data-driven, automated taxonomy construction in specialized domains.
This study addresses the challenge in attributed graph schema design of whether repeatedly occurring descriptive attributes should be embedded within nodes or externalized as reusable metadata. Building upon Fifth Normal Form (5NF), the authors propose a principled decision framework that systematically identifies metadata candidates based on semantic criteria rather than mere repetition frequency. The approach classifies attributes into characteristic nodes, embedded properties, or borderline cases using five key principles: cross-element occurrence frequency, conceptual independence, lossless externalizability, reuse potential, and governance relevance. Empirical validation through a library domain case study and an entity classification task demonstrates that repetition alone is insufficient for externalization decisions—semantic judgment is essential. The proposed method significantly enhances the accuracy, consistency, and reusability of metadata modeling in graph-based systems.
This study addresses the challenge of low-quality metadata that hinders dataset discoverability and reuse, particularly in the context of large language model (LLM)-generated descriptions lacking empirical guidance on context selection and its impact on quality. Building a literature-based framework for description quality assessment, the authors conduct systematic ablation experiments across 252 real-world CSV datasets. They uncover a previously unreported “table-structure penalty” phenomenon: relying solely on table structure significantly degrades narrative quality. While representative data samples aid semantic grounding, they do not improve overall human-rated quality. The work further reveals that different LLMs exhibit consistent descriptive styles. Through LLM-as-a-judge evaluations, semantic attribute analysis, and large-scale experimentation, the study offers key recommendations for LLM-assisted data publishing: concise, relevant context yields better results than redundant input, and table structure should be used cautiously as a basis for generation.