provenance tracking

Capturing, linking, and recording the origin and transformation history of data and artifacts to enable reproducibility, auditing, and responsible downstream use by producing verifiable audit trails and source linkages.

provenancetracking

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This work addresses the lack of traceable and tamper-resistant transparency mechanisms in large language models (LLMs) deployed in high-stakes decision-making contexts, which undermines accountability. To bridge this gap, the paper introduces the first LLM lifecycle auditing framework that integrates technical provenance with governance records. It proposes a reference architecture enabling cross-organizational traceability and implements a lightweight, open-source Python-based auditing layer. By leveraging append-only logs, event emitters, structured metadata, and an auditor interface, the system seamlessly integrates into existing LLM workflows with minimal intrusiveness. This design ensures complete, tamper-evident traceability across critical stages—including training, deployment, and monitoring—thereby facilitating robust accountability and responsibility attribution throughout the model’s lifecycle.

accountabilityaudit trailsgovernance

Towards dimensions and granularity in a unified workflow and data provenance framework

Apr 15, 2025
TA
Tanja Auge
🏛️ University of Regensburg | University of Rostock | Wismar University of Applied Sciences | Rostock University Library

In scientific provenance, the disconnection between workflow and data provenance—coupled with inconsistent dimensions and granularity—undermines trustworthiness and reproducibility. To address this, we propose the first unified framework that systematically integrates workflow and data provenance. Our approach introduces a dimension-granularity joint representation model, formally defines the W7+1 provenance problem, and enables domain-adaptable, end-to-end, fine-grained provenance modeling across the full research lifecycle. Evaluated on representative biomedical use cases, the framework achieves traceability from raw data and analytical steps to final results, significantly enhancing transparency, verifiability, and cross-study reproducibility. The core innovation lies in the first-ever orthogonal co-modeling and unified resolution of workflow and data provenance along both dimensional axes (e.g., who, what, when) and granularity levels (e.g., task-level, operation-level, byte-level).

Address dimensions and granularity in provenance trackingDevelop unified framework for workflow and data provenanceEnhance research credibility and reproducibility via provenance

This study addresses the longstanding fragmentation in software artifact traceability research, characterized by incomplete linkages, ambiguous techniques, and disconnected application contexts. Through a systematic literature review, it constructs the first comprehensive traceability landscape encompassing 22 artifact types and 23 relationship kinds, and introduces a technology decision map, a standardized evaluation benchmark, and a role-oriented dynamic path alignment framework. The work uncovers critical challenges: a pervasive code-centric bias, a reproducibility crisis stemming from only 37% of studies releasing open-source artifacts, and a significant adoption gap with 95% of proposed tools never deployed in industry. In response, it offers targeted strategies to bridge these gaps, establishing a unified knowledge foundation for future research and practical implementation in traceability.

artifact associationssoftware artifactssoftware traceability

Existing automated approaches for mapping cyber threat intelligence (CTI) to MITRE ATT&CK lack supporting evidence, provenance tracking, and validation history, making their credibility difficult to assess. This work proposes the first knowledge graph–driven framework for CTI governance that enables auditable management of TTP assertions through fine-grained evidence preservation, complete provenance chains, versioned trust decisions, and lossless revocation mechanisms. The framework integrates multi-extractor collaborative verification, assertion aggregation, consensus modeling, and policy-driven validation, all underpinned by versioned knowledge graph management. Evaluated on 65 CTI reports comprising 5,303 sentences, the approach achieves a precision of 90.6% under six-party consensus and efficiently supports seven categories of audit queries concerning provenance, trustworthiness, and versioning.

Cyber Threat IntelligenceMITRE ATT&CKprovenance

Trustworthy Provenance for Big Data Science: a Modular Architecture Leveraging Blockchain in Federated Settings

May 30, 2025
NG
Nicola Giuseppe Marchioro
🏛️ University of Trento | Oak Ridge National Laboratory | University of Chicago | Argonne National Laboratory

To address challenges of insufficient scientific data provenance integrity and weak cross-organizational interoperability in multi-institutional collaborative research, this paper proposes a federated provenance architecture integrated with a permissioned blockchain. The architecture adopts a modular, domain-agnostic design, incorporating persistent identifiers (PIDs), versioned provenance graph modeling, and federated computation mechanisms—ensuring decentralized interaction while guaranteeing immutability, long-term auditability, and cross-platform verifiability of provenance data. Unlike existing approaches, our work is the first to deeply embed a permissioned blockchain into the federated provenance workflow, thereby overcoming provenance consistency bottlenecks imposed by organizational boundaries. Evaluation of a prototype system demonstrates significant improvements in transparency, accountability, and reproducibility of cross-institutional research data, establishing foundational infrastructure for trustworthy large-scale scientific data analysis.

Ensuring trustworthiness in collaborative big data scienceGuaranteeing data integrity using blockchain in federated environmentsTracking provenance in distributed multi-institutional research settings

Latest Papers

What's happening recently
View more

This work addresses the problem of providing provably correct snapshot-equivalent change capture and replay for continuously written databases without relying on global snapshots or locking mechanisms. It introduces the “authenticated virtual slice” model, which combines interleaved log scanning with watermark-based positioning to enable incremental authentication over primary key ranges and advancing frontiers while preserving source continuity. For the first time, the paper formally defines and machine-verifies snapshot equivalence for DBLog, presenting a rigorous Isabelle/HOL proof that all well-formed executions satisfy per-key replay equivalence within their designated frontiers and key ranges, and that the authentication process yields valid virtual slices.

change-data-capturecorrectness formalizationdatabase replication

This work addresses the challenge of large language models generating row-level data in cross-Wiki table construction without verifiable source support. To mitigate this issue, the authors propose the first auditable framework that enforces separation of write permissions between curators and auditors, introduces a row-level source citation gating mechanism, and defines a comprehensive set of 12 audit categories spanning coverage keys, schema alignment, and source roles. This design ensures that every generated table row is explicitly grounded in and traceable to credible sources. Experimental evaluation on a benchmark of 51 instances demonstrates substantial improvements: source-front precision increases by 42% (from 0.356 to 0.505) and F1 score rises by 35% (from 0.334 to 0.451), significantly enhancing both source accuracy and overall auditability.

auditable curationcitation accuracycross-wiki tables

This study addresses the lack of empirical analysis on how quality defects in requirements documents affect the performance of automated trace link recovery (TLR). For the first time, it systematically annotates 28 types of requirements quality defects across 189 use cases from two datasets, evaluates five state-of-the-art TLR methods, and analyzes their performance impacts using both statistical significance and effect size measures. The findings reveal that specific defect types differentially influence TLR effectiveness: while certain defects significantly degrade performance, others unexpectedly enhance it. Beyond identifying key factors that either hinder or facilitate TLR accuracy, this work demonstrates that the choice of TLR method should be strategically tailored to the quality characteristics of the underlying requirements documentation.

empirical studyquality defectsrequirements quality

This study addresses the challenges of traceability in software engineering—stemming from fine-grained artifacts, heterogeneity of work products, and ambiguous responsibilities—by proposing Taxonomic Trace Links (TTL) as a complementary mechanism to traditional trace links. TTL leverages domain ontologies and taxonomies, integrated with automated classifiers, to establish early and structured traceability relationships among requirements, business use cases, and test cases. Empirical validation in an industrial case study at Ericsson demonstrates that TTL effectively supports traceability in real-world settings; however, its deployment is constrained by limitations in classifier accuracy and the complexity of ontology construction. The feasibility and applicability boundaries of TTL are rigorously assessed through a mixed-methods approach combining quantitative link evaluation with qualitative feedback from focus groups.

artifact structureempirical evaluationsoftware engineering

Design-oriented visualization research often struggles to meet conventional reproducibility standards due to its inherent subjectivity, contextual dependence, and iterative nature, thereby limiting its transparency and rigor. To address this challenge, this work proposes “traceability” as a viable alternative to traditional reproducibility. It presents the first systematic theoretical framework centered on three core components—recording, reporting, and reading—and introduces tRRRacer, a supporting tool implementing this framework. Through collaborative autoethnography, the authors reflect on practical applications of traceability in design-oriented research, demonstrating its feasibility and yielding actionable principles alongside theoretical insights. This approach offers a novel pathway to enhance the rigor and transparency of such studies without relying on strict reproducibility criteria.

design-oriented researchreproducibilitytraceability

Hot Scholars

SP

Silvio Peroni

University of Bologna
Semantic PublishingSemantic WebOpen ScienceScience of Science
ZZ

Zibin Zheng

IEEE Fellow, Highly Cited Researcher, Sun Yat-sen University, China
BlockchainSmart ContractServices ComputingSoftware Reliability
FB

Frank Breitinger

University of Augsburg
Digital forensicscybersecuritynetwork analysiscybersecurity education
TJ

Tong Jia

Peking University
AIOpsAnomaly DetectionLog AnalysisAI for Medical Research
TZ

Tiantian Zhu

Zhejiang University of Technology
Mobile SecuritySystem SecurityArtificial Intelligence