data sanitization

This study addresses a critical gap in data cleaning research—the lack of large-scale, real-world dirty datasets that hinder the effective evaluation of methods in practical settings. To bridge this gap, the authors construct the first large-scale dirty dataset comprising real postal addresses paired with their ground-truth counterparts. Leveraging this dataset, they conduct a systematic benchmarking study of state-of-the-art data cleaning approaches. Their experiments reveal substantial limitations of current methods when applied to real-world data, underscoring the need for more robust and context-aware techniques. The dataset and empirical findings not only establish a reliable benchmark for future research but also provide actionable insights to guide the development of data cleaning solutions tailored to real-world applications.

In highly regulated domains such as finance, data quality control (QC) is often fragmented into isolated preprocessing steps, undermining end-to-end trustworthy AI pipelines. To address this, we propose the first AI-driven DataOps framework that embeds QC as a system-level core component. Our framework deeply integrates rule-based engines, statistical analysis, and custom AI-powered anomaly detection across the entire data lifecycle—from ingestion and transformation to model deployment—enabling dynamic remediation, policy-configurable workflows, and end-to-end auditability. Technically, it unifies data profiling, stream processing, cloud-native storage interfaces, and a proprietary AI detection module. Evaluated in a real-world financial production environment, the framework achieves significantly improved anomaly recall, reduces manual intervention by 42%, and ensures audit completeness and full data traceability under high-throughput conditions, fully satisfying regulatory compliance requirements.

This work addresses the challenge of formally comparing the semantic differences among transaction isolation levels. To this end, it introduces Isolde, a tool that, for the first time, automatically constructs counterexamples demonstrating behavioral discrepancies between isolation levels. By modeling transaction executions through formal specifications, Isolde generates execution traces that are permitted under one isolation level but prohibited under another. This approach enables automated verification of isolation-level equivalence and falsification of claimed semantic properties. The method not only reproduces established theoretical results but also uncovers long-standing errors in the literature and previously unknown flaws in the specifications of widely used isolation checkers, thereby significantly advancing the automation and reliability of reasoning about transaction isolation semantics.

To address performance degradation, reliance on white-box model information, and high false-positive rates in dataset ownership verification, this paper proposes a black-box, lossless, and zero-false-positive verification framework. Methodologically, it introduces clean-label targeted poisoning to embed a secret key—comprising out-of-distribution samples and random labels—into the training data. Post-training, the model exhibits statistically detectable, significant responses to key samples, without requiring access to internal parameters. Our key contribution is the first non-backdoor-based verification mechanism, integrating statistical hypothesis testing with ViT/ResNet ensembles. On ImageNet-1K, it achieves >99.9% detection confidence and zero accuracy loss. Moreover, it remains robust against common defenses—including pruning, fine-tuning, and input preprocessing—outperforming existing backdoor watermarking approaches significantly.

Data cleaning remains highly manual, inefficient, and error-prone. This paper proposes the first goal-driven LLM-based framework for automatic workflow generation: given a dirty table and a target query, it end-to-end generates a minimal viable clean table along with executable cleaning steps—including deduplication, missing-value imputation, and format standardization. Our contributions are threefold: (1) We introduce the first benchmark dataset comprising annotated quadruples of (goal, dirty table, cleaning workflow, cleaned answer); (2) We design a zero-shot, multi-stage prompting framework—requiring no fine-tuning—that decomposes the task into goal column identification, data quality diagnosis, and operation-parameter generation; (3) We empirically validate that off-the-shelf LLMs possess inherent reasoning capabilities sufficient to generate high-quality, executable cleaning workflows across three major LLM families, significantly reducing human intervention.

Log anomaly detection suffers from a scarcity of high-quality, anomaly-free training data, as manual labeling is prohibitively expensive and existing automated cleaning methods neglect log structural characteristics and system-level semantics. To address this, we propose LogPurge, the first framework featuring rule-augmented, two-stage iterative purification. In Stage I, a large language model (LLM) identifies fine-grained anomaly patterns; in Stage II, system-aware rules—including timestamp consistency and template frequency constraints—are integrated for semantic validation and divide-and-conquer filtering of log segments. Evaluated on multiple benchmark datasets, LogPurge achieves an average anomaly removal rate of 98.74% while preserving 82.39% of normal sequences. Its purified logs boost downstream detector F1-scores by up to 149.72% over state-of-the-art methods, significantly enhancing detection performance.

This work addresses the challenges posed by the massive and unstructured time-series data generated in industrial cyber-physical systems (CPS), where existing preprocessing approaches rely on ad hoc scripts that suffer from poor readability, reusability, and maintainability. To overcome these limitations, the authors propose and implement CPSLint, the first domain-specific language (DSL) tailored for industrial CPS data preprocessing. CPSLint abstracts common data cleaning and validation operations into a concise and expressive syntax, enabling cross-scenario reuse and significantly improving both data preparation efficiency and team collaboration. The DSL has been open-sourced, and experimental results demonstrate that complex preprocessing tasks can be accomplished in just a few lines of code, substantially reducing redundant development efforts. CPSLint thus establishes a scalable and standardized paradigm for industrial time-series data processing.

This study addresses the compliance challenges faced by data practitioners in machine learning systems under regulations such as the GDPR and the AI Act, particularly concerning data quality. Through semi-structured interviews with practitioners in the European Union, combined with thematic analysis of regulatory texts and engineering workflows, the research systematically uncovers a structural disconnect between regulation-driven data quality requirements and ML engineering practices. It identifies five core challenges: misalignment between legal principles and engineering implementation, fragmented data pipelines, lack of purpose-built compliance tools, ambiguous accountability, and reactive responses to audits. Building on these findings, the work proposes directions for designing compliance-oriented tooling, establishing effective governance mechanisms, and fostering cultural transformation to bridge the gap between regulatory mandates and practical ML development.

Existing database implementations lack formal verification of isolation-level semantics. This work proposes a novel approach based on separation logic that directly encodes isolation levels from transactional consistency models as logical specifications, enabling comprehensive modeling and verification of all possible executions of both the database and its clients. For the first time, this method yields “free-theorem”-style formal guarantees for isolation levels: any implementation adhering to the specification automatically satisfies the corresponding isolation semantics. The entire theory has been mechanized in the Rocq proof assistant, yielding an end-to-end formally verified framework for reasoning about the correctness of database isolation levels, thereby substantially strengthening guarantees of system reliability.

This work addresses the gap between theory and practice in error detection and cleaning for tabular data by proposing and implementing an interactive web-based demonstration system that integrates error modeling, injection, and intelligent cleaning. For the first time, the system unifies machine learning–based data cleaning methods with dependency-aware error generation models within a visual platform, enabling users to upload tables, inject realistic errors, and interactively experience the cleaning process alongside interpretable explanations of the underlying mechanisms. By synergizing machine learning, database technologies, and web development, the project enhances intuitive understanding of error repair principles and provides a publicly accessible demo platform (https://cured.demo.calgo-lab.de/) that serves as an effective validation tool for both research and practical applications in data cleaning.