Benchmarking Content-Based Puzzle Solvers on Corrupted Jigsaw Puzzles

2025-07-10

AI Summary

This work addresses the limited robustness of content-based jigsaw solvers in realistic fragment reconstruction tasks—such as archaeological artifact reassembly and shredded document restoration—under physical degradation. We establish a comprehensive benchmark encompassing three realistic degradation categories: missing fragments, edge erosion, and content corruption. To systematically analyze performance bottlenecks, we propose a deep model optimization framework integrating targeted data augmentation and fine-tuning, and rigorously evaluate state-of-the-art methods. Experiments reveal that standard solvers suffer severe accuracy degradation under high corruption levels, whereas models trained with degradation-aware augmentation achieve substantial robustness gains; notably, the Positional Diffusion model demonstrates superior cross-degradation generalization. This study provides the first systematic quantification of content-based solvers’ sensitivity to diverse physical degradations and empirically validates the efficacy of generative modeling for enhancing robustness in fragment reassembly.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Content-based puzzle solvers have been extensively studied, demonstrating significant progress in computational techniques. However, their evaluation often lacks realistic challenges crucial for real-world applications, such as the reassembly of fragmented artefacts or shredded documents. In this work, we investigate the robustness of State-Of-The-Art content-based puzzle solvers introducing three types of jigsaw puzzle corruptions: missing pieces, eroded edges, and eroded contents. Evaluating both heuristic and deep learning-based solvers, we analyse their ability to handle these corruptions and identify key limitations. Our results show that solvers developed for standard puzzles have a rapid decline in performance if more pieces are corrupted. However, deep learning models can significantly improve their robustness through fine-tuning with augmented data. Notably, the advanced Positional Diffusion model adapts particularly well, outperforming its competitors in most experiments. Based on our findings, we highlight promising research directions for enhancing the automated reconstruction of real-world artefacts.

Problem

• Evaluating puzzle solvers on corrupted jigsaw puzzles
• Assessing robustness to missing pieces and erosion
• Improving automated reconstruction of real-world artefacts

Innovation

• Introducing three types of jigsaw puzzle corruptions
• Evaluating heuristic and deep learning-based solvers
• Fine-tuning deep learning models with augmented data