Problem
Research questions and friction points this paper is trying to address.
Real-World Image Restoration
Mixed Degradations
Agentic Methods
Multimodal Foundation Models
HarnessIR: Harnessing Multimodal Foundation Models for Universal Real-World Image Restoration
🤖 AI Summary
This study addresses the challenge of complex mixed degradations in real-world images, where conventional stepwise restoration paradigms relying on single models prove inadequate. To this end, we propose HarnessIR, an intelligent agent framework that transcends task-specific toolchain limitations by pioneering the use of multimodal foundation models (MFMs) as core executors. The framework establishes a closed-loop workflow encompassing perception, diagnosis, prompt composition, and verification, wherein validation feedback drives iterative optimization to achieve universal image restoration within a single inference pass. Extensive evaluations demonstrate that HarnessIR attains state-of-the-art performance on the MiO100 benchmark while exhibiting exceptional generalization capabilities and superior restoration quality across complex real-world scenarios.
📝 Abstract
Real-world low-quality images suffer from complex mixed degradations, including but not limited to noise, blur, atmospheric effects, etc. Recent agentic methods usually model real-world image restoration (Real-IR) as a sequential tool calling problem over task-specific single-degradation restoration models. This paradigm, however, is fundamentally limited because complex real-world degradations cannot be cleanly undone degradation by degradation, and the tool used for task-specific models caps the capability of the agent system. In this work, we present HarnessIR, an agentic framework for Real-IR by harnessing a multimodal foundation model (MFM) as the executor. HarnessIR consists of five stages: perception and diagnosis, on-demand tool invocation, prompt composition, execution, and verification-driven refinement. Unlike prior agentic Real-IR methods that rely on tool chains assembled from task-specific models, HarnessIR feeds the restoration requirements, the perceptual diagnosis, and the evidence into an MFM that performs restoration in a single pass, followed by verification stages to determine whether the result warrants further processing. Under our harness, off-the-shelf MFMs handle restoration tasks remarkably well, achieving state-of-the-art results on the widely used MiO100 synthetic benchmark. More importantly, by exploiting the strong generalization ability of MFMs, HarnessIR delivers compelling restoration quality on challenging real-world scenes where previous agentic IR systems often struggle. Codes is available at https://github.com/PolyU-VCLab/HarnessIR.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multimodal Foundation Models
Image Restoration
Agentic Framework
Mixed Degradations
Verification-driven Refinement
🔎 Similar Papers
No similar papers found.
