Learning from Adversity: Semantic-Aware Mask Refinement through Adversarial Perturbation

2026-07-31

AI Summary

This work addresses common limitations in image segmentation models—such as ambiguous boundaries, semantic inconsistency, and structural errors—by introducing the Phoenix framework. Phoenix generates semantically aware noise through adversarial mask perturbations to simulate realistic segmentation errors and employs a contrastive learning–based tripartite refinement mechanism that simultaneously enhances intra-class feature consistency and inter-class separability. Integrating adversarial learning, embedding attacks, and relational modeling, Phoenix operates as a plug-and-play module without requiring modifications to the backbone architecture. Extensive experiments demonstrate that Phoenix consistently outperforms existing approaches across diverse segmentation tasks, delivering substantial improvements in mask quality and reliably boosting the performance of state-of-the-art models.

Abstract

Despite significant advances in image segmentation, even state-of-the-art models produce masks with imperfect boundaries, semantic inconsistencies, and structural errors. Mask refinement addresses these limitations, yet current approaches rely on simplistic synthetic noise that fails to capture the complex error patterns of real segmentation models. We introduce Phoenix, a novel framework that leverages adversarial learning to generate semantically meaningful noise patterns and contrastive learning to model refinement relationships. Our approach consists of two key innovations: (1) Adversarial Mask Perturbation, which employs embedding attacks to create semantic-aware noise that mimics real segmentation errors, and (2) Contrastive Mask Refinement Learning, which establishes a tri-directional framework that ensures feature consistency within semantic regions while maintaining separation between classes. Experiments demonstrate that Phoenix significantly outperforms existing methods across diverse tasks, while consistently enhancing state-of-the-art segmentation models with substantial improvements. Our code and project page are publicly available at https://phoenix-eccv26.github.io.

Problem

• mask refinement
• semantic inconsistency
• adversarial perturbation
• segmentation errors
• boundary imperfection

Innovation

• Adversarial Mask Perturbation
• Contrastive Mask Refinement
• Semantic-Aware Noise
• Mask Refinement
• Embedding Attacks