Training-Free Multi-Concept Image Editing

2026-02-24

AI Summary

This work addresses the challenge of precisely manipulating hard-to-describe visual concepts—such as facial structure and material texture—in training-free image editing while preserving identity and fine details. The authors propose a training-free multi-concept editing framework that integrates an optimized DDS with LoRA-driven concept adapters, enabling the joint exploitation of textual semantics and low-level visual cues during diffusion. For the first time, this approach achieves controllable composition of multiple visual concepts without any training. Key innovations include ordered timesteps scheduling, regularization strategies, and negative prompt guidance, which collectively enhance editing stability and fidelity. Extensive evaluations on InstructPix2Pix and ComposLoRA benchmarks demonstrate that the method outperforms existing training-free approaches both quantitatively and qualitatively.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Editing images with diffusion models without training remains challenging. While recent optimisation-based methods achieve strong zero-shot edits from text, they struggle to preserve identity or capture details that language alone cannot express. Many visual concepts such as facial structure, material texture, or object geometry are impossible to express purely through text prompts alone. To address this gap, we introduce a training-free framework for concept-based image editing, which unifies Optimised DDS with LoRA-driven concept composition, where the training data of the LoRA represent the concept. Our approach enables combining and controlling multiple visual concepts directly within the diffusion process, integrating semantic guidance from text with low-level cues from pretrained concept adapters. We further refine DDS for stability and controllability through ordered timesteps, regularisation, and negative-prompt guidance. Quantitative and qualitative results demonstrate consistent improvements over existing training-free diffusion editing methods on InstructPix2Pix and ComposLoRA benchmarks. Code will be made publicly available.

Problem

• image editing
• diffusion models
• training-free
• visual concepts
• text prompts

Innovation

• training-free
• multi-concept editing
• LoRA
• diffusion models
• concept composition