MoEIoU: Rethinking Bounding-Box Regression as a Mixture of Experts

2026-05-30

AI Summary

This work addresses the limitation of existing IoU-based loss functions, which employ fixed geometric penalty terms and thus struggle to adapt to the dynamic optimization requirements of bounding box regression across different training stages. To overcome this, the authors propose MoEIoU, the first IoU loss grounded in a Mixture-of-Experts (MoE) mechanism, formulating regression as a mixture-of-experts problem. MoEIoU leverages the log-sum-exp function to smoothly and adaptively aggregate errors in overlap, center alignment, and aspect ratio, while integrating a curriculum learning strategy to dynamically adjust the weights of these error components during training. Extensive experiments on PASCAL VOC, HRIPCB, and MS COCO demonstrate that MoEIoU consistently outperforms current state-of-the-art losses, yielding faster convergence, improved localization accuracy, and notable performance gains across multiple YOLO architectures.

Abstract

Bounding-box regression is a fundamental component of object detection, playing a critical role in precise object localization. Existing Intersection-over-Union (IoU)-based loss functions extend the IoU objective by incorporating geometric penalties, such as center-distance and aspect-ratio mismatch, to improve bounding-box regression. However, these penalties typically remain fixed throughout training and do not account for the optimization dynamics in which predicted boxes initially exhibit large center-distance and shape errors, with later stages focusing on improving overlap with the ground truth. To address this limitation, we introduce MoEIoU, a mixture-of-experts based regression loss that jointly models overlap, center alignment, and aspect-ratio mismatch. MoEIoU aggregates these components using a log-sum-exp function, which emphasizes the dominant localization error while maintaining smooth contributions from other terms. Additionally, a curriculum-based weighting schedule is employed to prioritize correcting box position and shape in early training stages and improving overlap in later stages. We evaluated proposed MoEIoU on PASCAL VOC, HRIPCB, and MS COCO using multiple YOLO architectures, along with large-scale simulation experiments. It consistently outperforms standard and recent state-of-the-art losses, demonstrating faster convergence and improved localization accuracy. We further show that this adaptive aggregation improves existing IoU-based losses, yielding consistent gains and providing more effective optimization guidance for bounding-box regression in object detection frameworks.

Problem

• bounding-box regression
• IoU-based loss
• geometric penalties
• optimization dynamics
• object detection

Innovation

• Mixture of Experts
• IoU-based loss
• adaptive aggregation
• bounding-box regression
• curriculum weighting