DishSeg24k: A Large-Scale Benchmark for Food Segmentation with Stochastic Expert Decoding

2026-07-25

AI Summary

This work addresses the challenges of dense occlusion, fine-grained category similarity, and long-tailed distribution in real-world food segmentation by introducing DishSeg24k, a large-scale dish-level segmentation benchmark comprising 24,096 images, 112,281 instances, and 278 categories. The authors propose FEAST, a novel model that formulates query decoding as a Markov decision process and employs a reinforcement learning–guided mixture-of-experts architecture to jointly explore boundary uncertainty and promote expert specialization. A dual-critic decoupling mechanism is introduced to separate task-oriented query optimization from structure-aware expert routing, effectively mitigating expert collapse under long-tailed data distributions. Experiments demonstrate that FEAST significantly outperforms existing methods on DishSeg24k, achieving absolute gains of 3.21% in mIoU, 3.68% in mDice, and 4.00% in mAcc, while also showing strong generalization on FoodSeg103.

Abstract

Food segmentation is essential for applications such as intelligent catering, dietary assessment, and recommendation. However, existing benchmarks fail to capture the complexity of real-world dining scenes. The challenges of dense inter-dish overlap, fine-grained class similarity, and extreme long-tail class distributions exceed the fidelity of current datasets. To fill this gap, we introduce \textbf{DishSeg24k}, a large-scale dish-level segmentation benchmark with 24,096 images, 112,281 instances, and 278 fine-grained categories in real-world dining environments. Based on DishSeg24k, we further propose \textbf{Food Expert-Adaptive Segmentation Transformers (FEAST)} to address these challenges. FEAST models query-based decoding as a Markov Decision Process (MDP), where each decoder layer update is treated as a sequential decision step that explores uncertainty along dish boundaries. We further redesign the decoder with a reinforcement learning (RL)-guided Mixture-of-Experts (MoE) module, in which a dual-critic decoupled optimization scheme separates task-oriented query refinement from structure-aware expert routing. This design promotes expert specialization and prevents expert collapse under long-tail category distributions. Finally, extensive experiments on DishSeg24k demonstrate the state-of-the-art performance of FEAST, which outperforms previous methods by {+3.21\%} mIoU, {+3.68\%} mDice, and {+4.00\%} mAcc, respectively. We further validate the effectiveness of FEAST on FoodSeg103. The dataset and code will be publicly released.

Problem

• food segmentation
• dense inter-dish overlap
• fine-grained class similarity
• long-tail distribution
• real-world dining scenes

Innovation

• Food Segmentation
• Mixture-of-Experts
• Reinforcement Learning
• Markov Decision Process
• Long-tail Distribution