🤖 AI Summary
This work addresses the challenge faced by conventional vision-language models in fine-grained classification, particularly their difficulty in distinguishing visually similar species within the same genus or family. To overcome this limitation, the authors propose a reinforcement learning framework based on Group Relative Policy Optimization that structures the classification process as a hierarchical reasoning pipeline—progressing from species to genus to family—and incorporates an intermediate reward mechanism to guide the model toward generating interpretable and verifiable decision paths. By uniquely integrating hierarchical classification with intermediate rewards, the method achieves a state-of-the-art average accuracy of 91.7% on the Birds-to-Words dataset, substantially outperforming human experts (77.3%) and demonstrating strong cross-domain generalization capabilities on primate and marine species classification tasks.
📝 Abstract
Traditional vision-language models struggle with contrastive fine-grained taxonomic reasoning, particularly when distinguishing between visually similar species within the same genus or family. We introduce TaxonRL, a reinforcement learning approach using Group Relative Policy Optimization with intermediate rewards that decomposes the reasoning process into hierarchical taxonomic predictions. Our method incentivizes models to explicitly reason about species-level, genus-level, and family-level features before making final classifications. This structured approach is designed not only to boost accuracy but also to yield a transparent, verifiable decision-making process. On the challenging Birds-to-Words dataset, TaxonRL achieves 91.7\% average accuracy, exceeding human performance (77.3\%) while generating interpretable reasoning traces. We demonstrate strong cross-domain generalization, showing substantial gains in primate and marine species verification. Our results establish that enforcing structured, hierarchical reasoning provides a powerful and transferable framework for fine-grained visual discrimination.