Toward Unified Fine-Grained Vehicle Classification and Automatic License Plate Recognition

2026-04-06

AI Summary

Existing vehicle attribute extraction methods are limited in real-world surveillance scenarios due to idealized assumptions, insufficient attribute coverage, and the decoupling of fine-grained classification from license plate recognition. This work proposes a unified deep learning framework that jointly models vehicle color, make, model, type, and license plate recognition, introducing the large-scale real-world surveillance dataset UFPR-VeSV. This dataset comprises 24,945 images annotated with 13 colors, 26 makes, 136 models, 14 vehicle types, and both license plate text and corner coordinates—marking the first effort to synchronize such comprehensive annotations. Leveraging infrared image processing and cross-attribute validation via license plate information, the approach addresses practical challenges including multi-colored bodies, infrared imaging, and differentiation of same-platform models, demonstrating the effectiveness and potential of integrating fine-grained vehicle classification with automatic license plate recognition in real-world applications.

Abstract

Extracting vehicle information from surveillance images is essential for intelligent transportation systems, enabling applications such as traffic monitoring and criminal investigations. While Automatic License Plate Recognition (ALPR) is widely used, Fine-Grained Vehicle Classification (FGVC) offers a complementary approach by identifying vehicles based on attributes such as color, make, model, and type. Although there have been advances in this field, existing studies often assume well-controlled conditions, explore limited attributes, and overlook FGVC integration with ALPR. To address these gaps, we introduce UFPR-VeSV, a dataset comprising 24,945 images of 16,297 unique vehicles with annotations for 13 colors, 26 makes, 136 models, and 14 types. Collected from the Military Police of Paraná (Brazil) surveillance system, the dataset captures diverse real-world conditions, including partial occlusions, nighttime infrared imaging, and varying lighting. All FGVC annotations were validated using license plate information, with text and corner annotations also being provided. A qualitative and quantitative comparison with established datasets confirmed the challenging nature of our dataset. A benchmark using five deep learning models further validated this, revealing specific challenges such as handling multicolored vehicles, infrared images, and distinguishing between vehicle models that share a common platform. Additionally, we apply two optical character recognition models to license plate recognition and explore the joint use of FGVC and ALPR. The results highlight the potential of integrating these complementary tasks for real-world applications. The UFPR-VeSV dataset is publicly available at: https://github.com/Lima001/UFPR-VeSV-Dataset.

Problem

• Fine-Grained Vehicle Classification
• Automatic License Plate Recognition
• surveillance images
• real-world conditions
• vehicle attributes

Innovation

• Fine-Grained Vehicle Classification
• Automatic License Plate Recognition
• Infrared Surveillance
• Multi-Attribute Annotation
• Real-World Dataset