TriCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering

2026-08-04

AI Summary

This work addresses the challenges of low accuracy and lack of multimodal data in fine-grained aircraft classification on edge devices using only RGB images. To overcome these limitations, the authors propose a method that generates structure-preserving pseudo-thermal and pseudo-LiDAR depth maps from a single RGB image and integrates them into a lightweight Qwen3-VL backbone for trimodal fusion. The approach leverages task-specific instructions, rotation-invariant supervised fine-tuning, and a GSPO alignment strategy to map predictions onto an expert-defined taxonomy grounded in engineering semantics such as propulsion type and airframe configuration. Experiments demonstrate 88.33% and 78.00% classification accuracy on validation and test sets, respectively, with 94% of outputs in parseable format. After 4-bit quantization, the model runs efficiently at 1.48 seconds per sample on 8GB edge devices, marking the first implementation of RGB-driven trimodal semantic aircraft classification at the edge.

Abstract

Edge platforms used for aerial observation must interpret aircraft imagery under limited memory, limited compute, and intermittent connectivity. This setting is difficult for standard RGB-only recognition models and general-purpose vision-language models, especially when calibrated thermal and LiDAR aircraft data are unavailable. We present TriCLE, an application-oriented tri-modal vision-language system for aircraft taxonomic grouping under edge constraints. From a single RGB aircraft image, TriCLE generates a structure-preserving FLIR-style thermal view and a pseudo-LiDAR depth projection, then fuses the aligned views with task instructions in a compact Qwen3-VL backbone. The model is aligned to an expert aircraft taxonomy based on propulsion, airframe family, size, design era, and configuration, so its outputs reflect engineering-relevant similarity rather than only surface appearance. We evaluate supervised fine-tuning, rotation-preserving SFT, and three policy-alignment strategies: GRPO, GSPO, and DAPO. Sequence-level GSPO gives the strongest validation performance, reaching 88.33\% validation accuracy and 0.91 weighted F1 on valid aircraft outputs. On a held-out aircraft test partition, GSPO achieves 78.00\% accuracy and 0.793 weighted F1 while preserving 94.00\% parseable output formatting. After 4-bit quantization and attention-memory optimization, the aligned 4B model fits an 8GB deployment target and processes each tri-modal triplet in 1.48 seconds. These results support TriCLE as a practical prototype for interpretable, edge-feasible aircraft grouping, while emphasizing the need for further validation on real aligned thermal and LiDAR sensor streams.

Problem

• edge computing
• fine-grained clustering
• aircraft taxonomy
• vision-language reasoning
• multimodal perception

Innovation

• tri-modal reasoning
• edge-deployed clustering
• pseudo-thermal and pseudo-LiDAR generation
• vision-language alignment
• fine-grained aircraft taxonomy