GasTwinFormer: A Hybrid Vision Transformer for Livestock Methane Emission Segmentation and Dietary Classification in Optical Gas Imaging

2025-08-20

AI Summary

Methane emissions from livestock account for 32% of anthropogenic methane emissions, necessitating high-precision, real-time monitoring. To address this, we propose GasTwinFormer—the first hybrid vision Transformer architecture tailored for optical gas imaging (OGI). It introduces a novel Mix Twin encoder with alternating spatial-reduction global attention and local grouped attention, coupled with a lightweight LR-ASPP decoder and a multi-scale joint learning framework to simultaneously perform methane plume segmentation and feed-type classification. Evaluated on our newly established large-scale OGI dataset of cattle methane emissions—the first of its kind—GasTwinFormer achieves 74.47% mIoU and 83.63% mF1 for segmentation, with an inference speed of 114.9 FPS and only 3.348M parameters; feed classification accuracy reaches 100%. These results significantly outperform existing methods, establishing a new paradigm for precise carbon emission source attribution and dietary intervention in livestock management.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Livestock methane emissions represent 32% of human-caused methane production, making automated monitoring critical for climate mitigation strategies. We introduce GasTwinFormer, a hybrid vision transformer for real-time methane emission segmentation and dietary classification in optical gas imaging through a novel Mix Twin encoder alternating between spatially-reduced global attention and locally-grouped attention mechanisms. Our architecture incorporates a lightweight LR-ASPP decoder for multi-scale feature aggregation and enables simultaneous methane segmentation and dietary classification in a unified framework. We contribute the first comprehensive beef cattle methane emission dataset using OGI, containing 11,694 annotated frames across three dietary treatments. GasTwinFormer achieves 74.47% mIoU and 83.63% mF1 for segmentation while maintaining exceptional efficiency with only 3.348M parameters, 3.428G FLOPs, and 114.9 FPS inference speed. Additionally, our method achieves perfect dietary classification accuracy (100%), demonstrating the effectiveness of leveraging diet-emission correlations. Extensive ablation studies validate each architectural component, establishing GasTwinFormer as a practical solution for real-time livestock emission monitoring. Please see our project page at gastwinformer.github.io.

Problem

• Segmenting livestock methane emissions in optical gas imaging
• Classifying cattle dietary treatments from visual data
• Developing real-time monitoring solution for climate mitigation

Innovation

• Hybrid vision transformer with Mix Twin encoder
• Lightweight LR-ASPP decoder for feature aggregation
• Unified framework for simultaneous segmentation and classification