Exploiting In-Sensor Computing for Energy-Efficient Earth Observation

2026-05-31

AI Summary

This work addresses the challenge of redundant data transmission from satellites, which arises from the surge in remote sensing data coupled with limited downlink bandwidth, leading to substantial waste of energy and communication resources. The authors propose an end-to-end intelligent on-orbit observation framework that, for the first time, deeply integrates TinyML with Sony’s IMX500 intelligent vision sensor to achieve sensing-computing convergence under stringent 8 MB resource constraints. By deploying and optimizing lightweight models—including SqueezeNet, ShuffleNetV2, and MCUNetV1—the system attains 96.68% accuracy on the EuroSAT dataset, processes images at 17.40 frames per second, consumes only 14.19 mJ per inference, and achieves an energy efficiency of 42.26 GMAC/J, significantly alleviating main processor load and minimizing unnecessary data downlink.

Abstract

The rapid growth of the satellite industry has driven a significant increase in geospatial data acquisition, highlighting a critical bottleneck: the severe disparity between the volume of collected sensor data and the limited downlink bandwidth available to ground stations. While On-Board Computing (OBC) has helped address this by pre-processing data in orbit, this article further advances the paradigm by introducing an in-sensor computing framework. We present an optimized end-to-end Earth Observation (EO) pipeline tailored for strict computational constraints by integrating TinyML techniques with the Sony IMX500 Intelligent Vision Sensor. Specifically, our approach shifts processing directly to the sensor level, offloading the computation from the primary embedded device, and effectively mitigating the downlink transmission of noisy or irrelevant data. We evaluated several efficient Convolutional Neural Networks (ConvNets), i.e., SqueezeNet, ShuffleNetV2, and MCUNetV1, on the EuroSAT dataset. Experimental results show that, despite the optimizations required for deployment on the IMX500 platform, our models maintain a competitive 96.68% accuracy while operating within its 8 MB constraints. Specifically, the models reach an average processing throughput of 17.40 FPS with a latency of 27.43 ms. Furthermore, our system profile exhibits high energy efficiency, with a low energy footprint of 14.19 mJ per inference and an efficiency rating of 42.26 GMAC/J, demonstrating its viability for in-sensor deployment.

Problem

• Earth Observation
• downlink bandwidth
• sensor data
• energy efficiency
• in-sensor computing

Innovation

• in-sensor computing
• TinyML
• Earth Observation
• energy efficiency
• intelligent vision sensor