Tiny-DroNeRF: Tiny Neural Radiance Fields aboard Federated Learning-enabled Nano-drones

2026-03-02

AI Summary

This work proposes Tiny-DroNeRF, a lightweight neural radiance field model based on an optimized Instant-NGP architecture tailored for nano-drones operating under extreme resource constraints (<100 mW power and <100 MB memory). It presents the first successful implementation of NeRF training on the GAP9 ultra-low-power microcontroller. To overcome the limitations of onboard memory and computational capacity, the authors introduce a federated learning framework specifically designed for nano-drones, enabling collaborative multi-agent training and 3D scene reconstruction. Experimental results demonstrate a 96% reduction in memory footprint with only a 5.7 dB loss in reconstruction fidelity. Furthermore, the proposed federated learning approach significantly enhances reconstruction quality by alleviating the data storage bottleneck inherent to individual devices.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Sub-30g nano-sized aerial robots can leverage their agility and form factor to autonomously explore cluttered and narrow environments, like in industrial inspection and search and rescue missions. However, the price for their tiny size is a strong limit in their resources, i.e., sub-100 mW microcontroller units (MCUs) delivering $\sim$100 GOps/s at best, and memory budgets well below 100 MB. Despite these strict constraints, we aim to enable complex vision-based tasks aboard nano-drones, such as dense 3D scene reconstruction: a key robotic task underlying fundamental capabilities like spatial awareness and motion planning. Top-performing 3D reconstruction methods leverage neural radiance fields (NeRF) models, which require GBs of memory and massive computation, usually delivered by high-end GPUs consuming 100s of Watts. Our work introduces Tiny-DroNeRF, a lightweight NeRF model, based on Instant-NGP, and optimized for running on a GAP9 ultra-low-power (ULP) MCU aboard our nano-drones. Then, we further empower our Tiny-DroNeRF by leveraging a collaborative federated learning scheme, which distributes the model training among multiple nano-drones. Our experimental results show a 96% reduction in Tiny-DroNeRF's memory footprint compared to Instant-NGP, with only a 5.7 dB drop in reconstruction accuracy. Finally, our federated learning scheme allows Tiny-DroNeRF to train with an amount of data otherwise impossible to keep in a single drone's memory, increasing the overall reconstruction accuracy. Ultimately, our work combines, for the first time, NeRF training on an ULP MCU with federated learning on nano-drones.

Problem

• nano-drones
• neural radiance fields
• ultra-low-power computing
• 3D reconstruction
• resource-constrained devices

Innovation

• Tiny Neural Radiance Fields
• Federated Learning
• Ultra-Low-Power MCU
• Nano-drones
• 3D Scene Reconstruction