Self-Repairing Recurrent Ensembles for Real-Time Recovery from Distribution Shift

2026-10-02

AI Summary

This study addresses the severe performance degradation of pretrained controllers under distribution shifts, such as sensor drift, where online expert labels are unavailable. To overcome this challenge, we propose an unsupervised online recovery framework that leverages recurrent network ensembles and sequential Kalman fusion to generate self-supervised pseudo-labels. Furthermore, we introduce a novel self-repair mechanism based on masked observations and confidence weighting, unified within a real-time fine-tuning paradigm via the RFLO algorithm for online imitation learning. Experimental results on continuous control tasks demonstrate that the proposed approach enables models to rapidly recover near-original performance following sensor shifts, significantly outperforming full-observation ensemble baselines.

Abstract

Deploying a pretrained controller exposes it to conditions that are absent from its training data. Sensor drift, outright sensor failure and accumulating measurement noise all induce a distribution shift that can collapse an otherwise competent policy; typically at a point in time where no expert is available to supply corrective labels. We present a method that lets a policy recover from such shifts online and without supervision. Our controller is an ensemble of recurrent networks, each of which observes a randomly masked subset of the observation vector, and whose Gaussian outputs are combined through sequential Kalman fusion so that confident members dominate the consensus action. At deployment, we treat this consensus as a self-supervised label and fine-tune each member towards it, scaling each member's contribution proportional to the complement of its squared Kalman gain. Gradients are computed using RFLO, an efficient and biologically plausible approximation of Real-Time Recurrent Learning, so that a parameter update follows every environment step and the policy reacts to a shift as it unfolds. On a range of simulated continuous control tasks, our approach recovers close to the original performance after a sensor shift, while ensembles that see the full observation are unable to recover. The same framework subsumes fully online interactive imitation learning: when an expert is present, the consensus label is replaced by the expert action and the identical update rule refines the policy during teleoperation.

Problem

• distribution shift
• online recovery
• unsupervised adaptation
• sensor failure
• continuous control

Innovation

• Self-Repairing Recurrent Ensembles
• Distribution Shift
• Sequential Kalman Fusion
• Random Masking
• RFLO