🤖 AI Summary
This work addresses the challenge of transfer learning for modeling physical systems across disparate temporal scales under data-scarce conditions. The authors propose a time-warping–based recurrent neural network (RNN) transfer approach that adapts a pretrained long short-term memory (LSTM) model to target systems with different dynamic time scales by rescaling the time axis. Theoretically, they demonstrate that LSTMs can accurately approximate a class of delay differential equations and retain this approximation capability under time-warping transformations. By incorporating time warping into RNN-based transfer learning—a novel contribution—the method achieves high-accuracy predictions across time spans ranging from one hour to one thousand hours in fuel moisture content forecasting, matching or surpassing state-of-the-art transfer techniques while fine-tuning only a small subset of parameters.
📝 Abstract
Dynamical systems describe how a physical system evolves over time. Physical processes can evolve faster or slower in different environmental conditions. We use time-warping as rescaling the time in a model of a physical system. This thesis proposes a new method of transfer learning for Recurrent Neural Networks (RNNs) based on time-warping. We prove that for a class of linear, first-order differential equations known as time lag models, an LSTM can approximate these systems with any desired accuracy, and the model can be time-warped while maintaining the approximation accuracy.
The Time-Warping method of transfer learning is then evaluated in an applied problem on predicting fuel moisture content (FMC), an important concept in wildfire modeling. An RNN with LSTM recurrent layers is pretrained on fuels with a characteristic time scale of 10 hours, where there are large quantities of data available for training. The RNN is then modified with transfer learning to generate predictions for fuels with characteristic time scales of 1 hour, 100 hours, and 1000 hours. The Time-Warping method is evaluated against several known methods of transfer learning. The Time-Warping method produces predictions with an accuracy level comparable to the established methods, despite modifying only a small fraction of the parameters that the other methods modify.