Hamiltonian Metric Learning and Energy-Based Training: A Dissipative Geometric Framework for Optimization

2026-10-04

AI Summary

This study addresses the limitation of conventional optimization methods that lack a geometric perspective, hindering the analysis of curvature effects on optimization trajectories. By modeling parameter optimization as a dissipative dynamical system, this work proposes the HAMLET framework, which integrates Hamiltonian mechanics with Riemannian geometry to define local metric structures. Based on variational principles, it derives geometric forces and constructs metric-compatible dissipation laws, while incorporating matrix product state reparameterization and Poincaré recurrence maps to enhance dynamical analysis capabilities. Evaluated on the MNIST benchmark, the proposed method achieves a highest average test accuracy of 97.95% along with the lowest negative log-likelihood. These results validate the effectiveness of reconstructing optimization paradigms from a geometric dynamical systems perspective.

Abstract

Optimization in machine learning is usually expressed through iterative rules that update model parameters using information from the loss landscape. In this work, we study an alternative viewpoint in which parameter optimization is treated as the evolution of a dissipative dynamical system. The model parameters are regarded as generalized coordinates, the loss acts as a potential energy, and a positive-definite metric defines the local kinetic geometry of parameter space. Starting from a variational formulation, we derive the corresponding Hamiltonian dynamics, the geometric force generated by a position-dependent metric, and a metric-compatible Rayleigh dissipation law. The resulting continuous system satisfies a monotonic energy-dissipation relation, while its discrete form allows the influence of curvature on the optimization trajectory to be studied directly. We illustrate the framework using a controlled CIFAR-10 image-reconstruction problem for which the optimum is known analytically. With the image Hessian used as the metric, the curvature dependence of the quadratic modal dynamics is removed. We then reparameterize the same image as a matrix product state, producing a genuinely position-dependent metric and a nonzero geometric force. Poincar\'e return maps provide a complementary phase-space view of the resulting contraction dynamics. These examples establish HAMLET as a geometric, energy-based framework for studying optimization as dissipative motion in parameter space. On a five-seed MNIST MLP benchmark, HAMLET attains the highest mean test accuracy (97.95\%) and the lowest mean test negative log-likelihood (0.0696) among the three evaluated optimizers.

Problem

• Optimization
• Dissipative dynamical system
• Hamiltonian dynamics
• Metric learning
• Energy-based training

Innovation

• Hamiltonian dynamics
• dissipative optimization
• metric learning
• energy-based training
• geometric force