Form and Function: Machine Unlearning as a Problem of Misaligned States

2026-05-17

AI Summary

This study addresses the problem of machine unlearning in online L-BFGS optimizers, aiming to precisely reverse the influence of deleted samples on the model state and recover the counterfactual optimizer state—that is, the state that would have resulted had the sample never been observed. To this end, the work introduces a state-aware, multi-dimensional error metric and demonstrates for the first time that merely correcting model parameters is insufficient for effective unlearning; instead, both the optimizer’s memory and parameters must be jointly aligned with their counterfactual counterparts. The approach leverages online L-BFGS and integrates counterfactual event stream modeling, inverse Hessian action comparison, memory-operator error analysis, and recursive bias bound derivation. Experiments show that jointly updating memory and parameters significantly outperforms strategies that adjust only one component and closely approximates the performance of a counterfactual oracle.

Abstract

We formulate machine unlearning for online L-BFGS as a counterfactual state-alignment problem. Given an actual event stream and a deletion-edited counterfactual stream, the target of unlearning is the optimizer state that would have arisen had the deleted samples never been processed. We introduce state-aware metrics that separately measure parameter error, memory-operator error, combined state error, and update-direction error. The memory metric compares the inverse-Hessian actions induced by the o-L-BFGS memory, rather than treating curvature pairs as of finite influence. Under convexity assumptions, we derive a recursive bound on counterfactual state deviation. We then evaluate a state-aware benchmark of deletion interventions, including memory-only and parameter-only corrections, against an counterfactual oracle model. These results show that unlearning for online L-BFGS is not merely a parameter-correction problem: it requires alignment with a realizable counterfactual optimizer state.

Problem

• machine unlearning
• online L-BFGS
• counterfactual state
• optimizer state
• state alignment

Innovation

• machine unlearning
• state alignment
• online L-BFGS
• counterfactual optimization
• memory-operator error