Learning in Infinitesimal Non-Compositional Sketches

2026-07-16

AI Summary

This work addresses structural inconsistencies in machine learning models arising from non-compositional phenomena—specifically, when schemas cannot be decomposed via quotient sketches. To tackle this, the paper introduces a unified categorical framework that models learning tasks as sketches equipped with commutativity, limit, and colimit constraints. It formalizes non-compositionality not as predictive error but as an obstruction to universal decomposition, leveraging tangent learning sketches and an INC iterative endofunctor. By analyzing tangent lifts in slice categories, the approach characterizes how compositional constraints are preserved under infinitesimal perturbations, recasting the learning process as the search for a coalgebraic fixed point of the INC functor. Theoretically, the existence of a final INC coalgebra is established via the Aczel–Mendler theorem; empirically, the framework’s efficacy is validated across deep learning, large language models, and reinforcement learning settings.

Abstract

This paper develops a categorical framework -- Learning in Infinitesimal Non-Compositional Sketches (LINCS) -- as the repair of non-compositionality: failures of diagrams to factor through quotient sketches lifted to the tangent category setting. Machine learning problems are specified as sketches: graphs with commutativity conditions $\mathcal D$, limit cones $\mathcal L$, and colimit cocones $\mathcal K$, generalizing the usual scalarization of loss functions or vector space assumptions. Non-compositionality is defined purely as failure of a universal factorization problem, not as arithmetic error between the desired and actual predictions. Given a learning sketch $\mathbb S=(S,\mathcal D,\mathcal L,\mathcal K)$, whose underlying graph is $S$, and a model $D:J \rightarrow C$, the base defect is the obstruction to factorization $\mbox{Obs}(\mbox{Fact}_{\mathbb S}(D))$. The tangent lift applies the tangent functor $T$ to obtain $TD:J \rightarrow C$, and LINCS is defined as the obstruction $\mbox{Obs}(\mbox{Fact}_{\mathbb S}(TD))$ -- asking whether infinitesimal perturbations preserve the compositionality constraints.The paper also introduces Tangent Learning Sketches, which are sketches equipped with Cockett-Cruttwell tangent structure. The paper defines the INC endofunctor, which iterates the tangent lift, producing a tower $D,TD,T^2D, \cdots$ of factorization problems. ML is thereby formulated as the search for a coalgebraic fixed point where successive tangent unfoldings stabilize ($νT_{\mbox{INC}}$). Using the Aczel--Mendler theorem, we prove existence of a final INC coalgebra whenever $T_{\mbox{INC}}$ admits a set-based class realization that creates its final carrier. A detailed experimental evaluation of LINCS is underway in a number of concrete ML settings, including deep learning, large language models, and reinforcement learning, and is described in companion papers.

Problem

• non-compositionality
• factorization
• tangent category
• sketches
• machine learning

Innovation

• tangent categories
• non-compositionality
• categorical sketches
• coalgebraic learning
• factorization obstruction