The Discrete Charm of the MLP: Binary Routing of Continuous Signals in Transformer Feed-Forward Layers

2026-03-11

AI Summary

This work reveals that MLP layers in Transformers implement a binary routing mechanism via binarized neuron activations to determine whether to apply nonlinear transformations to input tokens. Through neuron activation analysis, causal interventions, comparisons between binary and continuous features, and polynomial fitting, the study provides the first evidence of functional binary routing in MLPs and uncovers an architectural evolution from shallow single-gateway to deep consensus-based configurations, thereby extending the piecewise affine theory of deep networks. Experiments on GPT-2 Small demonstrate that when consensus is intact, removing MLPs increases perplexity by only 10.1%, whereas under collapse it surges by 43.3%; moreover, the binary decisions incur negligible information loss, achieving 79.2% accuracy compared to 78.8% with full continuous representations.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We show that MLP layers in transformer language models perform binary routing of continuous signals: the decision of whether a token needs nonlinear processing is well-captured by binary neuron activations, even though the signals being routed are continuous. In GPT-2 Small (124M parameters), we find that specific neurons implement a consensus architecture -- seven "default-ON" neurons and one exception handler (N2123 in Layer 11) that are 93-98% mutually exclusive -- creating a binary routing switch. A cross-layer analysis reveals a developmental arc: early layers (L1-3) use single gateway neurons to route exceptions without consensus quorums; middle layers (L4-6) show diffuse processing with neither gateway nor consensus; and late layers (L7-11) crystallize full consensus/exception architectures with increasing quorum size (1 to 3 to 7 consensus neurons). Causal validation confirms the routing is functional: removing the MLP at consensus breakdown costs 43.3% perplexity, while at full consensus removing it costs only 10.1% -- exceeding a 4x difference. Comparing binary vs. continuous features for the routing decision confirms that binarization loses essentially no information (79.2% vs. 78.8% accuracy), while continuous activations carry additional magnitude information (R^2 = 0.36 vs. 0.22). This binary routing structure explains why smooth polynomial approximation fails: cross-validated polynomial fits (degrees 2-7) never exceed R^2 = 0.06 for highly nonlinear layers. We propose that the well-established piecewise-affine characterization of deep networks can be complemented by a routing characterization: along the natural data manifold, the piecewise boundaries implement binary decisions about which tokens need nonlinear processing, routing continuous signals through qualitatively different computational paths.

Problem

• binary routing
• continuous signals
• transformer MLP
• nonlinear processing
• token routing

Innovation

• binary routing
• MLP
• transformer
• neuron activation
• consensus architecture