NdLinear Is All You Need for Representation Learning

2025-03-21

AI Summary

Conventional neural networks often flatten multidimensional data (e.g., images, medical volumes, multivariate time series), discarding cross-dimensional structural information and introducing redundant computation. Method: We propose NdLinear—a native tensor-dimensional linear transformation module that applies independent linear mappings along each dimension, requiring zero additional parameters and no architectural modifications. It is plug-and-play compatible with CNNs, RNNs, and Transformers, and models high-order dimensional dependencies without attention mechanisms or modality-specific preprocessing. Contribution/Results: Across multiple benchmark tasks, NdLinear significantly improves representation capability and parameter efficiency. It demonstrates that native multidimensional modeling—preserving tensor structure without sacrificing computational tractability—is both effective and practical.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Multi-modal Vision
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Many high-impact machine learning tasks involve multi-dimensional data (e.g., images, volumetric medical scans, multivariate time-series). Yet, most neural architectures flatten inputs, discarding critical cross-dimension information. We introduce NdLinear, a novel linear transformation that preserves these structures without extra overhead. By operating separately along each dimension, NdLinear captures dependencies that standard fully connected layers overlook. Extensive experiments across convolutional, recurrent, and transformer-based networks show significant improvements in representational power and parameter efficiency. Crucially, NdLinear serves as a foundational building block for large-scale foundation models by operating on any unimodal or multimodal data in its native form. This removes the need for flattening or modality-specific preprocessing. Ndlinear rethinks core architectural priorities beyond attention, enabling more expressive, context-aware models at scale. We propose NdLinear as a drop-in replacement for standard linear layers -- marking an important step toward next-generation neural architectures.

Problem

• Preserves multi-dimensional data structure without overhead
• Captures cross-dimension dependencies ignored by standard layers
• Eliminates need for flattening or modality-specific preprocessing

Innovation

• NdLinear preserves multi-dimensional structure efficiently
• Operates separately along each dimension for dependencies
• Drop-in replacement for standard linear layers