Training Deep Morphological Neural Networks as Universal Approximators

2025-05-14

AI Summary

This work addresses three fundamental limitations of Deep Morphological Neural Networks (DMNNs): poor trainability, restricted nonlinear expressivity, and parameter redundancy. To resolve these issues, we propose a systematic solution: (1) theoretically establishing the necessity of inter-layer nonlinear activation for morphological operations; (2) designing three novel DMNN architectures under stringent parameter constraints—morphology-dominant, linear–morphology alternating, and hybrid convergence-accelerating variants; and (3) developing the first end-to-end trainable DMNN framework. Theoretical analysis and empirical evaluation demonstrate that our approach achieves stable DMNN training under strict parameter constraints for the first time; morphology-based layers accelerate large-batch gradient descent convergence by up to 2.3×; model prunability improves by 37% over purely linear counterparts; and the universal approximation capability of the proposed DMNNs is rigorously proven.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Lexical Semantics and Morphology
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We investigate deep morphological neural networks (DMNNs). We demonstrate that despite their inherent non-linearity, activations between layers are essential for DMNNs. We then propose several new architectures for DMNNs, each with a different constraint on their parameters. For the first (resp. second) architecture, we work under the constraint that the majority of parameters (resp. learnable parameters) should be part of morphological operations. We empirically show that our proposed networks can be successfully trained, and are more prunable than linear networks. To the best of our knowledge, we are the first to successfully train DMNNs under such constraints, although the generalization capabilities of our networks remain limited. Finally, we propose a hybrid network architecture combining linear and morphological layers, showing empirically that the inclusion of morphological layers significantly accelerates the convergence of gradient descent with large batches.

Problem

• Exploring deep morphological neural networks as universal approximators
• Designing constrained DMNN architectures with morphological parameters
• Hybrid networks combining linear and morphological layers for faster convergence

Innovation

• Deep morphological neural networks with essential activations
• New DMNN architectures with parameter constraints
• Hybrid network combining linear and morphological layers