VNODE: A Piecewise Continuous Volterra Neural Network

2025-09-29

AI Summary

This paper addresses the challenge of jointly achieving high modeling capacity and parameter efficiency in image classification. We propose VNODE, a piecewise-continuous hybrid architecture that integrates nonlinear Volterra filtering with Neural Ordinary Differential Equations (Neural ODEs). Its core innovation lies in embedding the Volterra series into the Neural ODE framework, enabling alternating discrete feature extraction and continuous latent-state evolution—thereby emulating hierarchical information processing in the visual cortex. This design preserves strong nonlinear representational power while substantially reducing parameter count. VNODE supports end-to-end training via gradient-based optimization. Experiments on CIFAR-10 and ImageNet-1K demonstrate state-of-the-art accuracy under significantly lower computational complexity compared to existing methods, achieving a favorable trade-off between model expressivity and efficiency.

Technology Category

Computer Vision: Learning & Optimization for CVMachine Learning: Deep Neural Architectures and Foundation ModelsCognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsUser Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendationEconomics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

This paper introduces Volterra Neural Ordinary Differential Equations (VNODE), a piecewise continuous Volterra Neural Network that integrates nonlinear Volterra filtering with continuous time neural ordinary differential equations for image classification. Drawing inspiration from the visual cortex, where discrete event processing is interleaved with continuous integration, VNODE alternates between discrete Volterra feature extraction and ODE driven state evolution. This hybrid formulation captures complex patterns while requiring substantially fewer parameters than conventional deep architectures. VNODE consistently outperforms state of the art models with improved computational complexity as exemplified on benchmark datasets like CIFAR10 and Imagenet1K.

Problem

• Integrating Volterra filtering with neural ODEs for image classification
• Combining discrete feature extraction with continuous state evolution
• Reducing parameters while outperforming state-of-the-art models

Innovation

• Integrates Volterra filtering with neural ODEs
• Alternates discrete feature extraction with ODE evolution
• Reduces parameters while outperforming state-of-the-art models