High-Performance Inference Graph Convolutional Networks for Skeleton-Based Action Recognition

2023-05-30

arXiv.org

AI Summary

To address the slow inference speed and high computational overhead of Graph Convolutional Networks (GCNs) in skeleton-based action recognition, this paper proposes two high-performance inference GCNs: HPI-GCN-RP, which decouples training and inference architectures via structural reparameterization to accelerate inference, and HPI-GCN-OP, which incorporates over-parameterization to enhance representational capacity and improve accuracy. This work is the first to synergistically integrate reparameterization and over-parameterization into skeleton GCNs, achieving joint optimization of accuracy and efficiency. HPI-GCN-OP achieves state-of-the-art (SOTA) accuracy of 93.0% on NTU-RGB+D 60 and 90.1% on NTU-RGB+D 120, while attaining a 5× inference speedup over existing SOTA methods—without sacrificing accuracy. The proposed framework establishes a new paradigm for lightweight and efficient skeleton-based action recognition.

Abstract

Recently, the significant achievements have been made in skeleton-based human action recognition with the emergence of graph convolutional networks (GCNs). However, the state-of-the-art (SOTA) models used for this task focus on constructing more complex higher-order connections between joint nodes to describe skeleton information, which leads to complex inference processes and high computational costs. To address the slow inference speed caused by overly complex model structures, we introduce re-parameterization and over-parameterization techniques to GCNs and propose two novel high-performance inference GCNs, namely HPI-GCN-RP and HPI-GCN-OP. After the completion of model training, model parameters are fixed. HPI-GCN-RP adopts re-parameterization technique to transform high-performance training model into fast inference model through linear transformations, which achieves a higher inference speed with competitive model performance. HPI-GCN-OP further utilizes over-parameterization technique to achieve higher performance improvement by introducing additional inference parameters, albeit with slightly decreased inference speed. The experimental results on the two skeleton-based action recognition datasets demonstrate the effectiveness of our approach. Our HPI-GCN-OP achieves performance comparable to the current SOTA models, with inference speeds five times faster. Specifically, our HPI-GCN-OP achieves an accuracy of 93% on the cross-subject split of the NTU-RGB+D 60 dataset, and 90.1% on the cross-subject benchmark of the NTU-RGB+D 120 dataset. Code is available at github.com/lizaowo/HPI-GCN.

Problem

• Graph Convolutional Networks
• Action Recognition
• Computational Efficiency

Innovation

• Graph Convolutional Networks
• Reparameterization
• Overparameterization