StrokeNet: Unveiling How to Learn Fine-Grained Interactions in Online Handwritten Stroke Classification

2025-12-05

AI Summary

Online handwritten stroke classification faces challenges in modeling fine-grained semantic relationships due to high variability in writing styles, ambiguous content, and dynamic spatial positioning. To address the limitation of existing methods in capturing local stroke interactions, this paper proposes a reference-point sequence coupling representation and an Inline Sequence Attention module, incorporating a Cross-Ellipse Query mechanism for multi-scale spatial feature aggregation. A joint optimization framework is introduced to simultaneously predict stroke categories and semantic transition relations. The method integrates dynamic reference-point selection, sequential modeling, spatial query clustering, and multi-task learning (primary classification + regression + auxiliary branch), enabling end-to-end training. Evaluated on public benchmarks including CASIA-onDo, it achieves state-of-the-art performance—improving accuracy from 93.81% to 95.54%—with significantly enhanced robustness and generalization capability.

Technology Category

• Machine Learning: Learning with Manifolds
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Stroke classification remains challenging due to variations in writing style, ambiguous content, and dynamic writing positions. The core challenge in stroke classification is modeling the semantic relationships between strokes. Our observations indicate that stroke interactions are typically localized, making it difficult for existing deep learning methods to capture such fine-grained relationships. Although viewing strokes from a point-level perspective can address this issue, it introduces redundancy. However, by selecting reference points and using their sequential order to represent strokes in a fine-grained manner, this problem can be effectively solved. This insight inspired StrokeNet, a novel network architecture encoding strokes as reference pair representations (points + feature vectors), where reference points enable spatial queries and features mediate interaction modeling. Specifically, we dynamically select reference points for each stroke and sequence them, employing an Inline Sequence Attention (ISA) module to construct contextual features. To capture spatial feature interactions, we devised a Cross-Ellipse Query (CEQ) mechanism that clusters reference points and extracts features across varying spatial scales. Finally, a joint optimization framework simultaneously predicts stroke categories via reference points regression and adjacent stroke semantic transition modeling through an Auxiliary Branch (Aux-Branch). Experimental results show that our method achieves state-of-the-art performance on multiple public online handwritten datasets. Notably, on the CASIA-onDo dataset, the accuracy improves from 93.81$%$ to 95.54$%$, demonstrating the effectiveness and robustness of our approach.

Problem

• Classifies online handwritten strokes with fine-grained interactions
• Models localized semantic relationships between strokes using reference points
• Improves stroke classification accuracy on diverse writing styles and positions

Innovation

• StrokeNet encodes strokes as reference pair representations
• It uses Inline Sequence Attention for contextual feature construction
• Cross-Ellipse Query mechanism captures spatial feature interactions