Pointer-Augmented Autoregressive Generation of Patent Claims with Joint Topology and Content Decoding

2026-07-27

AI Summary

This work addresses the challenge of generating patent claims that simultaneously satisfy hierarchical structure and content consistency constraints—a task at which existing autoregressive models struggle. The authors propose SPG, a novel method that jointly predicts claim topology and textual content during autoregressive decoding. SPG employs a pointer mechanism to dynamically select parent nodes for dependent claims and introduces a depth-adaptive span regularization term to constrain shared representations. Notably, it is the first approach to embed topological prediction directly within the decoding process and incorporates preference optimization with self-generated negative samples to mitigate the scarcity of negative examples in granted patents. Implemented on the Llama-3-8B-Instruct architecture, SPG recovers 79.0% of ground-truth parent links on the HUPD-DCG dataset and improves antecedent consistency from 0.292 to 0.478, with expert evaluation confirming its significant superiority over baseline methods.

Abstract

Autoregressive decoders emit flat token sequences and cannot enforce hierarchical constraints across output segments, a limitation that becomes acute in patent claim generation, where a claim set forms a dependency forest whose scope must narrow monotonically with depth. Topology and content are mutually dependent: a dependent claim's wording must reflect its parent's scope, yet the parent must be chosen before that wording exists, so neither post-hoc parsing nor grammar-constrained decoding suffices. We propose SPG (Structure-aware Patent Generation), which predicts topology inside the autoregressive pass. A pointer head selects each dependent claim's parent, and its gradients, together with a depth-adaptive scope regularizer, reshape the shared decoder's representations during training. A second stage then applies a violation-weighted preference objective over self-generated deficient candidates, supplying the negative signal that granted-patent corpora lack. On HUPD-DCG, SPG on Llama-3-8B-Instruct recovers 79.0\% of gold parent links, a quantity its training reward never supervises, and raises antecedent consistency from 0.292 to 0.478 over a supervised baseline of equal scale, with expert evaluation corroborating these gains.

Problem

• patent claim generation
• hierarchical constraints
• dependency forest
• autoregressive decoding
• scope consistency

Innovation

• pointer-augmented autoregressive decoding
• joint topology-content modeling
• structure-aware generation
• depth-adaptive regularization
• preference-based fine-tuning