ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review

2026-07-31

AI Summary

This work addresses the challenges of inconsistent, incomplete, and stylistically fragmented peer review comments by introducing ReVoicer, a conversational system that enables reviewers to naturally annotate paper excerpts via speech or text. Leveraging large language models, ReVoicer performs context-aware comment sanitization, categorization, and anchoring to source passages, then synthesizes these inputs into a structured review draft aligned with the reviewer’s historical writing style. A key innovation lies in its exclusive reliance on the reviewer’s own annotations—without incorporating external judgments—and its ability to automatically identify gaps in coverage according to conference-specific review criteria. The prototype generates coherent, comprehensive review drafts and is slated for user evaluation within the ISMAR community.

Abstract

We present ReVoicer, a prototype system that supports peer reviewers by letting them converse with a paper as they read it. The reviewer highlights a passage and speaks (or types) a train-of-thought comment. A large language model then cleans the comment using the surrounding prose as context, tags it by comment type, and anchors it to the passage. After the reviewer finishes reading, ReVoicer checks the accumulated notes against a venue-specific rubric, reports coverage gaps, and drafts a review composed only from the reviewer's own comments, written to a style guide distilled from the reviewer's past reviews. The system introduces no critiques of its own. We describe the system's design rationale and implementation, and we outline plans for future evaluations. With the ISMAR community, we will gather feedback and discuss the system design and ideas for additional features and evaluations.

Problem

• peer review
• voice annotation
• human-centered AI
• large language models
• review assistance

Innovation

• conversational voice annotation
• LLM-assisted peer review
• human-centered AI
• review synthesis
• context-aware comment processing