🤖 AI Summary
This study addresses the challenges of losing diverse perspectives and lacking traceability in the analysis of large-scale heterogeneous textual corpora. To this end, it proposes a structured reading approach grounded in large language models, which defers irreversible information compression by sequentially performing insight extraction, semantic clustering, theme generation, and iterative omission detection. This pipeline explicitly preserves divergent viewpoints, thereby enhancing both coverage and auditability of the analytical process. Evaluated on a corpus of 152 industrial policy documents, the method successfully extracted over 17,500 structured insights and constructed a comprehensive thematic map. The implementation has been open-sourced as the first end-to-end framework supporting large-scale qualitative synthesis.
📝 Abstract
ReadingMachine is a computational methodology for structured corpus reading that uses large language models to perform bounded reading operations over entire document collections. Rather than relying on retrieval or recursive summarization, the approach decomposes analysis into inspectable stages including insight extraction, semantic clustering, theme generation, and iterative omission detection. By delaying irreversible compression and explicitly tracking intermediate representations, the method prioritizes coverage, traceability, and preservation of disagreement across large corpora. The system is demonstrated on a heterogeneous corpus of 152 industrial policy documents, producing more than 17,500 extracted insights and a structured thematic map. ReadingMachine is released as an open-source experimental framework for large-scale qualitative synthesis and corpus analysis.