Leveraging Language Models and Machine Learning in Verbal Autopsy Analysis

2025-08-22

AI Summary

In regions with weak civil registration systems, verbal autopsy (VA) remains a critical tool for cause-of-death attribution; however, existing automated VA methods rely solely on structured questionnaire responses, neglecting the diagnostic value of free-text narratives. This study systematically validates, for the first time, the independent contribution of VA narrative text to cause-of-death classification. We propose a multimodal fusion framework integrating transformer-based pre-trained language models with machine learning to jointly model unstructured narratives and structured questionnaire items. Evaluated on empirical South African VA data, our narrative-only model outperforms state-of-the-art question-answer models; multimodal integration further improves overall accuracy—particularly for non-communicable disease identification. Additionally, we analyze how information sufficiency affects inter-rater agreement between physicians and models, offering methodological insights for VA instrument redesign and scalable global mortality surveillance.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

In countries without civil registration and vital statistics, verbal autopsy (VA) is a critical tool for estimating cause of death (COD) and inform policy priorities. In VA, interviewers ask proximal informants for details on the circumstances preceding a death, in the form of unstructured narratives and structured questions. Existing automated VA cause classification algorithms only use the questions and ignore the information in the narratives. In this thesis, we investigate how the VA narrative can be used for automated COD classification using pretrained language models (PLMs) and machine learning (ML) techniques. Using empirical data from South Africa, we demonstrate that with the narrative alone, transformer-based PLMs with task-specific fine-tuning outperform leading question-only algorithms at both the individual and population levels, particularly in identifying non-communicable diseases. We explore various multimodal fusion strategies combining narratives and questions in unified frameworks. Multimodal approaches further improve performance in COD classification, confirming that each modality has unique contributions and may capture valuable information that is not present in the other modality. We also characterize physician-perceived information sufficiency in VA. We describe variations in sufficiency levels by age and COD and demonstrate that classification accuracy is affected by sufficiency for both physicians and models. Overall, this thesis advances the growing body of knowledge at the intersection of natural language processing, epidemiology, and global health. It demonstrates the value of narrative in enhancing COD classification. Our findings underscore the need for more high-quality data from more diverse settings to use in training and fine-tuning PLM/ML methods, and offer valuable insights to guide the rethinking and redesign of the VA instrument and interview.

Problem

• Automating verbal autopsy cause classification using narratives
• Enhancing accuracy with language models and machine learning
• Improving multimodal fusion of narratives and structured questions

Innovation

• Transformer-based PLMs for narrative analysis
• Multimodal fusion combining narratives and questions
• Task-specific fine-tuning for enhanced classification accuracy