Survey and Improvement Strategies for Gene Prioritization with Large Language Models

2025-01-30

AI Summary

Low diagnostic accuracy, model bias, and poor interpretability hinder causal gene identification in rare genetic disease diagnosis. Method: We propose an HPO-driven divide-and-conquer multi-agent framework that semantically classifies patient phenotypes and stratifies case solvability using the Human Phenotype Ontology (HPO), then leverages collaborative multi-agent reasoning to mitigate research bias and input-order sensitivity of large language models (e.g., GPT-4) in gene prioritization. Contribution/Results: This is the first framework enabling phenotype-specific, causally grounded, and interpretable gene ranking. It achieves near 30% Top-1 accuracy—marking a substantial improvement—and significantly enhances detection of high-specificity phenotype–gene associations. The framework supports reanalysis of diagnostically challenging cases and facilitates targeted therapeutic development.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Reasoning under Uncertainty: Causality
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Rare diseases are challenging to diagnose due to limited patient data and genetic diversity. Despite advances in variant prioritization, many cases remain undiagnosed. While large language models (LLMs) have performed well in medical exams, their effectiveness in diagnosing rare genetic diseases has not been assessed. To identify causal genes, we benchmarked various LLMs for gene prioritization. Using multi-agent and Human Phenotype Ontology (HPO) classification, we categorized patients based on phenotypes and solvability levels. As gene set size increased, LLM performance deteriorated, so we used a divide-and-conquer strategy to break the task into smaller subsets. At baseline, GPT-4 outperformed other LLMs, achieving near 30% accuracy in ranking causal genes correctly. The multi-agent and HPO approaches helped distinguish confidently solved cases from challenging ones, highlighting the importance of known gene-phenotype associations and phenotype specificity. We found that cases with specific phenotypes or clear associations were more accurately solved. However, we observed biases toward well-studied genes and input order sensitivity, which hindered gene prioritization. Our divide-and-conquer strategy improved accuracy by overcoming these biases. By utilizing HPO classification, novel multi-agent techniques, and our LLM strategy, we improved causal gene identification accuracy compared to our baseline evaluation. This approach streamlines rare disease diagnosis, facilitates reanalysis of unsolved cases, and accelerates gene discovery, supporting the development of targeted diagnostics and therapies.

Problem

• Rare Genetic Diseases
• Pathogenic Gene Identification
• Large Language Models

Innovation

• Large Language Models
• Rare Genetic Diseases Diagnosis
• Gene Diversity Handling