LOCO-EPI: Leave-one-chromosome-out (LOCO) as a benchmarking paradigm for deep learning based prediction of enhancer-promoter interactions

2024-12-02

Applied intelligence (Boston)

AI Summary

Existing enhancer–promoter interaction (EPI) prediction models commonly employ random data splitting, causing leakage of homologous genomic regions across training and test sets and severely inflating estimates of generalization performance. Method: We propose leave-one-chromosome-out (LOCO) cross-validation as a new benchmarking paradigm for EPI prediction—first enforcing strict chromosomal-level separation between training and test data to eliminate genomic information leakage. Building on this, we design a hybrid deep neural network that jointly leverages k-mer sequence features and contextual modeling capabilities. Contribution/Results: Under LOCO evaluation, our model significantly outperforms state-of-the-art baselines, while most existing methods exhibit precipitous performance drops—revealing their poor generalizability. We release the first standardized EPI benchmark dataset with LOCO splits, establishing a biologically realistic evaluation framework and advancing EPI prediction research toward genuine genomic generalization.

Technology Category

• Machine Learning: Ensemble Methods
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In mammalian and vertebrate genomes, the promoter regions of the gene and their distal enhancers may be located millions of base-pairs from each other, while a promoter may not interact with the closest enhancer. Since base-pair proximity is not a good indicator of these interactions, there is considerable work toward developing methods for predicting Enhancer-Promoter Interactions (EPI). Several machine learning methods have reported increasingly higher accuracies for predicting EPI. Typically, these approaches randomly split the dataset of Enhancer-Promoter (EP) pairs into training and testing subsets followed by model training. However, the aforementioned random splitting causes information leakage by assigning EP pairs from the same genomic region to both testing and training sets, leading to performance overestimation. In this paper we propose to use a more thorough training and testing paradigm i.e., Leave-one-chromosome-out (LOCO) cross-validation for EPI-prediction. We demonstrate that a deep learning algorithm, which gives higher accuracies when trained and tested on random-splitting setting, drops drastically in performance under LOCO setting, confirming overestimation of performance. We further propose a novel hybrid deep neural network for EPI-prediction that fuses k-mer features of the nucleotide sequence. We show that the hybrid architecture performs significantly better in the LOCO setting, demonstrating it can learn more generalizable aspects of EP interactions. With this paper we are also releasing the LOCO splitting-based EPI dataset. Research data is available in this public repository: https://github.com/malikmtahir/EPI

Problem

• Predict enhancer-promoter interactions accurately across chromosomes
• Address performance overestimation due to random dataset splitting
• Develop hybrid deep neural network for generalizable EPI prediction

Innovation

• LOCO cross-validation for EPI-prediction benchmarking
• Hybrid deep neural network with k-mer features
• Public LOCO-split EPI dataset released