Drug-Target Interaction Prediction via Hierarchical Sequential Cross-Attention over Chemical and Protein Language Models

2026-09-28

AI Summary

This study addresses the limitation of existing drug-target interaction (DTI) models that independently encode sequences and struggle to explicitly model cross-molecular dependencies. Inspired by the induced-fit mechanism, we propose a bidirectional cross-attention framework. Methodologically, it integrates ChemBERTa and ESM-2 pretrained representations, employing hierarchical sequential cross-attention to enable fine-grained interactions between chemical substructures and protein regions. One-dimensional convolutions and attention pooling are then utilized to construct fixed-size interaction vectors. Experimental results demonstrate that the proposed model achieves state-of-the-art performance on the BIOSNAP dataset and matches SOTA AUROC on the Davis dataset. Notably, with only 25.2M parameters, it maintains strong competitiveness while exhibiting superior cold-start generalization capabilities.

Abstract

Predicting Drug-Target Interactions~(DTIs) is a central task in computational drug discovery, with direct applications in virtual screening, drug repurposing, and therapeutic candidate prioritization. Although recent deep learning methods have improved DTI prediction, many sequence-based models still process drugs and proteins independently and only combine their representations at a late prediction stage. This limits their ability to explicitly model cross-molecular dependencies between chemical substructures and protein sequence regions. In this paper, we propose a sequence-only DTI prediction architecture that combines two pre-trained language models, ChemBERTa for drug SMILES strings and ESM-2 for protein amino acid sequences, with a hierarchical interaction module. The proposed model first extracts contextual representations using pre-trained encoders, then applies 1D convolutional layers to condense local sequence patterns, followed by a sequential bidirectional cross-attention mechanism inspired by the induced-fit view of molecular recognition. Finally, attention-based pooling constructs fixed-size interaction-aware vectors for binary prediction. Experiments on BIOSNAP, Davis, and BindingDB show that the proposed model achieves the best performance on BIOSNAP, matches the best AUROC on Davis, and remains competitive on BindingDB while using only 25.2 million trainable parameters. Ablation results confirm the contribution of both the CNN and cross-attention modules, and cold-start experiments indicate promising generalization to unseen proteins and drugs.

Problem

• Drug-Target Interaction Prediction
• Cross-molecular Dependencies
• Sequence-based Models
• Computational Drug Discovery

Innovation

• Drug-Target Interaction Prediction
• Hierarchical Cross-Attention
• Pre-trained Language Models
• Induced-Fit Mechanism
• Sequence-based Modeling