Overview of BioASQ 2024: The twelfth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

2025-08-28

AI Summary

The BioASQ 2024 challenge addresses biomedical semantic indexing and question answering through four tasks: the continuing Task B (factoid QA) and Synergy (semantic indexing), alongside two novel tasks—MultiCardioNER, the first multilingual clinical named entity recognition (NER) task in cardiology spanning English, French, German, and Spanish; and BIONNE, a Russian–English bilingual nested NER task. Methodologically, participating systems integrated pretrained language models, cross-lingual transfer learning, domain adaptation, and nested NER modeling. Thirty-seven teams submitted over 700 systems, most substantially outperforming baselines. This edition marks the first integration of clinical domain adaptation and multilingual nested NER into biomedical evaluation benchmarks, significantly expanding linguistic coverage (adding Russian, French, German, and Spanish) and task dimensions (nested structures, cross-linguality, and specialty-domain focus). Collectively, BioASQ 2024 advances the state of the art in biomedical information extraction by establishing more realistic, clinically grounded, and linguistically diverse evaluation paradigms.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This is an overview of the twelfth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2024. BioASQ is a series of international challenges promoting advances in large-scale biomedical semantic indexing and question answering. This year, BioASQ consisted of new editions of the two established tasks b and Synergy, and two new tasks: a) MultiCardioNER on the adaptation of clinical entity detection to the cardiology domain in a multilingual setting, and b) BIONNE on nested NER in Russian and English. In this edition of BioASQ, 37 competing teams participated with more than 700 distinct submissions in total for the four different shared tasks of the challenge. Similarly to previous editions, most of the participating systems achieved competitive performance, suggesting the continuous advancement of the state-of-the-art in the field.

Problem

• Advancing biomedical semantic indexing through large-scale challenge tasks
• Improving multilingual clinical entity detection in cardiology domain
• Enhancing nested named entity recognition for Russian and English

Innovation

• New multilingual clinical entity detection task
• Introduced nested NER for Russian English
• Established biomedical QA semantic indexing tasks