uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking

2026-06-10

AI Summary

This work addresses the challenges of cross-domain retrieval and question answering in multi-turn dialogues, particularly concerning handling unanswerable queries and effectively integrating dialogue history. The paper proposes a multi-turn retrieval-augmented generation (RAG) framework that innovatively combines learned sparse retrieval with large language model (LLM)-driven listwise reranking. By fully leveraging the complete dialogue context throughout the pipeline, the approach enables dialogue-aware query rewriting, context-sensitive reranking, and answer generation. Evaluated on SemEval-2026 Task 8 across four diverse domains—finance, cloud documentation, government, and Wikipedia—the system demonstrates strong cross-domain generalization and robust performance in identifying and handling unanswerable questions.

Abstract

This report describes our participation in SemEval-2026 Task 8 on multi-turn retrieval and question answering. The task evaluates conversational systems across four domains (finance, cloud documentation, government, Wikipedia), and includes unanswerable queries where the available collection does not contain sufficient evidence to produce a complete response. We propose a multi-turn retrieval-augmented generation pipeline that combines learned sparse retrieval with LLM-based reranking and generation. Using sparse retrieval as the primary retrieval method, we leverage its strong generalization across domains. In addition, we make use of the long-context capabilities of LLMs for conversational query rewriting, pointwise and listwise reranking, and generating the final response, each conditioned on the full conversational history. This multi-step design enables effective integration of conversational context throughout retrieval and generation, improving robustness across domains.

Problem

• multi-turn retrieval
• question answering
• unanswerable queries
• conversational systems
• cross-domain

Innovation

• learned sparse retrieval
• listwise reranking
• multi-turn RAG
• conversational query rewriting
• long-context LLM