CRISS: A Retrieval-Augmented AI Chatbot for Assisting Cancer Registrars

2026-09-24

AI Summary

This study addresses the challenge cancer registrars face in navigating complex and frequently updated coding standards by proposing a domain-specific conversational assistant based on Retrieval-Augmented Generation (RAG). The approach constructs a specialized knowledge base that integrates large language models with dense vector indexing, enabling precise evidence anchoring and human–machine collaboration for highly challenging queries. Furthermore, an LLM-as-a-Judge evaluation protocol is introduced to rigorously assess system performance. Experimental results demonstrate that the proposed RAG-based system significantly outperforms non-RAG baselines in both response quality and semantic similarity. Ultimately, this work provides rapid, traceable access to clinical guidelines, offering practical decision support across diverse registration scenarios.

Abstract

Cancer registrars, including Oncology Data Specialists (ODSs), must interpret complex and frequently updated coding and staging standards. We developed CRISS (Cancer Registry Intelligent Support System), a retrieval-augmented generation (RAG) conversational assistant that provides rapid, citation-supported access to registry guidance. This study evaluated whether CRISS could (1) support accurate and citation-supported responses, (2) improve access to and interpretation of relevant guidance, and (3) support training/helpdesk use while preserving human oversight of final abstraction decisions. We built a domain-specific knowledge base from national cancer registry standards, segmented into metadata-tagged passages and indexed as dense embeddings. Retrieved passages were used to generate citation-grounded responses through a large language model (LLM). Open-weight, proprietary, and non-RAG baseline models across Gemini and GPT families were evaluated on easy, medium, and hard registry questions using an LLM-as-a-Judge protocols. RAG configurations consistently outperformed non-RAG approaches, especially as question difficulty increased. Mean grounding scores for RAG were 0.62/0.56/0.59 across easy/medium/hard tiers versus 0.29/0.26/0.29 for non-RAG. RAG models also achieved higher semantic-similarity scores overall. Proprietary RAG models performed strongest on easy and medium questions, while local RAG models ranked highest on hard questions and proprietary models were generally more cautious. Domain-specific RAG improved evidence grounding and response quality for cancer registry questions while enabling citation-supported assistance across complexity levels. CRISS demonstrates the potential of human-centered, citation-grounded AI to support cancer registrars while preserving human oversight for final coding decisions.

Problem

• Cancer registrars
• Retrieval-augmented generation
• Coding standards
• AI chatbot
• Evidence grounding

Innovation

• Retrieval-Augmented Generation
• Domain-Specific Knowledge Base
• Dense Embeddings
• LLM-as-a-Judge
• Citation-Grounded AI