← Research collection

Poster abstract · RRS 2026

Self-Evolving Agentic Retrieval-Augmented Generation Enables High-Accuracy and Confidence-Aware Radiation Oncology Reasoning

Co-author


Tingjun Liu, BS1,2, Xucheng Wang, BS1,3, Matthew Inkman, MS1, Andrew Willett, MD1,7, Weizhi Du1,2, Julian C. Hong, MD, MS4,5,6, Michael R. Waters, MD, PhD1,7, and Jin Zhang, PhD, MPHS1,7,8

Author affiliations

1. Department of Radiation Oncology, Washington University School of Medicine, St. Louis, MO, 63108.

2. Department of Computer Science, Washington University in St. Louis, St. Louis, MO, 63105.

3. Department of Biomedical Informatics, Harvard Medical School, Boston, MA, 02115.

4. Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA, 94143.

5. Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, 94143.

6. Joint Program in Computational Precision Health, University of California, Berkeley and San Francisco, Berkeley, CA, 94720.

7. Alvin J. Siteman Cancer Center, Washington University in St. Louis, St. Louis, MO, 63110.

8. Institute for Informatics, Data Science & Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO, 63110.

Abstract

Purpose/Objectives: Large language models (LLMs) have yet to reach their full potential in radiation oncology. In this highly technical subspecialty, clinical prompts remain particularly vulnerable to hallucination, outdated knowledge, and poor reliability estimation. To address these challenges, we developed eRT-RAG, a self-evolving multi-agent retrieval-augmented generation framework for radiation oncology, and evaluated whether evidence-grounded agentic retrieval can achieve high accuracy while providing confidence-aware validation.

Materials/Methods: eRT-RAG integrates multi-agent query generation, curated retrieval from radiation oncology textbooks and NCCN guidelines, evidence-weighted answer aggregation, source citation, and self-evolving literature updates. We evaluated eRT-RAG on the 2021 ACR Radiation Oncology In-Training Examination (TXIT), consisting of 300 textual and multimodal multiple-choice questions. Accuracy was determined by the official answer key and compared with a non-retrieval baseline. To further simulate real-world radiation oncology decision support, we also tested eRT-RAG on multiple contemporary open-ended, clinically debatable radiation oncology questions.

Results: eRT-RAG achieved 98.0% accuracy on TXIT, outperforming the non-retrieval baseline of 93% on GPT-5. Confidence scores were higher for correct than incorrect responses, and confidence-based discrimination of correctness achieved AUC of 0.79, suggesting that system confidence can serve as a practical signal for identifying outputs requiring automated knowledge base updates and clinician review. In debatable, timely clinical questions, expert radiation oncologists rate eRT-RAG as providing more direct and thorough clinical assessments compared to available GPT models.

Conclusion: eRT-RAG achieved high accuracy in radiation oncology and provided confidence scores that effectively disciminate the reliability of responses. These findings support confidence-aware, evidence-grounded agentic RAG as a pathway toward safer and more clinically useful LLM support in radiation oncology. More broadly, this framework could be expanded to other niche, knowledge intensive clinical fields, where accurate, current, and confidence-aware decision support are required.

System overview

eRT-RAG agent workflow, source retrieval, self-evolving updates, accuracy, and confidence analysisView full size ↗
eRT-RAG retrieval and answer aggregation, self-evolving knowledge updates, and evaluation results.
← Selected works

Scroll to inspect the full figure.