DiReCT RAG for Clinical Diagnostic Reasoning

Clinical documentation contains large volumes of unstructured text, making it difficult to retrieve relevant information efficiently and use it as context for language-model-generated responses. General-purpose language models may produce unsupported statements when answering clinical questions without access to relevant evidence. The challenge was to develop a retrieval pipeline that could identify useful clinical notes and provide supporting context for generated answers.

DiReCT was developed as a healthcare-focused Generative AI and information-retrieval project using the MIMIC-IV-Ext Direct dataset. The system combines traditional keyword-based retrieval with semantic search to identify relevant clinical documentation. A Retrieval-Augmented Generation architecture then supplies retrieved passages to an open-source large language model to produce context-aware responses. The project also incorporates evaluation methods and privacy-conscious data-handling practices.

Developed a modular RAG pipeline combining BM25 keyword retrieval with dense semantic retrieval using SentenceTransformers and FAISS. Retrieved clinical passages are incorporated into a structured prompt that provides context for an open-source language model to generate responses. A Streamlit frontend allows users to submit questions, inspect retrieved documents, and review generated answers. The system also supports retrieval evaluation using Precision@K and Recall@K.

The system follows a retrieval-first design in which user questions are processed by complementary keyword and semantic search components. Relevant passages are collected and supplied to the generation model as contextual evidence. The Streamlit interface separates user input, retrieved clinical information, and generated responses to improve transparency and usability. Privacy-conscious design principles keep raw clinical data outside the public repository.

The system consists of the following logical components:

1. User Interface: Streamlit-based interface for submitting queries and reviewing results.
2. Query Processing: Prepares user questions for retrieval.
3. Keyword Retrieval: BM25-based search for lexical matching.
4. Semantic Retrieval: SentenceTransformers embeddings with FAISS similarity search.
5. Context Assembly: Selects and prepares retrieved passages for the generation stage.
6. Answer Generation: An open-source large language model generates context-aware responses.
7. Evaluation: Precision@K and Recall@K measure retrieval performance.
8. Data Management: Local storage for authorized clinical data, with privacy-conscious handling.

The project was implemented in Python using a modular retrieval and generation workflow. SentenceTransformers supports dense text embeddings, FAISS enables efficient similarity search, and BM25 provides keyword-based retrieval. The retrieved passages are integrated into prompts for an open-source LLM. Streamlit provides the interactive frontend, while evaluation utilities support retrieval-quality analysis. The development workflow includes environment setup, dependency installation, local data storage, and documentation for responsible dataset use.

DiReCT brings together keyword retrieval, semantic search, and language-model-based answer generation in a single research-oriented application. It enables users to inspect retrieved clinical passages alongside generated responses and provides a foundation for evaluating retrieval quality using Precision@K and Recall@K. Quantitative performance claims should be added after running the evaluation pipeline and recording the actual results. The system is intended for research and educational use, not as a replacement for professional medical judgment.