An LLM-Based RAG System for Question Answering over User-Provided Documents
Abstract
The rapid growth of digital documents has created a need for efficient methods of accessing
and understanding information contained within them. Traditional document-reading
approaches require users to manually search through large amounts of text, which can be
time-consuming and inefficient. This project presents an LLM-based Retrieval-Augmented
Generation (RAG) system for question answering over user-provided documents, designed
to provide users with relevant and context-aware answers from their uploaded documents.
The proposed system combines semantic information retrieval with a Large Language
Model (LLM) to improve document-based question answering. Users can upload supported
documents to the system, after which the document content is extracted and divided into
smaller text chunks. These chunks are transformed into numerical vector representations
using a sentence-transformer-based embedding model and stored in ChromaDB, a vector
database. When a user submits a question, the system converts the query into an embedding
and performs semantic similarity search against the stored document chunks. The most
relevant chunks are then retrieved and provided as contextual information to the LLM. The
LLM generates an answer based primarily on the retrieved context, allowing the system to
respond to questions without requiring the entire document to be supplied to the model for
every query.
The system consists of a React-based frontend, a FastAPI backend, a ChromaDB vector
store, a sentence-transformer embedding model, and an LLM accessed through the Groq
API. The application also provides source information associated with retrieved document
content, allowing users to identify the document from which the answer was generated. The
separation between document processing, semantic retrieval, and answer generation
provides a modular architecture that can be extended with different language models or
retrieval configurations.
The developed system demonstrates the practical application of Retrieval-Augmented
Generation for document-based question answering while reducing dependence on
manually searching through lengthy documents. Although the current implementation has
limitations related to document formats, retrieval accuracy, context size, and dependence
on external LLM services, the architecture provides a strong foundation for future
improvements such as multi-document knowledge management, conversation memory,
improved retrieval techniques, document management interfaces, authentication, advanced
source citation, and more sophisticated question-answering capabilities.
Collections
- 2026-2030 [51]