Enhancing the Software Development Life Cycle with Conversational AI: LLM-Driven Documentation Q&A and Automated Model Evaluation

In software development, navigating code documentation is a time-consuming and cognitively demanding task. This research, investigates the potential of Large Language Models (LLMs) to improve documentation consultation, aiming to reduce developer efforts while increasing their understanding. The study introduces a novel system that integrates an existing LLM-based documentation generator with an advanced question-answering system. This system, composed of different Retrieval-Augmented Generation (RAG) models, is designed to understand the contextual needs of user queries and intelligently route them to the most appropriate RAG within the proposed architecture. This approach proved to effectively handle both fine-grained and coarse-grained questions, obtaining high Context Precision (0.955) and Answer Relevancy (0.900) according to the RAGAs metrics. The research also investigates automated project summarisation and automated evaluation strategies for natural language systems, proposing procedures to reduce human effort while ensuring reliable results. Overall, this study highlights the potential of LLMs and Generative AI to streamline software development workflows, reducing time and cost associated with code understanding and LLM evaluation. It contributes to the growing field of AI-driven software engineering tools, laying a foundation for future research and applications in documentation management and intelligent systems design. This thesis is the result of practical research conducted in collaboration with DATA Reply.