Enhancing Chatbot Efficacy in Italian Language through Retrieval Augmented Generation and LoRA Fine-Tuning

This thesis delves into the integration of Large Languages Models (LLM)within the implementation of conversational agents. Traditionally, chatbotsystems were built using rule-based or simple machine learning approachesthat heavily relied on predefined scripts and limited user input interpretation. Addressing these challenges, our research proposes a methodology thatleverages the advanced capabilities of LLMs, enhanced through Retrieval-Augmented Generation (RAG) systems and the Low-Rank Adaptation (LoRA)fine-tuning technique, to significantly improve chatbot performance on theItalian language. Through meticulous evaluation, including the most recent benchmarks de-signed for assessing LLMs, our research demonstrated the superiority of thefine-tuned model over the original one in handling Italian conversations byleveraging only a minimal subset of model’s parameters. This research was conducted internally the ML/AI team of Injenia, withinthe European project Masters in AI for Careers in EU (MAI4CAREU).