OrienterAI: An Efficient Hybrid RAG Chatbot for Orienting Citizens Toward the Emilia-Romagna Educational Offer

This thesis presents OrienterAI, a hybrid Retrieval-Augmented Generation (RAG) chatbot designed to help citizens navigate the vocational training catalogue of the Emilia-Romagna region. Large Language Models (LLMs) have made it possible to query specialized knowledge bases in natural language, with RAG techniques grounding answers in real documents to mitigate hallucination; yet public-sector catalogues list thousands of active courses across cities, sectors, and qualification levels, making it difficult to identify offerings that match a user's profile and eligibility constraints such as EQF level, minimum age, and target audience without expert guidance. OrienterAI addresses this problem through two main components. First, an LLM-based extraction module normalizes free-text course descriptions into structured fields such as EQF level, minimum age, required skills, and macro-sector. Second, a four-stage cascaded retrieval pipeline progressively relaxes geographic and topical filters before falling back to semantic search over a Milvus vector database indexed with BGE-M3 embeddings, followed by a generation stage that renders the most relevant courses while minimizing fabricated content. The system was evaluated on a catalogue of more than 15,000 active courses and a benchmark of 250 synthetic queries spanning five complexity levels, using Accuracy@k, NDCG@k, and Precision@k as ranking metrics. End-to-end evaluation yields a Precision@1 of 0.768, an Accuracy@10 of 0.876, and an NDCG@10 of 0.798, while sustaining acceptable latency on a single NVIDIA RTX 3090 GPU. These results confirm the viability of an efficient, domain-specific RAG approach for public-sector information access and point to a scalable model for similar multilingual catalogues.