Large Language Models for Interpretive Question Answering over Cross-Referenced Legal Documents in Evolving Knowledge Bases

This thesis investigates the application of Large Language Models (LLMs) and advanced retrieval architectures for interpretive question answering over interconnected legal documents in evolving regulatory corpora, focusing on Italian public procurement jurisprudence. While Retrieval-Augmented Generation (RAG) mitigates factual hallucinations by grounding responses in external knowledge, legal domains present two distinct challenges: normative cross-references require exact citation tracking beyond pure semantic similarity, and continuous legislative and judicial updates induce temporal retrieval obsolescence over time. We address these issues through an integrated pipeline comprising three core components. First, a customized crawling infrastructure extracts multi-source administrative rulings and statutory provisions into a relational Knowledge Graph, currently containing approximately 29,000 nodes and 90,000 semantic links. Second, a domain-adapted hybrid retrieval module combines dense vector search, fine-tuned to the domain, with exact lexical matching and graph-based citation traversal. Third, a tool-augmented agentic architecture exposed via REST APIs enables conversational LLMs to dynamically choose retrieval strategies and navigate legal precedents across the Knowledge Graph. To improve retrieval accuracy, we generated a synthetic dataset comprising over 700,000 questions using distributed endpoints across multiple machines. Furthermore, we evaluated various retrieval models and strategies to assess their performance in real-world scenarios across both static and evolving knowledge bases. This evaluation established an empirical retraining window of 18–24 months, confirming the viability of an efficient, obsolescence-aware legal intelligence system for dynamic regulatory domains.