Finetuning commercial Large Language Models with LoRA for enhanced Italian language understanding

In this thesis we took the first steps of creating a well functioning LLM for the Italian language. We finetune two open source commercially licensed LLMs, MPT and LLaMA 2 on an Italian instruction dataset, Stambecco. Although the models do not perform as well as initially aimed for, we did have findings that are broadly applicable and we believe that this work justifies the creation of an LLM pretrained on a majority of Italian data.