Improving Deep Question Answering: The ALBERT Model

Natural Language Processing is a field of Artificial Intelligence referring to the ability of computers to understand human speech and language, often in a written form, mainly by using Machine Learning and Deep Learning methods to extract patterns. Languages are challenging by definition, because of their differences, their abstractions and their ambiguities; consequently, their processing is often very demanding, in terms of modelling the problem and resources. Retrieving all sentences in a given text is something that can be easily accomplished with just few lines of code, but what about checking whether a given sentence conveys a message with sarcasm or not? This is something difficult for humans too and therefore, it requires complex modelling mechanisms to be addressed. This kind of information, in fact, poses the problem of its encoding and representation in a meaningful way. The majority of research involves finding and understanding all characteristics of text, in order to develop sophisticated models to address tasks such as Machine Translation, Text Summarization and Question Answering. This work will focus on ALBERT, from Google Research, which is one of the recently released state-of-the-art models and investigate its performance on the Question Answering task. In addition, some ideas will be developed and experimented in order to improve model's performance on the Stanford Question Answering Dataset (SQuAD), after exploring breakthrough changes that made training and fine-tuning of huge language models possible.