Distillation Knowledge applied on Pegasus for Summarization

Niccolai, Lorenzo (2020) Distillation Knowledge applied on Pegasus for Summarization. [Laurea], Università di Bologna, Corso di Studio in Informatica [L-DM270]
Documenti full-text disponibili:
[img] Documento PDF (Thesis)
Disponibile con Licenza: Creative Commons: Attribuzione - Non commerciale - Condividi allo stesso modo 4.0 (CC BY-NC-SA 4.0)

Download (886kB)

Abstract

In the scope of Natural Language Processing one of the most intricate tasks is Text Summarization, in human terms: writing an essay. Something that we learn in primary school is yet very difficult to reproduce for a machine, it was almost impossible before the advent of Deep Learning. The trending technology to face up Summarization - and every task that involves generating text - is the Transformer. This thesis aims to experiment what entails reducing the complexity of Pegasus, a huge state-of-the-art model based on Transformers. Through a technique called Knowledge Distillation the original model can be compressed in a smaller one transferring the knowledge, but without losing much efficiency. For the experimentation part the distilled replicas were varied in size and their performance assessed evaluating some suitable metrics. Reducing the computational power needed by the models is crucial to deploy such technologies in devices with poor capabilities and a not reliable enough internet connection to use cloud computing, like mobile devices.

Abstract
Tipologia del documento
Tesi di laurea (Laurea)
Autore della tesi
Niccolai, Lorenzo
Relatore della tesi
Scuola
Corso di studio
Ordinamento Cds
DM270
Parole chiave
neural,networks,nlp,transformers,summarization,abstractive,distillation,pegasus
Data di discussione della Tesi
16 Dicembre 2020
URI

Altri metadati

Statistica sui download

Gestione del documento: Visualizza il documento

^