Lyrics Instrumentalness: An Automatic System for Vocal and Instrumental Recognition in Polyphonic Music with Deep Learning

Human voice recognition is a crucial task in music information retrieval. In this master thesis we developed an innovative AI system, called Instrumental- ness, to address the instrumental song tagging. An extended pipeline was proposed to fit the Instrumentalness require- ments, w.r.t. the well known tasks of Singing Voice Detection and Singing Voice Segmentation. A deep research on the available datasets was made and two different approaches were tried. The first one involves strongly labeled datasets and tested different neural architectures, while the second one used an attention mechanism to address a weakly labeled dataset, experimenting on different loss functions. Transfer learning was used to take advantage of the most recent architec- tures in the music information retrieval field, keeping the model efficient and effective. This work demonstrates that the quality of data is as important as its quan- tity. Moreover, the architectures to address strongly labeled datasets achieved the best performance, but it is remarkable that the attention mechanism used to address the weakly labeled datasets seems to be effective, even if the dataset was imbalanced and small.