Decoding Unintelligible Infant Vocalizations During Early Communication using Deep Learning Methods

Typically developing infants begin producing unintelligible sounds within the first week after birth. These early vocalizations serve as a fundamental form of communication, allowing infants to express physical needs and communicative intentions long before their first words emerge. Although often difficult for adults to interpret, these sounds contain patterns that provide valuable insights into an infant's language and brain development. This thesis presents a deep learning (DL)-based approach for detecting infant vocalizations, such as babbling and cooing, using spectrogram and embedding features extracted from a dataset of 62 newborns without medical conditions. Convolutional neural networks (CNNs) and convolutional recurrent neural networks (CRNNs) were trained to determine the most effective architecture and features for vocalization detection. Among the tested models, the CRNN achieved the best performance (accuracy up to 0.91) when using spectrogram features as input, with a static threshold applied to the computed probabilities to segment infant vocalizations. Performance was evaluated using key metrics, including detection error rate (DER =0.84) and F1-score (0.54), demonstrating the potential of DL for automated vocalization analysis. Future research should enhance detection accuracy by acquiring larger datasets, including infants at risk of neurodevelopmental disorders. This could pave the way for integrating these tools into clinical practice to support early screening and intervention.