Water Supply Network Management: Sensor Analysis using Google Cloud Dataflow

The growing field of IoT increases the amount of time series data produced every day. With such information overload it is necessary to promptly clean and process those information extracting meaningful knowledge and avoiding raw data storage. Nowadays cloud infrastructures allow to adopt this processing demand by providing new models for defining data-parallel processing pipelines, such as the Apache Beam unified model which evolved from Google Cloud Dataflow and MapReduce paradigm. The projects of this thesis have been implemented during a three-month internship at Injenia srl, and face this exact trail, by processing external IoT-acquired data, going through a cleansing and a processing phase in order to obtain neural networks ready-to-feed data. The sewerage project acquires signals from IoT sensors of a sewerage infrastructure and aims at predicting signals' trends over close future periods. The aqueduct project acquires the same information type from aqueduct plants and aims to reduce the false alarm rate of the telecontrol system. Given the good results of both projects it can be concluded that the data processing phase has produced high-quality information which is the main objective of this thesis.