Deep Learning-based Recognition of Human Actions in a Collaborative Robotics Environment

The project of this thesis consists in the development of a software able to recognize and classify some actions performed in front of a 3D camera, within a context of collaborative robotics. Thisthesisistheresultofaninternshipprojectcarriedoutbytheauthorofthisthesis at the global company KUKA Deutschland GmbH, during the semester July 2019 - January 2020, under the supervision of Dr. Kirill Safronov and Eng. Pierre Venet. As regards the structure of the algorithm proposed in this thesis, the first elaboration consists in the recognition of the objects present in a scene. This step has been carried out through an artificial intelligence model, in particular a convolutional neural network, trained on a dataset prepared ad-hoc for the specific application. Once the objects were recognized, 3D point clouds were calculated for each detected element, through a projection method from 2D to 3D space. A tool for the recognition of the human body, in 3D, has been integrated with the overall algorithm, in order to allow an accurate understanding of the manipulations performed. Finally, the action recognition itself was achieved through a method inspired by a state machine, able to describe the evolution of the relations among objects, over time. Once the proposed method was developed, several tests were carried out, in order to quantitatively assess its accuracy and robustness. The performance of this algorithm has been shown to be coherent with the methods proposed by recent scientific articles on the subject. In addition, there are several aspects, which make the proposed solution innovative. First of all, the design of this technology was oriented to the classification of actions performed in an industrial context. Secondly, as introduced earlier, the algorithm uses three-dimensional data to perform its function. Finally, the last novel factor consists in the integration of the system with a 3D human body position estimation technology.