Markerless robot programming by demostration using a time-of-flight camera

Robot programming by demonstration is a technique that consists in automatically converting a demonstration of a certain task into a robot program ready to use. In particular, the purpose of this thesis is to exploit a 3D time-of-flight camera, used to acquire manipulation tasks performed by the human operator, to plan the trajectories that the robot will replicate. To this purpose, a demonstration device has been developed and used as the target to be tracked by the camera. The prototype is provided with a gripper similar to the end effector of an industrial robot so that, during the demonstration movement, the human operator approaches, picks and releases the objects to be manipulated just as the robot. No passive or active markers are used and, since the 3D time-of-flight camera represents the instantaneous image of the tracked object as a point cloud, the device has been designed with a special external geometry, easy to be detected thus providing univocally its pose in each time frame acquired. Successively, an Iterative Closest Point algorithm is applied to the acquired point clouds to calculate the homogeneous transformation matrixes that synthetically represent the relative displacement of the grippers between two consecutive poses. Therefore, the overall set of these matrices – properly referred to the robot reference frame – can be used to describe the reference trajectory to be performed by the robot starting from the initial pose of its end effector (which must be defined by the user). Indeed, as the final step of the proposed procedure, these data are converted into the specific programming language of the robot controller to make the machine replicate the movement performed by the human operator.