Testing of AI-driven gesture recognition techniques to support human-centered design of industrial systems

Work-related musculoskeletal disorders (WMSDs) of the hand and upper limb are among the most common occupational health problems in manufacturing and assembly. Their prevention requires reliable ergonomic assessment of manual tasks, still mainly performed through expert observation. Vision-based Hand Pose Estimation (HPE) can provide continuous, objective information about hand posture and movement without constraining the operator. However, current HPE models are trained mainly on bare-hand datasets and often fail in industrial settings, where gloves, tool occlusions, and clutter alter hand appearance. This thesis addresses this gap through a multi-modal acquisition and alignment strategy combining ZED-2 stereo RGB-D cameras with Quantum Manus instrumented gloves. A Kabsch-based spatio-temporal alignment algorithm was developed to synchronize the streams and map glove kinematics into the camera frame, enabling supervision for gloved hands under occlusion. The pipeline was applied to 18 Bonfiglioli gearbox reducer assembly trials, covering bare-hand and gloved conditions. It produced more than 35,000 hand-centred samples for training and evaluating the proposed ErgoHand model. MediaPipe, OpenPose, and WiLoR were benchmarked using a binary hand-presence formulation. Results confirmed a strong industrial domain gap: MediaPipe and OpenPose achieved recall below 0.07, while WiLoR reached 0.39 but still missed more than half of the hands present. Therefore, OCRA and HAL cannot be applied without critical data loss. This thesis contributes the data infrastructure, alignment methodology, and domain-specific learning strategy required for reliable hand monitoring in human-centred industrial systems, supporting the sustainability and human-centricity goals of Industry 5.0.