Single-Camera Vision System for Real-Time Manual Assembly Verification and Adaptive Operator Guidance

This thesis presents a single-camera RGB vision system for real-time monitoring, verification, and adaptive guidance of manual assembly operations. The system integrates ROI-based interaction detection, hand tracking, object detection, task-specific orientation classification, and a two-level finite-state machine architecture to interpret operator actions, verify procedural correctness, and provide configurable visual assistance at three levels. RF-DETR was selected for object detection, and pose estimation was reformulated as discrete orientation classification to handle symmetric components. The system was evaluated with eight participants across 70 trials. A linear mixed-effects analysis showed a significant effect of assistance level on motion quality (LR = 77.22, p < 0.001) and a significant condition × learning interaction (LR = 7.46, p = 0.024): full guidance supported sustained improvement while mid guidance plateaued. Perceived usability was similar for full and mid guidance despite the objective performance gap, indicating a dissociation between user perception and measured performance.