Implementation of a modular and versatile experimental apparatus to assess multisensory perception in space and time

Multisensory integration underlies our perception of the world: the brain com- bines information from different senses to help us interact with the environment more effectively. Better understanding this process is critical to developing new technologies in areas such as rehabilitation, human-machine interfaces, and robotics. This thesis chronicles the design and implementation of an experimental system designed to study how the brain integrates multi-modal stimuli in space and time. The apparatus is highly modular, expandable, and versa- tile for multisensory stimulation, particularly for this first version we focused on visual-acoustic stimulation. The device was built from scratch using affordable components, including three Raspberry Pi 4 microcomputers, RGB LED arrays, and an audio amplifier, all coordinated by a control PC with an intuitive graphical user interface. Using this infrastructure, sound and light stimuli can be precisely generated and synchronized, both individually and in combination. The entire software system, developed in C++ with Qt Creator, is based on TCP/ over IP local area network communication in which the four connected devices quickly exchange data from individual trials and record participants’ responses via USB keyboard. An initial psychophysical test confirmed that the setup works properly, paving the way for future validations and applications in advanced experimental protocols.