“Sensory Augmentation: Integration of an Auditory Compass Signal into Human Perception of Space”, 2017-02-04 (; similar):
Bio-mimetic approaches to restoring sensory function show great promise in that they rapidly produce perceptual experience, but have the disadvantage of being invasive. In contrast, sensory substitution approaches are non-invasive, but may lead to cognitive rather than perceptual experience.
Here we introduce a new non-invasive approach that leads to fast and truly perceptual experience like bio-mimetic techniques. Instead of building on existing circuits at the neural level as done in bio-mimetics, we piggy-back on sensorimotor contingencies at the stimulus level. We convey head orientation to geomagnetic North, a reliable spatial relation not normally sensed by humans, by mimicking sensorimotor contingencies of distal sounds via head-related transfer functions.
We demonstrate rapid and long-lasting integration into the perception of self-rotation. Short training with amplified or reduced rotation gain in the magnetic signal can expand or compress the perceived extent of vestibular self-rotation, even with the magnetic signal absent in the test. We argue that it is the reliability of the magnetic signal that allows vestibular spatial recalibration, and the coding scheme mimicking sensorimotor contingencies of distal sounds that permits fast integration.
Hence we propose that contingency-mimetic feedback has great potential for creating sensory augmentation devices that achieve fast and genuinely perceptual experiences.
…Our novel iPhone based sensory augmentation device [hearSpace] measures head orientation to North via orientation sensors (compass, gyro, accelerometer) integrated into a headphone and transforms their output into a spatial sound using a sound engine based on head-related transfer functions (HRTF) (Figure 1A, Figure 1B). A recording of a waterfall serves as the sound source which provides the ecological semantics of a natural sound coming from a distance. Further, the sound has a pink-noise like frequency spectrum which is pleasant to hear. The waterfall sound is reliably situated in the direction of magnetic North, moving in such a way as to compensate the movements of the head. This artificial sensorimotor contingency: (1) allows aligning the head with a global reference, creating a reliably stable artificial external reference for the eyes, ears and the vestibular system, and (2) provides an intuitive sensory code that mimics the acoustic characteristics of distal sounds.