“Measuring Motion-To-Photon Latency for Sensorimotor Experiments With Virtual Reality Systems”, Matthew Warburton, Mark Mon-Williams, Faisal Mushtaq, J. Ryan Morehead2022-10-10 ()⁠:

[VR headsets have actually gotten slower since the Oculus Rift!] Consumer virtual reality (VR) systems are increasingly being deployed in research to study sensorimotor behaviors, but properties of such systems require verification before being used as scientific tools.

The ‘motion-to-photon’ latency (the lag between a user making a movement and the movement being displayed within the display) is a particularly important metric as temporal delays can degrade sensorimotor performance. Extant approaches to quantifying this measure have involved the use of bespoke software and hardware and produce a single measure of latency and ignore the effect of the motion prediction algorithms used in modern VR systems. This reduces confidence in the generalizability of the results.

We developed a novel, system-independent, high-speed camera-based latency measurement technique to co-register real and virtual controller movements, allowing assessment of how latencies change through a movement. We applied this technique to measure the motion-to-photon latency of controller movements in the HTC Vive, Oculus Rift, Oculus Rift S, and Valve Index, using the Unity game engine and SteamVR.

For the start of a sudden movement, all measured headsets had mean latencies between 21 and 42 ms. Once motion prediction could account for the inherent delays, the latency was functionally reduced to 2–13 ms, and our technique revealed that this reduction occurs within ~25–58 ms of movement onset. Our findings indicate that sudden accelerations (eg. movement onset, impacts, and direction changes) will increase latencies and lower spatial accuracy.

Our technique allows researchers to measure these factors and determine the impact on their experimental design before collecting sensorimotor data from VR systems.

Figure 4: (a) The latency at the start (Sudden) and middle (Continuous) of the movement was measured. Note that the real controller positions were sampled every camera frame, whereas virtual controller positions could only be sampled every HMD frame. (b) Histograms of the measured latency for the different HMDs in the Sudden Movement (left panel) and Continuous Movement (right panel) conditions. The mean and standard deviation for each HMD and frame rate combination is shown beneath the histogram. The histogram bin widths were 4.17 ms, to match the camera frame rate, centered on a latency of 0 ms. Some of the variability present in the measurements is due to the stochasticity between the event occurring (the movement onset or the mid-point crossing), and when the camera captures a new frame or the HMD displays a new frame, as illustrated in Figure S2 in supplementary materials
Figure 5: Latency properties of the Oculus Rift and Valve Index at 90 Hz during a movement. Points show the mean value while the lines show individual movements. (a) Each movement was plotted relative to motion onset. Inset panels show a closer view of the motion onset. (b) The minimum latency that each real position could be displayed on the HMD was found. This was done for each sample after motion onset by projecting the real controller position forward in time until a greater virtual controller position was found, and then finding the next frame where the HMD was illuminated. (c) The difference between the virtual and real normalized positions was found at each time step to show the effect latency has on what is displayed by the HMD.