Michael Abrash, literally a genius programmer, recently posted on his blog at Valve regarding some of his VR (virtual reality) & AR (augmented reality) research. It’s chock-full of ideas and thoughts about the obstacles to overcome with VR/AR controller latency. That is making your control in the game world as instant and natural as it would be in real life. He says that even having controls be 99% accurate is not good enough, because it’s that 1% that causes your brain to register the motion as unrealistic. He gives some examples… normal PC games run with latency… about 50ms (and more sometimes) he calculates. In this context, it doesn’t matter really because the screen is your only reference point in the game world. With VR though, 50 milliseconds, that is .05 of a second, is enough latency (delay in movement) to make your brain not believe your eyes. It ruins the experience by feeling unresponsive or behind you. With AR, this delay causes graphics to not line up perfectly with real life (which has zero latency). He reckons that the acceptable threshold for VR latency could be between 7-15ms. He drills down into some ridiculously technical break downs of why current HMD (head mounted display) latency is no good and what the goals should be for future hardware. He also goes over an interesting anecdote of himself having a House figures it out moment… and it even includes a Star Trek reference. You can read it all here.
Meanwhile, in the Oculus VR lab, work continues on The Rift SDK, one of the only VR HMD’s that Michael Abrash (among others) has shown support for. Palmer Luckey and company are showing off their newly engineered motion sensor, which they boast “minimizes the time between the player’s head movement and the game engine receiving the sensor data to 2ms.” The sensor combines a gyroscope, accelerometer, and magnetometer. Through some pretty scientific data transfer and programming, they have effectively fused all this data together allowing for much more accurate and realtime head tracking. Due to the much faster speed of their self made sensor, they also claim that the track reduces drifting and loss of calibration (your center in the virtual world). They also discuss how this type of head tracking with The Rift will be included in the Rift SDK. When we tried the Rift back in Quakecon 2012, the reported latency was down to 38ms if my memory serves right. That was before the new and improved Rift hardware, which should prove quite an improvement, but as Abrash says “we’ve only scratched the surface.” He’s excited abou the Rift and believes that, if successful, could fuel the market for VR R&D which will eventually lead to the low latency HMD he dreams about. You can read the latest Oculus VR update here.
And guess what? Both Oculus VR and Valve are hiring, specifically for talented hardware designers and programmers who can over come VR & AR’s latency obstacles.