Virtual reality development has come a long way, but it’s always been restricted to the virtual realm. However, new technology from Disney allows for the physical world to blend with the virtual. The video below shows this research in action, as a participant catches an actual ball while wearing a VR headset.
Catching an object in the real world requires hand-eye coordination, something that has yet to be successfully translated into the VR realm. The Disney Research lab hopes that this type of coordination and other skills can be translated into virtual reality. Senior research scientist Günter Niemeyer said combining physical and virtual enhances user experience.
“Catching and feeling the real ball in your hand makes VR much richer, more believable, more exciting, more interactive, more dynamic, more real,” he said.
How did they do it?
The team used a motion capture system to track the ball, hence the reason the ball in ‘reality’ has more texture than the one in the virtual rendering. That system tracked the movement of the ball as well as the location of the catcher’s hands. The ball and subsequent actions then get relayed to the VR head-mounted display.
[Image source: DisneyResearchHub via YouTube]
The video shows three separate visualizations. The first seems the most normal and lifelike. The system renders the ball’s location in real time. The next shows the line of trajectory for the ball as predicted by the entire system. The last recommends a possible area where the catcher could catch the ball.
Research lab associate Mathew Pan said that all three renderings led to successful catches. The team reported 95 percent success rate with the first visualization of underhanded throws. However, the catching styles changed in the other two visualizations. The catcher used the VR’s predictions to ready his hands slightly earlier.
“The most apparent explanation is that, without information about the ball’s location, the catcher must rely on the identified target point, changing the task from one requiring higher brain functions to estimate trajectory to a simpler, visually guided pointing task,” Pan said.
This technology could make video games even more realistic
This technology could easily improve the interactivity of VR games or even create an entirely new genre. It’s easy for players to get lost in the virtual space created in their headsets. This new research could add to a video game’s elements and make the experience all the more realistic.
“With VR, we can show you the future by pre-rendering where the ball is going to be,” Niemeyer said. “For some types of interactions, game designers might choose to take advantage of VR to make certain tasks easier, just as using a net to catch balls might make some games more enjoyable.”
Disney is no stranger to VR tech. In September 2015, the media giant purchased VR startup Jaunt. Jaunt developed a custom camera that could shoot professional-grade 360-degree footage, and Disney initially wanted to use the technology to bring immersive experiences to ABC News. Disney has since used Jaunt to create renderings of all of its theme parks, especially in celebrating major milestones such as Disneyland’s 60th anniversary Diamond Celebration.