Augmented Reality Robotics

Mars rovers and your smartphone have the same problem: figuring out where they are without GPS.

Visual Odometry has been around for decades but is really taking off with mobile augmented reality. 

Take a look at all the references from Larry Matthies on using Visual Inertial Odometry on the Mars Exploration Rovers. Well, it should come as no surprise that his lab at NASA JPL was an early participant in Google’s Project Tango effort to give smartphones that same capability. 

Visual Odometry has been around for decades but is really taking off with mobile augmented reality. Take a look at all the references from Larry Matthies on using Visual Inertial Odometry on the Mars Exploration Rovers. Well, it should come as no surprise that his lab at NASA JPL was an early participant in Google’s Project Tango effort to give smartphones that same capability. Steve Goldberg, also with JPL and now full time at Google, is one of the few people who can claim to have optimized the VO pipeline for a rover on Mars, and the one powering AR on your your smartphone!

The higher level technology that JPL’s rovers and Tango (now known as ARCore) use is called SLAM, or Simultaneous Localization and Mapping. More specifically, they use VSLAM and VIO, where cameras and motion sensors come together to create Visual Inertial Odometry. Just like your eyes and ears work together, robots and augmented reality devices uses cameras and motion sensors.

These virtual characters that are supposed to be the future of our digital lives still have zero understanding of the world. They rely on imperfect mobile sensors that struggle with “optically uncooperative surfaces”. AR devices can’t yet recognize, track, and predict the movement of people or animals.

The big winners are going to be those embracing ARKit and ARCore, which make heavy use of cameras. Those two platforms alone are expected to power more than a billion devices in 2018!

As the “AR Cloud” grows to include more maps and semantics about the world, you can expect to be provided product-level navigation with centimeter level accuracy 

. Google is in the unique position of offering both LiDAR mapping, as used by Cartographer, and with VSLAM, underlying Project Tango / AR Core / VPS.

References:

https://medium.com/@ryanhickman/augmented-reality-robotics-bb6db40ab754