Teleoperated Robot Control with Augmented Reality

The University of Wyoming currently owns Jaguar 4x4 Wheel robots from Dr. Robot. The Jaguar has a high durability design with integrated camera, audio and optional manipulator arm. These features make it ideal for reconnaissance in the rugged terrain experienced by emergency response and military personnel. However, the robot is currently controlled through a game controller with robot data including video sent to a laptop. This inhibits the user's hands while also requiring the presence of a laptop to view vital information from the robot. This is less than ideal for the people likely to use this robot, as they require full use of their hands and carrying a computer is not always possible. The HoloLens is a wireless augmented reality device that generates 2-D and 3-D holograms on a transparent head mounted display (HMD). It utilizes on-board cameras and microphones to offer gesture and voice interaction with applications in a variety of lighting and sound conditions. It is a fully self-contained computer with a modified Windows 10 operating system. This project integrated the Microsoft HoloLens with the Jaguar 4x4 Wheel robot. It involved developing a HoloLens application to implement both voice and gesture control as well as a heads-up display with video and sensor information from the robot. This reduces encumbrance by eliminating the need for a game controller and laptop. Doing so helps accomplish the goal of providing safe and effective use to those most likely to use the Jaguar 4x4 Wheel robot.