As a team from the University of Pennsylvania, U.S. Army Research Laboratory and New York University demonstrated, supervising a drone could be as easy while using the pair of eye-tracking glasses and then consciously moving one’s gaze to lead it to where it wants to go.
The objective of the team was to produce an intuitive and non-invasive way for people to control an aerial vehicle at remote places. While there have been some past attempts at progressing the vision-based process of drone control, the difference is that this is a standalone system that doesn’t use any external sensors to track the drone, relative to the person who is in control.
The process of navigation is relative to the user, rather being relative to the drone, meaning that all points of orientation are taken from the user’s standpoint. For example, if a user asks the drone to go right, it moves to the user’s right, rather than going to the drone’s right, which will appear to the user as the drone going left.
The team has configured, without requiring to incorporate external systems like motion-capture technology or GPS to track the positions of user and drone relative to each other. The system involves some components like the Tobii Pro Glasses 2, gaze-tracking wearable which is designed with an inertial measurement unit (IMU), a high-definition camera, and an NVIDIA Jetson TX2 CPU and GPU to support with processing data, using a deep neural network.
The user can detect the drone via glasses and from the size of a quadrotor drone, the processor will examine approximately how long it’s relative position is.
This result gives the chance to establish new, non-invasive forms of interactions between a human and robots permitting the human to send new 3D-navigation waypoints to the robot. The team is now working to clear the way the system translates two-dimensional gaze coordinates into 3D navigational waypoints. According to the team, 3D navigation waypoint would come directly from the eye tracking glasses, but it was found that the depth component generated by the glasses was too noisy to use effectively.
In future, researchers are hoping to explore this issue in order to provide the user with more control over depth. The target is to generate new human-machine interfaces that are intuitive and responsive.
Watch a video of drone controlled by eye movements:
