New Research Probes Effectiveness of AR to Improve Self-Driving Car Safety

As self-driving cars become more common on our roads, a key question for future road safety is balancing passengers’ desire to relax during their trip with their awareness of road hazards and readiness to retake control.

Researchers from the University of Glasgow have been testing the potential of augmented reality technology to allow drivers to enjoy the benefits of being driven by an autonomous vehicle while enabling them to take the wheel quickly if necessary.

Their results suggest that placing attention-grabbing graphics over real-world views through car windscreens using augmented reality heads-up displays (HUDs) could help drivers use entertainment apps while maintaining awareness of the road. However, their design must be carefully managed to avoid overwhelming users with information at critical moments.

Their findings inform the development of safety features in future generations of self-driving cars as they move towards the complete autonomous control planned by manufacturers.

Thomas Goodge of the University of Glasgow’s School of Computing Science is one of the paper’s co-authors. He said: “One of the main attractions of autonomous vehicles is that drivers will eventually be able to take their focus off the road and place it on non-driving-related tasks like reading, playing games, or watching TV.

“Drivers can do that to some extent at the moment, but they still have to maintain an awareness of road conditions to take control in emergencies. It will likely be years before they can drive themselves entirely without human intervention.

“In the meantime, the people sitting behind the wheel are somewhere between drivers and passengers – not actively involved in steering but still required to supervise their vehicle to ensure they get safely to their destination.

“One big problem is that humans don’t deal very well with sustained supervisory tasks. They get bored and distracted, and when they do, they lose their awareness of the road. That could be dangerous if they react too slowly to a sudden change in the road conditions around them.

“What we wanted to do with this study was explore whether attention-grabbing augmented reality systems could allow people to perform non-driving tasks but quickly switch focus at critical moments.”

The team’s lab experiment put study participants in a driver’s seat in front of a steering wheel and computer monitors to simulate their view out of a car window. The screens displayed a series of 40 video clips of actual road situations. They asked them to complete tasks on either a tablet screen, similar to the large dashboard displays of current-generation vehicles or while using an augmented reality display overlaying functions on top of the road scene.

One task was a simple game where users tracked and ‘collected’ gems moving across the screen by gazing at them. The second task presented the participants with a more complicated number pad app on which they had to copy a phone number correctly.

In both scenarios, the video stopped abruptly before a potentially hazardous situation, like a pedestrian stepping into the road, to test situational awareness. The participants were asked to select from one of four predictions about what would happen next based on their understanding of the road conditions when the video cut off.

Their ability to correctly guess the outcome of the hazard was measured against an initial control experiment where they made predictions without performing a task. In both the heads-down portion of the task using the tablet and the heads-up section using the AR headset, participants were able to maintain some awareness of the conditions on the road. However, their ability to reliably predict what would happen next was significantly reduced in both setups.

In another section of the experiment, the AR headset displayed particular visual cues in participants’ eyes to draw their attention to developing road situations several seconds before the video stopped. Here, the participants showed increased awareness of the road conditions, improving performance more in the gem-collecting game than in the more demanding keypad app.

Study co-author Professor Frank Pollick of the University’s School of Psychology & Neuroscience said: “An important aspect of road safety is something called the ‘look but fail to see’ phenomenon, which happens when people fail to process what’s right in front of their eyes adequately.

This study aimed to replicate that in a simulated driving situation, where participants had all the relevant information on the screen. Still, they were being distracted to one degree or another, blurring their ability to judge the road conditions correctly.

“Augmented reality tech really does put information right in front of people’s eyes, so it could be a good fit for tackling the look but fail to see the problem. The study shows that adding visual cues to draw drivers’ attention to situations improves their ability to focus and understand situations quickly. However, the level of demand for their attention is important – they were more able to read the road correctly in the less cognitively challenging task.”

Professor Stephen Brewster leads the Multimodal Interaction Group at the University of Glasgow and is a co-author of the paper. He said, “Offering people the opportunity to distract themselves while in a self-driving car comes with potentially catastrophic risks, so we must understand as much as possible about how drivers can be supported to travel responsibly.

“This study suggests that there may be a kind of ‘goldilocks’ zone where people can be engaged in a task while still being kept informed about developing road conditions.

“Much more research will be required to delve into how augmented reality can help support users of self-driving cars in the future, but we hope that the outcomes of this study will help inform the development of augmented reality systems in the years to come.”

The team’s paper, titled ‘Can You Hazard a Guess? Evaluating the Effects of Augmented Reality Cues on Driver Hazard Prediction’ will be presented at the Association of Computing Machinery CHI conference on Human Factors in Computing Systems next month.

The research was supported by funding from the UKRI Centre for Doctoral Training in Socially Intelligent Artificial Agents and the European Research Council.