Immersive Simulator Tests ‘Trust’ Between Humans and Self-driving Cars

Mass production of autonomous vehicles is expected to soar by the early 2020s. Consumers have high expectations and a high degree of skepticism when it comes to self-driving cars. According to a World Economic Forum study, consumers are very reluctant to consider purchasing, or even trying autonomous vehicles because of safety, control, and faulty behavior concerns. The issue of trust is a primary challenge for industry professionals trying to popularize the use of fully autonomous systems.

Research on the physiological responses and fluctuations in trust levels of passengers of self-driving cars is sparse and most research to date has used online surveys and primitive simulations for their experiments rather than realistic, immersive simulations.

Researchers from Florida Atlantic University’s College of Engineering and Computer Science have introduced a new approach to measure trust between passengers and self-driving cars in real-time. Their unique, immersive virtual reality (VR) simulation combines visual, audio and movement engagement that provides a convincingly realistic simulation. The simulator they have developed provides a safe platform to expose human subjects to any trust-damaging incidents such as sharp turns, sudden stops, stoplight violations, speeding, tailgating, unexpected accidents, among others. The simulation incorporates a VR headset with a motion chair using 360-degree videos of actual driving scenarios recorded in South Florida’s roads and highways.

“The sequential and structured data collection, various trust states, and the realistic simulation platform we have developed are what makes our research methodology so novel,” said Mehrdad Nojoumian, Ph.D., an assistant professor in FAU’s Department of Computer and Electrical Engineering and Computer Science. “Our simulation approach is helping us to understand how the human mind goes from one specific trust-state to another one in a sequence of events as we gauge various levels of trust and distrust in real-time.”

Nojoumian’s first research project, in collaboration with FAU graduate students Shervin Shahrdar and Corey Park, was an empirical experiment on 50 human subjects, which demonstrated efficacy for collecting real-time data utilizing this simulation method.

“Results from our first experiment indicated that trust levels of humans change depending on the self-driving style. The majority of study participants were able to moderately rebuild their trust in the simulated self-driving car after faulty and erratic behaviors,” said Shahrdar. “The autonomous driving style directly influences the trust of the passengers in the system. For example, aggressive driving diminishes trust, and defensive, predictable driving increases and builds trust.”

This study laid the foundation for Nojoumian’s second experiment on a new group of 50 human subjects, which incorporated electroencephalography (EEG) to non-invasively monitor participants’ brainwave activity. Test subjects were exposed to scenarios designed to induce positive and negative emotional responses, quantified by the EEG beta wave to alpha wave power ratio. The simulation elicited negative emotions in order to assess the level of passenger fear, stress and anxiety in response to autonomous control actions of the self-driving cars. This study was recently featured by the Institute of Electrical and Electronics Engineers (IEEE) innovation spotlight.

“Our previous work required subjects to self-report, while the implementation of an EEG is providing more objective data,” said Nojoumian. “Analysis from our research demonstrates that the model we have developed is highly effective for collecting real-time data from human subjects and lays the foundation for more-involved future research in the domain of human trust and autonomous driving.”

Key future research directions for Nojoumian and his team include automatic collection of the physiological and psychological responses via EEG sensors, heartbeat sensors, facial recognition modules, and other methods during the simulated driving scenarios while analyzing them in real-time.

“Results from professor Nojoumian’s research will help to inform the design and operation of an EEG-based supervisory feedback control module or artificial intelligence that monitors the emotional state of passengers and adjusts the AI control parameters accordingly,” said Stella Batalama, Ph.D., dean of FAU’s College of Engineering and Computer Science. “The ultimate goal is to create a resilient supervisory feedback control module that monitors the passenger’s state and acts as a feedback loop to modulate the control actions of the self-driving car.”

The results of this study were presented in the Association for the Advancement of Artificial Intelligence/Association for Computing Machinery (AAAI/ACM) “Conference on Artificial Intelligence, Ethics and Society” and published in the AAAI/ACM AIES’19 proceedings.