Effects of Non-Driving Related Task Modality and Difficulty on Drivers’ Visual Attention and Cognitive Load During Automated Driving

This IAA funded driving simulator-based study on automated driving investigated the effect of task modality (auditory, heads-up display and tablet) and task difficulty (0-back and 2-back) of non-driving related task (NDRT), as well as a monitoring drive, on visual scanning patterns and cognitive load of the driver. 32 drivers with a valid UK drivers’ license were recruited for this study, and their eye-tracking, electrocardiogram and electrodermal activity data were collected. In addition to a monitoring drive where the drivers simply monitored the automated driving scenario, drivers were subject to a 3 x 2 repeated measures design of an NDRT in the form of a n-back task, having 3 modality levels (auditory, HUD and tablet) and 2 task difficulty levels (0-back and 2-back). Our results indicated that while auditory tasks did not seem to have a significant effect on drivers’ attention to the road centre region compared to their baseline monitoring drive, drivers had significantly lower glances to the road centre region during the two visual NDRTs, which deteriorated further as the task difficulty of the n-back task increased. Results from mean heart rate, skin conductance responses per minute and subjective workload ratings indicated that drivers’ cognitive load was not affected by the task modality. However, the drivers indicated significantly higher cognitive load during the more demanding 2-back task condition, compared to the 0-back condition. We also found that pupil-based measures were not as reliable as a stand-alone metric to indicate cognitive load, due to various confounding factors. Our results highlight the need for sensor fusion of eye-tracking and physiological sensors in future driver monitoring systems, to provide dedicated support in bringing the driver back into the driving control loop safely, for when they need to resume manual control of the vehicle.