Advanced Driver Assistance and Active Safety Systems through Driver’s Controllability Augmentation and Adaptation

Driven by federal directives and by public pressure to increase safety, several automotive manufacturers have declared their intention to radically reduce traffic accidents, or to eliminate them altogether, through on-board active safety control systems (ASCS). An accident-free vehicle, however, requires changing the way current ASCS operate and, above all, the way they interact with the human driver. The goal of this project was to introduce learning and adaptation into the next generation of ASCS, so that the assistance the vehicle offers is customized to the particular driver using it. The work was carried out in collaboration with Ford.

The sensor suite an active safety system can draw on, and the range each sensor covers, from the few metres immediately around the vehicle out to 300 metres ahead.

At the centre of the work is a control framework for the next generation of drive management systems (DMS) for passenger vehicles. Drawing on computational neuroscience, adaptive control, gain scheduling, model predictive control and situational-awareness sensing, the framework sets out to learn the driver’s habits, driving skills and comfort level; to model the driver’s cognitive state, such as whether the driver is attentive, along with the state of the vehicle, such as braking hard, and of the environment, such as being in a left turn; and then to adapt the DMS to the situation at hand. So configured, the DMS can keep the driver from taking an action that would violate the safe operating limits of the vehicle, and can help the driver recover once those limits have been crossed. The approach deliberately borrows validated techniques and practice from the aerospace industry, where fly-by-wire and flight management systems have improved the safety, reliability, performance and fuel efficiency of civil and military aircraft over the last three decades.

Architecture of the drive management system. An Intelligent Personal Minder, built from a driver reference model and a driver state model, informs the arbitration logic that mediates between the stability controller and the other on-board controllers before any command reaches the actuators.

To make this possible we developed methods for inferring both the long-term and the short-term behavior of drivers in traffic, using dynamic Bayesian networks and neuromorphic algorithms that estimate a driver’s skill and current state of attention from eye-movement data, together with motion cues taken from video of the driver’s view and from the steering and pedal inputs. We then gave a mathematically rigorous way of infusing that information into the active safety system, using parameter-dependent Lyapunov functions and results from adaptive, gain-scheduled and model predictive control.

Predicting where a driver looks. A bottom-up saliency model and a top-down model learned from gist features are combined into a priority map, and the predicted fixation is compared against the measured eye position. Figure courtesy of the iLab at the University of Southern California.

Two concrete systems came out of this line of work. The first is a lane-tracking steering assist built around a two-point visual driver model, in which the near and far points a driver actually fixates on while cornering become the inputs to the assist system, so that the assistance follows the same cues the driver uses. The second addresses the moment when a collision can no longer be avoided: rather than continuing to brake, the vehicle posture is controlled in minimum time, using only conventional control inputs, so that the vehicle meets the impact in the orientation that mitigates the consequences for its occupants.

Sponsors

This project is supported by NSF and Ford.

Collaborators

Laurent Itti, University of Southern California, and Ford