NASCAR was a MURI activity between Georgia Tech, MIT and the University of Southern California. Its objective was to develop perception and planning algorithms that would achieve a step change in the agility and speed of autonomous and semi-autonomous ground vehicles.
The bandwidth problem that motivates the project. A driver takes in gigabits per second of visual information and acts on it through a few kilobits per second of steering, throttle and braking commands, discarding almost everything in between.
The work proposed a departure from the standard practice of perception and sensing then control, promoting instead a paradigm of perception and sensing for control. The means were attention-focused, adaptive perception algorithms operating on actionable data in a timely manner. The inspiration came from human models of cognition and execution, and in particular from those of expert race drivers: by encapsulating their cognitive and reflexive planning layers, the algorithms could make extensive use of prior, context-dependent information at both the sensing and the execution level. Attention served as the mediator. Analyzing the saliency of a scene locates the hot-spots that serve as anchors for events, and where an expert driver looks can itself be learned from the expert.
The attention pipeline: a bottom-up saliency map identifies where to look, while the gist of the scene supports place recognition and localization. Figure courtesy of the iLab at the University of Southern California, a partner on the MURI.
Attention alone is not enough at the limits of handling, so the visual channel was fused with proprioceptive sensing to deduce terrain properties and friction characteristics, which then feed predictive and proactive control rather than being used reactively after the fact.
Exteroceptive sensing on the vehicle. The lidar returns are classified into drivable surface and obstacles, and the result is passed to the planner as it is produced rather than as a finished map.
Professional drivers use a repertoire of specialized maneuvers whose purpose is to shed excess speed on corner entry, to reorient the vehicle quickly inside the corner so that it can exit at maximum speed, or to connect two corners back to back, and they exploit the load transfer that braking and acceleration produce. As the way these skills are taught makes clear, such a maneuver is defined not only by the control actions that execute it but equally by the sensory cues and the attention it demands. In a departure from earlier work, we therefore defined motion primitives that integrate the sensing and the control technique together: the specialized techniques of expert rally drivers were studied, analyzed within a formal mathematical framework, examined for their optimality and robustness properties, and finally expressed as motion primitives combining an integrated sensing and control strategy.
A rally driver shedding speed and rotating the car on corner entry, one of the maneuvers formalized as a motion primitive.
Sponsors
Support for this project is provided by ARO.
Selected Publications and Presentations
Publications from this MURI are listed on the project website, amav.gatech.edu.