RAPID: Resilient multi-Agent Perception and Planning In Dynamic Domains

RAPID develops resilient, computationally-aware algorithms for distributed multi-agent perception and planning, for teams operating in large, dynamic and uncertain environments where centralized coordination is not available

Back to Research

In applications of interest to the US Navy, realizing the vision of intelligent autonomous systems capable of understanding and responding to mission-relevant objects, actors, and the environment still necessitates the solution to several open fundamental questions. Key challenges include dealing with dynamically evolving environments that require real-time extraction and processing of task-relevant information from heterogeneous tactical and intelligence streaming data obtained by sensors, machines, and humans; building hierarchical representations of the environment that allow reasoning at different levels of granularity; integrating perception and planning to support collaborative and distributed task execution with formal performance guarantees; resilient algorithms that go beyond offline training, which relies on massive amounts of data to account for the uncertainty of “known unknowns” but often fails to perform in op-tempo deployments in the field; computationally-aware, efficient algorithms that can predict actions of red forces in the field, can reason about significant changes over pre-computed plans and evolving scenarios, perform decision-making over short timespans, and deal with environments unlike those seen in simulation; and the orchestration of multi-agent teams to enable robust and distributed intelligence reaping the benefits of collaborative planning under partial and spatially fragmented information, different processing, sensing, communication, and control capabilities, and asynchronous decision-making. Current coordination algorithms for multi-agent perception, planning, and control do not scale well because they rely on centralized architectures. Such dependency is unrealistic in the scenarios of interest to the Navy, where agents operate in large dynamic and uncertain environments, over long time scales, and with limited communication.

A 3D scene graph of an indoor environment: a building node at the top branching into rooms, then places, then objects, drawn over a semantic reconstruction of the floor. The full graph on the left is dense with edges; the compressed graph on the right keeps only a sparse set of nodes, with purple circles marking the terminal nodes preserved by the compression 3D Scene Graph of an environment (left) and compressed version produced by D-Lite (right). The purple circles mark the terminal nodes: the proposed compression approach approximately preserves shortest path lengths between those locations of interest.

The proposed research on Resilient multi-Agent Perception and Planning In Dynamic Domains (RAPID) addresses the challenges laid out above to create high-confidence, high-reliability, minimal-supervision autonomous systems that can understand and act in high op-tempo missions. Our synergistic research plan is organized along four research thrusts that tackle key objectives pertaining to hierarchical, multi-scale, and resilient situational awareness in dynamic environments; the integration of perception and planning; distributed methods for perception and prediction; and distributed planning and control algorithms for multi-agent teams deployed in dynamic environments.

In particular, Thrust 1 focuses on hierarchical and resilient reasoning about dynamic environments at multiple temporal and spatial scales informed by metric and semantic information. Thrust 2 develops integrated hierarchical perception and planning to support decision-making at multiple levels of spatial, semantic, and temporal abstraction, providing the foundation to tackle communication-dependent multi-agent scenarios. Thrust 3 builds situational awareness and hierarchical abstractions from streaming data for multi-agent teams to enable operation in a distributed fashion. Thrust 4 complements the distributed perception techniques in Thrust 3 by focusing on the design of scalable distributed algorithms for multi-agent planning and optimal control under communication, dynamics, and perceptual constraints. The team’s expertise in machine perception, robot autonomy, multi-robot coordination, optimal control, planning, and distributed optimization is suited to achieve RAPID’s ambitious integrative research agenda.

This project is in collaboration with Jorge Cortés, Nikolay Atanasov, and Luca Carlone.

Sponsors

This project is funded by ONR.

Selected Publications and Presentations

1.

Knaup, J., D'sa J., Chalaki, B., Naes, T., Mahjoub, H.-N., Moradi-Pari, E., and Tsiotras, P., “Active Learning with Dual Model Predictive Path-Integral Control for Interaction-Aware Autonomous Highway On-ramp Merging,” IEEE International Conference on Robotics and Automation, Yokohama, Japan, May 13-17, 2024, pp. 14191-14197.

PDF DOI
2.

Psomiadis, E., Maity, D., and Tsiotras, P., “Communication-Aware Map Compression for Online Path-Planning,” IEEE International Conference on Robotics and Automation, Yokohama, Japan, May 13-17, 2024, pp. 12368-12374.

PDF DOI
3.

Tsiotras, P., Gombolay, M., and Foerster, J., “Editorial: Decision-Making and Planning for Multi-Agent Systems,” Frontiers in Robotics and AI, May 2024.

PDF DOI
4.

Afshari, M., Maity, D., and Tsiotras, P., “Communication- and Control-aware Optimal Quantizer Selection for Multi-agent Control,” IEEE Control Systems Letters, Vol. 8, pp. 2385-2390, 2024.

PDF DOI
5.

Psomiadis, E., Maity, D., and Tsiotras, P., “Multi-agent Task-Driven Exploration via Intelligent Map Compression and Sharing,” 17th International Symposium on Distributed Autonomous Robotic Systems (DARS), Cornell Tech, New York, NY, Oct. 28-30, 2024.

6.

Yin, J., Tsiotras, P., and Berntorp, K., “Chance-Constrained Information-Theoretic Stochastic Model Predictive Control with Safety Shielding,” 63rd IEEE Conference on Decision and Control, Milan, Italy, Dec. 16-19, 2024, pp. 653-658.

PDF DOI
7.

Knaup, J., and Tsiotras, P., “Adaptive Dual Covariance Steering with Active Parameter Estimation,” 63rd IEEE Conference on Decision and Control, Milan, Italy, Dec. 16-19, 2024, pp. 659-664.

PDF DOI
8.

Psomiadis, E., Pedram, A.-R., Maity, D., and Tsiotras, P., “Communication-Aware Iterative Map Compression for Online Path-Planning,” IEEE International Conference on Robotics and Automation, Atlanta, GA, May 19-23, 2025, pp. 6580-6586.

PDF DOI
9.

Parashar, A., Yin, J., Dawson, C., Tsiotras, P., and Fan, C., “Learning-based Bayesian Inference for Testing of Autonomous Systems,” IEEE International Conference on Robotics and Automation, Atlanta, GA, May 19-23, 2025.

DOI
10.

Psomiadis, E. and Tsiotras, P., “HCOA*: Hierarchical Class-ordered A* for Navigation in Semantic Environments,” IEEE Robotics and Automation Letters, Vol. 10, No. 10, pp. 10282-10289, 2025.

PDF DOI
11.

Knoedler, L., So, O., Yin, J., Black, M., Serlin, Z., Tsiotras, P., Alonso-Mora, J., and Fan, C., “Safety on the Fly: Constructing Robust Safety Filters via Policy Control Barrier Functions at Runtime,” IEEE Robotics and Automation Letters, Vol. 10, No. 10, pp. 10058-10065, 2025.

PDF DOI
12.

Pedram, A.-R., Psomiadis, E., Maity, D., and Tsiotras, P., “Communication-Aware Map Compression for Online Path-Planning: A Rate-Distortion Approach,” 2025.

arXiv