[NeurIPS2022] Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps

Abstract

Multi-agent collaborative perception could significantly upgrade the perception performance by enabling agents to share complementary information with each other through communication. It inevitably results in a fundamental trade-off between perception performance and communication bandwidth. To tackle this bottleneck issue, we propose a spatial confidence map, which reflects the spatial heterogeneity of perceptual information. It empowers agents to only share spatially sparse, yet perceptually critical information, contributing to where to commu- nicate. Based on this novel spatial confidence map, we propose Where2comm, a communication-efficient collaborative perception framework. Where2comm has two distinct advantages, i) it considers pragmatic compression and uses less communication to achieve higher perception performance by focusing on perceptually critical areas; and ii) it can handle varying communication band- width by dynamically adjusting spatial areas involved in communication. To evaluate Where2comm, we consider 3D object detection in both real-world and simulation scenarios with two modalities (camera/LiDAR) and two agent types (cars/drones) on four datasets, OPV2V, V2X-Sim, DAIR-V2X, and our origi- nal CoPerception-UAVs. Where2comm consistently outperforms previous meth- ods; for example, it achieves more than 100, 000× lower communication volume and still outperforms DiscoNet and V2X-ViT on OPV2V.