Imagine you are driving toward an intersection. A truck parked on the corner blocks your view of a car speeding toward the intersection from the right. You can’t see it, but the car next to you can. What if that neighboring vehicle could simply tell your car,

This simple idea of cars helping each other by describing what they see in natural language motivates UNCAP, our new framework for cooperative autonomous driving. But there’s an important twist: not every neighboring vehicle is worth listening to, and not every message deserves the same level of trust.

UNCAP teaches autonomous vehicles both who to talk to as well as how much to trust what they hear.

Researchers have since long envisioned fleets of autonomous vehicles cooperating to make driving safer. Blind corners, occlusions, and hidden pedestrians will become much less dangerous if nearby cars can share what they observe.

The traditional approach is straightforward: cars can exchange raw sensor data such as camera images, LiDAR point clouds, or radar measurements. However, this quickly becomes impractical to be implemented at scale for a variety of reasons. Raw sensor streams consume enormous bandwidth, require significant computation to process stream data, and assume that every vehicle has comparable sensing hardware. In reality, with the rise of autonomous driving, the roads are instead observing a mixture of manufacturers, sensor configurations, and heterogenous software stacks. Natural language offers an intriguing alternative to sensor data sharing. Instead of transmitting megabytes of images, a vehicle might simply say, “pedestrian crossing behind bus” or “motorcycle approaching from the left, fast-moving” .

Source link