15 · What the LiDAR Sees
Spin a real laser scan of a busy Boston intersection: which road users does the self-driving car actually see?
A LiDAR on the roof of a self-driving car fires laser pulses all around it and measures where each one bounces back: a point cloud. Below are 20 seconds of real scans from a car arriving at a busy intersection in Boston, one scan per second. People labelled every road user by hand, as a box. A computer can only detect what the laser actually hits: the boxes are green when they hold at least the number of points you choose, and red when they hold fewer.
A car 10 metres away and a car 40 metres away, both in plain view. The far one is four times as far. How many fewer laser points land on it: 4 times fewer, 16 times fewer, or more than 100 times fewer?
Data: nuScenes (Caesar et al., 2020), scene-0757, Boston Seaport, licensed CC BY-NC-SA 4.0. One scan per second, points within 40 m, thinned to 5,000 per scan for the web; the point counts in the boxes are nuScenes' own counts from the full scans.
What it shows
Distance alone would cost a far car only 16 times fewer points at four times the distance (the laser beams spread out in two directions). The real loss is far bigger, because far objects are also hidden: behind other cars, buses and street furniture, and between the laser’s rings: its beams fan out, and at 40 m neighbouring rings are about a metre apart. A road user the LiDAR misses is not in the self-driving car’s picture of the street, however real it is. This is why automated vehicles combine LiDAR with cameras and radar, and why roadside sensors help (Lecture 2).
Data: Caesar, H. et al. (2020). nuScenes: A multimodal dataset for autonomous driving. CVPR 2020. Licensed under CC BY-NC-SA 4.0.