Shadow-Catcher: Looking into Shadows to Detect Ghost Objects in Autonomous Vehicle 3D Sensing

Abstract

LiDAR-driven 3D sensing allows new generations of vehicles to achieve advanced levels of situation awareness. However, recent works have demonstrated that physical adversaries can spoof LiDAR return signals and deceive 3D object detectors to erroneously detect “ghost" objects. Existing defenses are either impractical or focus only on vehicles. Unfortunately, it is easier to spoof smaller objects such as pedestrians and cyclists, but harder to defend against and can have worse safety implications. To address this gap, we introduce Shadow-Catcher, a set of new techniques embodied in an end-to-end prototype to detect both large and small ghost object attacks on 3D detectors. We characterize a new semantically meaningful physical invariant (3D shadows) which Shadow-Catcher leverages for validating objects. Our evaluation on the KITTI dataset shows that Shadow-Catcher consistently achieves more than 94% accuracy in identifying anomalous shadows for vehicles, pedestrians, and cyclists, while it remains robust to a novel class of strong “invalidation” attacks targeting the defense system. Shadow-Catcher can achieve real-time detection, requiring only between 0.003 s–0.021 s on average to process an object in a 3D point cloud on commodity hardware and achieves a 2.17x speedup compared to prior work.

Appendices

Appendices

A Limitation of Prior Art

Recently, Sun et al. proposed CARLO [24], a system for detecting model-level LiDAR spoofing attacks. CARLO consists of two components. The first, Laser Penetration Detection (LPD), serves as a quick anomaly detector to filter fake and valid objects. Objects for which LPD is not confident in its decision are sent for further analysis to a second component, the Free Space Detection (FSD), which is computationally more expensive. LPD’s design intuition is that points in the frustum correlate with occlusion patterns, and hence, uses the ratio of the number of points behind the object’s bounding box over the total the number of points in the frustum of an object; objects with high ratio are classified as suspicious or definitely fake. This approach uses points in the bounding box (as part of the frustum), and for smaller objects, the ratio is small and heavily influenced by noisy LiDAR measurements. Moreover, the approach does not take into account the location and characteristics of points in the region behind the bounding box, and could be susceptible to false positives from noise artifacts. FSD’s detection is based on the intuition that genuine vehicles have high density of points and hence, low free space in the bounding boxes as most of the space in the bounding box should be occluded by points in front. However, for smaller objects, this approach might be ineffective as the original space in the bounding box is small and mostly occupied by the points. Hence there are limited regions for analysis of free space. We implemented CARLO and evaluated its effectiveness to distinguish genuine from spoofed pedestrian objects.

LPD Evaluated on Pedestrians. To evaluate the LPD ratios of genuine and spoofed pedestrians, we collected the LPD ratios of genuine pedestrian objects in the KITTI dataset as well 200 spoofed front-near pedestrians (6 m in front of ego-vehicle). Figure 6 shows the distribution of LPD ratios of genuine and spoofed pedestrians. We observe that there is an overlap of the two distributions from 0.5 to 0.8, which presents opportunities for attackers to invoke FSD. Additionally, as the LPD ratio’s denominator accounts for all the points in the frustum, and for small objects the number of points in frustum is small, there is a possibility of an attacker to inject points (within the total adversary $\mathcal {A}$ budget) in the frustum to lower the ratio to trigger FSD.

Fig. 6.

LPD ratio distribution of genuine and spoofed pedestrian objects.

Fig. 7.

FSD ratio distribution of pedestrian objects with full and down-sampled point cloud.

FSD Evaluated on Pedestrians. We randomly sampled 60 genuine pedestrian objects from KITTI and injected their point cloud 6 m in front of the ego-vehicle to spoof a front-near obstacle. Using the same 60 pedestrian objects, these objects’ point clouds were also down-sampled to the size of 60 points (below adversary $\mathcal {A}$’s budegt of 200 points) and were similarly injected (Point-GNN detected all down-sampled traces as pedestrians). We then used the implementation of Free Space Detection (FSD) in CARLO to evaluate the FSD ratio of spoofed objects with the full-sized and down-sampled point clouds. Figure 7 shows that the distribution of FSD ratio overlaps for pedestrians objects of full and down-sampled point clouds, with the majority of them having an FSD ratio of 0. This shows that the approach of FSD will not result in a separable distribution to effectively distinguish small spoofed pedestrians from genuine pedestrians. FSD expects ghost objects to result in very high FSD ratios which as we showed does not happen for small objects.

B 2D Shadow Region Estimation

We analyze Shadow-Catcher’s accuracy of 2D shadow region generation by comparing it with the 597 manually labeled shadows (see Sect. 4). We evaluate the 2D region generation separately since 3D regions build on top of it. The significance of 2D vs 3D region estimations in the detection performance is evaluated separately in Subsect. 6. To quantify how closely Shadow-Catcher can match the objects’ observed shadows, we measure their Intersection over the Union (IoU) and perform a Procrustes shape analysis. An IoU value of 1 means that the two regions are perfectly matched and 0 means the two regions are disjoint. Procrustes provides us with two metrics: (a) similarity of the shapes; and (b) scale differences of the shapes [1, 12, 19]. For similarity, values close to 1 mean that the shapes are identical. For scale, a value of 1 means that the size of the shapes are identical and anything less than 1 means the ground-truth shadow shape is smaller, and larger than 1 is the opposite.

Table 5. Aggregated correspondence metrics of all objects

Table 5 summarizes our results across all object types. Detailed results are deferred to the project website [2]. From the median values of the corresponding metrics, it can be observed that, for more than half the objects, the computed shadow matches closely with the ground-truth shadow—IoU, Similarity and Scale values are well above 0.5 which indicates a good prediction (object detection bounding box accuracy is commonly evaluated at IoU $\ge 0.5$  [8, 18]). We do observe some variation in the results which can be attributed to measurement inaccuracies and human-errors in the labeling process, and to over-estimation of shadow areas (Illustration provided on the project’s website [2]). Shadow-Catcher uses bounding boxes which are larger than the actual objects and this results in larger shadow regions. However, Shadow-Catcher’s exponential decay approach to weighting the significance of 3D points in shadows (see Sect. 5) compensates for this. This is verified with Shadow-Catcher’s overall accuracy in detecting genuine shadows, ghost and invalidation attacks (see Sect. 6).