Using Augmentation Techniques for Performance Evaluation in Automotive Safety

Abstract

This chapter describes a framework which uses augmentation techniques for performance evaluation of mobile computer vision systems. Computer vision systems use primarily image data to interpret the surrounding world, e.g. to detect, classify and track objects. The performance of mobile computer vision systems acting in unknown environments is inherently difficult to evaluate since, often, obtaining ground truth data is problematic. The proposed novel framework exploits the possibility to add new agents into a real data sequence collected in an unknown environment, thus making it possible to efficiently create augmented data sequences, including ground truth, to be used for performance evaluation. Varying the content in the data sequence by adding different agents or changing the behavior of an agent is straightforward, making the proposed framework very flexible. A key driver for using augmentation techniques to address computer vision performance is that the vision system output may be sensitive to the background data content. The method has been implemented and tested on a pedestrian detection system used for automotive collision avoidance. Results show that the method has potential to replace and complement physical testing, for instance by creating collision scenarios, which are difficult to test in reality, in particular in a real traffic environment.