Abstract
Open-set object detection better simulates the real world compared with close-set object detection. Besides the classes of interest, it also pays attention to unknown objects in the environment. We extend the previous concept of open-set object detection, aiming to detect both known and unknown objects. Because unknown objects have different textural features from known classes and the background, we assume that detecting unknown instances will generate high uncertainty. Therefore, in this paper, we propose an uncertainty-aware open-set object detection framework based on faster R-CNN. We introduce evidential deep learning to the field of object detection to estimate the uncertainty of the predictions and perform more accurate classification in open-set conditions. The obtained uncertainty will be utilized to pseudo-label unknown instances in the training data. We also introduce a contrastive clustering module to separate the feature representations of each class during the training phase. We set an uncertainty-based unknown identifier at the inference phase to enhance the generalization of the detector. We conduct experiments on three different data splits, and our method outperforms the recent SOTA method. We also demonstrate each component in our method is effective and indispensable in our ablation studies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bendale, A., Boult, T.E.: Towards open set deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1563–1572 (2016)
Chen, L., Lou, Y., He, J., Bai, T., Deng, M.: Evidential neighborhood contrastive learning for universal domain adaptation (2022)
Dempster, A.P.: A generalization of Bayesian inference. J. Roy. Stat. Soc.: Ser. B (Methodol.) 30(2), 205–232 (1968)
Dhamija, A., Gunther, M., Ventura, J., Boult, T.: The overlooked elephant of object detection: open set. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1021–1030 (2020)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059. PMLR (2016)
Gawlikowski, J., et al.: A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 (2021)
Geng, C., Huang, S.J., Chen, S.: Recent advances in open set recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3614–3631 (2020)
Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)
Han, Z., Zhang, C., Fu, H., Zhou, J.T.: Trusted multi-view classification. arXiv preprint arXiv:2102.02051 (2021)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., Makedon, F.: A survey on contrastive self-supervised learning. Technologies 9(1), 2 (2020)
Joseph, K., Khan, S., Khan, F.S., Balasubramanian, V.N.: Towards open world object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5830–5840 (2021)
Jøsang, A.: Subjective Logic: A Formalism for Reasoning Under Uncertainty. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42337-1
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Liu, W., Yue, X., Chen, Y., Denoeux, T.: Trusted multi-view deep learning with opinion aggregation. In: The 36th AAAI Conference on Artificial Intelligence (AAAI-2022), vol. 36, pp. 7585–7593 (2022)
Malinin, A., Gales, M.: Predictive uncertainty estimation via prior networks. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Miller, D., Nicholson, L., Dayoub, F., Sünderhauf, N.: Dropout sampling for robust object detection in open-set conditions. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3243–3249. IEEE (2018)
Miller, D., Sünderhauf, N., Milford, M., Dayoub, F.: Uncertainty for identifying open-set errors in visual object detection. IEEE Robot. Autom. Lett. 7(1), 215–222 (2022). https://doi.org/10.1109/LRA.2021.3123374
Miller, D., Sünderhauf, N., Milford, M., Dayoub, F.: Uncertainty for identifying open-set errors in visual object detection. IEEE Robot. Autom. Lett. 7(1), 215–222 (2021)
Molchanov, D., Lyzhov, A., Molchanova, Y., Ashukha, A., Vetrov, D.: Greedy policy search: a simple baseline for learnable test-time augmentation. arXiv preprint arXiv:2002.09103 (2020)
Neal, L., Olson, M., Fern, X., Wong, W.-K., Li, F.: Open set learning with counterfactual images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 620–635. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_38
Oberdiek, P., Rottmann, M., Gottschalk, H.: Classification uncertainty of deep neural networks based on gradient information. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS (LNAI), vol. 11081, pp. 113–125. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99978-4_9
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Salimans, T., Kingma, D.P.: Weight normalization: a simple reparameterization to accelerate training of deep neural networks. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Scheirer, W.J., Jain, L.P., Boult, T.E.: Probability models for open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2317–2324 (2014)
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2012)
Sensoy, M., Kaplan, L., Kandemir, M.: Evidential deep learning to quantify classification uncertainty. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Stevens, W.: Efficient uncertainty estimation for open-set object detection. In: Epistemic Uncertainty Estimation for Object Detection in Open-Set Conditions, p. 91 (2021)
Valdenegro-Toro, M.: Deep sub-ensembles for fast uncertainty estimation in image classification. arXiv preprint arXiv:1910.08168 (2019)
Welling, M., Teh, Y.W.: Bayesian learning via stochastic gradient Langevin dynamics. In: Proceedings of the 28th International Conference on Machine Learning (ICML-2011), pp. 681–688 (2011)
Wen, Y., Liu, W., Weller, A., Raj, B., Singh, R.: SphereFace2: binary classification is all you need for deep face recognition. arXiv preprint arXiv:2108.01513 (2021)
Wen, Y., Tran, D., Ba, J.: BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715 (2020)
Yue, X., Chen, Y., Yuan, B., Lv, Y.: Three-way image classification with evidential deep convolutional neural networks. Cogn. Comput. 1–13 (2021). https://doi.org/10.1007/s12559-021-09869-y
Zhou, X., Yue, X., Xu, Z., Denoeux, T., Chen, Y.: Deep neural networks with prior evidence for bladder cancer staging. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1221–1226. IEEE (2021)
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)
Acknowledgments
This work was supported by National Natural Science Foundation of China (Serial Nos. 61976134, 61991410, 61991415), Natural Science Foundation of Shanghai (Serial No. 21ZR1423900) and Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province, China (No. CICIP2021001).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hang, Q., Li, Z., Dong, Y., Yue, X. (2022). Uncertainty-Aware Deep Open-Set Object Detection. In: Yao, J., Fujita, H., Yue, X., Miao, D., Grzymala-Busse, J., Li, F. (eds) Rough Sets. IJCRS 2022. Lecture Notes in Computer Science(), vol 13633. Springer, Cham. https://doi.org/10.1007/978-3-031-21244-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-21244-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21243-7
Online ISBN: 978-3-031-21244-4
eBook Packages: Computer ScienceComputer Science (R0)