Abstract
Recently, adversarial networks have attracted increasing attentions for the promising results of generative tasks. In this paper we present the first application of conditional adversarial networks to stereo matching task. Our approach performs a conditional adversarial training process on two networks: a generator that learns the mapping from a pair of RGB images to a dense disparity map, and a discriminator that distinguishes whether the disparity map comes from the ground truth or from the generator. Here, both the generator and the discriminator take the same RGB image pair as an input condition. During this conditional adversarial training process, our discriminator gradually captures high-level contextual features to detect inconsistencies between the ground truth and the generated disparity maps. These high-level contextual features are incorporated into loss function in order to further help the generator to correct predicted disparity maps. We evaluate our model on the Scene Flow dataset and an improvement is achieved compared with the most related work pix2pix.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
Bromley, J., Guyon, I., LeCun, Y., et al.: Signature verification using a s̈iameseẗime delay neural network. In: Advances in Neural Information Processing Systems, pp. 737–744 (1994)
Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. Appl. 6, 35–49 (1993)
Gidaris, S., Komodakis, N.: Detect, Replace, Refine: Deep Structured Prediction for Pixel Wise Labeling. arXiv preprint arXiv:1612.04770 (2016)
Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (2014)
Hirschmuller, H.: Accurate and efficient stereo processing by semiglobal matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 807–814. IEEE (2005)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Isola, P., Zhu, J.Y., Zhou, T., et al.: Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Luc, P., Couprie, C., Chintala, S., et al.: Semantic Segmentation using Adversarial Networks. arXiv preprint arXiv:1611.08408 (2016)
Luo, W., Schwing, A.G., Urtasun, R.: Efficient deep learning for stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5695–5703 (2016)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Mayer, N., Ilg, E., Hausser, P., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), arXiv:1512.02134 (2016)
Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1520–1528 (2015)
Park, M.G., Yoon, K.J.: Leveraging stereo matching with learning-based confidence measures. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 101–109 (2015)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi:10.1007/978-3-319-24574-4_28
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2007)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)
Seki, A., Pollefeys, M.: Patch based confidence prediction for dense disparity map. In: British Machine Vision Conference, 10 September 2016
Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1592–1599 (2015)
Acknowledgments
This paper is supported by NSFC(No.61772330, 61272247, 61533012, 61472075), the 863 National High Technology Re-search and Development Program of China (SS2015AA020501), the Basic Research Project of Innovation Action Plan (16JC1402800) and the Major Basic Research Program (15JC1400103) of Shanghai Science and Technology Committee.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Huang, H., Huang, B., Lu, H., Weng, H. (2017). Stereo Matching Using Conditional Adversarial Networks. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10636. Springer, Cham. https://doi.org/10.1007/978-3-319-70090-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-70090-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70089-2
Online ISBN: 978-3-319-70090-8
eBook Packages: Computer ScienceComputer Science (R0)