Skip to main content
SpringerLink
Log in

Deblurring Shaken and Partially Saturated Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We address the problem of deblurring images degraded by camera shake blur and saturated (over-exposed) pixels. Saturated pixels violate the common assumption that the image-formation process is linear, and often cause ringing in deblurred outputs. We provide an analysis of ringing in general, and show that in order to prevent ringing, it is insufficient to simply discard saturated pixels. We show that even when saturated pixels are removed, ringing is caused by attempting to estimate the values of latent pixels that are brighter than the sensor’s maximum output. Estimating these latent pixels is likely to cause large errors, and these errors propagate across the rest of the image in the form of ringing. We propose a new deblurring algorithm that locates these error-prone bright pixels in the latent sharp image, and by decoupling them from the remainder of the latent image, greatly reduces ringing. In addition, we propose an approximate forward model for saturated images, which allows us to estimate these error-prone pixels separately without causing artefacts. Results are shown for non-blind deblurring of real photographs containing saturated regions, demonstrating improved deblurred image quality compared to previous work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. We note that the regulariser \(\varvec{\phi }\) in Eq. (26) is not continuously differentiable, however this can be avoided by using the standard approximation \( |x|\simeq \sqrt{\epsilon +x^2}\), for some small number \(\epsilon \).

  2. http://cg.postech.ac.kr/research/deconv_outliers/ (accessed November 12, 2011).

References

  • Afonso, M., Bioucas-Dias, J., & Figueiredo, M. (2010). Fast image recovery using variable splitting and constrained optimization. IEEE Transactions on Image Processing, 19(9), 2345–2356.

    Article  MathSciNet  Google Scholar 

  • Almeida, M. S. C., & Figueiredo, M. A. T. (2013). Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Transactions on Image Processing, 22(8), 3074–3086.

    Article  MathSciNet  Google Scholar 

  • Bar, L., Sochen, N., & Kiryati, N. (2006). Image deblurring in the presence of impulsive noise. International Journal of Computer Vision, 70(3), 279–298.

    Article  Google Scholar 

  • Blake, A., & Zisserman, A. (1987). Visual Reconstruction. London: MIT Press.

    Google Scholar 

  • Bouman, C., & Sauer, K. (1993). A generalized Gaussian image model for edge-preserving MAP estimation. IEEE Transactions on Image Processing, 2(3), 296–310.

    Article  Google Scholar 

  • Cai, J.-F., Ji, H., Liu, C., Shen, Z (2009). Blind motion deblurring from a single image using sparse approximation. In: Proc. CVPR

  • Chen, C., & Mangasarian, O. L. (1996). A class of smoothing functions for nonlinear and mixed complementarity problems. Computational Optimization and Applications, 5(2), 97–138.

    Article  MathSciNet  MATH  Google Scholar 

  • Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145–148.

    Article  Google Scholar 

  • Cho, S., Wang, J., & Lee, S. (2011). Handling outliers in non-blind image deconvolution. In:Proc ICCV

  • Chou, P. B., & Brown, C. M. (1990). The theory and practice of Bayesian image labeling. International Journal of Computer Vision, 4(3), 185–210.

    Article  Google Scholar 

  • Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794.

    Article  Google Scholar 

  • Gamelin, T. W. (2001). Complex Analysis. New York: Springer-Verlag.

    Book  MATH  Google Scholar 

  • Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., Curless, B. (2010). Single image deblurring using motion density functions. In: Proc. ECCV

  • Harmeling, S., Hirsch, M., Schölkopf, B (2010a) Space-variant single-image blind deconvolution for removing camera shake. In:NIPS

  • Harmeling, S., Sra, S., Hirsch, M., Schölkopf, B (2010b) Multiframe blind deconvolution, super-resolution, and saturation correction via incremental EM. In: Proc. ICIP.

  • Ji, H., & Wang, K. (2012). Robust image deblurring with an inaccurate blur kernel. IEEE Transactions on Image Processing, 21(4), 1624–1634.

    Article  MathSciNet  Google Scholar 

  • Joshi, N., Kang, S. B., Zitnick, C. L., & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29(4), 30–39.

    Article  Google Scholar 

  • Krishnan, D., Fergus, R. (2009). Fast image deconvolution using hyper-Laplacian priors. In NIPS.

  • Krishnan, D., Tay, T., Fergus, R (2011). Blind deconvolution using a normalized sparsity measure. In: Proc. CVPR.

  • Levin, A., Fergus, R., Durand, F., & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. ACM Transactions on Graphics, 26(3), 70–79.

    Article  Google Scholar 

  • Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. MIT: Technical report.

  • Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Efficient marginal likelihood optimization in blind deconvolution. CVPR: In Proc.

  • Lucy, L. B. (1974). An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 745–754.

    Article  Google Scholar 

  • Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. The Journal of the Optical Society of America, 62(1), 55–59.

    Article  Google Scholar 

  • Schultz, R. R., & Stevenson, R. L. (1994). A bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing, 3(3), 233–242.

    Article  Google Scholar 

  • Shan, Q., Jia, J., & Agarwala, A. (2008). High-quality motion deblurring from a single image. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 73-1–73-10.

  • Tai, Y.-W., Du, H., Brown, M. S., & Lin, S. (2008). Image/video deblurring using a hybrid camera. CVPR: In Proc.

  • Tai, Y.-W., Tan, P., & Brown, M. S. (2011). Richardson-Lucy deblurring for scenes under a projective motion path. IEEE PAMI, 33(8), 1603–1618.

    Article  Google Scholar 

  • Wang, Y., Yang, J., Yin, W., & Zhang, Y. (2008). A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences, 1(3), 248–272.

    Article  MathSciNet  MATH  Google Scholar 

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Welk, M. (2010). Robust variational approaches to positivity-constrained image deconvolution. Technical Report 261, Saarland University, Saarbrücken, Germany.

  • Whyte, O. (2012). Removing camera shake blur and unwanted occluders from photographs. PhD thesis, ENS Cachan.

  • Whyte, O., Sivic, J., Zisserman, A. (2011). Deblurring shaken and partially saturated images. In: Proc. CPCV, with ICCV.

  • Whyte, O., Sivic, J., Zisserman, A., Ponce, J. (2010). Non-uniform deblurring for shaken images. In: Proc. CVPR.

  • Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2012). Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2), 168–186.

    Article  MathSciNet  MATH  Google Scholar 

  • Wiener, N. (1949). Extrapolation, interpolation, and smoothing of stationary time series. London: MIT Press.

    MATH  Google Scholar 

  • Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proc. ECCV

  • Yang, J., Zhang, Y., & Yin, W. (2009). An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. The SIAM Journal on Scientific Computing, 31(4), 2842–2865. doi:10.1137/080732894.

    Article  MathSciNet  MATH  Google Scholar 

  • Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. ACM Transsactions on Graphics (Proc. SIGGRAPH 2007), 26(3), 1-1–1-10.

  • Yuan, L., Sun, J., Quan, L., & Shum, H.-Y. (2008). Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics (Proc. SIGGRAPH 2008), 27(3), 74-1–74-10.

  • Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In Proc. ICCV

Download references

Acknowledgments

This work was partly supported by the MSR-INRIA laboratory, the EIT ICT labs and ERC grant VisRec no. 228180.

Author information

Authors and Affiliations

  1. Microsoft Corporation, Redmond, WA, USA

    Oliver Whyte

  2. INRIA - Willow Project Laboratoire d’Informatique de l’Ecole Normale Supérieure (CNRS/ENS/INRIA UMR 8548), Paris, France

    Josef Sivic

  3. Visual Geometry Group, Department of Engineering Science, University of Oxford, Oxford, UK

    Andrew Zisserman

Authors
  1. Oliver Whyte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josef Sivic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew Zisserman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver Whyte.

Additional information

Communicated by Dr. Srinivas Narasimhan, Dr. Frédo Durand and Dr. Wolfgang Heidrich.

Parts of this work were previously published in the IEEE Workshop on Color and Photometry in Computer Vision, with ICCV 2011 Whyte et al. (2011).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Whyte, O., Sivic, J. & Zisserman, A. Deblurring Shaken and Partially Saturated Images. Int J Comput Vis 110, 185–201 (2014). https://doi.org/10.1007/s11263-014-0727-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-014-0727-3

Keywords

Search

Navigation