Skip to main content
SpringerLink
Log in

Lightweight refined networks for single image super-resolution

  • 1188: Artificial Intelligence for Physical Agents
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, software architectures applied to physical agents have become a boost from the emerging Artificial Intelligence(AI). In these smart physical agents, simultaneously image information processing appears vital in particular. Single Image Super Resolution(SISR) serves as the foundation of the image process, presenting its prospects driven by deep learning(DL) methods. In these DL methods, convolutional layers are stacked to implement a mapping between the original low resolution(LR) image and the high resolution(HR) image. Despite improved performances, convolution neural networks(CNN) based methods with large parameters are so complex and time-consuming, which is difficult to apply in mobile devices. To tackle the above issue, we propose Distillation Information Block(DIB) and Feature Information Refined Block(FIRB). In our proposed Lightweight Refined Networks for single image Super-Resolution(LRSR), to lengthen our network with fewer parameters increased, the proposed DIB grasps more information by using a large receptive field. To enhance the information utilization, we build FIRB to refine advanced features and recover more details. Furthermore, with the compact structure, the execution time can be comparatively reduced with higher performance. We conduct extensive experiments on different datasets, which demonstrate that the proposed method performs comparatively better compared with the state-of-the-art lightweight methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVC, Surrey, UK, p 1–10

  2. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition 1251–1258

  3. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In David J. Fleet, Tom as Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision - ECCV 2014 -13th European Conference, Zurich, Switzerland. Proceedings, Part IV, volume 8692 of Lecture Notes in Computer Science, p 184-199

  4. Ellery A (2016) Autonomous navigation-self-localization and mapping (slam). Springer, Berlin Heidelberg

    Book  Google Scholar 

  5. Foley JD, Dam AV, Feiner SK, Hughes JF (1990) Computer graphics, principle and practice, 2nd edition

  6. Forsyth DA, Ponce J (2012) Computer Vision: A Modern Approach. Computer Vision: A Modern Approach, Second Edition. Pitman

  7. Franois R (2010) A non-local approach for image super-resolution using intermodality priors. Medical image analysis 14(4):594–605

    Article  Google Scholar 

  8. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, p 349–356

  9. Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020) Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) p 1577–1586

  10. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. Comput Sci 3(4):212–223

    Google Scholar 

  11. Hu J, Shen L, Albanie S, Sun G, Wu E (2017) Squeeze-and-excitation networks. CVPR p 7132-7141

  12. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE conference on computer vision and pattern recognition 5197–5206

  13. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR p 723-731

  14. Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S, Tian Q (2020) Video super-resolution with temporal group attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR p 8005–8014

  15. Keys RG (2003) Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing 29

  16. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 1646–1654

  17. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 1637–1645

  18. Kim SK, Duh H, Sarhan NJ, Hahanov V (2013) Real-time multimedia computing. Multimed Tools Appl 65(2):181–186

    Article  Google Scholar 

  19. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, ICLR 

  20. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution 5835–5843

  21. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang ZA (2016) Photo-realistic single image super-resolution using a generative adversarial network

  22. Levine S, Pastor P, Krizhevsky A, Ibarz J, Quillen D (2016) Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int J Robot Res 37(4–5):421–436

    Google Scholar 

  23. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  24. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops 136–144

  25. Liu F, Yu Q, Chen L, Jeon G, Albertini MK, Yang X (2021) Aerial image super-resolution based on deep recursive dense network for disaster area surveillance. Pers Ubiquit Comput 1–10

  26. Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV)

  27. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 2:416–423. IEEE

  28. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Article  Google Scholar 

  29. Mattmann CA (2013) Computing: A vision for data science. Nature 493(7433):473–5

    Article  Google Scholar 

  30. Molchanov P, Tyree S, Karras T, Aila T, Kautz J (2016) Pruning convolutional neural networks for resource efficient inference. OpenReview.net 

  31. Namhyuk A, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV) 252–268

  32. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition 1874–1883

  33. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In IEEE Conference on Computer Vision and Pattern Recognition CVPR p 2798–2798

  34. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. ICCV p 4549–4557

  35. Thurnhofer-Hemsi K, Lopez-Rubio E, Roe-Vellve N, Molina-Cabello MA (2020) Multiobjective optimization of deep neural networks with combinations of lp-norm cost functions for 3d medical image super-resolution. Integrated Computer-Aided Engineering 27(3):233–251

    Article  Google Scholar 

  36. Timofte R, Agustsson E, Gool LV, Yang MH, Zhang L, Lim B et al (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) p 852–863

  37. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In International conference on curves and surfaces 711–730. Springer

  38. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV) 286–301

  39. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. CVPR 2472–2481

  40. Zhou W, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

Download references

Acknowledgements

This work was partially supported by the funding from Sichuan University under grant 2020SCUNG205.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, Sichuan, 610064, China

    Jiahui Tong, Haoran Yang & Xiaomin Yang

  2. School of Aeronautics & Astronautics, Sichuan University, Chengdu, Sichuan, 610065, China

    Jiahui Tong, Haoran Yang & Xiaomin Yang

  3. The Center of Gerontology and Geriatrics, West China Hospital, Sichuan University, Chengdu, China

    Qingyu Dou

  4. School of Electronic Engineering, Xidian University, Xi’an, 710071, China

    Gwanggil Jeon

  5. Department of Embedded Systems Engineering, Incheon National University, Incheon, 22012, Korea

    Gwanggil Jeon

Authors
  1. Jiahui Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingyu Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoran Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gwanggil Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaomin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gwanggil Jeon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tong, J., Dou, Q., Yang, H. et al. Lightweight refined networks for single image super-resolution. Multimed Tools Appl 81, 3439–3458 (2022). https://doi.org/10.1007/s11042-021-11318-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11318-9

Keywords

Search

Navigation