Skip to main content
SpringerLink
Log in

Thermography data fusion and nonnegative matrix factorization for the evaluation of cultural heritage objects and buildings

  • Published:
Journal of Thermal Analysis and Calorimetry Aims and scope Submit manuscript

Abstract

The application of the thermal and infrared technology in different areas of research is considerably increasing. These applications involve nondestructive testing, medical analysis (computer aid diagnosis/detection—CAD), and arts and archeology, among many others. In the arts and archeology field, infrared technology provides significant contributions in terms of finding defects of possible impaired regions. This has been done through a wide range of different thermographic experiments and infrared methods. The proposed approach here focuses on application of some known factor analysis methods such as standard nonnegative matrix factorization (NMF) optimized by gradient-descent-based multiplicative rules (SNMF1) and standard NMF optimized by nonnegative least squares active-set algorithm (SNMF2) and eigen-decomposition approaches such as principal component analysis (PCA) in thermography, and candid covariance-free incremental principal component analysis in thermography to obtain the thermal features. On the one hand, these methods are usually applied as preprocessing before clustering for the purpose of segmentation of possible defects. On the other hand, a wavelet-based data fusion combines the data of each method with PCA to increase the accuracy of the algorithm. The quantitative assessment of these approaches indicates considerable segmentation along with the reasonable computational complexity. It shows the promising performance and demonstrated a confirmation for the outlined properties. In particular, a polychromatic wooden statue, a fresco, a painting on canvas, and a building were analyzed using the above-mentioned methods, and the accuracy of defect (or targeted) region segmentation up to 71.98%, 57.10%, 49.27%, and 68.53% was obtained, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The computational load is processing time measured while the computer system performs (CPU time).

References

  1. Scudieri F, Mercuri F, Volterri R. Solar loading thermography: time-lapsed thermographic survey and advanced thermographic signal processing for the inspection of civil engineering and cultural heritage structures. J Therm Anal Calorim. 2001;66(1):307.

    Article  CAS  Google Scholar 

  2. Mercuri F, Zammit U, Orazi N, Paoloni S, Marinelli M, Scudieri F. Active infrared thermography applied to the investigation of art and historic artefacts. J Therm Anal Calorim. 2011;104(2):475.

    Article  CAS  Google Scholar 

  3. Šulcová P, Šesták J, Menyhárd A, Liptay G. Some historical aspects of thermal analysis on the mid-European territory. J Therm Anal Calorim. 2015;120(1):239.

    Article  CAS  Google Scholar 

  4. Davin T, Serio B, Guida G, Pina V. Spatial resolution optimization of a cooling-down thermal imaging method to reveal hidden academic frescoes. Int J Therm Sci. 2017;112:188.

    Article  CAS  Google Scholar 

  5. Yao Y, Sfarra S, Ibarra-Castanedo C, You R, Maldague XP. The multi-dimensional ensemble empirical mode decomposition (MEEMD). J Therm Anal Calorim. 2017;128:1841–58.

    Article  CAS  Google Scholar 

  6. Nolesini T, Frodella W, Bianchini S, Casagli N. Detecting slope and urban potential unstable areas by means of multi-platform remote sensing techniques: the Volterra (Italy) case study. Remote Sens. 2016;8(9):746.

    Article  Google Scholar 

  7. Becherini F, Bernardi A, Di Tuccio MC, Vivarelli A, Pockelè L, De Grandi S, Fortuna S, Quendolo A. Microclimatic monitoring for the investigation of the different state of conservation of the stucco statues of the Longobard Temple in Cividale del Friuli (Udine, Italy). J Cult Herit. 2016;18:375.

    Article  Google Scholar 

  8. Tang Z, Dai SB. Wavelet analysis applied to thermographic data for the detection of sub-superficial flaws in mosaics. Stud Conserv. 2016;61(sup1):37.

    Article  Google Scholar 

  9. Cadelano G, Bison P, Bortolin A, Ferrarini G, Peron F, Girotto M, Volinia M. Monitoring of historical frescoes by timed infrared imaging analysis. Opto Electron Rev. 2015;23(1):102.

    Article  Google Scholar 

  10. Sfarra S, Ibarra-Castanedo C, Paoletti D, Maldague X. Infrared vision inspection of cultural heritage objects from the city of L’Aquila, Italy and its surroundings. Mater Eval. 2013;71(5):561.

    Google Scholar 

  11. Yousefi B, Sfarra S, Maldague XP. Quantitative assessment in thermal image segmentation for artistic objects. In: Optics for arts, architecture and archaeology (SPIE, 2017). pp. 10,331–7.

  12. Sfarra S, Ibarra-Castanedo C, Ridolfi S, Cerichelli G, Ambrosini D, Paoletti D, Maldague X. Holographic interferometry (HI), infrared vision and X-ray fluorescence (XRF) spectroscopy for the assessment of painted wooden statues: a new integrated approach. Appl Phys A Mater Sci Process. 2014;115(3):1041.

    Article  CAS  Google Scholar 

  13. Ibarra-Castanedo C, Sfarra S, Paoletti D, Bendada A, Maldague XP. Nondestructive testing of externally reinforced structures for seismic retrofitting using flax fiber reinforced polymer (FFRP) composites. In: Proceedings of SPIE. vol. 8705, pp. 87,050U–1.

  14. Thakur VK, Thakur MK, Kessler MR. Handbook of composites from renewable materials, biodegradable materials, vol. 5. Hoboken: Wiley; 2017.

    Google Scholar 

  15. Rockinger O. Image sequence fusion using a shift-invariant wavelet transform. In: Proceedings of international conference on image processing. IEEE; 1997. vol. 3, pp. 288–91.

  16. Rajic N. Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures. Compos Struct. 2002;58(4):521.

    Article  Google Scholar 

  17. Zhang H, Robitaille F, Grosse CU, Ibarra-Castanedo C, Martins JO, Sfarra S, Maldague XP. Optical excitation thermography for twill/plain weaves and stitched fabric dry carbon fibre preform inspection. Compos Part A Appl Sci Manuf. 2018;107:282–93.

    Article  CAS  Google Scholar 

  18. Zhang H, Hassler U, Genest M, Fernandes H, Robitaille F, Ibarra-Castanedo C, Joncas S, Maldague X. Comparative study on submillimeter flaws in stitched T-joint carbon fiber reinforced polymer by infrared thermography, microcomputed tomography, ultrasonic c-scan and microscopic inspection. Opt Eng. 2015;54(10):104109.

    Article  Google Scholar 

  19. Zhang H, Sfarra S, Sarasini F, Ibarra-Castanedo C, Perilli S, Fernandes H, Duan Y, Peeters J, Avdelidis NP, Maldague X. Optical and mechanical excitation thermography for impact response in basalt–carbon hybrid fiber-reinforced composite laminates. IEEE Trans Ind Inf. 2018;14(2):514–22.

    Article  Google Scholar 

  20. Ibarra-Castanedo C, Piau JM, Guilbert S, Avdelidis NP, Genest M, Bendada A, Maldague XP. Comparative study of active thermography techniques for the nondestructive evaluation of honeycomb structures. Res Nondestruct Eval. 2009;20(1):1.

    Article  Google Scholar 

  21. Lee DD, Seung HS. Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems; 2001. pp. 556–562.

  22. Yousefi B, Fleuret J, Zhang H, Maldague XP, Watt R, Klein M. Automated assessment and tracking of human body thermal variations using unsupervised clustering. Appl Opt. 2016;55(34):D162.

    Article  PubMed  Google Scholar 

  23. Avdelidis N, Moropoulou A. Emissivity considerations in building thermography. Energy Build. 2003;35(7):663.

    Article  Google Scholar 

  24. Yousefi B, Sfarra S, Ibarra-Castanedo C, Maldague XP. Thermal NDT applying candid covariance-free incremental principal component thermography (CCIPCT). In: Thermosense 2017 (SPIE, 2017). pp. 10,214–58.

  25. Yousefi B, Sfarra S, Ibarra-Castanedo C, Maldague XP. Comparative analysis on thermal non-destructive testing imagery applying candid covariance-free incremental principal component thermography (CCIPCT). Infrared Phys Technol. INFPHY-2017-146. 2017.

  26. Sfarra S, Marcucci E, Ambrosini D, Paoletti D. Infrared exploration of the architectural heritage: from passive infrared thermography to hybrid infrared thermography (HIRT) approach. Materiales de Construcción. 2016;66(323):094.

    Article  Google Scholar 

  27. Mallat SG. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell. 1989;11(7):674.

    Article  Google Scholar 

  28. Stollnitz EJ, DeRose AD, Salesin DH. Wavelets for computer graphics: a primer. 1. IEEE Comput Graph Appl. 1995;15(3):76.

    Article  Google Scholar 

  29. Li H, Manjunath B, Mitra SK. Multi-sensor image fusion using the wavelet transform. In: Proceedings of IEEE international conference on image processing, 1994, ICIP-94. IEEE; 1994. vol. 1, pp. 51–55.

  30. MacQueen J et al. Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley symposium on mathematical statistics and probability, Oakland, CA, USA; 1967. vol. 1, pp. 281–297.

  31. Tillmann AM, Pfetsch ME. The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans Inf Theory. 2014;60(2):1248.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would thank anonymous reviewers and Journal of Thermal Analysis and Calorimetry’s editor for their constructive comments. We would like to thank Geltrude Di Matteo (director) and Alessandro Verrocchia (chief restorer), of the Musè (Il Museo delle Paludi di Celano) Italy, for their kind cooperation in this work. The authors also want to acknowledge the support of the Multipolar Infrared Vision Canada Research Chair (MIVIM), tier 1. A special thank to Mr. Giovanni Pasqualoni, the University of L’Aquila (L’Aquila, Italy), for the technical support during the thermographic acquisitions.

Author information

Authors and Affiliations

  1. Computer Vision and System Laboratory (CVSL), Department of Electrical and Computer Engineering, Laval University, Quebec City, QC, G1V 0A6, Canada

    Bardia Yousefi, Clemente Ibarra-Castanedo, Nicolas P. Avdelidis & Xavier P. V. Maldague

  2. Department of Industrial and Information Engineering and Economics (DIIIE), University of L’Aquila, Piazzale E. Pontieri no. 1, Monteluco di Roio, Roio Poggio, 67100, L’Aquila, AQ, Italy

    Stefano Sfarra

  3. Tomsk Polytechnic University, Lenin Av., 30, Tomsk, Russia, 634050

    Stefano Sfarra

  4. Aerospace Integration Research Centre (AIRC), College Road, Cranfield, MK43 0AL, UK

    Nicolas P. Avdelidis

Authors
  1. Bardia Yousefi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Sfarra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clemente Ibarra-Castanedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolas P. Avdelidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xavier P. V. Maldague
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bardia Yousefi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yousefi, B., Sfarra, S., Ibarra-Castanedo, C. et al. Thermography data fusion and nonnegative matrix factorization for the evaluation of cultural heritage objects and buildings. J Therm Anal Calorim 136, 943–955 (2019). https://doi.org/10.1007/s10973-018-7644-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10973-018-7644-6

Keywords

Search

Navigation