Skip to main content
SpringerLink
Log in

Segmentation of scanning tunneling microscopy images using variational methods and empirical wavelets

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In the fields of nanoscience and nanotechnology, it is important to be able to functionalize surfaces chemically for a wide variety of applications. Scanning tunneling microscopes (STMs) are important instruments in this area used to measure the surface structure and chemistry with better than molecular resolution. Self-assembly is frequently used to create monolayers that redefine the surface chemistry in just a single-molecule-thick layer (Love et al. in Chem Rev 105(4):1103–1170, 2005; Nuzzo and Allara in J Am Chem Soc 105(13):4481–4483, 1983; Smith et al. in Prog Surf Sci 75(1):1–68, 2004). Indeed, STM images reveal rich information about the structure of self-assembled monolayers since they convey chemical and physical properties of the studied material. In order to assist in and to enhance the analysis of STM and other images (Thomas et al. in ACS Nano 10(5):5446–5451, 2016; Thomas et al. in ACS Nano 9(5):4734–4742, 2015), we propose and demonstrate an image processing framework that produces two image segmentations: One is based on intensities (apparent heights in STM images) and the other is based on textural patterns. The proposed framework begins with a cartoon + texture decomposition, which separates an image into its cartoon and texture components. Afterward, the cartoon image is segmented by a modified multiphase version of the local Chan–Vese model (Wang et al. in Pattern Recognit 43(3):603–618, 2010), while the texture image is segmented by a combination of 2D empirical wavelet transform and a clustering algorithm. Overall, our proposed framework contains several new features, specifically in presenting a new application of cartoon + texture decomposition and of the empirical wavelet transforms and in developing a specialized framework to segment STM images and other data. To demonstrate the potential of our approach, we apply it to raw STM images of various monolayers and present their corresponding segmentation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. https://www.mathworks.com/help/images/texture-segmentation-using-gabor-filters.html

  2. https://www.mathworks.com/matlabcentral/fileexchange/52753-kolian1-texture-segmentation-lbp-vs-glcm

References

  1. Ambrosio L, Tortorelli VM (1990) Approximation of functional depending on jumps by elliptic functional via t-convergence. Commun Pure Appl Math 43(8):999–1036

    MathSciNet  MATH  Google Scholar 

  2. Arivazhagan S, Ganesan L (2003) Texture segmentation using wavelet transform. Pattern Recognit Lett 24(16):3197–3203

    MATH  Google Scholar 

  3. Arivazhagan S, Ganesan L, Kumar TS (2006) Texture classification using curvelet statistical and co-occurrence features. In: ICPR 2006. 18th international conference on pattern recognition, 2006, vol 2, pp 938–941. IEEE

  4. Arthur D, Vassilvitskii S (2007) K-means++: the advantages of careful seeding. In: Proceedings of the 18th annual ACM-SIAM symposium on discrete algorithms

  5. Averbuch A, Coifman RR, Donoho DL, Elad M, Israeli M (2006) Fast and accurate polar Fourier transform. Appl Comput Harmon Anal 21(2):145–167

    MathSciNet  MATH  Google Scholar 

  6. Averbuch A, Coifman RR, Donoho DL, Israeli M, Shkolnisky Y (2008) A framework for discrete integral transformations I-the pseudopolar Fourier transform. SIAM J Sci Comput 30(2):764–784

    MathSciNet  MATH  Google Scholar 

  7. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends® Mach Learn 3(1):1–122

    MATH  Google Scholar 

  8. Buades A, Le T, Morel JM, Vese L (2011) Cartoon + texture image decomposition. Image processing on line 1

  9. Buades A, Lisani JL (2016) Directional filters for cartoon + texture image decomposition. Image Process Line 6:75–88

    MathSciNet  Google Scholar 

  10. Buades A, Lisani JL (2016) Directional filters for color cartoon + texture image and video decomposition. J Math Imag Vis 55(1):125–135

    MathSciNet  Google Scholar 

  11. Bumm LA, Arnold JJ, Cygan MT, Dunbar TD, Burgin TP, Jones L, Allara DL, Tour JM, Weiss PS (1996) Are single molecular wires conducting? Science 271:1705–1707

    Google Scholar 

  12. Candes E, Demanet L, Donoho D, Ying L (2006) Fast discrete curvelet transforms. Multiscale Model Simul 5(3):861–899

    MathSciNet  MATH  Google Scholar 

  13. Candes EJ, Donoho DL (2005) Continuous curvelet transform: I. Resolution of the wavefront set. Appl Comput Harmon Anal 19(2):162–197

    MathSciNet  MATH  Google Scholar 

  14. Chambolle A (2004) An algorithm for total variation minimization and applications. J Math Imag Vis 20(1):89–97

    MathSciNet  MATH  Google Scholar 

  15. Chan TF, Esedoglu S (2005) Aspects of total variation regularized \({L}_1\) function approximation. SIAM J Appl Math 65:1817–1837

    MathSciNet  MATH  Google Scholar 

  16. Chan TF, Sandberg BY, Vese LA (2000) Active contours without edges for vector-valued images. J Vis Commun Image Represent 11(2):130–141

    Google Scholar 

  17. Claridge SA, Liao WS, Thomas JC, Zhao Y, Cao HH, Cheunkar S, Serino AC, Andrews AM, Weiss PS (2013) From the bottom up: dimensional control and characterization in molecular monolayers. Chem Soc Rev 42(7):2725–2745

    Google Scholar 

  18. Dameron AA, Charles LF, Weiss PS (2005) Structures and displacement of 1-adamantanethiol self-assembled monolayers on Au 111. J Am Chem Soc 127(24):8697–8704

    Google Scholar 

  19. Dragomiretskiy K, Zosso D (2014) Variational mode decomposition. IEEE Trans Signal Process 62(3):531–544

    MathSciNet  MATH  Google Scholar 

  20. Dragomiretskiy K, Zosso D (2015) Two-dimensional variational mode decomposition. In: International workshop on energy minimization methods in computer vision and pattern recognition, pp 197–208. Springer

  21. Dunn D, Higgins WE (1995) Optimal gabor filters for texture segmentation. IEEE Trans Image Process 4(7):947–964

    Google Scholar 

  22. Dunn D, Higgins WE, Wakeley J (1994) Texture segmentation using 2-d gabor elementary functions. IEEE Trans Pattern Anal Mach Intell 16(2):130–149

    Google Scholar 

  23. Dunn D.F, Higgins W.E (1993) Optimal gabor-filter design for texture segmentation. In: 1993 IEEE international conference on acoustics, speech, and signal processing, 1993. ICASSP-93., vol 5, pp 37–40. IEEE

  24. Esedoglu S, Tsai YHR (2006) Threshold dynamics for the piecewise constant Mumford-Shah functional. J Comput Phys 211(1):367–384

    MathSciNet  MATH  Google Scholar 

  25. Fowlkes C, Belongie S, Chung F, Malik J (2004) Spectral grouping using the Nyström method. IEEE Trans Pattern Anal Mach Intell 26(2):214–225

    Google Scholar 

  26. Gabay D, Mercier B (1976) A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math Appl 2(1):17–40

    MATH  Google Scholar 

  27. Garcia-Cardona C, Merkurjev E, Bertozzi AL, Flenner A, Percus AG (2014) Multiclass data segmentation using diffuse interface methods on graphs. IEEE Trans Pattern Anal Mach Intell 36(8):1600–1613

    MATH  Google Scholar 

  28. Gethers ML, Thomas JC, Jiang S, Weiss NO, Duan X, Goddard WA III, Weiss PS (2015) Holey graphene as a weed barrier for molecules. ACS Nano 9(11):10909–10915

    Google Scholar 

  29. Getreuer P (2012) Chan-Vese segmentation. Image Process Line 2:214–224

    Google Scholar 

  30. Gilles J (2013) Empirical wavelet transform. IEEE Trans Signal Process 61(16):3999–4010. https://doi.org/10.1109/TSP.2013.2265222

    Article  MathSciNet  MATH  Google Scholar 

  31. Gilles J, Heal K (2014) A parameterless scale-space approach to find meaningful modes in histograms-application to image and spectrum segmentation. Int J Wavel Multiresolut Inf Process 12(6):14500441–145004417. https://doi.org/10.1142/S0219691314500441

    Article  MathSciNet  MATH  Google Scholar 

  32. Gilles J, Tran G, Osher S (2014) 2D empirical transforms. Wavelets, ridgelets and curvelets revisited. SIAM J Imag Sci 7(1):157–186. https://doi.org/10.1137/130923774

    Article  MathSciNet  MATH  Google Scholar 

  33. Glowinski R, Marroco A (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique 9(R2): 41–76

  34. Gooding JJ, Mearns F, Yang W, Liu J (2003) Self-assembled monolayers into the 21st century: recent advances and applications. Electroanalysis 15(2):81–96

    Google Scholar 

  35. Guttentag AI, Barr KK, Song TB, Bui KV, Fauman JN, Torres LF, Kes DD, Ciomaga A, Gilles J, Sullivan NF, Yang Y, Allara DL, Zharnikov M, Weiss PS (2016) Hexagons to ribbons: flipping cyanide on Au \(\{\)111\(\}\). J Am Chem Soc 138(48):15580–15586

    Google Scholar 

  36. Guttentag AI, Wachter T, Barr KK, Abendroth JM, Song TB, Sullivan NF, Yang Y, Allara DL, Zharnikov M, Weiss PS (2016) Surface structure and electron transfer dynamics of the self-assembly of cyanide on Au \(\{\)111\(\}\). J Phys Chem C 120(47):26736–26746

    Google Scholar 

  37. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 3(6):610–621

    Google Scholar 

  38. Huang Y, De Bortoli V, Zhou F, Gilles J (2018) Review of wavelet-based unsupervised texture segmentation, advantage of adaptive wavelets. IET Image Process 12(9):1626–1638

    Google Scholar 

  39. Jain AK, Farrokhnia F (1991) Unsupervised texture segmentation using gabor filters. Pattern Recognit 24(12):1167–1186

    Google Scholar 

  40. Kim M, Hohman JN, Cao Y, Houk KN, Ma H, Jen AKY, Weiss PS (2011) Creating favorable geometries for directing organic photoreactions in alkanethiolate monolayers. Science 331(6022):1312–1315

    Google Scholar 

  41. Kim M, Hohman JN, Serino AC, Weiss PS (2010) Structural manipulation of hydrogen-bonding networks in amide-containing alkanethiolate monolayers via electrochemical processing. J Phys Chem C 114(46):19744–19751

    Google Scholar 

  42. Labate D, Lim W-Q, Kutyniok G, Weiss G (2005) Sparse multidimensional representation using shearlets. In: Wavelets XI, vol 5914. International Society for Optics and Photonics

  43. Love JC, Estroff LA, Kriebel JK, Nuzzo RG, Whitesides GM (2005) Self-assembled monolayers of thiolates on metals as a form of nanotechnology. Chem Rev 105(4):1103–1170

    Google Scholar 

  44. March R (1992) Visual reconstruction with discontinuities using variational methods. Image Vis Comput 10(1):30–38

    Google Scholar 

  45. Merkurjev E, Garcia-Cardona C, Bertozzi AL, Flenner A, Percus AG (2014) Diffuse interface methods for multiclass segmentation of high-dimensional data. Appl Math Lett 33:29–34. https://doi.org/10.1016/j.aml.2014.02.008

    Article  MathSciNet  MATH  Google Scholar 

  46. Merriman B, Bence J.K, Osher S (1992) Diffusion generated motion by mean curvature. In: Proceedings of the geometry center workshop

  47. Merriman B, Bence JK, Osher SJ (1994) Motion of multiple junctions: a level set approach. J Comput Phys 112(2):334–363

    MathSciNet  MATH  Google Scholar 

  48. Meyer Y (2001) Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. American Mathematical Society, Boston, MA, US

    MATH  Google Scholar 

  49. Modica L (1987) The gradient theory of phase transitions and the minimal interface criterion. Arch Ration Mech Anal 98(2):123–142

    MathSciNet  MATH  Google Scholar 

  50. Modica L, Mortola S (1977) Un esempio di \(\Gamma\)-convergenza. Bollettino della Unione Matematica Italiana B 14(5):285–299

    MathSciNet  MATH  Google Scholar 

  51. Mumford D, Shah J (1989) Optimal approximations by piecewise smooth functions and associated variational problems. Commun Pure Appl Math 42(5):577–685

    MathSciNet  MATH  Google Scholar 

  52. Nuzzo RG, Allara DL (1983) Adsorption of bifunctional organic disulfides on gold surfaces. J Am Chem Soc 105(13):4481–4483

    Google Scholar 

  53. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    MATH  Google Scholar 

  54. Osher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J Comput Phys 79(1):12–49

    MathSciNet  MATH  Google Scholar 

  55. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Google Scholar 

  56. Poirier GE (1997) Characterization of organosulfur molecular monolayers on Au (111) using scanning tunneling microscopy. Chem Rev 97(4):1117–1128

    Google Scholar 

  57. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenom 60(1–4):259–268

    MathSciNet  MATH  Google Scholar 

  58. Shen L, Yin Q (2009) Texture classification using curvelet transform. In: Proceedings of the international symposium on information processing (ISIP09), pp 319–324. Citeseer

  59. Smith RK, Lewis PA, Weiss PS (2004) Patterning self-assembled monolayers. Progress Surf Sci 75(1):1–68

    Google Scholar 

  60. Strang G (1993) Wavelet transforms versus fourier transforms. Bull Am Math Soc 28(2):288–305

    MathSciNet  MATH  Google Scholar 

  61. Thomas JC, Goronzy DP, Dragomiretskiy K, Zosso D, Gilles J, Osher SJ, Bertozzi AL, Weiss PS (2016) Mapping buried hydrogen-bonding networks. ACS Nano 10(5):5446–5451

    Google Scholar 

  62. Thomas JC, Goronzy DP, Serino AC, Auluck HS, Irving OR, Jimenez-Izal E, Deirmenjian JM, Machacek J, Sautet P, Alexandrova AN et al (2018) Acid-base control of valency within carboranedithiol self-assembled monolayers: molecules do the can-can. ACS Nano 12(3):2211–2221

    Google Scholar 

  63. Thomas JC, Schwartz JJ, Hohman JN, Claridge SA, Auluck HS, Serino AC, Spokoyny AM, Tran G, Kelly KF, Mirkin CA, Gilles J, Osher SJ, Weiss PS (2015) Defect-tolerant aligned dipoles within two-dimensional plastic lattices. ACS Nano 9(5):4734–4742

    Google Scholar 

  64. Unser M (1995) Texture classification and segmentation using wavelet frames. IEEE Trans Image Process 4(11):1549–1560

    MathSciNet  Google Scholar 

  65. Vese LA, Chan TF (1997) Reduced non-convex functional approximations for image restoration and segmentation. UCLA CAM Reports pp 97–56

  66. Vese LA, Chan TF (2002) A multiphase level set framework for image segmentation using the Mumford and Shah model. Int J Comput Vis 50(3):271–293

    MATH  Google Scholar 

  67. Wang XF, Huang DS, Xu H (2010) An efficient local Chan-Vese model for image segmentation. Pattern Recognit 43(3):603–618

    MATH  Google Scholar 

  68. Weldon TP, Higgins WE, Dunn DF (1996) Efficient gabor filter design for texture segmentation. Pattern Recognit 29(12):2005–2015

    Google Scholar 

  69. Yugay D, Goronzy DP, Kawakami LM, Claridge SA, Song TB, Yan Z, Xie YH, Gilles J, Yang Y, Weiss PS (2016) Copper ion binding site in \(\beta\)-amyloid peptide. Nano Lett 16(10):6282–6289

    Google Scholar 

  70. Zhang H, Fritts J.E, Goldman S.A (2003) An entropy-based objective evaluation method for image segmentation. In: Storage and Retrieval Methods and Applications for Multimedia 2004, vol 5307, pp. 38–50. International Society for Optics and Photonics

  71. Zhang H, Fritts JE, Goldman SA (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280

    Google Scholar 

  72. Zosso D, An J, Stevick J, Takaki N, Weiss M, Slaughter LS, Cao HH, Weiss PS, Bertozzi AL (2017) Image segmentation with dynamic artifacts detection and bias correction. Inverse Probl Imaging 11(3):577–600

    MathSciNet  MATH  Google Scholar 

  73. Zosso D, Dragomiretskiy K, Bertozzi AL, Weiss PS (2017) Two-dimensional compact variational mode decomposition. J Math Image Vis 58(2):294–320

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the W.M. Keck Foundation Center for Leveraging Sparsity. The self-assembly and STM imaging were supported by the US Department of Energy (Grant #DE-SC-1037004). L. Torres Mandiola was partially supported by NSF DMS-1312361. K. Bui, J. Fauman, and D. Kes were partially supported by NSF DMS-1045536. A. Bertozzi was supported by Simons Math + X Investigator Award Number 510776. The authors thank the editor and anonymous reviewers for their helpful comments in improving the quality of the manuscript.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of California, Irvine, 340 Rowland Hall, Irvine, CA, 92697-3875, USA

    Kevin Bui

  2. Department of Applied Mathematics, 526 UCB, ECOT 225, University of Colorado Boulder, Boulder, Colorado, 80309-0526, USA

    Jacob Fauman

  3. College of Arts and Sciences, University of San Francisco, USF Downtown Campus, 101 Howard Street, San Francisco, CA, 94105, USA

    David Kes

  4. Department of Mathematics, South Kensington Campus, Imperial College London, London, SW7 2AZ, UK

    Leticia Torres Mandiola

  5. Sorbonne Paris Cité, Laboratoire Jacques-Louis Lions, UMR 7598, UPMC, CNRS, University Paris Diderot, 75205, Paris, France

    Adina Ciomaga

  6. Department of Mathematics, University of California, Los Angeles, 520 Portola Plaza, Los Angeles, CA, 90095-1555, USA

    Ricardo Salazar & Andrea L. Bertozzi

  7. Department of Mathematics and Statistics, San Diego State University, 5500 Campanile Dr, San Diego, CA, 92181-7720, USA

    Jérôme Gilles

  8. Department of Chemistry and Biochemistry, University of California, Los Angeles, Los Angeles, California, 90095, USA

    Dominic P. Goronzy, Andrew I. Guttentag & Paul S. Weiss

  9. California NanoSystems Institute, University of California, Los Angeles, Los Angeles, California, 90095, USA

    Dominic P. Goronzy, Andrew I. Guttentag & Paul S. Weiss

  10. Department of Materials Science and Engineering, University of California, Los Angeles, Los Angeles, California, 90095, USA

    Paul S. Weiss

Authors
  1. Kevin Bui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacob Fauman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Kes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leticia Torres Mandiola
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adina Ciomaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ricardo Salazar
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrea L. Bertozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jérôme Gilles
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dominic P. Goronzy
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Andrew I. Guttentag
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Paul S. Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kevin Bui.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bui, K., Fauman, J., Kes, D. et al. Segmentation of scanning tunneling microscopy images using variational methods and empirical wavelets. Pattern Anal Applic 23, 625–651 (2020). https://doi.org/10.1007/s10044-019-00824-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-019-00824-0

Keywords

Search

Navigation