Skip to main content
SpringerLink
Log in

Automated inspection in robotic additive manufacturing using deep learning for layer deformation detection

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In this paper, an automated layer defect detection system for construction 3D printing is proposed. Initially, a step-by-step procedure is implemented to develop a deep convolutional neural network that receives images as input and is able to distinguish concrete layers from other surrounding objects through semantic pixel-wise segmentation. Using data augmentation techniques, 1M labeled images are generated and used to train and test the CNN model. Then, a defect detection module is developed which is able to detect deformations in the printed concrete layers extracted from the images using the CNN model. The evaluation results based on metrics such as accuracy, F1 score, and miss rate verify the acceptable performance of the developed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., & Devin, M. (2016). Tensorflow: A system for large-scale machine learning. In In 12th {USENIX} Symposium on Operating Systems Design and Implementation, 2016.

  • Badmos, O., Kopp, A., Bernthaler, T., & Schneider, G. (2020). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing, 31, 885–897.

    Article  Google Scholar 

  • Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495.

    Article  Google Scholar 

  • Bali, R., & Sarkar, D. (2016). R Machine Learning By Example, Packt Publishing Ltd, 2016.

  • Baslamisli, A. S., Hoang-An, L., & Gevers, T. (2018). CNN based learning using reflection and retinex models for intrinsic image decomposition. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.

  • Canny, J. (1986). A computational approach to edge detection. In IEEE Transactions on pattern analysis and machine intelligence 6, 1986.

  • Chen, F.-C., & Jahanshahi, M. R. (2017). NB-CNN: deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics, 65(5), 4392–4400.

    Article  Google Scholar 

  • Chollet, F. (2015). Keras: Deep learning library for theano and tensorflow. 2015. [Online]. https://keras.io/k.

  • Craveiroa, F., Duarte, J. P., Bartoloa, H., & Bartolod, P. J. (2019). Additive manufacturing as an enabling technology for digital construction: A perspective on Construction 4.0. Automation in Construction, 103, 251–267.

    Article  Google Scholar 

  • Davtalab, O., Kazemian, A., & Khoshnevis, B. (2018). Perspectives on a BIM-integrated software platform for robotic construction through Contour Crafting. Automation in Construction, 89, 13–23.

    Article  Google Scholar 

  • Ding, L., Wei, R., & Che, H. (2014). Development of a BIM-based automated construction system. Procedia Engineering, 85, 123–131.

    Article  Google Scholar 

  • Feng, Z. -H., Kittler, J., & William Christmas, P. H. A. X. -J. W. (2017). Dynamic attention-controlled cascaded shape regression exploiting training data augmentation and fuzzy-set sample weighting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.

  • Ghaffar, S. H., Corker, J., & Fan, M. (2018). Additive manufacturing technology and its implementation in construction as an eco-innovative solution. Automation in Construction, 93, 1–11.

    Article  Google Scholar 

  • Hojati, M., Nazarian, S., Duarte, J., Radlińska, A., Ashrafi, N., Craveiro, F., & Bilén, S. (2018). 3D Printing of concrete: A continuous exploration of mix design and printing process. In 42st International Association for Housing (IAHS), Naples, 2018.

  • Jie, C., Tan, C.-H., Hou, J., Chau, L.-P., & Li, H. (2018). Robust video content alignment and compensation for rain removal in a CNN framework. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.

  • Kazemian, A., Yuan, X., Cochran, E., & Khoshnevis, B. (2017). Cementitious materials for construction-scale 3D printing: Laboratory testing of fresh printing mixture. Construction and Building Materials, 145, 639–647.

    Article  Google Scholar 

  • Kazemian, A., Yuan, X., Davtalab, O., & Khoshnevis, B. (2019). Computer vision for real-time extrusion quality monitoring and control in robotic construction. Automation in Construction, 101, 92–98.

    Article  Google Scholar 

  • Khoshnevis, B., & Kazemian, A. (2020). Contour crafting: A revolutionary platform technology. Construction Printing Technology Magazine,pp. 48–53, 2020.

  • Khoshnevis, B., Yuan, X., Zahiri, B., Zhang, J., & Xia, B. (2015). Deformation analysis of sulfur concrete structures made by contour crafting. In AIAA SPACE 2015 Conference and Exposition. California: Pasadena.

    Google Scholar 

  • Khoshnevis, B., Yuan, X., Zahiri, B., Zhang, J., & Xia, B. (2016). Construction by Contour Crafting using sulfur concrete with planetary applications. Rapid Prototyping Journal, 22(5), 848–856.

    Article  Google Scholar 

  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436.

    Article  Google Scholar 

  • Li, X., Jia, X., Yang, Q., Lee, J. (2020). Quality analysis in metal additive manufacturing with deep learning. Journal of Intelligent Manufacturing,2020.

  • Lin, H., & Li, B. (2019). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30, 2525–2534.

    Article  Google Scholar 

  • Liu, N., Han, J., & Yang, M.-H. (2018). PiCANet: Learning pixel-wise contextual attention for saliency detection. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.

  • Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.

  • Perrot, A., Rangeard, D., & Pierre, A. (2016). Structural built-up of cement-based materials used for 3D-printing extrusion techniques. Materials and Structures, 49(4), 1213–1220.

    Article  Google Scholar 

  • Roussel, N. (2018). Rheological requirements for printable concretes. Cement and Concrete Research, 112, 76–85.

    Article  Google Scholar 

  • Teizer, J., Blickle, A., King, T., Leitzbach, O., Guenther, D., Mattern, H., & König, M. (2018). BIM for 3D Printing in Construction. In Building Information Modeling (pp. 421–446).

  • Tibaut, A., Rebolj, D., & Perc, M. N. (2016). Interoperability requirements for automated manufacturing systems in construction. Journal of Intelligent Manufacturing, 27, 251–262.

    Article  Google Scholar 

  • Tomasi, C., & Manduchi, R. (1998) Bilateral filtering for gray and color images. ICCV,vol. 98, no. 1, 1998.

  • Van Der Putten, J., Deprez, M., Cnudde, V., De Schutter, G., & Van Tittelboom, K. (2019). Microstructural characterization of 3D printed cementitious materials. Materials, 12(18), 2993.

    Article  Google Scholar 

  • Viktor, M., Naidu Nerella, V., Will, F., Näther, M., Otto, J., & Krause, M. (2019). Large-scale digital concrete construction–CONPrint3D concept for on-site, monolithic 3D-printing. Automation in Construction ,vol. 107, 2019.

  • Wolfs, R., Bos, F., & Salet, T. (2018). Early age mechanical behaviour of 3D printed concrete: Numerical modelling and experimental testing. Cement and Concrete Research, 106, 103–116.

    Article  Google Scholar 

  • Wolfs, R. J. M., Bos, F. P., & Salet, T. A. M. (2019). Triaxial compression testing on early age concrete for numerical analysis of 3D concrete printing. Cement and Concrete Composites, 104, 103344.

    Article  Google Scholar 

  • Wu, P., Wang, J., & Wang, X. (2016). A critical review of the use of 3-D printing in the construction industry. Automation in Construction, 68, 21–31.

    Article  Google Scholar 

  • Xia, M., Nematollahi, B., & Sanjayan, J. (2019). Printability, accuracy and strength of geopolymer made using powder-based 3D printing for construction applications. Automation in Construction, 101, 179–189.

    Article  Google Scholar 

  • Xiaodong, T., & Eades, A. (2005). Errors, artifacts, and improvements in EBSD processing and mapping. Microscopy and Microanalysis, 11(1), 79–87.

    Article  Google Scholar 

  • Yin, X., Liu, H., Chen, Y., & Al-Hussein, M. (2019). Building information modelling for off-site construction: Review and future directions. Automation in Construction, 101, 72–91.

    Article  Google Scholar 

  • Zhang, J., & Khoshnevis, B. (2013). Optimal machine operation planning for construction by Contour Crafting. Automation in Construction, 29, 50–67.

    Article  Google Scholar 

  • Zhang, L., Yang, F., Zhang, Y. D., & Julie Zhu, Y. (2016). Road crack detection using deep convolutional neural network. In 2016 IEEE international conference on image processing (ICIP), 2016.

  • Zhu, B., Pan, J., Nematollahi, B., Zhou, Z., Zhang, Y., & Sanjayan, J. (2019). Development of 3D printable engineered cementitious composites with ultra-high tensile ductility for digital construction. Materials & Design,vol. 181, 2019.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Southern California, Los Angeles, CA, USA

    Omid Davtalab

  2. Contour Crafting Corporation, El Segundo, CA, USA

    Omid Davtalab, Xiao Yuan & Behrokh Khoshnevis

  3. Department of Construction Management, Louisiana State University, Baton Rouge, LA, USA

    Ali Kazemian

  4. Department of Industrial and Systems Engineering, University of Southern California, Los Angeles, CA, USA

    Xiao Yuan & Behrokh Khoshnevis

Authors
  1. Omid Davtalab
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Kazemian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Behrokh Khoshnevis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omid Davtalab.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Davtalab, O., Kazemian, A., Yuan, X. et al. Automated inspection in robotic additive manufacturing using deep learning for layer deformation detection. J Intell Manuf 33, 771–784 (2022). https://doi.org/10.1007/s10845-020-01684-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-020-01684-w

Keywords

Search

Navigation