Skip to main content
SpringerLink
Log in

CNN and bi-LSTM based 3D golf swing analysis by frontal swing sequence images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, the method to overcome the limitations of the existing three-dimensional golf swing analysis system by using deep learning technology, and analyze the three-dimensional quantitative information through sequence images acquired with a single camera is studied. In this paper, CNN was used to extract the appropriate features from the image of the golf frontal swing sequence, and a regression model based on Bi-LSTM was used to predict the correct information in each sequence. This classifies the major swing section, and analyzes the quantitative status of the twisting angles of the upper body, head, shoulder and pelvis for body-sway, head-up and X-factor analysis. For the experiment, in this paper, a total of 520 times swing data were obtained using no. 1 wood club and no. 7 iron club from five subjects. In the major swing section classification experiment, each swing section was classified with an average accuracy of about 95.44%. Quantitative analysis results from each analysis model showed that the upper body motion prediction RMSE averaged 4.23 degrees, the head motion prediction RMSE averaged 5.18 degrees, and the shoulder and pelvis twisting angle prediction RMSE averaged 3.86 degrees. As a result, it was confirmed that a three-dimensional quantitative analysis based on sequence images is possible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. An J, Wulf G, Kim S (2013) Increased carry distance and X-factor stretch in golf through an external focus of attention. J Mot Learn Dev 1(1):2–11

    Article  Google Scholar 

  2. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Article  Google Scholar 

  3. Bourgain M, Hybois S, Thoreux P, Rouillon O, Rouch P, Sauret C (2018) Effect of shoulder model complexity in upper-body kinematics analysis of the golf swing. J Biomech 25(75):154–158. https://doi.org/10.1016/j.jbiomech.2018.04.025

  4. Chen HT, Huang TW, Chou CL, Tsai HC, Lee SY (2015) Improving golf swing skills using intelligent glasses, In Proceedings of IEEE Visual Communications and Image Processing, pp. 1–4. https://doi.org/10.1109/VCIP.2015.7457819

  5. Chiu JPC, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist 4:357–370

    Article  Google Scholar 

  6. Chotimanus P, Cooharojananone N, Phimoltares S (2012) Real swing extraction for video indexing in golf practice video, In Proceedings of the Computing Communications and Applications Conference, pp. 420–425. https://doi.org/10.1109/ComComAp.2012.6154885

  7. Fradkin A (2017) Routledge International Handbook of Golf Science(1st Edition), Routledge, pp. 490

  8. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning (vol. 1). Cambridge: MIT press.

  9. Ha CS (2008) The kinematic and kinetic variables of head up putting involved in golf. Korea J Sports Sci 17(1):675–684

    Google Scholar 

  10. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  11. Hsu YL, Chen YT, Chou PH, Kou YC, Chen YC, Su HY (2016) Golf swing motion detection using an inertial-sensor-based portable instrument. In Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Taiwan, pp. 1–2. https://doi.org/10.1109/ICCE-TW.2016.7521016

  12. Hua Y, Mou L, Zhu XX (2019) Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multilabel aerial image classification. ISPRS J Photogramm Remote Sens 149:188–199

    Article  Google Scholar 

  13. Huang SY, Kuo KP, Lin YH (2015) A golf swing analysis system using wii balance board and kinect sensors for novice players. Multimed Tools Appl 74(23):10679–10696

    Article  Google Scholar 

  14. Ji S, Zhang C, Xu A, Shi Y, Duan Y (2018) 3D convolutional neural networks for crop classification with multi-temporal remote sensing images, Remote Sensing, vol. 10, no. 1, 75

  15. Jiao L, Bie R, Wu H, Wei Y, Kos A, Umek A (2018) Golf swing data classification with deep convolutional neural network. The IPSI BgD Trans Int Res 14(1):29–34

    Google Scholar 

  16. Jin S, Lee W, Kim Y, Lee D, Jung S (2017) Deep-learning based adaptive power control on common data link system. J Inst Electron Inf Engin 54(11):33–44

    Google Scholar 

  17. Kelly P, Healy A, Moran K, O’Connor NE (2010) A virtual coaching environment for improving golf swing technique, In Proceedings of the 2010 ACM Workshop on Surreal Media and Virtual Cloning, pp. 51-56

  18. Kim PK (2014) Automatic extraction of golf swing features using a single Kinect. J Korea Soc Comput Inf 19(12):197–207

    Article  Google Scholar 

  19. Kim HJ (2017) Deep learning city: a big data analytics framework for smart cities. Inf policy 24(4):79–92

    Google Scholar 

  20. Kim SJ, Shin YS, Choi JY (2002) A comparative study of kinematic variables related with address and impact in golf driver swing. J Spor Leis Stud 17:145–158

    Google Scholar 

  21. Kim YJ, Kim KD, Kim SH, Lee S, Lee HS (2017) Golf swing analysis system with a dual band and motion analysis algorithm. IEEE Trans Consum Electron 63(3):309–317

    Article  Google Scholar 

  22. Ko KR (2018) A study on golf swing analysis based on sequence images using CNN and LSTM, PhD diss., Graduate School of Chosun University

  23. Ko KR, Pan SB (2015) Feature Extraction and Classification of Posture for Four-Joint based Human Motion Data Analysis, J Ins Elec Inf Eng 52(6):117–125. https://doi.org/10.5573/ieie.2015.52.6.117

  24. Ko KR, Pan SB (2018) Automatic classification of golf swing section based on single camera using transfer learning. In Proceedings of the KIIT Summer Conference, pp. 37–39

  25. Ko KR, Pan SB (2018) Golf swing motion analysis based on inertial sensors. J Korean Inst Inf Tech 16(1):125–131

    Article  Google Scholar 

  26. Lee SH, Cho HY (2009) An condition analysis about the exercise injury factors and treatment of golf club members. J Spor Leis Stud 37(2):1303–1314

    Google Scholar 

  27. Lee KI, Park JJ (2006) The kinematic analysis of golf swing by amateur female golfers. J Spor Leis Stud 28:349–362

    Google Scholar 

  28. Lindsay DM, Vandervoort AA (2014) Golf-related low back pain: a review of causative factors and prevention strategies. Asian J Sports Med 5(4):e24289. https://doi.org/10.5812/asjsm.24289

  29. Liu Q, Zhou F, Hang R, Yuan X (2017) Bidirectional-convolutional LSTM based spectralspatial feature learning for hyperspectral image classification, Remote Sensing, vol. 9, no. 12, 1330

  30. McGill S (2010) Core training: evidence translating to better performance and injury prevention. Strength Cond J 32(3):33–46

    Article  Google Scholar 

  31. McNally W, Vats K, Pinto T, Dulhanty C, McPhee J, Wong A (2019) “GolfDB: a video database for golf swing sequencing,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops

  32. Mun F, Suh SW, Park HJ, Choi A (2015) Kinematic relationship between rotation of lumbar spine and hip joints during golf swing in professional golfers. Biomed Eng Online 14(14):41. https://doi.org/10.1186/s12938-015-0041-5

  33. Najafi B, Lee-Eng J, Wrobel JS, Goebel R (2015) Estimation of center of mass trajectory using wearable sensors during golf swing. J Sports Sci Med 14(2):354–363

    Google Scholar 

  34. Nam CNK, Kang HJ, Suh YS (2014) Golf swing motion tracking using inertial sensors and a stereo camera. IEEE Trans Instrum Meas 63(4):943–952

    Article  Google Scholar 

  35. Nike Sport Research Lab (n.d.), [online] Available at: http://news.nike.com/news/a-look-inside-nike-s-sport-research-lab

  36. O’Donoghue P (2014) An introduction to performance analysis of sport, Routledge, pp. 274

  37. Park J, Kwak S (2013) Golf swing classification using fuzzy system. J Broadcast Eng 18(3):380–392

    Article  Google Scholar 

  38. Park S, Chang JY, Jeong H, Lee JH, Park J (2017) Accurate and efficient 3D human pose estimation algorithm using single depth images for pose analysis in golf. In Proceedings of 2017 IEEE Conf Comput Vis Pattern Recogn Workshops, pp. 105-113. https://doi.org/10.1109/CVPRW.2017.19

  39. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Article  Google Scholar 

  40. Sim FK, Sundaraj K (2010) Human motion tracking on broadcast golf swing video using optical flow and template matching, Comput Appl Ind Electron, pp. 169–173

  41. Smith AC, Roberts JR, Wallace ES, Kong P, Forrester SE (2016) Comparison of two-and three-dimensional methods for analysis of trunk kinematic variables in the golf swing. J Appl Biomech 32(1):23–31

    Article  Google Scholar 

  42. Smith AC, Roberts JR, Kong PW, Forrester SE (2017) Comparison of center of gravity and center of pressure patterns in the golf swing. Eur J Sport Sci 17(2):168–178

    Article  Google Scholar 

  43. Tucker CJ (2016) Golf injuries. Sports Med Update:2–6

  44. Ueda M, Negoro H, Kurihara Y, Watanabe K (2013) Measurement of angular motion in golf swing by a local sensor at the grip end of a golf club. IEEE Trans Hum Mach Syst 43(4):398–404

    Article  Google Scholar 

  45. Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2017) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166

    Article  Google Scholar 

  46. Watanabe K, Hokari M (2006) Kinematical analysis and measurement of sports form. IEEE Trans Syst Man Cybern Syst Hum 36(3):549–557

    Article  Google Scholar 

  47. Zhang Z, Zhang Y, Kos A, Umek A (2017) A sensor-based golfer-swing signature recognition method using linear support vector machine. Elektrotehniski Vestnik 84(5):247–252

    Google Scholar 

  48. Zhang Z, Zhang Y, Kos A, Umek A (2018) Strain gage sensor based golfer identification using machine learning algorithms. Procedia Compu Sci 129:135–140

    Article  Google Scholar 

  49. Zheng N, Barrentine SW, Fleisig GS, Andrews JR (2008) Kinematic analysis of swing in pro and amateur golfers. Int J Sports Med 29(6):487–493

    Article  Google Scholar 

  50. Zhong L, Hu L, Zhou H (2019) Deep learning based multi-temporal crop classification. Remote Sens Environ 221:430–443

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (No. 2017R1A6A1A03015496) and by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2018R1A2B6001984).

Author information

Authors and Affiliations

  1. IT Research Institute of Chosun University, Gwangju, Republic of Korea

    Kyeong-Ri Ko & Sung Bum Pan

  2. Department of Electronics Engineering, Chosun University, Gwangju, Republic of Korea

    Sung Bum Pan

Authors
  1. Kyeong-Ri Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung Bum Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung Bum Pan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ko, KR., Pan, S.B. CNN and bi-LSTM based 3D golf swing analysis by frontal swing sequence images. Multimed Tools Appl 80, 8957–8972 (2021). https://doi.org/10.1007/s11042-020-10096-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10096-0

Keywords

Search

Navigation