Elektrotehniški vestnik 84(5): 247-252, 2017 Original scientific paper A sensor-based golfer-swing signature recognition method using linear support vector machine Zhichao Zhang1, Yuan Zhang1,*, Anton Kos2, Anton Umek2 1Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University ofJinan, Jinan, China 2Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia *Corresponding author e-mail: yzhang@ujn.edu.cn Abstract. Golf performance varies from person to person because of the differences in physical features of golfer's body and skill level. Recognizing golf swings of an individual golf player is essential to help improving the golf skill level. This can be done through feedback information provided by a specialized equipment or by a personal coach. Based on the classification of the golfer-swing shapes, this work analyses golf-swings. A sensor-based golfer-swing signature-recognition method is performed by using linear support vector machine (LSVM). Golf-swing signals are acquired by a strain-gage sensor fitted to the golf club that measures the club bend. To classify each golfer-swing multi-class classifier is built by combining binary LSVM models with an error-correcting-output-codes multi-class strategy. The experiment results of the training accuracy, testing accuracy and training time are compared with the results of other models including decision-tree algorithms, discriminant-analysis algorithms, other support vector machine algorithms, k-nearest neighbor (KNN) classifiers, and ensemble classifiers. A comparison shows that by using the strain-gage sensor and multi-class LSVM model, the golfer-swing signature is recognized accurately and effectively. Keywords: strain-gage sensor; golf swing; golfer-swing signature recognition; linear support vector machine Prepoznava golfskega zamaha na osnovi senzorskih signalov in metode linearnih podpornih vektorjev Izvedba golfskega zamaha se razlikuje glede na fizične sposobnosti igralca in nivoja njegovega znanja. Prepoznava posameznikovega golfskega zamaha je pomembna za pomoč pri njegovi izboljšavi. Le ta je navadno izvedena na osnovi povratne informacije, ki jo priskrbi posebej za to razvita oprema ali osebni trener. Članek obravnava razvrščanje oblik golskega zamaha, ki je osnova za personalizirano analizo izvedbe zamaha. V članku je opisana izvedba prepoznave posameznikovega golskega zamaha (osebni podpis) na osnovi senzorskih signalov in metode linearnih podpornih vektorjev (linear support vector machine - LSVM). Signali golfskih zamahov so pridobljeni s pomočjo strain gage senzorja pričvrščenega na palico za golf, ki meri upogib palice med izvedbo zamaha. Za razvrščanje posameznikovega zamaha je uporabljen model LSVM razvrščevalnika z več razredi in strategijo izhodnih kod za odpravljanje napak (error-correcting-output-codes). Rezultati poskusov kažejo, da naša metoda z uporabo strain gage senzorjev in razvrščanja po metodi LSVM z več razredi prepozna posameznikov golfski zamah natančno in učinkovito. 1 Introduction Golf is popular all over the world since the 20th century [1]-[3]. Golfers usually pay quite a lot for personal coach fees to learn playing golf well. Thus smart golf systems are expected to help golf players to reduce the coach fee costs [4]-[5]. The computer science and sensor technology play a vital role in sports activity analysis [6]. Multiple fields of technology has been applied in golf swing detecting and recognition; e.g. sensor, video analysis and image processing, machine learning, signal processing, etc. In Lv Dongyue's paper, the RGB-D images of the golfers and golf swings were generated by Kinect [7]. Since the resolution of the depth images is quite low, the author designed a dynamic Bayesian Network model to restore the joints positions from the original images. Video is also a popular choice for the data source in detecting and analyzing the golf swings or golfer performance [8]-[9] Golf-swing instructions for users with different physical features are provided in [8]. A professional's golferswings video was recorded to extract the specific postures, and a Gaussian process-regression model was built to predict the posture of other golfers (especially the non-professional players). In [9], the features of joints position, velocity, and angulation were extracted from golfer three-dimensioned skeleton coordination acquired by Kinect. The author designed a model combined with the Hidden Markov Model and Fuzzy Neural Network to classify the feature sequences into the performance groups of excellent, good, medium, and bad with the accuracy of 80%. There is another interesting work using video as the data source in which the authors analyzed the putter video to recognize the putting of each golfer [10]. They detected putter head, signaled by a red marker with a digital camera when the golfers performed putting. Six golfers perform 30 putting movements each. A simple image-processing algorithm was used to get the main point clusters and the Darwinian particle swarm optimization was applied to fit the function of each trial. The parameters of the function were the input of five classifiers: linear discriminant analysis, quadratic discriminant analysis, naive Bayes with a normal distribution, naive Bayes with a kernel smoothing density estimate, and least-squares support vector machines (LS-SVM). The LS-SVM yielded the best classification accuracy of 74.11%. From the above works we notice that a high-quality camera is beneficial for the golf-swing analysis but the system costs quite a lot. Thus Kinect, a game controller designed by Microsoft, is popular for taking golf-swing videos. However, using Kinect to take golf-swing videos, still costs much more than using pressure sensors [7], [11]. In [11], the author designed a system that combines a Wii balance board with Kinect sensors to detect the golfer's defined common mistakes in gravity and posture movement. There are four pressure sensors in the four corners of a Wii balance board, whereas 20 different joints of the human skeleton are provided by Kinect. Wii balance board pressure-sensor signals are compared to videos taken by Kinect. Their experiment shows that the cost of using Wii balance board pressure-sensor is lower at a higher accuracy compared to cameras. In our work we use a strain-gage (SG) sensor to acquire golf-swing signals. Our system is relatively low-cost and has a high classification accuracy. Authors [12] developed a portable instrument composed of a microcontroller, six-axis inertial sensor (MPU-6050), and Bluetooth wireless transmission module. MPU-6050 integrates a triaxial accelerometer and a triaxial gyroscope to detect the accelerations and angular velocities generated from golf swing movements. The authors designed an algorithm to recognize the golf swing motion with seven stages: address, backswing, top of the swing, downswing, impact, follow-through, and finish. This work shows that recognizing the signal/golf swing collected by their sensors is meaningful and feasible, and the recognized signal/golf swing in different stages can be interpreted by the mechanic. Different from the works mentioned above, our original golf-swing signal is acquired from a strain-gage (SG) sensor, fitted to the golf club, accurately measuring the golf-club bend [13]-[14]. Our prime goal is to recognize the golf swing of each golfer. This is essential for a later analysis of golf-swing performance of individual golfers [13]. For golf players, one of the most important steps of the training process is to achieve consistency; that means repeating the proper gestures and club positions many times. Consequently, our focus is on recognizing individual golfer-swing (signature) to provide the basis for the analysis of their different type of swings and possible feedback about their performance and progress. The contributions of our work are: 1) High-precision golf swing signal acquisition using SG sensor fitted to the golf club for a measurement of the golf-club bend. 2) Golfer-swing signature recognition using a multi-class LSVM model. The rest of the paper is organized as follows. Section 2 presents the golf swing/signal acquisition. Section 3 introduces the LSVM model and the multi-class strategy. Details of our experiment and comparison are explained in Section 4. The discussion is demonstrated in Section 5. Section 6 presents the conclusion and future work. 2 Golf swing signal acquisition In this work, the signal (golf swing) is collected by a SG sensor fitted to the golf club [13]-[14]. The golf club bend is measured using the SG sensor SGD-3/350-LY11 from Omega (Norwalk, CT, USA) shown in Figure 1 [13]. The cRIO professional measurement system combined with module 9237 (National Instruments Corporation, Austin, TX, USA) is used for connection with the SG sensor and acquiring the golf-swing signal as seen in Figure 2. Figure 1. A train gage sensor fitted to the shaft of the golf club measures its bend during execution of the swing. In our study, the golf-swing signals are collected from four skilled golf players. The sampling frequency of the system is 500 Hz and we record 2 seconds of each golf swing. The acquired SG golf-swing signal representing the golf-club shaft bend is shown in Figure 3. In Figure 3, there is an obvious variation seen from around 0.7 s to 1.25 s between the signals of players 2, 3, and 4. Meanwhile, from 1.8 s to the end, the signals of player 1 are different from those of the other three players. Thus, separateness in the time domain of the acquired golf-swing signals is noticed. In addition, there are 1000 points of each of the acquired 134 swings indicating a high dimension of data set. Accordingly, we adopt the Principle Component Analysis (PCA) to reduce the data dimension before classification. Figure 2. Golf-swing measurement system. Figure 4. Example of a binary SVM with a soft margin for a two-dimension of input vector xi e {xi,i-x2,J [17] The main idea of SVM is to find the optimal hyperplane shown as a solid line to separate the points of these two classes [18]. To maximize the distance from the hyperplane to the points of each class, a parallel support vector marked with a dotted line of each class is fixed by the closest points, whereas the distance between the 1 support vector and the hyperplane called margin j^y is doubled and maximized. Equation (1) obtained from the above analysis [17]: 2 max-.- w,b ||w| S. t. ' wTXi + b>+l, yt = +l . wTXi + b < -l, ji = —l (l) Figure 3. Golf swing signal representing the bend of the golf club during the swing execution. equaling 3 Multi-class LSVM model SVM is an algorithm used to solve a convex quadratic programming designed according to the knowledge of statistical learning. The algorithm was proposed by C. Cortes in 1995 [15]. LSVM is one of the variants of the SVM algorithms combined with the linear kernel function that is usually faster and simpler than the nonlinear function [16]. 3.1 LSVM introduction As shown in Figure 4, the points with the output space of two classes are marked with '+' and '-' separately [17]. The hyperplane is defined aswrx + b = 0, whereas the support vectors of a positive and negative class are wTx + b = 1 and wTx + b = -1, respectively. Variable x is the input space, whereas w and b are the constant parameters. Supposing the points with an ^-dimension input and output space of a binary support vector machine (SVM) model, the data set can be defined as D = {(xi,yi),(x2,y2), ■■■, (xm,ym)}, yt e {1,-1}. The value of yt indicates which class the input vector of xt belongs to. min—nwl w,b 2 (2) s.t. yi(wTXi + b) > l, i = 1,2,...,m. However, sometimes it is impossible to find a hyperplane to separate all the points, e.g. in Figure 4, there are a few points marked with red circles which are classified incorrectly even with the found optimal hyperplan [18]. Therefore, the soft margin is designed so that some points are allowed to violate yi(wTXi + b) > 1. Therefore, yt (wTxt + b) > 1 - with the slack variable > 0 is used to replace the previous constraint of equation (2). Accordingly, we get: • l|, I 12 „ min—Uwll + C w,b 2 i=1 (3) s. t.yi(wTxi +b)>1-ft, ft > 0, i=1, 2, ..., m. We notice that equation (3) is a constrained optimization problem that can be dealt with Lagrangian equation (4): 2 l Iwl i=1 i=1 m + ^ai(l-ti-yi(wTxi+b)) (4) max a m mm ......- i=1 m ^llaiUjyiyjxixj î=I j=i (5) s. t, 1 ViYi = 0, 0 < at < C,i = 1,2 ... ,m We can find the Karush-Kuhn-Tucker (KKT) complementarity conditions as seen in equation (6) [19]. The Sequential Minimal Optimization (SMO) is a classical effective algorithm usually used to calculate a [20]. The main idea of SMO is to introduce a constraint as seen in equation (7), when calculating equation (5). (ai(yif(xi)-1 + ^i) = 0 { b(C-ai) = 0 ,i = 1,z-,m (6) Table 1. Each learner is a binary LSVM model of f(x) = wTx + b that marks the positive class and the negative class with '1' and '-1', respectively. Coding matrix M composed of elements m(k,/) (the variables of / and k indicate the number of learners and classes, respectively) is shown below, whereas /=1, 2, 3, 4, 5, 6 and k=1, 2, 3, 4. i=i s. t. at> 0,^i> 0 To find a saddle point, we set the derivative of L(w, b, a, %, n) with respect to variables w, b and ^ to be zero, and combine the results with Lagrangian equation (4), and after that we can get a dual problem as seen in equation (5). M = ll -l 0 00 ll l0 0 -1 l 0 l 0 -1 -1. The final predicted observation calculated by equation (8) is assigned to class k that minimizes the aggregation of the losses for the six binary learners. T.^!=1lmkll^hinge(mkl, sl) k = arg min k hinget) = max(0,1 - z) 4 Experiment and comparison (8) In the experiment, we acquired 134 swings from 4 golf players. The swings of each individual are divided into training set and testing set. Figure 5 shows the framework of the experiment. After collecting the golf swings, six confidences are calculated to be the input of the multi-class LSVM model by using the Principle Component Analysis (PCA). In addition, the classification results can be given in the output of the multi-class LSVM model. 2 m m i = 1 y1a1+y2a2 = k (7) After getting a^nd ^, variables w and b are calculated, whereas hyperplane wTx + b = 0 is found, and function f(x) = wTx + b is the one used to predict the response (output) for the predictor space (input). 3.2 Multi-class strategy An error-correcting output codes multi-class model is used in our work since it can improve the classification accuracy [21]. Table 1. The one-versus-one coding design Classl Class 2 Class 3 Class 4 Learner 1 1 -1 0 0 Learner 2 1 0 -1 0 Learner 3 1 0 0 -1 Learner 4 0 1 -1 0 Learner 5 0 1 0 -1 Learner 6 0 0 1 -1 The swings in this work are acquired from four golfers, which mean that 4 classes of swings are included into the dataset. Therefore, the one-versus-one coding design is chosen in the first step with Cf=6 learners as shown in Golf-swing acquisition Figure 5. Framework of the experiment We compare the multi-class LSVM model in terms of the training accuracy, testing accuracy and training time with 18 other models which include decision-tree algorithms, discriminant-analysis algorithms, other support vector machine algorithms, k-nearest neighbor (KNN) classifiers, and ensemble classifiers (see Table 2 to Table 6). We can see that the multi-class LSVM model gets a 100% training accuracy and 100% testing accuracy with the minimum training time of four seconds, which performs the best within all 19 models. The comparison result shows that our method using the SG sensor and the multi-class model is effective and little time consuming in the golfer-swing signature classification. Table 2. The performance of decision tree Classifier Training Testing Training accuracy_accuracy_time (s) Complex tree 93.9% 89.79% 7 Medium tree 93.9% 89.79% 4 Simple tree 93.9% 89.79% 5 Table 3. The performance of discriminant analysis Classifier Training Testing Training _accuracy_accuracy_time (s) Linear discriminant 100%_100%_7_ Quadratic discriminant 98.5%_98.53%_4_ Table 4. The performance of support vector machine Classifier Training Testing Training _accuracy_accuracy_time (s) LSVM 100% 100% 4 Quadratic SVM_100%_98.53% 4_ Medium Gaussian SVM 59.1%_57.35% 4_ Coarse Gaussian SVM 59.1% 57.35% 4 Table 5. The performance of nearest neighbor classifier Classifier Training Testing Training _accuracy_accuracy_time (s) Fine KNN 100% 100% 6 ~ Medium KNN_75.6%_83.82% 4_ Coarse KNN_59.1%_57.35% 4_ Cosine KNN_84.8%_94.12% 4_ Cubic KNN 75.8% 77.94% 4 Table 6. The performance of ensemble classifier Classifier Training Testing Training accuracy_accuracy_time (s) Boosted trees 59.1% 57.35% 6 Bagged trees 98.5% 97.06% 7 Subspace discriminant 100% 100% 10 Subspace KNN 100% 100% 6 Rusboost 92.4% 92.66% 6 5 Discussion This work addresses the issues of recognizing golf-swing signature of individual golfers. The result of using linear discriminant, LSVM, and fine KNN classifiers with which the 100% training accuracy and 100% testing accuracy are achieved demonstrates the usefulness and applicability of machine learning algorithm in identifying the golfer-swing signature. The multi-class LSVM performs the best with the 100% testing accuracy and minimum training time compared with the other 18 models listed in Table 2 - Table 6. The conclusion drawn from our analysis is that the golf swings vary from one golfer to the other, and that our SG sensor-based LSVM model works well in golfer-swing signature recognition. Based on the current golfer-swing analysis and personal golfer-swing history, our next step will be providing a (real-time) feedback to golfers. 6 Conclusion and future work In this work we designed a method to recognize golferswing signature of individual golf players using a SG sensor and the multi-class LSVM model. A comparison with other models shows that the presented multi-class LSVM model is effective in recognizing golfer-swing signature and that it outperforms other comparable classifiers. Our experiment shows that each golfer has his/her own distinguishable swing signature. The aim of our future work is to invite a large number of golf players to analyze their golf swings and to build models to classify different types of swings and identify possible errors in swing execution. Acknowledgement This work is supported in part by the National Natural Science Foundation of China under Grant 61572231, in part by the Shandong Provincial Key Research & Development Project under Grant 2017GGX10141, and in part by the Slovenian Research Agency within the research program Algorithms and Optimization Methods in Telecommunications. The corresponding author is Yuan Zhang. References [1] Theriault, Germain, and Pierre Lachance. "Golf injuries." Sports Medicine 26, no. 1 (1998): 43-57. [2] Ong, T.F.,and Mansor, S.H., 2014. Travel Motivation and Points of Attachment Among Golf Spectators. In Proceedings of the International Colloquium on Sports Science, Exercise, Engineering and Technology 2014 (ICoSSEET 2014) (pp. 581-590). Springer Singapore. [3] Stenner, Brad J., Amber D. Mosewich, and Jonathan D. Buckley. "An exploratory investigation into the reasons why older people play golf." Qualitative Research in Sport, Exercise and Health 8, no. 3 (2016): 257-272. [4] Wu, Hsien-Huang P., and Hong-Yi Guo. "Automatic Optical Inspection for steel golf club." In Fuzzy Systems and Knowledge Discovery (FSKD), 2015 12th International Conference on, pp. 2332-2336. IEEE, 2015. [5] Natsagdorj, Sainbuyan, John Y. Chiang, Che-Han Su, Chih-chung Lin, and Cheng-yen Chen. "Vision-based Assembly and Inspection System for Golf Club Heads." Robotics and Computer-Integrated Manufacturing 32 (2015): 83-92. [6] Thomas, Graham, Rikke Gade, Thomas B. Moeslund, Peter Carr, and Adrian Hilton. "Computer vision for sports: current applications and research topics." Computer Vision and Image Understanding (2017). [7] Lv, Dongyue, Zhipei Huang, Lixin Sun, Nenghai Yu, and Jiankang Wu. "Smart motion reconstruction system for golf swing: a DBN model based transportable, non-intrusive and inexpensive golf swing capture and reconstruction system." Multimedia Tools and Applications 76, no. 1 (2017): 1313-1330. [8] Sugimura, Daisuke, Hiroharu Tsutsui, and Takayuki Hamamoto. "Detecting flaws in golf swing using common movements of professional players." Machine Vision and Applications 27, no. 1 (2016): 13-22. [9] Zhang,Lichao, Jui-Chien Hsieh, and Jiangping Wang. "A Kinect-based golf swing classification system using HMM and neuro-fuzzy." In Computer Science and Information Processing (CSIP), 2012 International Conference on, pp. 1163-1166. IEEE, 2012. [10] Couceiro, M.S., Dias, G., Mendes, R. and Araujo, D., 2013. Accuracy of pattern detection methods in the performance of golf putting. Journal of motor behavior, 45(1), pp.37-53. [11] Huang, Shih-Yu, Kuei-Pin Kuo, and Yi-Hsuan Lin. "A golf swing analysis system using Wii balance board and Kinect sensors for novice players." Multimedia Tools and Applications 74, no. 23 (2015): 10679-10696. [12] Hsu, Yu-Liang, Yu-Tai Chen, Po-Huan Chou, Yu-Chen Kou, Yi-Chung Chen, and Heng-Yi Su. "Golf swing motion detection using an inertial-sensor-based portable instrument." In Consumer Electronics-Taiwan (ICCE-TW), 2016 IEEE International Conference on, pp. 1-2. IEEE, 2016. [13] Umek, Anton, Yuan Zhang, Sašo Tomažič, and Anton Kos. "Suitability of Strain Gage Sensors for Integration into Smart Sport Equipment: A Golf Club Example." Sensors 17, no. 4 (2017): 916. [14] Kos, Anton, Sašo Tomažič, and Anton Umek. "Suitability of smartphone inertial sensors for real-time biofeedback applications." Sensors 16, no. 3 (2016): 301. [15] Corinna Cortes, Vladimir Vapnik, "Support vector networks", Machine Learning, 20(3), str. 273-297, 1995. [16] Andras kocsor , Janos csirik, "Fast Independent Component Analysis in Kernel Feature Spaces", International Conference on Current Trends in Theory and Practice of Computer Science, 2001(2234), str: 271-281, 2001. [17] Zhihua Zhou, Machine Learning, Tsinghua University Press, 2016, pp. 121-140. [18] Scholkopf, Bernhard, and Alexander J. Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002, pp.6-19. [19] Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004, pp.244. [20] Platt, John. "Sequential minimal optimization: A fast algorithm for training support vector machines." (1998). [21] Furnkranz, Johannes, "Round Robin Classification." J. Mach. Learn. Res., Vol. 2, 2002, pp. 721-747. Zhichao Zhang received her BS in Computer Science in 2015 from University of Jinan, China. She is a master student majoring in computer science at Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, and was a visiting student sponsored by the EU's Erasmus+ program at the Faculty of Electrical Engineering, University of Ljubljana, Slovenia. As a first author, she currently has four papers published in biomedical signal processing and M-Health. She is a member of both ACM and CCF. Her current research interests include biomedical signal processing, machine learning, and pattern recognition. Yuan Zhang received his Ph.D. degree in Control Theory & Engineering from Shandong University, China, in 2012. He is currently an Associate Professor at University of Jinan (UJN), China. Dr. Zhang was a visiting professor at Computer Science Department, Georgia State University, USA, in 2014. As the first author or corresponding author he has published more than 40 peer reviewed papers in international journals and conference proceedings, one book chapters, and five patents in the areas of biomedical engineering and m-Health. He has served as Leading Guest Editor for four special issues of IEEE, Elsevier, Springer and InderScience publications, and has served on the technical program committee for numerous international conferences. He is an associate editor for IEEE Access. Dr. Zhang's research interests are in biomedical signal processing and m-Health. His research has been extensively supported by the Natural Science Foundation of China, China Postdoctoral Science Foundation, and Natural Science Foundation of Shandong Province with total grant funding over one million RMB. Dr. Zhang is a Senior Member of both IEEE and ACM. Anton Kos received his Ph.D. degree in electrical engineering from the University of Ljubljana, Slovenia, in 2006. He is currently an assistant professor at the Faculty of Electrical Engineering, University of Ljubljana. He is a member of the Laboratory of Information Technologies at the Department of Communication and Information Technologies. Between 2000 and 2009 he was the head of IT at GlaxoSmithKline d.o.o., Ljubljana, Slovenia. He is the (co)author of fourteen papers appeared in the international engineering journals and of more than thirty papers presented at international conferences. He is a member of the research program Algorithms and Optimization Methods in Telecommunications, which is one of the two research programs that are every year listed among the best research programs in Slovenia. He is the leader of two industrial research and development projects in designing of sensor-based smart sport equipment and forestry machinery. His professional work encompasses co-organizing international VIPSI conferences, serving as a program committee member at several international conferences (IIKI, FABULUOS, and ICIST), serving as a guest editor and reviewer at several international scientific journals. He is the Maxeler ambassador for Slovenia (Dataflow computing). He is a member of IEEE, Union of Electrical Engineering of Slovenia, and SICOM. Anton Umek received his Ph.D. degree in electrical engineering from the University of Ljubljana, Slovenia, in 1999. He is currently a Senior Lecturer at the Faculty of Electrical Engineering, University of Ljubljana. He is a member of the Laboratory of Information Technologies at the Department of Communication and Information Technologies. He is a member of the research program Algorithms and Optimization Methods in Telecommunications that was two years in a row the best research program financed by the Slovenian research agency. Since last year he is in charge of industrial research and development projects in designing of sensor-based smart sport equipment and sensor-based forestry machinery. His teaching and research work includes signal processing, digital communication, secure communications, access network technologies and design of sensor-supported sport training systems. He is the (co)author of eight papers appeared in the international engineering journals and of more than thirty papers presented at international conferences. He is also a member of IEEE and since 2015 the Slovenian section ComSOC chapter chair.