Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 24 Evaluation of Key Performance Indicators of Logistics Firms Nazlıcan GÖZAÇAN 1* and Çisem LAFCI 2 1 Yaşar University/ International Logistics Management, Izmir, Turkey 2 Yaşar University/ International Logistics Management, Izmir, Turkey [Corresponding Author indicated by an asterisk *] Abstract — Key Performance Indicators (KPI) has been outlined for implementing total quality management (TQM) across logistics sector. This study constituted on the quality values of logistics firms in the logistics sector, which is examined with key performance indicators through the integrated method of Analytic Hierarchy Process (AHP) and SMART Goal Setting. The calculations were performed for logistics firms. The method used in this study is the integrated method of the AHP Method and SMART Goal Setting. The results highlight the most mentioned key performance indicators in the literature in a prioritized version also during the prioritizing process via AHP Method, the SMART Goal Setting approach also is applied. Key words - Analytic Hierarchy Process (AHP), Key performance indicators (KPI), SMART Goal Setting, Total Quality Management. I. INTRODUCTION The significance of logistics is growing in this era of continuous growth. The industry demonstrates ongoing growth in many fields. Furthermore, in response to these radical adjustments influencing the logistics industry, the performance word with which the logistics companies are involved is becoming more crucial. The performance knowledge of the logistics process must, therefore, accommodate itself to these latest adjustments. The Council of Logistics Management describes Logistics as “…is part of the supply chain process that plans, implements, and controls the efficient, effective flow and storage of goods, services, and related information from the point of origin to the point of consumption in order to meet customers’ requirements”. With this definition, companies have recognized that logistics management plays a vital role in previous decades. Another important point regarding the concepts of logistics performance state is the transportation of the products to the receiver on time with the quality of the logistics services and competence of it. In addition, the logistic performance of the logistics operations is a measure of success and effectiveness [14]. Therefore, quality control in logistics has a high value [4]. There is a significant amount of KPIs for logistics services in the literature, and this surplus is highly dispersed. This disorganization makes quality control difficult. KPIs should be prioritized to solve this problem and to comply with changing quality criteria. The objective of this research is to evaluate the KPIs for Total Quality Management (TQM) of Logistics. An extended literature review is conducted to find the KPIs of logistics in several fields. And, a table of all KPIs found is designed to choose the most mentioned ones. The method used to prioritize the most mentioned KPIs is the integrated method of the AHP and SMART Goal Setting proposed by Shain and Mahbod in 2007. AHP is used as a multi-criteria decision-making method to analyze the literature data with the help of SMART (Specific, Measurement, Attainable, Realistic, and Time-Sensitive) goal-setting method. This paper is organized as follows. The KPIs in logistics are summarized in Section 2. The methodology is explained in Section 3. The analysis is presented in Section 4. Results and conclusions are summarized in Section 5. II. KPI IN LOGISTICS KPIs are excellent indicators to monitor the performance. Key performance indicators vary following nature, and strategy and contribute to calculating the success in achieving long -term goals, also take into account the difficult-to-measure criteria [7]. Databases such as Science Direct, Taylor & Francis, Emerald Insight, and Google Scholar have been used for the research. In this research, a broad literature review is performed. As a result of the literature review, 116 KPIs have been observed, and the first five the most mentioned ones are used as the KPIs in this study. KPIs found throughout the research can be seen in Table 1. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 25 Table 1 KPIs observed throughout the Literature Review Ability to deal with returns Flexibility to changes Product configuration Accounts receivable turnover Geographical location Purchase order cycle time Accuracy of forecasting Government satisfaction Quality of delivery documentation Accurate billing Interest coverage ratio Quality system certifications Accurate documentation Inventory accuracy Quality when receiving goods in the warehouse Advance notice on shipping delays Inventory days of supply Receiving and assessing of return shipments Advance shipment notification Inventory management and Relationships with other stakeholders registration Asset turns Inventory turn-around times Research and development capability Available stock level Investor (financier) satisfaction Response accuracy Backorder/stock-outs IT Infrastructure Response time to inquiries Building product displays Labor cost Responsiveness to changes Cash flow Labor efficiency Return on investments Cash-to-cash cycle time Lead time Return processing cost Circumstance of delivery Loss/damage experience Revenue growth Community satisfaction Management accounting techniques Right equipment supplied Complete orders Managerial skills Sales force complaints Cost Market share Sales force feedback Cost of returned goods Non-government organization Sales growth satisfaction Cultural match Number of customer complaints Shipping error Customer complaints handling Number of customer returns Shipping/transportation Cost Customer inquiry response Number of dollars shipped Social media usage for brand building Customer satisfaction Number of dollars/unit shipped Stock-holding cost Customer service level Number of kilos/units shipped Supplier base size Delivery defect Number of shipments per vehicle-mile Supplier reliability Delivery reliability On-time pick-up Supplier satisfaction Delivery security On-time shipment-delivery The transport and warehouse capacity utilization Educated employee Order accuracy Timeliness Effectiveness in distribution operations Order entry methods Total / overall satisfaction Effectiveness of delivery invoice Order procedure convenience Total logistics management costs methods Emergency shipping Overhead cost Total order cycle time Employee satisfaction Packaging activities Total Supply chain cycle time Environmental Past performance Transport capacity awareness/understanding Environmental group satisfaction Perceived quality Value added cost Equipment cleanliness Perceived value of product Value-added activities Equity ratio Personnel attitude Warehouse capacity Ethical responsibility Picking/shipping accuracy Warehouse cycle time Fill rates Process cycle time Warehouse labor productivity Financial perspective-profitability Product and service variety Warranty cost Product clarity Willingness for information sharing The on-time shipment, lead time, backorder or stock-outs, shipping error, and fill rate are top five KPIs mentioned according to the literature research of KPIs in international logistics firms (See Table 2). Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 26 Table 2 the Most Mentioned First Five KPIs of Logistics Literature KPIs Mention Ratio (%) On-time Shipment/delivery 52 Lead Time 32 Backorder/Stock-outs 32 Shipping error 28 Fill Rates 24 On-time shipment/delivery: The right products are delivered correctly at the right time. Lead time: The time between the orders initiated and the right product delivered without damage. Backorder/stock-out: the order cannot be delivered due to the lack of adequate products available in stock. Shipping error: the number related to shipping mistakes such as mislabeled packaging, neglecting package dimension, etc. Fill rates: the ratio of the volume of the vehicle utilized to the total volume. III. METHODOLOGY Shain and Mahbod proposed the combined method of the AHP and SMART Goal Setting. The combined method is used in this research to prioritize the most cited KPIs. AHP is used as a multi - criteria decision-making approach to prioritize the top five KPIs mentioned in the literature with the help of SMART (Specific, Measurement, Attainable, Realistic, and Time-Sensitive) goal-setting method. The KPIs should be consistent with the SMART acronym. The integration of the AHP and SMART Goal Setting will be an assistant to determine the consistent KPIs in the logistics firms. The process starts with prioritizing the goals with AHP. After all of the goals are prioritized, the following steps can be seen in Fig. 1. (a) First of all, all of the KPIs are defined; as the second stage The AHP Hierarchy based on SMART characteristics is built; pairwise comparison matrices are generated between alternatives however in this case which are the KPIs defined; global weights are calculated; (b) the KPIs that are more relevant to the goals are selected. A. Analytical Hierarchy Process (AHP) AHP, a multi-criteria decision-making method is a great way to evaluate KPIs due to all the risks participating [1]. AHP can be a convincing technique when it is applied for complicated decision - making problem for TQM implementation [3]. One of the advantages of AHP is to make both qualitative and quantitative decision qualities consistent, and to be flexible in terms of determining objectives [5]. AHP which is the decision-making tool proposed to effectively aid in making the best solution of complex multiple criteria problems by handling combining both tangible and intangible factors and sub-factors to weight and prioritize is introduced by Thomas L. Saaty [10]. Decomposition of structural hierarchy, pair wise comparative judgments, and synthesis of priorities and the measurement of consistency are the three steps of the AHP [9]. The first step is to decompose the structural hierarchy of the problem into a hierarchical framework such as goal, criteria, and alternatives. AHP problems are defined in at least three levels for complex decisions to be structured as a hierarchy from the general goal to various "criteria", "sub-criteria", and so on. The main purpose is placed at the top of the hierarchy. Criteria follow the main purpose as in intermediate level. Sub-criteria may occupy as a level between criteria and alternatives in some cases. Finally, the decision alternatives are found on the lowest level of the hierarchy. As a creative task of the process, to establish a structural hierarchy problem may require creative thinking depending on the nature or type of managerial decision using different perspectives [10]. Pair wise comparison matrices are created to each element to determine the priorities of the elements in the hierarchical framework in Equation (1); A= [aij]nxn i=1,...,n j=1,...,n (1) For each element in each level of the hierarchy, a set of matrices of relative rankings is obtained for the element of the immediately higher level to conduct pairwise comparisons. Such Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 27 pairwise comparison matrices are structured to prioritize and transform individual comparative assessments between the elements via ratio scale measurements. The significance of each criterion is subjective. The significance values are given by the decision-maker, such as supervisors or experts, and written on the matrices [10]. The scale that is used to determine the importance is a nine- point scale (See Table 3). Table 3 the Nine- Point Scale [10] Intensity of Importance Definition Explanation 1 Equal importance Two activities contribute equally to the objective 3 Moderately preferred Experience and judgment slightly favor one activity over another 5 Strongly preferred Experience and judgment strongly or essentially favour one activity over another 7 Very strongly preferred An activity is strongly favored over another and its dominance demonstrated in practice 9 Extremely preferred The evidence favoring one activity over another is of the highest degree possible of affirmation 2, 4, 6,8 Intermediates values Used to represent compromise between the preferences listed above Reciprocals Reciprocals for inverse comparison After the pair wise comparison matrices are obtained, eigenvectors or in other word, the relative weights (the relative importance of the elements compared to each other) as in Equation (2), global weights in Equation (3), and the maximum eigenvalue that is also known as λmax for each matrix are calculated in Equation (4) by solving the matrices of order n and a new matrix is created as a result; Bi=[bij]nx1 i=1,...,n bij= aij (2) n ∑ i=1 aij Ci=[bij]nxn i=1,...,n j=1,...,n (3) Wi=[wi]nx1 D=[aij]nxn x [wi]nx1 = [di]nx1 (4) di ∑ i=1 n Ei Ei= wi i=1,...,n λmax= n When the eigenvectors (the relative weights), global weights and the λ max are achieved, the consistency index is calculated for the matrices of order n by the formula in Equation (5); CI = (λmax -n)/(n-1) (5) The value of the random consistency index (RI) based on matrix size from 1 to 10 obtained by approximating random indices using a sample size of 500 (See Table 4) [10]. Table 4 Average Random Index (RI) based on matrix size [10] n 1 2 3 4 5 6 7 8 9 10 Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 28 RI 0 0 0.52 0.89 1.11 1.25 1.35 1.40 1.45 1.49 The consistency ratio is then calculated using the Equation (6); The acceptable CR range is different depending on the size of matrix i.e. 0.05 for a 3 by 3 matrix, 0.08 for a 4 by 4 matrix and 0.1 for all larger matrices, n≥ 5 [12]. If the value of the CR is more than the acceptable range, the evaluation is inconsistent. The process should be reviewed. B. SMART Goal Setting Goal setting is one of the key points to take “right” steps on the way of success, and it is one of the first steps that should be held. The most significant advantage of goal setting is that it is a great tool to guarantee that work is completed on the correct intention, at the accurate time, by the proper staff, in the right utilization of assets [1]. The SMART acronym was first introduced by Doran in 1981. Doran said that “The establishment of objectives of the development of their respective action plans is the most critical step in a company’s management process.” The acronym that is effective to be associated with setting the ideal goal is SMART. The acronym should be specific on the target area to improve, measurable to be an indicator of progress, assignable to specify who is in charge to manage the process, realistic to see if the goal is achievable within the current resources, time-based to see the due date. The acronym is not limited with the five criteria; also SMARTER acronym is used [8]. KPIs reveal the success level on the way to achieve the goal/s. The goal/s should be clear and concrete. Table 5 displays the words associated with each letter of the SMART acronym that is taken into consideration while determining the KPIs in the paper with the reasons. Table 5 Smart Goal Setting for Logistics Sector SPECIFIC Each criterion in terms of clear and detailed feasibility (which must be able to response such questions as who, where, when, how) and will be evaluated in comparison with the corresponding criteria. MEASURABLE Each criterion will be evaluated in comparison with the criteria in terms of concrete measurability by defining the physical manifestations of the goal to make it easier to reach. ATTAINABLE Each criterion in terms of being reasonable and achievable, and will be evaluated in comparison with the corresponding criteria. REALISTIC Each criterion will be assessed in terms of its realism and the availability of its resources in comparison with the corresponding criteria. TIME-SENSITIVE Each criterion will be evaluated in comparison with the corresponding criteria by having a time interval. KPI should provide the structure and depend on this time interval so the analyst is able to monitor the progress. CR = CI/RI (6) Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 29 Fig. 1 Integration of The AHP Method and SMART Goal Setting [1] Define and this all of the KPIs Build AHP hierarchy based on SMART characteristics Pairwise comparison Calculate global weight Select relev ant KPIs a) Select KPIs Specific Measurable Attainable Realistic Time-Sensitive b) KPI1 KPI2 KPI3 ... KPIn Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 30 IV. ANALYSIS OF THE MODEL Table 6 The Pair wise Comparison Matrices of KPIs With Relative Importance Weights On-time Lead Backorder Shipping Shipment- Fill Rates Relative Importance Time /Stock-outs error delivery Specific: On-time delivery- 1.000 0.497 0.446 0.735 0.393 0.106 shipment Lead Time 2.010 1.000 0.378 0.633 0.218 0.118 Backorder/Stock-outs 2.241 2.646 1.000 0.699 0.523 0.205 Shipping error 1.361 1.579 1.432 1.000 0.398 0.183 Fill Rates 2.546 4.583 1.911 2.515 1.000 0.388 Measurable: On-time delivery- shipment 1.000 0.498 0.386 0.664 0.369 0.093 Lead Time 2.006 1.000 0.218 0.622 0.198 0.098 Backorder/Stock-outs 2.590 4.583 1.000 0.751 0.467 0.224 Shipping error 1.506 1.607 1.332 1.000 0.188 0.149 Fill Rates 2.711 5.045 2.141 5.318 1.000 0.436 Attainable: On-time delivery- 1.000 0.244 0.225 0.386 0.237 0.061 shipment Lead Time 4.091 1.000 0.293 0.595 0.440 0.142 Backorder/Stock-outs 4.450 3.409 1.000 0.633 0.565 0.243 Shipping error 2.590 1.682 1.579 1.000 0.427 0.208 Fill Rates 4.213 2.272 1.769 2.340 1.000 0.346 Realistic: On-time delivery- shipment 1.000 0.298 0.481 0.790 0.595 0.111 Lead Time 3.350 1.000 0.435 0.841 0.595 0.183 Backorder/Stock-outs 2.079 2.300 1.000 0.773 0.523 0.211 Shipping error 1.266 1.189 1.294 1.000 0.263 0.160 Fill Rates 1.682 1.682 1.911 3.807 1.000 0.335 Time-Sensitive: On-time delivery- 1.000 0.485 0.485 1.189 1.012 0.153 shipment Lead Time 2.060 1.000 0.355 0.849 0.816 0.172 Backorder/Stock-outs 2.060 2.817 1.000 1.027 0.726 0.258 Shipping error 0.841 1.178 0.974 1.000 0.355 0.153 Fill Rates 0.988 1.225 1.377 2.817 1.000 0.264 The analysis starts with pair wise comparison matrices (see Table 6). The importance of each selected KPI is evaluated in terms of their conformity to SMART characteristics by experts. The evaluations of each logistics expert were obtained by taking geometric means, and pairwise comparison matrices were created. Eigenvectors or relative importance values are calculated as the next step after obtaining the pairwise comparison matrices by summing up each. Later, each element is divided by the sum of its column. The same process is implemented for the SMART acronym, as can be seen in Table 7. Time-Sensitivity is the most significant criterion for logistics companies. Table 7 The Pair wise Comparison Matrices of SMART Acronym With Relative Importance Weights Specific Measurable Attainable Realistic Time-Sensitive Relative Importance Specific 1.000 0.261 0.411 0.863 0.411 0.098 Measurable 3.834 1.000 0.809 0.719 0.439 0.190 Attainable 2.432 1.236 1.000 1.158 0.695 0.210 Realistic 1.158 1.390 0.863 1.000 0.293 0.153 Time-Sensitive 2.432 2.280 1.439 3.409 1.000 0.348 Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 31 The final weights for each KPI are calculated as: 0.098*0.106+0.190*0.089+0.210*0.61+0.153*0.106+0.348*0.153 The results of the calculation of final weights of each KPI are seen in Table 8. V. RESULTS AND CONCLUSION In this study, Microsoft Excel is utilized for the computations. Consistency rates of each comparison matrix are therefore below 0.1 in the calculations of pairwise comparison matrices, so that all the comparison matrices are consistent. The results show that the most important KPI, which is compatible with SMART criteria, are fill rates, backorder/stock -outs, shipping error, lead time, and on-time delivery, as seen in Table 8. Table 8 The Final Weights KPIs Final Weights Ranking On-time delivery-shipment 0.109 5 Lead Time 0.146 4 Backorder/Stockouts 0.234 2 Shipping error 0.169 3 Fill Rates 0.341 1 AHP/SMART Method is used because of its advantage of being able to consider both qualitative and quantitative factors. The hierarchy applied at the beginning of the method describes the general situation more clearly and also explains the possible communication between the criteria and the alternatives better. In this case, KPIs are considered as alternatives. As conclusion, the five most important KPIs of logistics are ranked in Table 8. As mentioned at the beginning of the paper, KPIs are essential for TQM. In the logistics sector, Logistics 4.0 has become even more critical in today's conditions and technology. Accordingly, the preferences of the logistics companies, depending on the preferences of quality perception, and KPIs change and develop depending on the understanding of quality. This research provides the basis for the next study. The next research should concentrate on the evolving concept of performance along with the requirements of the ever- evolving idea of Logistics 4.0 and how the present circumstances will evolve. REFERENCES 1. A. Shahin and M. A. Mahbod, “Prioritization of key performance indicators: An integration of analytical hierarchy process and goal setting” International Journal of Productivity and Performance Management, vol. 56, PP. 226 -240, 2007. 2. J. Kangas, “Multiple ‐use planning of forest resources by using the analytic hierarchy process”, Scandinavian Journal of Forest Research, vol. 7, pp. 259-268, 1992. 3. K. S. Chin, K. F. Pun and Y. Xu and J. S. F. Chan, “An AHP based study of critical factors for TQM imp lementation in Shanghai manufacturing industries”, Technovation, vol. 22, pp. 707-715, 2002. 4. M. Christopher, “Logistics & supply chain management”, Pearson UK, 1998. 5. M. Kurtilla, M. Personnen, J. Kangas and M. Kajanus, “Utilizing the analytic hierarchy process (AHP) in SWOT ana lysis —a hybrid method and its application to a forest-certification case”, Forest policy and economics, vol. 1, pp. 41-52, 2000. 6. M. Lambert, J. R. Stock, and L. M. Ellram, “Fundamentals of logistics management”, McGraw-Hill/Irwin, 1998. 7. Ö. B. Tek and İ. Karaduman, “Tedarik Zinciri Bakış Açısıyla Lojistik Yönetimi: Küresel Yönetimsel Yaklaşım Türkiye Uygulamaları”, İstanbul: Ekonomi Yayınları, 2012. 8. T. Doran, “There’sa SMART way to write management’s goals and objectives”, Management review, vol.70, p p. 35- 36, 1981. 9. T. L. Saaty, “The Analytical Hierarchy Process, Planning, Priority. Resource Allocation.”, RWS Publications, USA, 1980. 10. T. L. Saaty, “Fundamentals of decision making and priority theory”, 2nd ed., Pittsburgh, PA: RWS Publications, 2000. 11. T. L. Saaty and L. G. Vargas, “Models, methods, concepts & applications of the analytic hierarchy p rocess”, S p ri nger Science & Business Media, vol. 175, 2012. 12. W. Atthirawong, and B. MacCarthy, “An application of the analytical hierarchy process to international location decision- making,” In Proc. of the 7th International Sym. on Manufacturing, September 2002. 13. W. Cheng, and H. Li, “Information priority-setting for better resource allocation using analytic hierarchy process (A HP)”, Information Management & Computer Security, vol. 9, pp. 61-70, 2001. 14. Y. Bayraktutan, and M. Özbilgin, “Uluslararası ve Yurtiçi Ticarette Taşıma Türlerinin Payı: Bir Analitik Hiyerarşi Prosesi (A H P) Uygulaması”, Journal of Institute of Social Sciences, vol. 6, pp 405-436, 2015. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 24-32 doi: 10.2478/jlst-2020-0002 32 AUTHORS A. Nazlıcan GÖZAÇAN is with Yaşar University/ International Logistics Management, Izmir, Turkey (e- mail: nazlicangozacan@outlook.com). B. Çisem LAFCI is with Yaşar University/ International Logistics Management, Izmir, Turkey (e-mail: cisemlafci@gmail.com). Manuscript received by 20 Feb 2019. Published as submitted by the author(s).