DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING ORODJA ZA UMERJANJE DISTOX IN POTREBA PO PREVERJANJU UMERITVE Matija PERNE 1,2* Abstract UDC 53.089.6:551.44 Matija Perne: DistoX calibration tools and the need for cali- bration checking For proper cave surveying using DistoX, the device needs to be calibrated with adequate accuracy. Calibrating does not require any tools; but, tools to make calibration easier have been devel- oped. Theoretical consideration shows that the use of certain tools enables one to introduce a type of calibration error that goes undetected by the calibration software. In this study, the existence of such errors is experimentally confirmed and their magnitude is estimated. It is demonstrated to be crucial that the DistoX is calibrated and that the calibration is valid, that is, that the device has not changed since it was last calibrated. No part of the DistoX must have moved or changed its magnetization since calibration, not even the battery. The calibration method used and the quality of the resulting calibration are important too. It is highly recommended that the DistoX be checked im- mediately before surveying a cave and thus avoid the possibil- ity of using an uncalibrated, not validly calibrated, or poorly calibrated device. To complete the check, a few survey shots are measured multiple times with the device at different roll angles, and the back shot of one of the shots is measured. If the device is properly calibrated, the measurements will agree with each other within the acceptable measurement error. This is not the case for a device that is not properly calibrated. Key words: cave surveying, DistoX, statistics. Izvleček UDK 53.089.6:551.44 Matija Perne: Orodja za umerjanje DistoX in potreba po pre- verjanju umeritve Pri merjenju jam z orodjem DistoX je pomembno, da je to umerjeno in da je umeritev zadovoljivo natančna. Umerjanje ne zahteva dodatnih orodij, vendar so na voljo pripomočki, ki naj bi umerjanje olajšali. Teoretični razmislek pokaže, da uporaba nekaterih pripomočkov povzroča napake, ki jih programska oprema za umerjanje ne zazna. V raziskavi eksperimentalno potrdim obstoj teh napak in ocenim njihovo velikost. Pokažem, da je ključno, da je DistoX umerjen in da je umeritev veljavna, kar pomeni, da se lastnosti naprave od umerjanja do merjenja niso spremenile. Med umerjanjem in merjenjem se ne sme pre- makniti noben del DistoX, niti ta ne sme spremeniti magneti- zacije, niti baterija. Pomembna sta tudi način umerjanja in ka- kovost umeritve. Toplo priporočam preizkus delovanja DistoX neposredno pred merjenjem jame, da bi se izognili merjenju z neumerjeno, neveljavno umerjeno ali slabo umerjeno napravo. Napravo preizkusimo tako, da nekaj vizur izmerimo večkrat z napravo v različnih položajih, da je tipkovnica enkrat zgo- raj, enkrat spodaj in enkrat na vsaki strani. Poleg tega eno od preizkusnih vizur izmerimo v obe smeri. Če je orodje DistoX ustrezno umerjeno, se bodo meritve ujemale v okviru dopustne merske napake, v nasprotnem primeru pa načelno ne. Ključne besede: merjenje jam, DistoX, statistika. 1 Društvo za raziskovanje jam Ljubljana, Luize Pesjakove 11, SI-1000 Ljubljana, Slovenia 2 Department of Systems and Control, Jožef Stefan Institute, Jamova cesta 39, SI-1000 Ljubljana, Slovenia, e-mail: matija.perne@ijs.si * Corresponding author Received/Prejeto: 23.02.2021 DOI: 10.3986/ac.v50i1.9653 ACTA CARSOLOGICA 50/1, 183-194, POSTOJNA 2021 COBISS: 1.01 MATIJA PERNE INTRODUCTION DistoX is a device that measures distance, azimuth, and inclination of a survey shot between two stations at a press of a button and can wirelessly transmit the measurements to another device. It is a custom modification of a Disto™ laser distance meter designed for cave surveying (Heeb 2016; Heeb 2019a). It is regularly used in cave surveying (Bedford 2012; Albert 2017; Kukuljan 2019; White 2019) and has been mentioned often in the literature (Gázquez & Calaforra 2013; Bessone et al. 2016; Ćalić et al. 2016; Mouici et al. 2017; Heggset 2019) even though its typical use does not result in a publication. It has been used in archaeology (Ortiz et al. 2013; Trimmis 2018) and mining (Sovero Vargas 2013). The angular accuracy of the device is reported to be 0.5 degree RMS (Heeb 2015). And, it compares favorably to the use of a compass and clinom- eter in cave surveys (Redovniković et al. 2014). The version DistoX2, studied in this paper, is ob- tained by replacing the main circuit board of a Leica Dis- to X310 with a board that contains a STMicroelectron- ics LIS3LV02DL accelerometer and a PNI Geomagnetic Sensor (STMicroelectronics 2008; PNI Sensor Corpora- tion 2016; Heeb 2019b). These sensors enable the device to measure its own orientation relative to the direction of gravity and the Earth’s magnetic field (Heeb 2009), deter- mining the direction of the laser beam in space. Manufacturing tolerances and external influences cause measurement errors in determining angles with DistoX. The main errors are eliminated by calibrating the instrument (Heeb 2008; Heeb 2009). The calibration procedure consists of measuring multiple unidirectional groups. A unidirectional group is a set of measurements of a fixed but not a priori known direction with various roll angles, turning the device around the beam (Heeb 2009). Based on these measurements, calibration coeffi- cients are calculated (Heeb 2015). The coefficients deter - mine a linear correction function that is applied to the sensor values before evaluation (Heeb 2009). It is recom- mended to have 14 unidirectional groups of 4 measure- ments each that are well spread out (Heeb 2008; Heeb 2009). After following the recommended procedure, the calibration errors contribute less than 10% of the total survey error when calibration and survey shots are taken with the same accuracy (Heeb 2009). Calibration requires no special tools. The user manual remarks that “for best performance, the device should be calibrated in regular intervals” (Heeb 2015). The need for regular, precise, time-consuming calibration is an annoyance and mistakes can be made in the process. Several authors using DistoX in their research reassure the reader that the instrument was properly calibrated (Грачев 2010; Domínguez-Cuesta et al. 2012; Gázquez & Calaforra 2013; Sovero Vargas 2013; Pennos et al. 2016; Heggset 2019). Calibration is one of the reasons for upgrading the DistoX2 to a re- chargeable LiPo battery, which eliminates battery re- placement as a reason for re-calibrating the device, al- though the primary reason to use a LiPo battery is to improve compass precision (Heeb 2014). At a workshop on cave survey organized by Društvo za raziskovanje jam Ljubljana on April 20, 2019, it was noticed that a number of DistoX devices being used in cave surveys reported azimuths that varied by several degrees when the same shot was measured at different roll angles. It became apparent that there was a need for better cali- bration of the devices. Several tools that hold the DistoX steady and only allow rotation around the long axis have been developed to make calibration more convenient (Regala 2016; Ko- zlov 2018). It is not clear whether their use results in cor- rect unidirectional groups and in a valid calibration. One of them, the DistoX2 Calibration Cube (Roberson 2019), was obtained and tested for the study presented in this paper. The version of the tool, current as of September 2019, has two parts. One part is fixed with respect to the ground and consists of a 3D printed cube with holes and three aluminum rod legs. The other part attaches to the DistoX and provides a rod that fits into one of the holes of the cube for every unidirectional group, allowing rota- tion only around a single axis. Three of the components of the moving part are 3D printed, two are aluminum rods, and one is a brass bolt. In this study, two DistoX2 devices were calibrated using both the classical procedure from the manual (Heeb 2008) and the DistoX2 Calibration Cube (Rob- erson 2019). The resulting calibrations were tested. The conclusion based on these tests is that it is beneficial to check the DistoX calibration before every use of the in- strument. The recommended calibration check proce- dure is based on suggestions from Heeb (2008) and Cor- vi (2020). The calibration check procedure takes much less time than calibrating a DistoX does. ACTA CARSOLOGICA 50/1 – 2021 184 DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING THEORETICAL BACKGROUND A calibrated DistoX uses sensor signals to determine the length, azimuth, and inclination of a survey shot, providing all the measurements required for a cave survey. The distance is measured with a laser distance meter. The azimuth and inclination are deduced from the direction of gravity and the direction of the Earth’s magnetic field relative to the laser beam. The direction of gravity and of the magnetic field are calculated from the signals of built-in accelerometers and geomagnetic sensors. These signals are affected by systematic meas - urement errors, such as those resulting from the follow- ing (Heeb 2009): • the offset and gain errors of the sensors, • incorrect mounting angles of the magnetic sensors and accelerometers in relation to one another and the laser beam, • influence of the metallic parts of the instrument, such as the battery, on the magnetic field. These errors can be eliminated with a linear function, and the coefficients of the necessary transformation can be calculated from a series of calibration measurements (Heeb 2009). The correction to gravity is applied with the formula [1] in which is the resulting gravity and is a vector with three components, is a 3 by 3 transformation matrix, is the vector of the sensor values, and is the offset. The transformation is determined by 12 coefficients, that is, 9 elements of and 3 elements of , that have to be calculated from calibration shots. By convention, the x direction of the coordinate system of is pointing along the laser beam, y is to the right and z down (Fig. 1). A rigorous analysis of computing the calibration co- efficients from various types of calibration shots is given in Heeb (2009). Heeb analyzed the three following pos- sibilities: • using shots of known directions, • using random free measurements of unknown di- rections, and • using unidirectional groups of several shots in the same direction at different roll angles. Heeb (2009) determined that the known directions meth- od is impractical because stations in known relative posi- tions are typically not available. And, free measurements are not sufficient for determining the angles between the sensors and the laser beam. The method used in prac- tice is the unidirectional group method. Unidirectional group measurements are sufficient for determining all of the calibration coefficients except for one parameter that is related to the roll angle and is ambiguous. It does not influence the use of the device though and the choice is made to set it (Heeb 2009). The equations for the magnetic field are of the same form as the equations for gravity and are used the same way. According to Heeb (2009), 4 evenly spread out unidirectional groups of 4 shots at different roll angles, combined with free measurements, are enough to deter- mine the calibration coefficients with a good precision. More groups increase the precision; so, 14 unidirectional Fig. 1: DistoX on the calibration tool in the field work location. The coordinate system is defined as in Heeb (2009) (Photo: M. Perne). ACTA CARSOLOGICA 50/1 – 2021 185 MATIJA PERNE groups of four shots with evenly spread out roll angles around the beam are recommended. The coefficients are determined with an optimization method from all the shots, averaging out the random errors. Corvi (2017) reports that 24 or 27 coefficients re- sult from calibration. The 12 for gravity and the 12 for magnetism add up to 24, while the extra three are for the nonlinear correction, if used. It was determined that nonlinearity of the accelerometers may cause a signifi- cant systematic error after linear calibration, so support for a simple second order correction function was added to the firmware (Heeb 2014). WHEN TO RE-CALIBRATE THE DISTOX? When any source of error that is corrected by the cali- bration changes in size, the calibration coefficients cease to be valid and the instrument must be re-calibrated. The parts of the instrument must not move relative to one another in order for the calibration coefficients to be constant. If the device is jostled enough that critical parts move without getting loose, re-calibration will help. The influence of the device’s metallic parts on the magnetic field can change if the parts either physically move or change their magnetization. A re-calibration after a battery change is unavoidable (Heeb 2009); any movement of the batteries may change the coefficients. A serious magnetization of the ferromagnetic material in the instrument by an invisible magnetic field can harm the instrument’s precision (Heeb 2016) and a magnetiza- tion change orders of magnitude smaller would suffice to noticeably change the calibration coefficients. Due to component drift and aging, the offset and gain errors of the sensors change with time and calibra- tion has to be repeated occasionally (Heeb 2009). Travelling for a long distance does not necessitate re-calibration in itself though. The relationship between the external fields and the sensor values is independent from the location. A calibrated device works equally well everywhere in the world (Heeb 2009), even though grav- ity, the strength of the geomagnetic field, and the mag- netic inclination are different in different places. Mag- netic declination, however, is a separate and unrelated is- sue. Devices like DistoX measure the magnetic azimuth. When a different azimuth is needed, conversion is neces- sary. QUALITY MEASURES AND THEIR LIMITATIONS When the calibration coefficients are calculated from the unidirectional groups, it is checked how accurately the roll axes x of the shots within each group point in the same direction. If the spread is big, the calibration coef- ficients are likely not accurate. The calibration software computes several quality measures that quantify the av- erage error. An automated warning is provided if certain ones are above a set threshold (Corvi 2020). The laser beam is assumed to be pointing along the x axis (Heeb 2009). If, due to random errors, the laser is not pointing exactly in the same direction in all of the shots within each unidirectional group, the algorithm will be less precise in matching the x axis and the beam. At the same time, the determined x axis will be found to not have pointed in a constant direction within each unidirectional group, so the quality measure will be big. A calibration tool provides an x axis that is indepen- dent from the laser beam. Correct calibration with a tool depends on the match between the x axis provided by the tool and the laser beam. The mismatch between the two does not contribute to the quality measure – as long as the x axis is pointing in the same direction in all the shots of each group, the quality measure will be small. In the case of a stable x axis and a poor match between it and the laser beam, a poor calibration with a good quality measure and no warning will result. The mismatch angle is likely to be dependent on the device and the calibration tool and may vary between calibrations. Some of the calibration tools, for example the Cali- bration Cube, require attaching parts to the DistoX for the duration of the calibration. The instructions correctly specify that all the parts should be non-magnetic – plas- tic, aluminum, brass (Roberson 2019) – but a slight lapse of attention could result in the use of e.g., a steel washer. The calibration would faithfully include its influence on the magnetic environment in the instrument, so the quality measure would be small. However, the calibration coefficients would only be valid until the magnetic part is removed. ACTA CARSOLOGICA 50/1 – 2021 186 DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING EXPERIMENTAL To test the difference between the calibrations using the method described in the manual and using the Calibra- tion Cube, the following procedure was followed: 1. Obtain a DistoX. 2. Take it to a wooded area with an even magnetic en- vironment. 3. Test the DistoX on a triangular test course, survey- ing in both directions, measuring each shot four times with evenly spread roll angles. 4. Calibrate the DistoX with the Calibration Cube. 5. Repeat the test under number 3. 6. Calibrate the DistoX following the DistoX Calibra- tion Manual (Heeb 2008). 7. Repeat the test under number 3. 8. Repeat some of the points 4–7 depending on the collected data. 9. Go back to 1 if necessary. First, a Calibration Cube had to be acquired. The holes of the 3D model were changed into metric sizes and the model was printed. As soon as a DistoX was obtained, a mistake in the assumptions became apparent. The assumption was that the brass screw of the Calibration Cube that holds the DistoX does so by pinching the device. Therefore, the hole in the 3D printed part was increased in diameter to 8 mm and another 4 mm hole was added next to it, re- flecting the sizes of brass bolts at hand. However, the bolt is supposed to engage an internal 1/4-inch thread in the DistoX. In the first tests, the DistoX was pinched, while for the later ones, a custom brass bolt was made – 1/4- inch brass bolts are indeed not easily available in Slovenia where the work was performed – and the hole in the tool was made smaller with several layers of nail polish (Fig. 2). For this reason, we have not tested the calibration tool exactly as it was designed, but it functioned as intended. The field work was performed with two DistoX de- vices in a forest close to location 33 T 447502 E 5113134 N (WGS 84 46.169761 14.319938). Three test course sta- tions were marked on tree trunks 5 to 8 m apart and at in- clinations under 30° from one another. The Cube calibra- tions were done in the immediate vicinity (Fig. 1) and the Cube was always moved between successive calibrations. The classical calibrations were performed in the immedi- ate surroundings as well, following the DistoX Calibra- tion Manual of Heeb (2008). The unidirectional groups of shots were taken between tree trunks, from the tree trunks to the ground, and from the ground to the trees. As several of the calibration stations were temporary fea- tures or out of reach, they were not documented and not systematically reused from one calibration to another. No part of the test course was used in any calibration. Both tested devices use a built-in LiPo battery. The DistoX work was carried out over four separate days and at different times of day. Several technical issues bothered us on the first day, including too much ambient light to see the laser dot easily, not enough light to see the target easily, and DistoX pinched to the Calibration Cube using a 4 mm bolt. The majority of the field work was performed on the following three days, after resolv- ing the technical issues, with optimal equipment and workforce and mostly optimal weather. All of the work was done with care, none of it was performed when the circumstances felt unsuitable for accurate work. Different calibration methods and their tests were performed in a random fashion with the number of classical and Cube calibrations and the numbers of tests of both balanced on each day of field work. Any effect of the date on the measurement accuracy would thus not correlate with the calibration method. The log with the details is provided with the data set (Perne 2021). Fig. 2: Attaching the DistoX to the cali- bration tool with an almost correct bolt. The bolt is slightly too long and we avoid- ed sawing it by adding an aluminum washer. The hole on the tool, which was originally too big, was shrunk using a few layers of nail polish (Photo: M. Perne). ACTA CARSOLOGICA 50/1 – 2021 187 MATIJA PERNE The azimuth and inclination angles of the test course were checked with a Suunto handheld compass and cli- nometer so that the DistoX results can be compared with values obtained with an independent method. The field experience has shown that the central cube of the tool is very stable when the legs are stuck in the soil (Fig. 1). In the Calibration Cube created for this study, the DistoX-containing moving part is less stable than the central cube and wobbles significantly. Care was taken to minimize the wobble by supporting the DistoX with hands but it was not clear how much of the wobble re- mained. A smartphone with Android and TopoDroid ver- sion 4.1.4G was used for computing the calibration coef- ficients from the calibration shots and for recording the test data. DATA ANALYSIS AND RESULTS Raw measurement data with detailed metadata and the code used for calculating the values presented in the ta- bles and the text is freely available in the online data set of Perne (2021). Data processing is done in R (R Core Team 2020) using the package dplyr (Wickham et al. 2020). DATA EXPLORATION Tab. 1 contains several statistics of DistoX tests. The most important numbers are reproduced in the graph in Fig. 3. Tab. 2 presents the data on the instrument calibrations themselves. The names of the calibration files (Tab. 2) start with the label of the device, which is either “n1” or “n2” . The label of the calibration follows, consisting of the letter “k” for a Calibration Cube calibration or the letter “g” for the classical calibration, followed by the consecutive number of the given type of calibration on the particular device. The names of the test files (Tab. 1) start with the name of the calibration file, which is followed by the letter “t” and the consecutive number of the test of the given calibra- tion. For the files that required a manual correction of a mistake in the raw data, the letter “p” follows, and all the Tab. 1: Results of the tests of the calibrations. All the σ’s and the Error stddev are in angular degrees, all the Δ’s are in meters. Each row represents one test, the more or less strong yellow ones are tests of the calibrations with the tool, the red ones refer to the classical calibrations from one tree to the other. When neighboring lines are of the same shade, they test the same calibration. We measured six test shots (a triangle in both directions), each one four times with the DistoX in different orientations. In the columns <σ α > and <σ φ > are the average standard deviations in the azimuth and the inclination, averaged over the six shots. The columns max( σ α ), max(σ φ ), min(σ α ) and min(σ φ ) contain the biggest and the smallest standard deviations of both angles picked from the data on the six shots. The columns rmsΔ tot , rmsΔ vert and rmsΔ hor are the root-mean-square mismatches of the closure of the triangle: total, vertical, and horizontal, respec- tively. As each side is measured eight times (four times in each direction), we get 8 3 = 512 possible surveys of the triangle and average over them all to obtain these numbers. The remaining columns are the total, vertical, and horizontal closure mismatches of the averaged triangle survey. The table is generated by the script “DistoX.R” (Perne 2021). Test <σ α > <σ φ > max(σ α ) max(σ φ ) min(σ α ) min(σ φ ) rmsΔ tot rmsΔ vert rmsΔ hor Δ tot Δ vert Δ hor Calibra- tion Error stddev n1original 1.717 0.255 3.021 0.386 0.685 0.171 0.289 0.047 0.285 0.029 0.010 0.028 n2original 3.402 0.629 5.690 0.730 1.061 0.556 0.697 0.107 0.688 0.037 -0.006 0.037 n1k1t1p 0.390 0.288 0.532 0.419 0.216 0.173 0.128 0.087 0.094 0.047 -0.027 0.038 0.24 n1k1t2p 0.364 0.321 0.457 0.443 0.200 0.250 0.119 0.071 0.095 0.031 -0.010 0.030 0.24 n1k1t3p 0.369 0.274 0.658 0.311 0.222 0.222 0.109 0.063 0.089 0.021 -0.001 0.021 0.24 n1k2t1p 0.458 0.403 0.915 0.465 0.208 0.356 0.138 0.083 0.110 0.028 -0.010 0.026 0.18 n1k2t2 0.477 0.390 0.835 0.443 0.250 0.346 0.123 0.074 0.098 0.014 0.001 0.014 0.18 n2k1t1 0.853 0.786 0.954 0.826 0.751 0.735 0.200 0.133 0.150 0.022 0.014 0.016 0.17 n1g1t1 0.282 0.187 0.574 0.299 0.058 0.096 0.117 0.084 0.082 0.027 0.013 0.023 0.18 n1g1t2p 0.236 0.127 0.379 0.171 0.096 0.082 0.105 0.065 0.083 0.021 0.019 0.009 0.18 n1g1t3 0.264 0.110 0.392 0.183 0.129 0.058 0.092 0.049 0.077 0.015 -0.001 0.015 0.18 n1g2t1 0.316 0.124 0.457 0.150 0.189 0.082 0.084 0.050 0.067 0.019 0.009 0.016 0.29 n2g1t1 0.163 0.057 0.299 0.096 0.082 0.000 0.041 0.018 0.037 0.020 -0.002 0.020 0.05 ACTA CARSOLOGICA 50/1 – 2021 188 DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING manual corrections are documented in the data set. The file name structure is different for the initial testing of the preexisting calibration of each instrument, and consists of the label of the device followed by the word “original” . STATISTICAL ANALYSIS A rigorous statistical analysis is a reliable and accurate way of quantifying the studied effects and is complemen- tary with data exploration. The measured angle value in a single test shot is written as [2] where is the measured value of the angle (azimuth or inclination), is the true value, and is the measurement noise. Noise is a random variable that is sampled from an unknown distribution. The possible hypotheses for the measurement noise of differently calibrated DistoXs are the following: 1. The noise distributions have the same variance for both calibration methods. 2. The devices calibrated either way are equally accu- rate and any difference in the variance of the noise distribution arises during testing from causes unre- lated to calibration. 3. The calibrations are different and influence the vari - ance of the noise distribution observed by the tests but the difference in the calibration method is not the cause, both calibration methods are equally good. 4. The Cube calibration method is not as good as the classical one, increasing the variance observed in the tests. It will be shown that hypotheses 1 to 3 can be rejected. To measure the noise, the true value has to be known, but it is not. It is thus approximated with a mean value. Five different mean values are considered: 1. Arithmetic mean of all the measurements of a shot over all tests. 2. Weighted arithmetic mean of all the measurements of a shot, where the weight is the inverse of the vari- ance of all the shot angles in a given test. 3. Arithmetic mean of the four measurements of a shot in the particular test. 4. Average of the arithmetic mean and the inverted arithmetic mean of the back shot. 5. Average of the weighted arithmetic mean and the inverted weighted arithmetic mean of the back shot. Tab. 2: TopoDroid’s criticism of the calibrations. According to the criteria of the app, all are reasonably good. It seems that the wobble of the DistoX on the Calibration Cube is small enough that it does not harm the result. Name Average error [°] BH delta Error stddev [°] Max. error [°] Itera- tions n1g1 0.28 0.566 0.18 1.01 39 n1g2 0.28 0.806 0.29 2.45 36 n1k1 0.36 0.688 0.24 1.18 34 n1k2 0.29 0.555 0.18 1.01 34 n2g1 0.1 0.214 0.05 0.23 31 n2k1 0.25 0.438 0.17 0.78 28 Fig. 3: Graphical representation of the σ columns of Tab. 1. ACTA CARSOLOGICA 50/1 – 2021 189 MATIJA PERNE Each approximation ascribes a different meaning to the true value and it is not evident which one is the most meaningful. The calculations are repeated with all of them and the full results are reported for the approxi- mate true value no. 2 because it is the most intuitive one. Hypothesis 1 is tested statistically with the modified robust Brown-Forsythe Levene-type test based on the absolute deviations from the median as implemented in levene.test function of Gastwirth et al. (2020). The noise variance in the 5 tests of classical calibrations is compared to the one in the 6 tests of Cube calibrations, and the p- value for the azimuth is found to be 3.6·10 -6 . According to the same Brown-Forsythe test, the noise standard de- viation in the tests of the Cube calibrations is most likely 1.6 times larger than the noise standard deviation in the tests of the classical calibrations. The ratio of the noise standard deviations is at least 1.35 at 95% confidence level. For inclination, the equivalent p-value is 1.3·10 -11 , the most likely ratio is 2.2, and the ratio is 1.85 or above at 95% confidence level. The result is robust with respect to the choice of the mean value used to approximate the true value; at 95% confidence, the noise standard devia- tion ratio is 1.22 or above in azimuth and 1.84 or above in inclination with every mean value used. Hypothesis 1 that the noise distributions have equal variances is thus firmly rejected and it is shown that the difference in vari- ances is not small. When the calibration tests are ranked in DistoX performance (Tab. 1), all 5 tests of classical calibrations are better than any of the 6 tests of the calibrations with the Cube. The claim holds true for every angle deviation measure apart from the maximum 1 . Only one combina- tion in behaves this way, making the occur- rence unlikely if the DistoX and its calibration had no in- fluence on the test results. Hypothesis 2 is thus rejected. The 3 classical calibrations perform better in test- ing than any of the 3 with the Cube. The probability of this happening if the calibration method had no influ- ence on calibration quality is one in , which is too high to confidently reject hypothesis 3. However, calibration is designed to contribute less than 10% of the survey error (Heeb 2009) so its influence should be below the detection limit of the tests. The observed difference is too large. Hypothesis 3 that the calibration methods are both good but the Cube resulted in worse calibrations by chance is therefore rejected as an explanation for the observed result. It follows that the remaining hypothesis 4 that the Cube leads to worse calibration than the classical method should be accepted. V ALIDATION AGAINST OTHER MEASUREMENT DEVICES The survey of the test course with handheld compass and clinometer and the code comparing it with the DistoX measurements is provided in the data set (Perne 2021). No significant difference between DistoX and Suunto an- gles is observed. The differences between the mean val- ues of each method are within 1 angular degree, which is within the measurement error of Suunto instruments. The systematic error of DistoX compared to Suunto is not detectable. DISCUSSION OF THE RESULTS The statistical tests show that the observed Cube calibra- tions negatively influence the performance of DistoX com- pared to classical calibrations but are not recognized as poor by the calibration quality measures. The finding can be ex- plained with the theoretical prediction of the consequences of providing an x axis independent from the laser beam. Inspecting the test results (Tab. 1; Fig. 3), the cali- bration that performs worst is n2k1, a Calibration Cube calibration. Its standard deviations of the angles are very similar for every test shot and in both azimuth and in- clination, the difference between max(σ α ) and min(σ φ ) is small. This is the expected error pattern if the DistoX follows a conical surface in each shot when the roll angle changes, as if the calibration x axis was not parallel to the beam. It is thus in good agreement with the predicted side effect of the use of calibration tools. As expected, the errors of Calibration Cube calibra- tions do not influence the quality measures (T ab. 2), some Cube calibrations are graded better than some classical calibrations but perform worse in tests. This is because the error cannot be detected in the calibration shots if it results from the x axis being at an angle to the beam. The data on the mismatch of the triangles is pro- vided to illustrate the influence of calibration on cave surveys. It is interesting to see how the errors average out if the DistoX is being rotated. The DistoX owners were not asked about the cali- bration state of their devices, reflecting the fact that the 1 The maximum is the most sensitve to outliers and thus the least likely to give the true picture. ACTA CARSOLOGICA 50/1 – 2021 190 DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING devices would likely be assumed to be calibrated if bor- rowed for cave surveying purposes. The owner of n1 mentioned that he compared it with a compass and was not impressed, while n2 was reportedly being used as if it was calibrated well. According to the test results (Tab. 1), both devices when borrowed seem to have achieved UIS survey grade 3 (Häuselmann 2011) accuracy but n2 seems not to have met BCRA survey grade 3 (British Cave Research Association 2002) standards. The calibration n1g2 was performed very carefully with the goal of achieving similar results as in the excel- lent n2g1, but it did not work out regarding the calibration quality measures. It does not seem to be a result of human error. More likely it is a consequence of a subtle difference between the DistoXs, perhaps just a fussy measure but- ton. The calibration n1g2 nevertheless achieved acceptable quality measures and performed flawlessly in the test. PRACTICAL IMPLICATIONS OF THE RESULTS One should regularly check the calibration of a DistoX used for cave surveying. Inferring that it is calibrated well based on its history is not reliable because one may not have the complete information. Calibration should be checked at least at the beginning of every survey so that a possible problem is detected before causing damage. It may be beneficial to check the calibration again at the end of the survey to better constrain the quality of the device throughout the survey work. One may even vary the roll angle whenever taking a multiple measurement of a survey shot in order to detect any issues as quickly as possible, although it is not clear whether the benefit outweighs the additional cost. One should not use a DistoX calibration tool with- out checking the calibration before use. A tool increases the likelihood of substantial calibration errors that are not detected by the calibration quality measures, so a calibration check is the only way of detecting them. When the same shot is measured multiple times at evenly spread out roll angles, i.e., with the display of the DistoX pointing in different directions, the measurement result variation should be within the desired survey accu- racy. The calibration can be checked by measuring several shots with varying roll angles and measuring a shot in both directions, checking the agreement of the values. The test with the back shot is necessary because changing the roll does not detect offsets in the x direction. 2 A rigorous derivation of the necessary and sufficient conditions for the correctness of the calibration is given by Corvi (2018). This calibration check method has been proposed before. The Calibration Manual (Heeb 2008) recommends per- forming the procedure after calibration as a quick check. TopoDroid User Manual (Corvi 2020) mentions that cal- ibration-check shots should be taken at different roll an- gles and describes the functionality of TopoDroid related to check shots. However, the need for regular calibration checks seems to not have been emphasized enough. The recommendation of checking the calibration with check survey shots is not contradicted by the claim that calibration only contributes up to 10% to the total survey error (Heeb 2009). This type of calibration check cannot see the difference between an excellent and a me- diocre calibration, but it can detect one that is so poor that it would importantly influence the survey accuracy. If a survey is performed with a poorly calibrated DistoX, one may want to calibrate the device and cor- rect the calibration errors after the survey. The survey azimuth and inclination values are not enough to make it possible because the conversion of the 6 raw sensor values into the 2 angles is irreversible. Additional infor- mation on the sensor readings or DistoX orientation is necessary for a successful correction. CONCLUSIONS It is crucial for a DistoX in cave survey use to have a fresh, valid calibration. The quality of the calibration, which depends on the method used, is somewhat important as well. Calibration tools offer opportunities for making the rotation axis not parallel to the laser beam or for having a magnetic part attached to the instrument during calibra- tion, increasing the calibration errors. These errors are not detected in the computation of the calibration co- efficients. Such mistakes do not occur if the calibration manual is followed literally and no tool is used. Never- theless, several mechanisms that can render calibration coefficients invalid unbeknownst to the user do exist re- gardless of the calibration method. Regular checking of the calibration, at least at the beginning of every survey, is thus highly recommended. If calibration is checked regularly, use of calibration tools may not be problematic as any sizeable calibration error would be detected and corrected before causing damage. 2 The author thanks Beat Heeb for pointing it out while taking full responsibility for the claim. ACTA CARSOLOGICA 50/1 – 2021 191 MATIJA PERNE NOTATIONS LIST <·> Average Matrix of calibration coefficients [-] Offset of accelerometers [m/s 2 ] Resulting gravity [m/s 2 ] Sensor signal of gravity [m/s 2 ] max(·) Maximum min(·) Minimum rmsΔ hor Horizontal root-mean-square mismatch of the triangle closure [m] rmsΔ tot , Total root-mean-square mismatch of the triangle closure [m] rmsΔ vert Vertical root-mean-square mismatch of the triangle closure [m] x Coordinate along the laser beam y Coordinate to the right with respect to the DistoX z Coordinate down with respect to the DistoX Δ hor Horizontal root-mean-square mismatch of the closure of the average triangle [m] Δ tot , Total root-mean-square mismatch of the closure of the average triangle [m] Δ vert Vertical root-mean-square mismatch of the closure of the average triangle [m] α true value of an angle (azimuth or inclination)[°] measured value of an angle [°] ν measurement noise [°] σ α Standard deviation in azimuth for a shot [°] σ φ Standard deviation in inclination for a shot [°] ACKNOWLEDGMENT Marjan Baričič, Matic Di Batista, Gregor Pintar, Pe- ter Prevec, Janez Strojan, Stephanie Sullivan and Rafko Urankar-Cile assisted in completing the work. Ljubljana Cave Exploration Society and its journal Glas podzem- lja have allowed the article Perne (2020) to be translated, modified and extended. I sincerely thank Paul Burger, Marco Corvi, Beat Heeb, and an anonymous reviewer for substantial comments on earlier drafts of the manuscript. Editing and revisioning of the article was financially sup- ported by the Slovenian Research Agency through re- search core funding No. P2-0001. REFERENCES Albert, G., 2017: Aspects of Cave Data Use in a GIS.- In: Karabulut, S. & M. Cengiz Çinku (eds.) Cave Inves- tigation. IntechOpen, pp. 25–47, Rijeka. https://doi. org/10.5772/intechopen.68833 British Cave Research Association, 2002: BCRA Survey- ing Pages.- [Online] Available from: http://bcra.org. uk/surveying/ [Accessed 17th June 2020]. Bedford, M., 2012: Cave Technology Symposium 2012.- CREG Journal, 79, 9-13. Bessone, L., Cossu, A., De Waele, J., Marcia, P., Sanna, L., Sauro, F. & S. Taiti, 2016: ESA CAVES: un pro- gramma di monitoraggio sotterraneo sviluppato per l’addestramento di astronauti.- In: Gili, R. et al. (eds.) Atti del Convegno Nazionale “La ricerca carso- logica in Italia”, 22 nd -23 rd June 2013, Frabosa Sopra- na. Laboratorio carsologico sotterraneo di Bossea, 215-219, Bossea. Corvi, M., 2017: DistoX calibration with TopoDroid.- [Online] Available from: http://marcocorvi.alterv- ista.org/caving/TDpdf/213-Calibration.pdf [Ac- cessed 15th June 2020]. Corvi, M., 2018: Survey accuracy and DistoX.- [Online] Available from: http://marcocorvi.altervista.org/ caving/ [Accessed 8th December 2019]. Corvi, M., 2020: TopoDroid User Manual, Version 5.0.4j - 2 Jul, 2020.- [Online] Available from: https:// github.com/marcocorvi/topodroid/raw//master/ manual.pdf [Accessed 23rd October 2020]. Ćalić, J., Mandić M., Gajović V. & P. Stošić, 2016: Speleološka istraživanja na Belavi, Šljivovičkom vrhu i Malom Stolu.- Pirotski zbornik, 41, 91-114. https://doi.org/10.5937/pirotzbor1641091C Domínguez-Cuesta, M., Ballesteros, D., Jiménez-Sán- chez, M. & P. González-Pumariega, 2012: Posicio- namiento 3D de cavidades subterráneas mediante topografía espeleológica. El ejemplo de la Cueva de El Pindal (Norte de España).- In: Berrezueta Alvara- do, E. & M. J. Domínguez-Cuesta (eds.) Técnicas ACTA CARSOLOGICA 50/1 – 2021 192 DISTOX CALIBRATION TOOLS AND THE NEED FOR CALIBRATION CHECKING Aplicadas a la Caracterización y aprovechamiento de Recursos Geológico-Mineros: Volumen III: Inter- acción con La Sociedad. Red Minería, XXI, CYTED and Instituto Geológico y Minero de España, pp. 74- 87, Oviedo. Gastwirth, J.L., Gel, Y.R., Hui, W.L.W., Lyubchich, V., Miao, W. & K. Noguchi, 2020: lawstat: Tools for Biostatistics, Public Policy, and Law: R package ver- sion 3.4.- [Online] Available from: https://CRAN.R- project.org/package=lawstat [Accessed 21st Octo- ber 2021]. Gázquez, F. & J.M. Calaforra, 2013: Origin of double‐ tower raft cones in hypogenic caves.- Earth Sur- face Processes and Landforms, 38, 14, 1655-1661. https://doi.org/10.1002/esp.3399 Гр ач е в, А.П., 2010: Топографо-геодезические работы в горизонтальных пещерах: Практические рекомендации для спелеотопографа.- [Online] Available from: https://hinko.org/hinko/Dow- loads/11/1/XI-1-21.pdf [Accessed 15th June 2020]. Häuselmann, Ph., 2011: UIS Mapping Grades.- Interna- tional Journal of Speleology, 40, 2, IV-VI. Heeb, B., 2008: DistoX Calibration Manual, 11/27/2008.- [Online] Available from: https://paperless.bheeb. ch/download/CalibrationManual.pdf [Accessed 15th June 2020]. Heeb, B., 2009: A General Calibration Algorithm for 3-Axis Compass/Clinometer Devices.- CREG Jour- nal, 73, 12-18, http://paperless.bheeb.ch/download/ Calibration.pdf [Accessed 15th June 2020]. Heeb, B., 2014: The Next Generation of the DistoX Cave Surveying Instrument. CREG Journal, 88, 5-8, https://paperless.bheeb.ch/download/DistoX2.pdf [Accessed 15th June 2020]. Heeb, B., 2015: DistoX2 User Manual, Leica Disto X310 based DistoX, Firmware Version 2.4, 2015/02/22.- [Online] Available from: https://paperless.bheeb. ch/download/DistoX2_UserManual.pdf [Accessed 15th June 2020]. Heeb, B., 2016: DistoX2 Assembly Manual, Leica Disto X310 based DistoX, Version 1.1 2016/08/25.- [On- line] Available from: http://paperless.bheeb.ch/ download/DistoX2_AssemblyManual.pdf [Ac- cessed 15th June 2020]. Heeb, B., 2019a: Paperless Cave Surveying.- [Online] Available from: http://paperless.bheeb.ch/ [Ac- cessed 15th June 2020]. Heeb, B., 2019b: DistoX2 Hardware Information Pack- age.- [Online] Available from: http://paperless. bheeb.ch/download/DistoX2Hardware.zip [Ac- cessed 15th June 2020]. Heggset, O.E., 2019: Kartlegging av Oppsalgrotta i Gild- eskål, Nordland.- Master's thesis. Department of Ge- ography, University of Bergen, pp. 119. Kozlov, M., 2018: OpenSCAD models for distox2 calibra- tion device by Mykyta Kozlov. Version 0.0.5.- [On- line] Available from: https://bitbucket.org/ngry/dis- tox2_cube/src/master/ [Accessed 15th June 2020]. Kukuljan, L., 2019: Seminar o digitalnom topografskom snimanju speleoloških objekata.- Subterranea Cro- atica, 17, 2, 63. Mouici, R., Baali, F., Hadji, R., Boubaya, D., Audra, P., Fehdi, C-É., Cailhol D., Jaillet S. & B. Arfib, 2017: Geophysical, geotechnical, and speleologic assess- ment for karst-sinkhole collapse genesis in Cheria plateau (NE Algeria).- Mining Science, 24, 59-71. https:doi.org/10.5277/msc172403 Ortiz, A.A., Cano, R.B., Cobo, F.R.R. & F.B. Jiménez, 2013: Espeleología y arqueología. Cueva del Higuer- alguardia, ejemplo de un trabajo interdisciplinar. 1963−2013.- Gota a Gota, 3, 42-51. Pennos, C., Lauritzen, S.E., Pechlivanidou, S. & Y . Sotiri- adis, 2016: Geomorphic constrains on the evolution of the Aggitis river basin Northern Greece (a pre- liminary report).- Bulletin of the Geological Society of Greece, 50, 1, 365-373. https://doi.org/10.12681/ bgsg.11737 Perne, M., 2020: Kako in zakaj umerjati DistoX.- Glas podzemlja 2020, 61-65, https://www.dzrjl.si/ wp-content/uploads/2020/03/GP-2020_splet- Page-64-68.pdf [Accessed 15th June 2020]. Perne, M., 2021: DistoX2 Calibration Cube Test (Ver- sion 1.2.0).- [Data set] Zenodo, doi:10.5281/zeno- do.4121748. PNI Sensor Corporation, 2016: RM3100 & RM2100 Sen- sor Suite User Manual.- [Online] Available from: https://www.pnicorp.com/wp-content/uploads/ RM3100-Sensor-Suite-User-Manual-R07-1-2.pdf [Accessed 15th June 2020]. R Core Team, 2020: R: A language and environment for statistical computing.- R Foundation for Statisti- cal Computing, Vienna, Austria. [Online] Available from: https://www.R-project.org/ [Accessed 23rd February 2021]. Redovniković, L., Ivković, M., Cetl, V. & I. Sambunjak, 2014: Testing DistoX device for measuring in the unfavourable conditions.- In: INGEO 2014 – 6 th International Conference on Engineering Surveying, 3 rd -4 th April 2014, Prague. International Federation of Surveyors, FIG, 269-274, Copenhagen. Regala, F.T., 2016: AESDA Calib - a calibrator for Dis- toX/X2.- [Online] Available from: https://www. aesda.org/wp-content/uploads/2016/09/Calibrator- EuroSpeleo2016-adapted.pdf [Accessed 15th June 2020]. ACTA CARSOLOGICA 50/1 – 2021 193 MATIJA PERNE Roberson, P., 2019: DistoX2 Calibration Cube.- [On- line] Available from: https://www.thingiverse. com/thing:3776523 [Accessed 11th September 2019]. Sovero Vargas, L.E., 2013: Aplicación del distox para le- vantamiento topográfico terráneo de labores angos- tas en la UP Carahuacra–Volcan Cia. Minera SAA Junín.- Bachelor’s thesis. Facultad de ingeniería de minas, Universidad Nacional del Centro del Perú, Huancayo, pp. 184. STMicroelectronics, 2008: LIS3LV02DL: MEMS in- ertial sensor 3-axis - ±2g/±6g digital output low voltage linear accelerometer.- [Online] Available from: https://www.st.com/resource/en/datasheet/ lis3lv02dl.pdf [Accessed 15th June 2020]. Trimmis, K.P ., 2018: Paperless mapping and cave archae- ology: A review on the application of DistoX survey method in archaeological cave sites.- Journal of Ar- chaeological Science: Reports, 18, 399-407. https:// doi.org/10.1016/j.jasrep.2018.01.022 White, W.B., 2019: Chapter 123 - Surveying caves.- In: White, W.B. et al. (eds.) Encyclopedia of Caves. Third Edition, Academic Press, pp. 1063-1070, San Diego. https://doi.org/10.1016/B978-0-12-814124- 3.00123-0 Wickham, H., François, R., Henry, L. & K. Müller, 2020: dplyr: A Grammar of Data Manipulation: R pack- age version 1.0.2.- [Online] Available from: https:// CRAN.R-project.org/package=dplyr [Accessed 9th October 2020]. ACTA CARSOLOGICA 50/1 – 2021 194