Kandidatka: PATRICIA COTIČ SINTEZA VEČSENZORSKIH NEPORUŠNIH PREISKAV GRADBENIH KONSTRUKCIJSKIH ELEMENTOV Z UPORABO METOD GRUČENJA Doktorska disertacija štev.: 9/GO THE SYNTHESIS OF MULTISENSOR NON-DESTRUCTIVE TESTING OF CIVIL ENGINEERING STRUCTURAL ELEMENTS WITH THE USE OF CLUSTERING METHODS Doctoral thesis No.: 9/GO Soglasje k temi doktorske disertacije je dala Komisija za doktorski študij Univerze v Ljubljani na 28. redni seji, 16. maja 2012. Za mentorja je bil imenovan izr. prof. dr. Zvonko Jagličić, za somentorja izr. prof. dr. Vlatko Bosiljkov. Odobreno je bilo pisanje disertacije v angleškem jeziku. Ljubljana, 6. maj 2014 Blank page Komisijo za oceno ustreznosti teme doktorske disertacije v sestavi: • izr. prof. dr. Zvonko Jagličić, • izr. prof. dr. Vlatko Bosiljkov, • prof. dr. Goran Turk, • doc. dr. Mitja Lakner, • prof. dr. Janez Dolinšek, UL FMF je imenoval Senat Fakultete za gradbeništvo in geodezijo na 29. redni seji, 28. marca 2012. Poročevalce za oceno doktorske disertacije v sestavi: • prof. dr. Janez Dolinšek, UL FMF, • dr. rer. nat. Ernst Niederleithinger, BAM Federal Institute for Materials Research and Testing, Berlin, • doc. dr. Mitja Lakner, • prof. dr. Goran Turk je imenoval Senat Fakultete za gradbeništvo in geodezijo na 8. redni seji, 26. februarja 2014. Komisijo za zagovor doktorske disertacije v sestavi: • prof. dr. Matjaž Mikoš, dekan UL FGG, predsednik, • izr. prof. dr. Zvonko Jagličić, • izr. prof. dr. Vlatko Bosiljkov, • prof. dr. Janez Dolinšek, UL FMF, • dr. rer. nat. Ernst Niederleithinger, BAM Federal Institute for Materials Research and Testing, Berlin, • doc. dr. Mitja Lakner, • prof. dr. Goran Turk je imenoval Senat Fakultete za gradbeništvo in geodezijo na 4. dopisni seji, ki je potekala od 24. do 28. aprila 2014. Blank page IZJAVA O AVTORSTVU Podpisana Patricia COTIČ izjavljam, da sem avtorica doktorske disertacije z naslovom Sin- teza večsenzorskih neporušnih preiskav gradbenih konstrukcijskih elementov z uporabo metod gručenja. Izjavljam, da je elektronska različica v vsem enaka tiskani različici. Izjavljam, da dovoljujem objavo elektronske različice v digitalnih repozitorijih. Ljubljana, 6. maj 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (podpis) Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. I Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. ERRATA Page with error Line with error Incorrect Correct II Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. III Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. BIBLIOGRAPHIC-DOCUMENTALISTIC INFORMATION AND ABSTRACT UDC: 551.509.313:624.012.82:624.012.4:(043.3) Author: Patricia Cotič, B.Sc. Supervisor: Assoc. prof. Zvonko Jagličić, Ph.D. Co-advisor: Assoc. prof. Vlatko Bosiljkov, Ph.D. Title: The synthesis of multisensor non-destructive testing of civil engi- neering structural elements with the use of clustering methods Document type: doctoral dissertation Notes: 100 p., 6 tab., 37 fig., 72 eq., 4 ann., 161 ref. Keywords: non-destructive testing, civil engineering, ground penetrating radar, infrared thermography, ultrasonic, complex resistivity, concrete, masonry, data fusion, image fusion, clustering methods Abstract: In the thesis, clustering-based image fusion of multi-sensor non-destructive (NDT) data is stud- ied. Several hard and fuzzy clustering algorithms are analysed and implemented both at the pixel and feature level fusion. Image fusion of ground penetrating radar (GPR) and infrared thermography (IRT) data is applied on concrete specimens with inbuilt artificial defects, as well as on masonry specimens where defects such as plaster delamination and structural cracking were generated through a shear test. We show that on concrete, the GK clustering algorithm exhibits the best performance since it is not limited to the detection of spherical clusters as are the FCM and PFCM algorithms. We also prove that clustering-based fusion outperforms supervised fusion, especially in situations with very limited knowledge about the material prop- erties and depths of the defects. Complementary use of GPR and IRT on multi-leaf masonry walls enabled the detection of the walls’ morphology, texture, as well as plaster delamination and structural cracking. For improved detection of the latter two, we propose using data fusion at the pixel level for data segmentation. In addition to defect detection, the effect of moisture is analysed on masonry using GPR, ultrasonic and complex resistivity tomographies. Within the thesis, clustering is also successfully applied in a case study where a multi-sensor NDT data set was automatically collected by a self-navigating mobile robot system. Besides, the classification of spectroscopic spatial data from concrete is taken under consideration. In both applications, clustering is used for unsupervised segmentation of data. IV Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. V Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. BIBLIOGRAFSKO-DOKUMENTACIJSKA STRAN IN IZVLEČEK UDK: 551.509.313:624.012.82:624.012.4:(043.3) Avtor: Patricia Cotič, univ. dipl. inž. grad. Mentor: izr. prof. dr. Zvonko Jagličić Somentor: izr. prof. dr. Vlatko Bosiljkov Naslov: Sinteza večsenzorskih neporušnih preiskav gradbenih konstruk- cijskih elementov z uporabo metod gručenja Tip dokumenta: doktorska disertacija Obseg in oprema: 100 str., 6 pregl., 37 sl., 72 en., 4 pril., 161 vir. Ključne besede: neporušne preiskave, gradbeništvo, georadar, infrardeča termo- grafija, ultrazvočna metoda, kompleksno-uporovna metoda, be- ton, zidovje, združevanje podatkov, združevanje slik, metode gručenja Izvleček: V disertaciji predstavimo uporabo postopka za združevanje slik večsenzorskih neporušnih preiskav, ki temelji na metodah gručenja. Za združevanje na nivoju posamezne slikovne točke in z uporabo značilnic analiziramo algoritme trdega in mehkega gručenja. Sintezo georadarskih in termograf- skih podatkov opravimo na rezultatih preiskav betonskih preizkušancev z vgrajenimi anomalijami ter na rezultatih preiskav zidovja s prisotnimi razpokami in odstopanjem ometa zaradi delovanja strižne obremenitve. Na betonskih preizkušancih najboljše deluje algoritem gručenja GK, ker prepoznava razrede gručenja poljubne oblike in ne le sferične kot algoritma FCM in PFCM. V primerih s še posebno omejenim vedenjem o materialnih lastnostih in globini anomalij združe- vanje z uporabo metod gručenja deluje bolje kot metode nadzorovanega združevanja podatkov. Na večslojnih kamnitih zidovih lahko z uporabo georadarja in infrardeče termografije zaznamo morfologijo in teksturo zidov ter odstopanje ometa in nastanek razpok. Za izboljšano zaznavanje odstopanja ometa in razpok predlagamo združevanje podatkov na nivoju posamezne slikovne točke za segmentacijo slik. Z georadarsko, ultrazvočno in geoelekrično tomografijo opravimo na zidovju tudi raziskavo vpliva stopnje vlažnosti zidovja na občutljivost neporušnih metod. Metode gručenja uporabimo tudi za združevanje neporušnih podatkov navigacijskega večsen- zorskega robotnega sistema. Poleg tega izvedemo tudi klasifikacijo spektroskopskih podatkov betonskih preizkušancev. V obeh primerih metode gručenja uporabimo za segmentacijo po- datkov. VI Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. VII Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. ACKNOWLEDGEMENT I would like to express my deepest appreciation to my supervisor, Dr. Zvonko Jagličić, and my co-advisor, Dr. Vlatko Bosiljkov, for guidance during my research. Without you, this dissertation would not have materialized. I would particularly like to express my warmest gratitude to Zvonko, who allowed me to spend hours on my Physics study. Without your optimism, encouragement and support both studies would hardly have been completed. I am deeply grateful to my supervisor at BAM, Dr. Ernst Niederleithinger, who guided me into the world of NDT and cluster analysis and gave me insightful comments and suggestions on my research. Only with your leading charm and the splendid working atmosphere from all the colleagues at BAM, Div. 8.2, the work done was made possible. I would particularly like to thank Dr. Christiane Trela, Dr. Ute Effner and Dr. Sabine Kruschwitz for the extensive support with the work done on the masonry specimen “Asterix”. Special thanks also to Gerd Wilsch for the LIBS data, as well as to Dr. Markus Stoppel for the BetoScan data. The valuable ideas on data fusion from Dr. Parisa Shokouhi are acknowledged. Marco Lange, Sean Smith and Jens Wöstmann were the guys willing to help me at any time. You taught me many tricks in the lab and contributed to almost all the work I did at BAM. You and especially Christian Köpp and Sebastian Goldschmidt, were my best German teachers. I would also like to express my special thanks to Julia Wolf, Maria Grohmann and Sabine Müller for the friendship and the time we spent in Berlin. I would also like to thank my colleagues at UL FGG. Particularly, Dr. Violeta Bokan-Bosiljkov for her valuable instructions and generous support, as well as Dr. Dejan Kolarič for his contribution on the numerical simulation on heat propagation. Special thanks also to Franci Čepon, Damjan Špeglič and Primož Murn for extensive measurement support. A big thank goes to the colleagues at IMFM, who contributed to my work in a number of ways. Part of the experimental work could not have been done without the cooperation with people from other institutes. I thank Dr. Aleksander Zidanšek (Jožef Stefan International Postgraduate School) and his group for the use of their radar system, as well as Vid Peterman (Modri Planet, d.o.o.) for performing DIC. I sincerely thank Dr. David Antolinc, Petra Štukovnik and Meta Kržan from UL FGG, with whom discussions illuminated my work significantly. You’ve become my dear friends that also contribute to my life experiences. Finally, I would like to express my warmest gratitude to my parents, who were greatly tolerant and supportive during these three and half years. Hvala vama! I owe my deepest thank to my dear sister Karolina, who gave me insightful comments and suggestions on my English writing. Jože, without you, the thesis in this form would not become true. Your love and encouragement has inspired my work enormously. The thesis is dedicated to You. VIII Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. IX Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. “I think nature’s imagination is so much greater than man’s, she’s never going to let us relax” – Richard Feynman X Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XI Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. TABLE OF CONTENTS Bibliographic-documentalistic information and abstract . . . . . . . . . . . . . . . . . . . III Bibliografsko-dokumentacijska stran in izvleček . . . . . . . . . . . . . . . . . . . . . . . . V Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VII 1 Introduction 1 1.1 Scientific background and motivation . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Aims and scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Basic principles of selected NDT methods 7 2.1 Overview of NDT methods used in civil engineering . . . . . . . . . . . . . . . . . 7 2.2 Ground penetrating radar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.1 Electromagnetic properties of building materials and the energy storage- loss mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.2 Wave nature of electromagnetic fields . . . . . . . . . . . . . . . . . . . . 10 2.2.3 Reflection and refraction of electromagnetic waves . . . . . . . . . . . . . 12 2.2.4 Horizontal and vertical resolution . . . . . . . . . . . . . . . . . . . . . . . 13 2.3 Complex resistivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 Electrical polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.2 Apparent and true resistivities . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.3 Electrode configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Infrared thermography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4.1 Interaction of incident radiation with matter . . . . . . . . . . . . . . . . 19 2.4.2 Thermal emission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.4.3 Conduction heat transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.4.4 Quantitative data analysis in pulsed thermography . . . . . . . . . . . . . 22 2.4.4.1 Thermal contrast . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4.4.2 Pulsed phase thermography . . . . . . . . . . . . . . . . . . . . . 24 2.5 Ultrasonic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.1 Acoustic wave theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.2 Reflection and refraction of acoustic waves . . . . . . . . . . . . . . . . . . 29 2.5.3 Acoustic attenuation and dispersion . . . . . . . . . . . . . . . . . . . . . 29 2.5.4 Piezoelectric transducers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3 NDT Data Fusion with Clustering 31 XII Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 3.1 Introduction to data fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2 A data fusion framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.3 Cluster analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.1 Distance and similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.2 Objective function-based clustering – a general note . . . . . . . . . . . . 34 3.3.3 Objective function-based clustering algorithms . . . . . . . . . . . . . . . 36 3.3.3.1 Hard clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.3.3.2 Fuzzy clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3.4 Validity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.3.5 Objective function-based clustering algorithm scheme . . . . . . . . . . . 42 4 Non-destructive testing of laboratory concrete specimens with inbuilt de- fects and inclusions 45 4.1 Experimental . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.1.1 Specimen description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.1.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.2.1 Characterization of defects and inclusions using infrared thermography . . 48 4.2.1.1 Thermal contrast . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.2.1.2 Pulsed phase thermography . . . . . . . . . . . . . . . . . . . . . 52 4.2.1.3 Numerical simulation . . . . . . . . . . . . . . . . . . . . . . . . 55 4.2.2 Sensitivity evaluation of ground penetrating radar and infrared thermography 58 5 Conclusion and Outlook 61 Summary 65 6 Razširjeni povzetek v slovenskem jeziku (Extended abstract in Slovene) 67 6.1 Uvod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.2 Osnove izbranih neporušnih metod . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.3 Združevanje podatkov z gručenjem . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.4 Neporušne preiskave laboratorijskih betonskih preizkušancev z vgrajenimi anoma- lijami . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6.4.1 Eksperimentalne metode in materiali . . . . . . . . . . . . . . . . . . . . . 74 6.4.2 Rezultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6.5 Neporušne preiskave laboratorijskih ometanih večslojnih kamnitih zidov . . . . . 76 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XIII Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 6.5.1 Eksperimentalne metode in materiali . . . . . . . . . . . . . . . . . . . . . 76 6.5.2 Rezultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.6 Neporušne preiskave laboratorijskega opečnatega zidu pod vplivom različne vlažnosti 78 6.6.1 Eksperimentalne metode in materiali . . . . . . . . . . . . . . . . . . . . . 78 6.6.2 Rezultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.7 Neporušne preiskave z večsenzorskim robotnim sistemom BetoScan . . . . . . . . 79 6.7.1 Eksperimentalne metode in materiali . . . . . . . . . . . . . . . . . . . . . 80 6.7.2 Rezultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.8 Uporaba metod gručenja za avtomatsko klasifikacijo spektroskopskih podatkov na betonskih preizkušancih . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.8.1 Eksperimentalne metode in materiali . . . . . . . . . . . . . . . . . . . . . 81 6.8.2 Rezultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.9 Zaključki in obeti . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 References 85 List of Publications 95 XIV Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XV Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. LIST OF TABLES Table 2.1: The typical range of relative dielectric permittivity εr, electric conductivity σ, wave velocity v and attenuation coefficient α of most common building materials measured at 100 MHz (Daniels, 2004: 74, 90). The values for the materials marked with an asterisk (∗) are taken from Wendrich (2009: 21). . 12 Table 2.2: Thermal properties (mass density ρm, thermal conductivity k, specific heat cp, thermal diffusivity α) of most common building materials (Brink, 2005: 119). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Table 2.3: The typical range of longitudinal sound wave velocity vl of some building materials (Wendrich, 2009: 11). . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Table 4.1: Material and concrete cover of inbuilt defects in specimens S0–S3. . . . . . . 46 Table 4.2: The frequency of the maximal phase contrast fch for defects in specimen S0. 53 Table 4.3: The simulation results of Cmax, tCmax and fch for defects at depths of 1.5, 3.5, 6 and 7.5 cm (see Fig. 4.17). . . . . . . . . . . . . . . . . . . . . . . . . . 57 XVI Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XVII Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. LIST OF FIGURES Figure 1.1: Failure mechanisms on concrete structures: (a) a schematic presentation (Banville, 2008: 2), (b) spalling of concrete due to a chloride attack of em- bedded reinforcement (Banville, 2008: 2), (c) rebar corrosion with visible cracking of concrete. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Figure 2.1: Schematic illustration of the GPR signal reflection from boundaries with different dielectric properties, and the different visualization modes – an A- scan, B-scan and C-scan (Kohl, 2005: 819). . . . . . . . . . . . . . . . . . . . 9 Figure 2.2: Common electrode arrangements in resistivity surveys and their configuration factors K. In the figure, a refers to the dipole separation length, n is the dipole separation factor, C1 and C2 are the current injection electrodes, and P1 and P2 measure the potential. . . . . . . . . . . . . . . . . . . . . . . . . 16 Figure 2.3: The arrangement of electrodes for a 2D resistivity survey of a brickwork specimen and the sequence of measurements used to build up a pseudo-section with the Wenner array (modified after Loke, 2011: 26). . . . . . . . . . . . . 17 Figure 2.4: The principle of optical IRT techniques in reflection mode – pulsed ther- mography (PT), square pulse thermography (SPT), lock-in thermography (LT) and step heating thermography (SHT). Particularly, the different used waveforms for heat generation are presented. . . . . . . . . . . . . . . . . . . 18 Figure 2.5: The spectral distribution of EM radiation from a blackbody at different tem- peratures as described by Planck’s law. The arrows mark the wavelength where the blackbody at a particular temperature radiates the most. From the inset (y-axis is logarithmic), it can be seen that a blackbody at room temperature (300 K) radiates much less than bodies of some 1000 K even in the IR region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Figure 2.6: The definition of the thermal contrast C(t) as the difference in surface tem- peratures above the defect Tdef and sound area Tsound. Here, Cmax refers to the maximal thermal contrast and tCmax is its corresponding time. . . . . . 24 Figure 2.7: The Fourier transform of a pulse with amplitude Ap and duration tp. . . . . 24 Figure 2.8: (a) The principle of pulsed phase thermography (PPT) and (b) presentation of the phase contrast curve with the frequency of the maximal phase contrast fch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Figure 3.1: Data fusion schemes at pixel-level fusion, feature-level fusion and symbol- level fusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Figure 3.2: Schematic diagram of the architecture of a generalized system for NDT data fusion (modified after Gros et al., 1995: 2). . . . . . . . . . . . . . . . . . . . 33 Figure 3.3: The distance norms used in fuzzy clustering algorithms (Babuška, 2009: 70). 35 Figure 3.4: Representation of partitioning a data set of two variables into three clusters. The assignment of observations (data points) to a specific cluster (here, red, green or blue) is associated by the colour. For a specific observation xj, the distances to the three cluster centres v1, v2, v3 are marked as well. . . . . . 36 Figure 3.5: The principle of hard and fuzzy clustering. . . . . . . . . . . . . . . . . . . . 37 Figure 3.6: Flowchart of a generalized clustering algorithm. . . . . . . . . . . . . . . . . 43 XVIII Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.1: Sketches of laboratory concrete test specimens with inbuilt defects. . . . . . 46 Figure 4.2: The IR camera FLIR A320 used for the thermographic measurements. . . . 47 Figure 4.3: The radar system used for the GPR measurements. . . . . . . . . . . . . . . 47 Figure 4.4: Thermograms of specimen S0 at different cooling down times. Each image has a different colour scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Figure 4.5: The time dependent thermal contrast C(t) for defects in specimen S0 after 30 min of heating. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Figure 4.6: The time dependent thermal contrast C(t) for defects in specimens S1 and S2 after 30 min of heating: (a) for a water and an air defect at a depth of 3 cm, and (b) for a polystyrene defect at a depth of 4.6 cm with present reinforcement of two densities (5 and 10 cm). . . . . . . . . . . . . . . . . . . 49 Figure 4.7: (a) A contrast image and (b) a timegram of specimen S0 after 30 min of heating. 50 Figure 4.8: (a) A contrast image and (b) a timegram of specimen S1 after 30 min of heating. 50 Figure 4.9: (a) A contrast image and (b) a timegram of specimen S2 after 30 min of heating. 50 Figure 4.10: (a) A contrast image and (b) a timegram of specimen S3 after 30 min of heating. 50 Figure 4.11: Dependence of (a) the maximal thermal contrast Cmax and (b) its corre- sponding time tCmax on the depth of defects for different heating times (15, 30 and 45 min). The dotted lines are guides to the eye. . . . . . . . . . . . . 51 Figure 4.12: The fitting result for Cmax and tCmax using the model proposed by Balageas et al. (2.36). Experimental results for defects in specimen S0 obtained at a heating time of 30 min were used. . . . . . . . . . . . . . . . . . . . . . . . . 52 Figure 4.13: The frequency dependent phase contrast ∆φ for (a) the defect at a depth of 1.5 cm and (b) the defect at a depth of 3 cm in specimen S0 after 45 min of heating and 150 min of cooling. . . . . . . . . . . . . . . . . . . . . . . . . . 53 Figure 4.14: The fitting result for fch using the model proposed by Arndt et al. (2.44). Experimental results for defects in specimen S0, obtained after 45 min of heating and 150 min of cooling, were used. . . . . . . . . . . . . . . . . . . . 54 Figure 4.15: IR images obtained for specimen S3: (a) the thermogram obtained directly after heating, (b) the contrast image and (c) the phase contrast image at a frequency of 3.7 · 10−3 Hz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Figure 4.16: The phase contrast images at (a) a frequency of 6.2 · 10−4 Hz for specimen S1 and (b) at a frequency of 3.0 · 10−4 Hz for specimen S2. In Fig. 4.16a, the upper defect refers to air (b), the lower to water (d). In Fig. 4.16b, the right part of the image refers to the part of the specimen with dense reinforcement. The red squares mark the defects position, labelled according to Table 4.1. . 55 Figure 4.17: A 2D numerical model simulating two cross-sections of a concrete specimen with inbuilt polystyrene defects (white colour). The points on the surface where temperature was monitored are marked with red dots. . . . . . . . . . 56 Figure 4.18: (a) The time dependent thermal contrast C(t) and (b) the frequency depen- dent phase contrast ∆φ for defects at depths of 1.5, 3.5, 6 and 7.5 cm after 30 min of simulated heating and 120 min of simulated cooling. The inset in Fig. 4.18a presents the time evolution of thermal contrast for the whole cooling period of 120 min. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XIX Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.19: (a) The fitting results for Cmax and tCmax using the model proposed by Balageas et al. (2.36) and (b) the fitting results for fch using the model proposed by Arndt et al. (2.44). . . . . . . . . . . . . . . . . . . . . . . . . . 58 Figure 4.20: A schematic presentation of true positive (T P ), true negative (T N ), false positive (F P ) and false negative (F N ) calls for the detection of a defect. . . 59 Figure 4.21: The results of sensitivity evaluation in terms of c_sens (in %) for GPR (full marks) and IRT (empty marks) with respect to the material and depth of the defects in specimens S0–S2. The dotted lines are guides to the eye. . . . 60 Figure 4.22: The imaging results of specimen S3 with marked polystyrene plates according to Table 4.1: (a) a GPR depth slice at a depth of 1.5 cm and (b) an IR phase contrast image at a frequency of 3.7 · 10−4 Hz. . . . . . . . . . . . . . . . . . 60 XX Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XXI Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. KAZALO PREGLEDNIC Preglednica 2.1: Okvirne vrednosti relativne dielektrične konstante εr, električne pre- vodnosti σ, hitrosti valovanja v in atenuacijskega koeficienta α za naj- pogostejše gradbene materiale pri frekvenci valovanja 100 MHz (Daniels, 2004: 74, 90). Vrednosti za materiale označene z zvezdico (∗) so vzete po Wendrich (2009: 21). . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Preglednica 2.2: Termične lastnosti (gostota ρm, toplotna prevodnost k, specifična toplota cp, termična difuzivnost α) najpogostejših gradbenih materialov (Brink, 2005: 119). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Preglednica 2.3: Okvirne vrednosti hitrosti longitudinalnih zvočnih valov vl v nekaterih gradbenih materialih (Wendrich, 2009: 11). . . . . . . . . . . . . . . . . 29 Preglednica 4.1: Material in krovni sloj betona vgrajenih anomalij v preizkušancih S0–S3. 46 Preglednica 4.2: Frekvenca pri maksimalnem faznem kontrastu fch za anomalije v preiz- kušancu S0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Preglednica 4.3: Rezultat simulacije za Cmax, tCmax in fch za anomalije na globinah 1,5, 3,5, 6 in 7,5 cm (glej sliko 4.17). . . . . . . . . . . . . . . . . . . . . . . 57 XXII Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XXIII Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. KAZALO SLIK Slika 1.1: Mehanizmi propadanja betonskih konstrukcij: (a) shematičen prikaz (Banville, 2008: 2), (b) odstopanje betona zaradi korozije armature kot posledica vdora kloridnih ionov (Banville, 2008: 2), (c) korozija armaturne palice z vidnim razpokanjem betona. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Slika 2.1: Shematični prikaz odboja georadarskega signala na mejah materialov z ra- zličnimi dielektričnimi konstantami. Prikazani so še načini grafičnega prika- zovanja rezultatov – grafični prikaz refleksijske sledi (A-scan), radargram geo- radarskega profila (B-scan) in grafični prikaz prereza (C-scan) (Kohl, 2005: 819). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Slika 2.2: Tipične geometrijske razporeditve elektrod in ustrezni konfiguracijski faktorji K pri geoelektričnih meritvah. Na sliki parameter a označuje razdaljo med dipoloma, n je faktor razdalje med diploma, C1 in C2 sta tokovni elektrodi, P1 in P2 pa merita napetost. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Slika 2.3: Razporeditev elektrod pri geometrijski razporeditvi Wenner za 2D geoelektri- čno meritev opečnatega preizkušanca ter sosledje meritev, ki ustvari psevdo prerez (prirejeno po Loke, 2011: 26). . . . . . . . . . . . . . . . . . . . . . . . 17 Slika 2.4: Princip delovanja tehnik optične IRT v odbojnem načinu merjenja – pulzne termografije (PT), termografije s kvadratnim pulzom (SPT), termografije z odzivom na periodično motnjo (LT) in termografije s stopničastim pulzom (SHT). Za vsako tehniko je prikazan način ogrevanja. . . . . . . . . . . . . . 18 Slika 2.5: Spektralna porazdelitev sevanja črnega telesa pri različnih temperaturah telesa kakor ga opisuje Planckov zakon. Navpične puščice označujejo valovno dolžino, pri kateri telo z izbrano temperaturo seva največ energije. Iz vložene slike (skala na ordinatni osi je logaritemska) je razvidno, da telesa s sobno tempe- raturo (300 K) tudi v IR območju sevajo precej manj kot telesa s temperaturo nekaj tisoč Kelvinov. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Slika 2.6: Določitev temperaturnega kontrasta C(t) iz razlike med površinsko temperatu- ro nad anomalijo Tdef in površinsko temperaturo nad homogenim področjem Tsound. Pri tem Cmax označuje maksimalni temperaturni kontrast, tCmax pa čas nastopa le-tega. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Slika 2.7: Fourierjeva transformacija pulza z amplitudo Ap in dolžine tp. . . . . . . . . . 24 Slika 2.8: (a) Princip delovanja pulzno-fazne termografije (PPT) in (b) prikaz krivulje faznega kontrasta in frekvence, pri kateri nastopi maksimalni fazni kontrast fch. 26 Slika 3.1: Združevanje podatkov na nivoju slikovne točke, z uporabo značilnic in na simbolnem nivoju. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Slika 3.2: Shematični prikaz arhitekture splošnega sistema za združevanje podatkov (prire- jeno po Gros et al., 1995: 2). . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Slika 3.3: Norme uporabljene v algoritmih za mehko združevanje (Babuška, 2009: 70). . 35 Slika 3.4: Prikaz gručenja množice podatkov dveh spremenljivk v tri razrede. Razpore- ditev posameznega podatka v ustrezen razred je na sliki ponazorjena z barvo razreda (rdeči, zeleni ali modri). Za podatek xj so prikazane razdalje do središč razredov v1, v2, v3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Slika 3.5: Princip delovanja trdega in mehkega gručenja. . . . . . . . . . . . . . . . . . . 37 Slika 3.6: Diagram poteka splošnega algoritma za gručenje podatkov. . . . . . . . . . . . 43 XXIV Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Slika 4.1: Skice laboratorijskih betonskih preizkušancev z vgrajenimi anomalijami. . . . 46 Slika 4.2: Termokamera FLIR A320, ki smo jo uporabili pri raziskavah. . . . . . . . . . 47 Slika 4.3: Georadarski sistem MALÅ Geoscience, ki smo ga uporabili pri raziskavah. . . 47 Slika 4.4: Termogrami za preizkušanec S0 pri različnih časih ohlajanja. Vsaka slika ima svojo temperaturno skalo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Slika 4.5: Potek časovno odvisnega temperaturnega kontrasta C(t) za različno globoke anomalije v preizkušancu S0 po 30 min segrevanju. . . . . . . . . . . . . . . . 48 Slika 4.6: Potek časovno odvisnega temperaturnega kontrasta C(t) za anomalije v preiz- kušancih S1 in S2 po 30 min segrevanju: (a) za vodno in zračno anomalijo na globini 3 cm ter (b) za anomalijo iz stiropora na globini 4,6 cm in ob prisotni armaturi rastra 5 oz. 10 cm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Slika 4.7: (a) Kontrastna slika in (b) časovna slika za preizkušanec S0 po 30 min segrevanju. 50 Slika 4.8: (a) Kontrastna slika in (b) časovna slika za preizkušanec S1 po 30 min segrevanju. 50 Slika 4.9: (a) Kontrastna slika in (b) časovna slika za preizkušanec S2 po 30 min segrevanju. 50 Slika 4.10: (a) Kontrastna slika in (b) časovna slika za preizkušanec S3 po 30 min segrevanju. 50 Slika 4.11: Odvisnost (a) maksimalnega temperaturnega kontrasta Cmax in (b) časa nje- govega nastopa tCmax od globine anomalije za različne čase gretja (15, 30 in 45 min). Črta le povezuje merske točke za lažje spremljanje rezultatov. . . . . 51 Slika 4.12: Rezultat prilagajanja eksperimentalnih podatkov za Cmax in tCmax z modelom po enačbi (2.36). Vključeni so rezultati za anomalije v preizkušancu S0 za čas gretja 30 min. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Slika 4.13: Potek frekvenčno odvisnega faznega kontrasta ∆φ za (a) anomalijo na globini 1,5 cm in (b) za anomalijo na globini 3 cm v preizkušancu S0 po 45 min segrevanju in merjenju ohlajanja za 150 min. . . . . . . . . . . . . . . . . . . 53 Slika 4.14: Rezultat prilagajanja eksperimentalnih podatkov za fch z modelom po enačbi (2.44). Vključeni so rezultati za anomalije v preizkušancu S0, dobljeni po 45 min segrevanju in merjenju ohlajanja za 150 min. . . . . . . . . . . . . . . 54 Slika 4.15: IR slike za preizkušanec S3: (a) termogram ob začetku snemanja, (b) kon- trastna slika in (c) fazno-kontrastna slika pri frekvenci 3, 7 · 10−3 Hz. . . . . . 54 Slika 4.16: Fazno-kontrastna slika pri (a) frekvenci 6, 2 · 10−4 Hz za preizkušanec S1 in pri (b) frekvenci 3, 0 · 10−4 Hz za preizkušanec S2. Na sliki 4.16a je zračna anomalija zgoraj (b), vodna anomalija pa spodaj (d). Na sliki 4.16b je gostejša armatura na desni strani. Rdeči kvadrati označujejo območje anomalij (za oznake anomalij glej preglednico 4.1). . . . . . . . . . . . . . . . . . . . . . . 55 Slika 4.17: 2D numerični model, ki ponazarja dva prereza betonskega preizkušanca z vgra- jenimi anomalijami iz stiropora (belo območje). Točke na sprednji strani mo- dela, kjer smo spremljali temperaturo, so označene z rdečimi pikami. . . . . . 56 Slika 4.18: (a) Potek časovno odvisnega temperaturnega kontrasta C(t) in (b) potek frekvenčno odvisnega faznega kontrasta ∆φ za anomalije na globinah 1,5, 3,5, 6 and 7,5 cm po 30 min simuliranem segrevanju in 120 min simuliranem ohla- janju. Vložena slika na sliki 4.18a prikazuje potek temperaturnega kontrasta za čas celotnega ohlajanja (120 min). . . . . . . . . . . . . . . . . . . . . . . . 57 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XXV Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Slika 4.19: (a) Rezultat prilagajanja podatkov za Cmax in tCmax z modelom po enačbi (2.36) in (b) rezultat prilagajanja podatkov za fch z modelom po enačbi (2.44). . . . 58 Slika 4.20: Shematični prikaz kazalcev za vrednotenje zaznavanja anomalij: pravilno po- zitivni (T P ), pravilno negativni (T N ), napačno pozitivni (F P ) in napačno negativni (F N ). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Slika 4.21: Rezultat občutljivostne analize za georadar (polni znaki) in IRT (prazni znaki) v odvisnosti od materiala in globine anomalij v preizkušancih S0–S2. Rezultati so podani v % parametra c_sens. Pikčasti črti povezujeta merske točke za lažje spremljanje rezultatov. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Slika 4.22: Grafični prikaz rezultatov za preizkušanec S3 z označenimi ploščami iz stiropora (preglednica 4.1): (a) georadarski grafični prikaz prereza na globini 1,5 cm in (b) IR fazno-kontrastna slika pri frekvenci 3, 7 · 10−4 Hz. . . . . . . . . . . . . 60 XXVI Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XXVII Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. LIST OF APPENDICES APPENDIX A: Article 1 – Image fusion for improved detection of near-surface defects in NDT-CE using unsupervised clustering methods APPENDIX B: Article 2 – Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value APPENDIX C: Article 3 – Effect of moisture on the reliability of void detection in brick- work masonry using radar, ultrasonic and complex resistivity tomography APPENDIX D: Article 4 – Unsupervised clustering of laser-induced breakdown spec- troscopy data from concrete XXVIII Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Blank page Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. XXIX Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. ABBREVIATIONS AC Alternating current AE Acoustic emission AFC Adaptive fuzzy clustering algorithm CE Civil engineering CFRP Carbon fibre reinforced plastic CR Complex resistivity DC Direct current DIC Digital image correlation D-S Dempster-Shafer EM Electromagnetic EMW Electromagnetic waves FCM Fuzzy c-means algorithm FFT Fast Fourier transform FPCM Fuzzy-possibilistic c-means algorithm FT Fourier transform GK Gustafson-Kessel algorithm GKAρ Gustafson-Kessel algorithm with adaptive cluster volumes GPR Ground penetrating radar IE Impact-echo IRT Infrared thermography IT Inductive thermography LIBS Laser-induced breakdown spectroscopy LMMSE Linear minimum mean square error LT Lock-in thermography NCFCM Noise handling fuzzy c-means algorithm NDE Non-destructive evaluation NDT Non-destructive testing NDT-CE Non-destructive testing in civil engineering PCM Possibilistic c-means algorithm PFCM Possibilistic-fuzzy c-means algorithm PPT Pulsed phase thermography PT Pulsed thermography PZT Lead zirconate titanate XXX Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. S Separation index SAFT Synthetic aperture focusing technique SC Partition index SHT Step heating thermography SPT Square pulse thermography TWT Two-way travel time UT Ultrasound thermography XB Xie and Beni’s index Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 1 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 1 INTRODUCTION 1.1 Scientific background and motivation Among the numerous methods developed for the inspection of materials and structures, non- destructive testing (NDT) techniques provide the advantage of preserving the inspected object undamaged [1]. Their application to civil engineering (CE) structures originally emerged in the 1940s for the assessment of homogeneity and compressive strength of concrete using techniques such as rebound hammer, a windsor probe test, a pull-out test and ultrasonic pulse velocity method, which are today standardized methods [2, 3]. Due to the fast development of both new building materials and construction techniques, the development of potential non-destructive methods for controlling the quality of a building process is nowadays still an important research field (e.g. for the detection of aggregate segregation, excessive amount of entrapped air voids and honeycombs1 due to inappropriate material properties and/or construction techniques). Relatively recently attention has focused on the loss of durability of existing building structures, caused by both material degradation and impacts of e.g. seismic action in relation to inappro- priate construction. To ensure accurate repair and retrofitting of such structures rather than demolition and construction of new ones, the evaluation of their current state together with damage assessment with minimal possible interference, is urgently required. To overcome the limits of visual inspection and testing samples extracted from the structure by drilling, many NDT methods have been proposed. Acoustic, dielectric, electric and thermal methods have arisen mainly from geophysics, as well as from NDT in aerospace for the inspection of compos- ite structures. Currently, only a few standards for particular application of NDT-CE methods are available [1, 3, 5]. Hence, to bring NDT into practice, an extensive validation process of the methods performance with respect to the various possible defect assessment applications is required [6]. Damage of concrete structures (mostly referred to bridges, parking garages and tunnels) reflects in cracking, spalling and delamination [7] (see Fig. 1.1). In most cases it is a result of deterioration mechanisms such as frost action, a sulphate attack, an alkali-silica reaction as well as reinforcement corrosion [4, 7]. Except from frost action, all processes are induced by the ingress of de-icing salts into the pore system of the cement matrix in combination with insufficient concrete quality and poor workmanship, which results in excessive amount of entrapped air voids and honeycombs as well as insufficient concrete cover above the reinforcement [8]. Damage as- sessment of masonry structures, among which a large proportion represents cultural heritage assets, is apart from the effect of de-icing salts related to the presence of high moisture content with even trapped water and internal voids. Moreover, particularly stone masonry buildings in European Mediterranean countries are highly vulnerable to seismic action, resulting in leaf detachment and cracks [9]. 1Segregation refers to separation of the components of fresh concrete, resulting in a non-uniform arrangement of aggregates, whereas honeycombs occur in case the space between aggregates is not filled [4]. 2 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 1.1: Failure mechanisms on concrete structures: (a) a schematic presentation (Banville, 2008: 2), (b) spalling of concrete due to a chloride attack of embedded reinforcement (Banville, 2008: 2), (c) rebar corrosion with visible cracking of concrete. Slika 1.1: Mehanizmi propadanja betonskih konstrukcij: (a) shematičen prikaz (Banville, 2008: 2), (b) odstopanje betona zaradi korozije armature kot posledica vdora kloridnih ionov (Banville, 2008: 2), (c) korozija armaturne palice z vidnim razpokanjem betona. To visualize the internal structure and to detect most of the above mentioned damages affecting CE structures, methods such as ground penetrating radar (GPR), ultrasonic, impact-echo (IE) and infrared thermography (IRT) are most commonly used [10, 11, 12]. Contrary to the inspec- tion of metal and composite structures, where most research is focused on the detection of cracks, the application of the above mentioned methods in CE has to account for both the heterogeneous material structure and a variety of possible defects and inclusions. With respect to this, NDT- CE methods are characterized by different penetration depths, as well as different sensitivity to the physical properties of the material. However, this requires extensive validation processes of the methods performance with respect to the type of defects and material inspected [6]. In the recent two decades, an outstanding progress has been made in the characterisation of NDT-CE methods with respect to specific testing problems, for which a literature overview is given below. GPR proves to be a highly effective method for the detection of reinforcement and metal in- clusions, but at the same time dense reinforcement greatly decreases the penetration capability of electromagnetic waves (EMW) and thus decreases the efficiency in detection of defects below reinforcement [13, 14]. In case the thickness of a structural element is known, the determination of the moisture and salt concentration in the material is possible since moisture greatly affects the velocity of EMW. Apart from velocity, the moisture and salt concentration largely affect the attenuation of the radar signal [13, 15, 16], whereas pore size distribution and distribution of aggregates in the material affect the penetration capabilities to a smaller extent [14]. The effect of unfilled and grouted joints as well as the presence of larger voids in different layered brick masonry was investigated in [17]. Here, the researchers concluded that unfilled joints can be observed in the radargrams as small hyperbolas, as well as the modifications resulting after the joints were grouted. Moreover, it was found that the detection of the multi-layered structure is based on the measurements of the travel time, amplitude and phase of the reflected radar signal. Ultrasonic has been employed for the thickness estimation of concrete structural elements [18], as well as for the detection of reinforcement and tendon ducts [18, 19, 20] and for the detection Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 3 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. of surface cracks [20, 21]. The physical basis for most of the experimental ultrasonic results obtained from the investigation of grouting faults in tendon ducts was explained by Krause et al. [22] and Im et al. [23] and further confirmed by Krause et al. using modelling techniques [24]. For the investigation of grouting faults in tendon ducts and thickness estimation of structural elements, IE was proposed as an alternative method to ultrasonic [23, 25, 26, 27]. However, for the inspection of thin structures, IE was found to perform poor since most of the input energy is transformed into shear and surface waves producing interference due to reflections at edges [6]. Apart from the investigation of concrete structures, IE was applied for evaluating the integrity of bonding in stone masonry structures and was in addition recommended for the detection of larger voids and for the monitoring of the quality of grouting repairs [28]. However, in gen- eral, GPR, the sonic pulse velocity method and IRT were proposed for assessing the bonding condition, monitoring the quality of grouting repairs, detecting the multi-layered structure and leaf detachment, as well as the presence of larger voids and higher moisture content in masonry structures [9, 29, 30, 31, 32, 33]. It should be noticed that for the inspection of historic masonry buildings, particularly the non-contact IRT has gained interest. Here, pulsed thermography was applied for the characterization of the surface and interfaces, more exactly for the detection of delamination, the masonry texture, filled vs. unfilled mortar joints [9, 31, 34, 35], as well as for the investigation of plastered mosaics [36, 37, 38]. Apart from pulsed thermography, lock-in thermography was applied on masonry for moisture measurements [39], as well as for the structural visualization at different depths [40]. Basic research on quantitative analysis of ex- perimental and numerical pulsed thermography data was performed on concrete specimens with inbuilt defects [41, 42, 43, 44, 45]. For the monitoring of micro-cracking activity at different levels of loading of concrete structures, acoustic emission (AE) was proposed [46, 47]. For the on-site investigation of historic structures on the other hand, IRT and ultrasonic were used for assessing the depth of surface cracks [48], as well as their orientation [49]. However, up till now, no methods have been proposed for the detection of internal cracks in masonry. A surface crack activation could only be observed from the displacement field when applying digital image correlation (DIC) to masonry walls subjected to shear loading [50, 51]. The given review indicates the capabilities and limitations of the currently used NDT techniques. Apart from the fact that each method can only measure certain physical properties and penetrate to a certain depth range, the methods’ performance is also greatly affected by the heterogeneous nature of building materials and structures. Therefore, the utilization of more than one method or one method in different modes, i.e. multi-sensor testing [52, 53], would generally be required to fully characterize the objects under inspection and to increase the accuracy of the evaluation. The first example of a multi-sensor inspection of concrete structures appears in the works by Krause et al. [18, 54] and Pöpel and Flohrer [55], where thickness measurements of structural elements and the detection of reinforcement are performed using GPR, ultrasonic and IE, as well as GPR and covermeter, respectively. Dérobert et al. [56] stress the benefits of combined use of GPR, IE, covermeter and radiography for the inspection of a post-tensioned beam before its 4 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. autopsy by means of hydro-demolition, whereas Naumann and Haardt [57] study the potential of combined use of GPR, acoustic, electrical and magnetic flux methods for the inspection of high- way structures. Further studies on multi-sensor inspection of concrete structures cover research on the detection, localization and characterization of surface-breaking cracks using electrical re- sistivity and ultrasonic [58], as well as the detection and visualization of inclusions using GPR, ultrasonic and IE [27]. On masonry, a combination of GPR, electrical and acoustic methods have been employed for the detection of moisture, inclusions, as well as defects such as leaf detachment and voids [9, 59, 60, 61]. Although the aforementioned studies have shown the benefit of multi-sensor testing to overcome the limitations of individual techniques, data processing and interpretation of the results were performed for each NDT technique individually. In practice, where large inspection areas and/or data sets should be evaluated, this is both costly and time consuming and is additionally sub- jected to human error interpretation and does not provide a straightforward evaluation of the structural condition. To overcome these constraints, current and future research shed light on au- tomated and computer controlled data analysis with systematic integration of information from multiple sources – data fusion [62]. Data fusion has long been used for military applications [52]. However, in the field of non-destructive evaluation (NDE), it was first introduced in 1994 in the conference papers by Georgel and Lavayssière [63] and Gros et al. [64]. Since then, research on data fusion has emerged rapidly for NDE applications in aerospace and nuclear industry. Different fusion algorithms have been proposed based on optimization methods, multi-resolution approaches [65, 66, 67], heuristic methods [68], probabilistic methods such as Bayesian inference and Dempster-Shafer (D-S) evidence [69, 65, 70, 71, 72, 73, 74], as well as fuzzy logic [71, 75]. 2 Taking into account that NDT results are commonly presented in 2D or 3D image formats rather than as numerical data, special interest has been devoted to the development of fully automated image processing and computer visualization [62]. However, only few of the above presented references report on the application of image fusion, based on eddy current and IRT images from composite and steel structures [69, 65, 73, 67]. In NDT-CE applications, image fusion was performed on GPR data acquired at different frequencies and polarizations, and on GPR and ultrasonic data on prestressed concrete bridge beams and test specimens for the detection of grouting faults in tendon ducts, as well as for the detection of reinforcement placed at larger depths [14, 76, 77]. However, here, the applied fusion techniques were restricted to simple arithmetical methods such as minimal or maximal amplitude and averaging. 1.2 Aims and scope The main goal of this thesis was to develop an image fusion methodology based on clustering techniques, which would enable more efficient and reliable evaluation of multi-sensor NDT data in CE applications. For image fusion, clustering methods have already been applied in medical imaging for pattern recognition, e.g. for the detection of cancerous cells from magnetic resonance images [78] and mammograms [79], as well as for subsurface zonation by cooperative inversion of 2For a comprehensive state-of-the-art on data fusion in NDE, refer e.g. to [53] and [62]. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 5 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. multi-method geophysical data [80, 81]. Their main advantage is that they can work fully unsu- pervised. This is of particular interest in implementing NDT in practice in terms of automated application by trained engineers and technicians rather than scientists. In the thesis, the aim was fulfilled by the implementation of hard – the k-means algorithm [82] – as well as different fuzzy clustering algorithms. For fuzzy clustering, the conventional proba- bilistic fuzzy c-means (FCM) by Bezdek [83], as well as its noise handling variant (NCFCM) [84] and the distance adaptive Gustafson-Kessel (GK) algorithm [85], were used. In addition, two fuzzy possibilistic clustering algorithms were analysed: the possibilistic c-means (PCM) with repulsion [86] and the possibilistic-fuzzy c-means (PFCM) [87]. Apart from developing and testing the image fusion technique for multi-sensor NDT, the thesis aimed at studying the NDT-CE methods performance with respect to different defects present in the structure as well as the material’s condition. Sensitivity analyses were performed on GPR and IRT measurements on laboratory concrete specimens with inbuilt defects and stone masonry walls subjected to plaster delamination and crack propagation. Moreover, the effect of moisture on the reliability of void detection in brickwork masonry was studied using GPR, ultrasonic and complex resistivity (CR) tomography. Four laboratory concrete specimens served for basic studies on the detection of near-surface defects in CE such as voids, localized water and delamination, as well as for the detection of inclusions such as pipes and reinforcement. GPR and IRT were chosen since they were regarded as complementary methods in the near-surface region (up till a depth of 10 cm). Both methods were also chosen for non-destructive assessment of laboratory plastered multi-leaf stone masonry walls. Here, the measurements aimed at evaluating the masonry texture and morphology, as well as plaster detachment and structural cracking due to simulated seismic activity through stepwise in-plane cyclic shear test. A complementary use of GPR and IRT was particularly considered in order to enable a relatively fast inspection with minimal impact on the damaged structure during the shear test. In practice, the material characterization with defect detection by means of NDT plays a crucial role for masonry with artistic or cultural heritage value. Here, the evaluation of the structural condition with minimal interference is of particular importance. As already mentioned, moisture is considered as an important parameter that may influence the state and degradation of masonry. Therefore, the performance of NDT-CE methods with respect to the moisture of the material was studied on a laboratory brickwork specimen. For this task, GPR, ultrasonic and CR tomographies were investigated. Since the tomographic tech- nique delivers a 3D visualization of the structure, the detection of a larger void was particularly considered with respect to the humidity level of masonry. During the course of this work, the developed clustering algorithms were implemented into two data fusion frameworks for practical application: the framework for a multi-sensor robot system – the BetoScan robot system, and for a fully automated classification of spectroscopic spatial data from concrete for cement identification. 6 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. The thesis is organized as follows. The following chapter introduces the NDT-CE methods. Particularly, the physical concepts of GPR, IRT, ultrasonic and CR are presented. Chapter 3 briefly describes the aspects of image fusion, focusing on cluster analysis. Hard and several fuzzy clustering methods are presented and followed by the algorithm derivation. Chapter 4 presents the results obtained from the sensitivity evaluation of GPR and IRT on laboratory concrete specimens, focusing on the characterization of defects and inclusions using IRT. Further fusion of this data set is covered in the article given in Appx. A. This article also gives the results from the BetoScan robot system. The main outcomes from the application of GPR and IRT on stone masonry walls are presented in the article given in Appx. B. Experimental measurement results from the investigation of brickwork masonry are covered in the article in Appx. C. Clustering of laser-induced breakdown spectroscopy (LIBS) data from a concrete specimen is presented in Appx. D. The thesis is summarized by the main findings of the presented research and outlooks in chapter 5. The main part of the thesis is followed by an extended abstract of the thesis in Slovene, given in chapter 6. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 7 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 2 BASIC PRINCIPLES OF SELECTED NDT METHODS 2.1 Overview of NDT methods used in civil engineering NDT-CE methods can be broadly classified into two categories – acoustic and electromagnetic methods. Both are based on the wave propagation with energy transfer that can be mathe- matically described by the second order partial differential wave equation. While for acoustic methods, the propagation of mechanical acoustic waves through a medium is of importance, electromagnetic methods are based on the propagation and radiation of electromagnetic waves. Therefore, different physical properties of the material are measured – with acoustic methods the information about the materials elastic properties is obtained, whereas electromagnetic methods are sensitive to the material electromagnetic properties. All acoustic methods have in common the basic principle, which is recording acoustic waves propagating in the material after the generation of a mechanical disturbance. For recording, a piezoelectric transducer is most commonly used. However, according to [11], factors such as the used frequency range and the type of the acoustic wave generator (elastic impact hammer and spheres, as well as piezoelectric transducers) further distinguish the acoustic methods used in CE: • Micro-seismic methods that operate in the frequency range f = 100 Hz – 1 kHz [11] and are mostly applied to the investigation of masonry arch bridges where deep penetration is essential. • Sonic pulse velocity method where an elastic hammer generates acoustic waves in the fre- quency range f = 500 Hz – 10 kHz [11]. Due to the method’s poor resolution, but deep penetration (results from the low frequency range), the method is particularly recom- mended for qualitative condition evaluation of masonry structures and for control of the quality of grouting repairs [88]. • Impact-echo (IE), which emerged from pile testing using frequencies lower than 1 kHz and is nowadays also used for thickness estimation of concrete slabs using frequencies up to 80 kHz [89]. The method is based on the measurement of resonant frequencies, resulting mainly from the reflections from the external boundary [12]. • Ultrasonic, which uses piezoelectric transducers for acoustic wave generation in the fre- quency range above 20 kHz [11]. • Acoustic emission (AE), which is based on the recording of acoustic waves generated within the material undergoing stress. This principle is contrary to the recording of actively generated waves with transducers or a mechanical impact that is common in other acoustic methods [46, 90]. The classification of various electromagnetic methods is, however, much broader than it is for acoustic methods. Here, the techniques are not only classified by the part of the electromag- netic spectrum they employ, but they also differ greatly in their sensitivity to different material properties and thus the used instrumentation. With respect to the measurement of dielectric, magnetic and electric properties, the following classification is most common [11]: 8 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. • Ground penetrating radar (GPR), which is based on antenna emissions of radio frequency signals and their penetration through a medium, as well as on the recording of reflected signals from irregularities with different dielectric properties in the material. • Eddy current or commercially also known as the covermeter method, which uses the prin- ciple of electric induction to determine the thickness of the concrete cover above the rein- forcement [11]. • Electrical resistivity methods (also geoelectrics) such as the conventional resistivity method and complex resistivity (CR)1 that are based on the measurement of potential difference between two electrodes as the result of an electric current injection [91]. • Electromagnetic potential methods which measure the electric potentials resulting from an applied electromagnetic field. In CE, the most common is the half-cell potential method, which is based on the measurement of the electrochemical potentials of the embedded reinforcement relative to a reference electrode, known as a half-cell placed at the concrete surface [11, 92]. Apart from the above methods, infrared thermography (IRT) is based on the recording of IR radiation from the previously heated surface of a specimen and is thus sensitive to the material thermal properties, whereas radiography measures the absorption of very high frequency electro- magnetic radiation (X-rays and gamma rays) or the absorption of a neutron flux, with respect to the density of the material [11]. The following sections present the basic physics related to four NDT methods taken into account in the main part of the thesis – GPR, CR, IRT and ultrasonic. The techniques’ fundamental theoretical concepts will be introduced, as well as their challenges and limitations. 2.2 Ground penetrating radar The first use of pulsed EMW to detect the structure of buried objects dates back to 1926 [93]. However, it was not before the 1980s when the first commercially available GPR system ap- peared [94], and since then the technique has emerged in a variety of other applications [95]. GPR is based on the generation and propagation of EM pulses in the radio-frequency range within the material, and the detection of either reflected or transmitted EM signals using an antenna [94]. For CE applications, the typical frequency of EM pulses is between 300 MHz and 2.5 GHz [76]. Upon the signal propagation into a dielectrically lossy material, the wave exhibits scattering and attenuation, as well as refraction and reflection from dielectric discontinuities enabling the detection of defects, which is the primary aim of NDT inspections. The estimated defect depths depends both on the waves’ velocity as well as on the travel time needed from transmission to reflection and recording of the signal, i.e. the two-way travel time (TWT) [94]. A point measurement of the reflected EM energy from internal discontinuities builds up a GPR trace – a time dependant variation of the signals energy (with amplitude and polarity). However, 1CR may in general refer to both the induced polarization method as well as to spectral induced polarization. Spectral induced polarization refers to the induced polarization method performed at several frequencies. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 9 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. during a radar measurement, several measurement points are collected along the scanning line – a GPR profile. The visualization of the GPR trace refers to an A-scan, whereas the visualization of the GPR profile refers to a B-scan, also termed radargram [76]. The GPR principle of the signal reflection from boundaries with different dielectric properties, as well as different visualization modes (an A-scan, B-scan and time slice or C-scan), are presented in Fig. 2.1. It is worth noticing that the GPR trace does not only include reflections from discontinuities, but is also affected by the direct air wave which appears first in the A-scan, as well as by the direct ground wave and the critically refracted lateral wave [96]. Figure 2.1: Schematic illustration of the GPR signal reflection from boundaries with different dielectric properties, and the different visualization modes – an A-scan, B-scan and C-scan (Kohl, 2005: 819). Slika 2.1: Shematični prikaz odboja georadarskega signala na mejah materialov z različnimi dielektričnimi konstantami. Prikazani so še načini grafičnega prikazovanja rezultatov – grafični prikaz refleksijske sledi (A-scan), radargram georadarskega profila (B-scan) in grafični prikaz prereza (C-scan) (Kohl, 2005: 819). 2.2.1 Electromagnetic properties of building materials and the energy storage-loss mechanisms Quantities that determine the materials’ response to an EM field are dielectric permittivity (ε), magnetic permeability (µ) and electric conductivity (σ), where all quantities determine the stored and released energy. ε is related to the constrained charge displacement in the electric field – the polarization process – and thus to the stored electric field energy, µ is related to the distortion of intrinsic atomic and molecular magnetic moments in the magnetic field – magnetisation – and thus to the stored magnetic field energy, and σ is related to the free charge movement in the electric field, which results in electric field energy released as heat. The statements can be mathematically described by the constitutive equations [96] D = εE, (2.1a) B = µH, (2.1b) J = σE, (2.1c) where D is the dielectric displacement field vector, E is the electric field strength vector, B is the magnetic flux density vector, H is the magnetic field strength vector and J is the electric current density vector. 10 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. In the general case of an inhomogeneous, anisotropic and nonlinear medium, ε, µ and σ are ten- sors, dependent on the position, direction and on the strength of the EM field. However, in the context of GPR applications, a homogeneous, isotropic and linear material can be assumed, re- sulting in scalar quantities [96]. Nonetheless, in real lossy (dispersive) materials, D, B and J may not be directly related to the energy field strength components according to Eq. (2.1a)–(2.1c), but exhibit a particular time-history dependence on the temporarily preceding field (radiated field due to moving charges) with respect to the frequency dependent polarization or magneti- zation phenomena. Hence, the material properties are frequency-dependent complex quantities, such that ε(ω) = ε0 − i ε00, (2.2a) µ(ω) = µ0 − i µ00, (2.2b) σ(ω) = σ0 − i σ00, (2.2c) where the imaginary dielectric permittivity ε00 and magnetic permeability µ00 relate to the re- laxation energy losses in the polarization and magnetization processes, respectively. Similarly, the imaginary electric conductivity σ00 quantifies the energy stored during polarization of free charges. In the context of GPR applications, only ε and σ are of importance as most building materials are generally nonmagnetic [96]. In general, polarization processes in materials can be of various types: electronic (circular orbits of electrons become elliptical), atomic (distortion of atoms), dipolar (the rotation of dipole molecules into alignment with the electric field) and Max-Wagner (accumulation of free ionic charges at boundaries at frequencies below 10 MHz) [96, 97]. All polarization processes are frequency dependent, where the imaginary dielectric constant relates to the energy released in the relaxation process [94]. However, for the GPR frequency range, the most important polarization process is the dipolar due to free and bound water molecules in the material. Here, the maximal released energy occurs around 10–20 GHz, which limits the high-frequency range of GPR [94]. Similarly, energy losses due to electric conductivity of ionic charge transport in water are most pronounced at the frequency range of 10–300 MHz, which defines the low-frequency limit of GPR [94]. However, the high-frequency limit is apart from the polarization processes determined by scattering losses, which become apparent at wavelengths approaching the size of particles [97]. In addition to material attenuation losses due to σ0 and ε00, as well as target scattering losses, the GPR gain decreases along the path also as a result of energy spreading loss [94, 95]. 2.2.2 Wave nature of electromagnetic fields The foundations of GPR are governed by the Maxwell equations coupling the electric and mag- netic fields, as well as by the constitutive laws describing the materials’ response to an EM field [96]. By coupling both of them, the wave equation for the electric and magnetic field is Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 11 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. obtained, which for the electric field vector E yields [96] ∂E ∂2E ∇ × ∇ × E + µσ + µε = 0. (2.3) ∂t ∂2t For a conducting dielectric, the resulting electric field can be described as a function of distance, z, and time, t, by [95] E(z, t) = E0 e−αzei(ωt−βz), (2.4) where the first exponential term relates to attenuation and the second to propagation of EMW. Parameter α is the material attenuation coefficient, β the phase constant, ω the angular fre- quency, ω = 2πf , and f is the frequency of EMW. By relating α and β to a complex propagation p factor for a dispersive medium, k, such that ik = α + iβ = iω µ (ε0 − iε00) ,2 the attenuation coefficient α and wave velocity v can be expressed as [95, 96] r σ µ µε0 p 1/2 α = = ω 1 + tan2 δ − 1 , (2.5a) 2 ε 2 1 µ p −1/2 v = √ = c r ε0r 1 + tan2 δ + 1 , (2.5b) µε 2 where c is the velocity of EMW in free space, εr is the relative dielectric permittivity of the medium, related to ε and the free-space dielectric permittivity, ε0, by εr = ε/ε0, µr is the relative magnetic permeability of the medium, related to µ and the free-space magnetic permeability, µ0, by µr = µ/µ0, and tan δ is the loss tangent quantifying the ratio between the total energy loss and the total energy storage due to both dipolar and conductivity effects as [96] σ0 + ωε00 tan δ = . (2.6) ωε0 − σ00 According to Eq. (2.5a), higher frequencies result in greatly attenuated EM propagation. Thus, lower frequencies have to be used with GPR to penetrate deeper. For low-loss conditions, the following can be assumed σ0 tan δ ≈ ≤ 1, (2.7) ωε0 which yields the low-loss conditions for the frequency independent attenuation coefficient and the wave velocity3 r σ µ α = , (2.8a) 2 ε c v = √ . (2.8b) εr 2Note that according to Daniels [95] and other literature on GPR, the term complex propagation factor k is mostly used for the relation ik = α + iβ, although strikly speaking k is the complex wavenumber and is related to the complex propagation factor ¯ γ by ¯ γ = −ik. 3Eq. (2.8b) is obtained by assuming µr = 1, which holds for most building materials. 12 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. From Eq. (2.8a) it can be seen that the electric conductivity affects the EMW attenuation the most. Using Eq. (2.8b), the wavelength of the EMW in low-lossy materials can be estimated according to v c λ = = √ . (2.9) f f εr Table 2.1 summarizes the typical range of relative dielectric permittivity εr, electric conductiv- ity σ, wave velocity v and attenuation coefficient α of various building materials measured at 100 MHz. Table 2.1: The typical range of relative dielectric permittivity εr, electric conductivity σ, wave velocity v and attenuation coefficient α of most common building materials measured at 100 MHz (Daniels, 2004: 74, 90). The values for the materials marked with an asterisk (∗) are taken from Wendrich (2009: 21). Preglednica 2.1: Okvirne vrednosti relativne dielektrične konstante εr, električne prevodnosti σ, hitrosti valovanja v in atenuacijskega koeficienta α za najpogostejše gradbene materiale pri frekvenci valovanja 100 MHz (Daniels, 2004: 74, 90). Vrednosti za materiale označene z zvezdico ( ∗) so vzete po Wendrich (2009: 21). Material εr σ (S/m) v (m/ns) α (dB/m) Air 1 0 0.300 0 Freshwater 81 10−6–10−2 0.033 0.01 Sea water 81 100 0.01 100 Concrete dry 4–10 10−3–10−2 0.11 2–12 Concrete wet 10–20 10−2–10−1 0.08 10–25 Brick dry∗ 4–8 < 10−3 0.12 < 2 Brick wet∗ 8–12 10−2 0.09 9–11 Limestone dry 7 10−8–10−6 0.11 0.5–10 Limestone wet 8 10−2–10−1 0.11 1–20 Sandstone dry 2–5 10−6–10−5 0.15 2–10 Sandstone wet 5–10 10−4–10−2 0.11 4–20 2.2.3 Reflection and refraction of electromagnetic waves When the GPR signal encounters a subsurface discontinuity, both reflection and refraction of EMW occur with respect to both geometric characteristics of the target and its dielectric prop- erties as well as the dielectric properties of the medium. The change in amplitude of the reflected signal from a planar boundary is governed by the reflection coefficient r Z r = 2 − Z1 , (2.10) Z2 + Z1 where Z1 and Z2 are the electromagnetic impedances of medium 1 and 2 and Z = E/H. In a nonmagnetic and low-lossy medium and a single frequency of radiation, the above expression may be simplified to [95] √ √ ε ε r = r2 − r1 √ √ , (2.11) εr2 + εr1 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 13 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. where εr1 and εr2 are the relative dielectric permittivities of medium 1 and 2. Considering that the reflection coefficient has a positive value when εr2 > εr1 and otherwise a negative one, amplitude changes can be detected with respect to the dielectric contrasts at boundaries. 2.2.4 Horizontal and vertical resolution The resolution of an antenna is defined by the ability to resolve two closely spaced features [94]. For GPR, two resolution components are important: vertical (range or depth) – ∆V – and horizontal (plan) – ∆H – resolution. The vertical resolution relates to the temporal resolution to differentiate two adjacent signals in time, whereas the horizontal resolution is the ability to differentiate two targets placed at the same depth [95]. Both are dependent on the antenna characteristics (system bandwidth B) and material properties by [96] v λ ∆ c V ≥ = , (2.12a) 4B 4 r r vd λ ∆ cd H ≥ = , (2.12b) 2B 2 where B = fc for impulse GPR [96] and fc is the centre frequency of the antenna signal, λc is related to fc according to Eq. (2.9) and d is the distance to the target. It should be noted that the expression for the horizontal resolution (2.12b) results from the Fresnel zone concept, i.e. the interference of reflected signals from targets placed at a distance d with the radiated EM field occurs in an ellipsoidal volume of radius ∆H (the Fresnel zone) [96]. The interference effects hence directly limit the ability to resolve two closely spaced targets at the same depth. However, in addition to the vertical resolution, the shallowest targets to be detected are deter- mined by the GPR near-field region, which extends up till the depth of 1.5λc [95]. However, the strongest antenna induction/coupling effects appear in the reactive near-field region. Thus, for the minimal detectable distance to targets, a recommendation of dmin = λc/2π has been given as the boundary between the reactive and radiating near-field region for an electrically small antenna [96]. For most GPR applications in CE, the use of a small bow-tie dipole or butterfly antennae is the most common. However, the antenna characteristics and the GPR system will not be reviewed herein. An excellent description is given in [95] and [96]. 2.3 Complex resistivity In 1912 the Schlumberger brothers performed the first electrical resistivity survey to estimate the subsurface conductivity over a region of interest [98]. A four-electrode array was used, where the outer two electrodes served for current injection and the inner two measured the resulting electrical potential difference. Nowadays, several electrode array configurations can be used and combined in a multi-electrode acquisition survey producing 2D and 3D data sets. With this resistivity imaging or geoelectric tomography, the resistivity distribution of a medium can be 14 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. obtained using tomographic inversion routines. The two electrically based resistivity methods – the resistivity and the CR method – determine the resistivity of a material from the measured potential difference between two electrodes af- ter current injection according to Ohm’s law. However, their main difference is that CR uses alternating currents (AC) of high frequencies and is more sensitive than the normal resistivity method, which uses either direct currents (DC) or AC of low frequencies (up to 20 Hz) [91]. Therefore, CR is able to detect materials of very low conductivity that can hardly be detected using the conventional resistivity method [99]. 2.3.1 Electrical polarization The term CR directly reflects that when AC is used, the materials’ frequency dependent complex resistivity (also electrical impedance) is of significance, where the real component ρ0 refers to the electrical resistivity and the imaginary component ρ00 to the reactivity and polarization phenomenon, such that4 1 ρ(ω) = ρ0 − i ρ00 = |ρ| eiδ = , (2.13) σ(ω) where |ρ| is the resistivity magnitude – the in-phase amplitude ratio between the measured voltage waveform and the transmitted current waveform, and δ is the phase angle – the time shift of both waveforms. It yields: p |ρ| = ρ02 + ρ002 , (2.14a) δ = arg(ρ(ω)). (2.14b) The complex conductivity and resistivity result from the polarization phenomenon of porous building materials when subjected to an electric current, which is called induced polarization. According to Marshall and Madden, the two main sources of low-frequency induced polarization are electrolytic and electrode polarizations [100, 101]. For building materials, the electrolytic polarization is the most important due to ions in the pore space water [101]. However, correct understanding of induced polarization is much more complex and is not within the scope of this thesis. To combine the above introduced induction polarization with the preceding section on GPR, it should be noted that in a general case, polarization phenomena occur due to both separation of constrained charges – dielectric polarization (see section 2.2.1), as well as induced polarization. The two processes are coupled through the Maxwell generalization of Ampère’s law ∂D ∇ × H = J + , (2.15) ∂t where the term ∂D/∂t also refers to the displacement current density vector. Using the consti- 4Note that as for GPR applications, electrical conductivity and resistivity are assumed to be scalar values, independent of the position, direction and on the strength of the electric field. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 15 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. tutive equations for D (2.1a) and J (2.1c), a relation between the total current density vector and a time-varying electric field, E = E0 eiωt, may be written as [101] Jtot = [1/ρ(ω) + iω ε(ω)] E. (2.16) However, for frequencies used in CR measurements (lower than 105 Hz), the displacement cur- rent contribution is usually ignored [101]. Thus, the above equation generalizes to Ohm’s law, expressed by the constitutive equation for J (2.1c). Relating E to the electric potential Φ, by the Poisson’s equation, E = −∇Φ, Ohm’s law writes as J = −σ ∇Φ. (2.17) 2.3.2 Apparent and true resistivities The aim of a resistivity survey is to obtain the resistivity of a medium from measured potential distribution after current injection. For a point current source located at position (xs, ys, zs), the relationship between the electric current density and the injected current I, yields [99, 102] I ∇ · J = · δ(x − x dV s) δ(y − ys) δ(z − zs), (2.18) where dV is the elemental volume surrounding the current source and δ is the Dirac delta function. Using Eq. (2.17), the above equation can be rewritten to I −∇ · [σ(x, y, z) ∇Φ(x, y, z)] = · δ(x − x dV s) δ(y − ys) δ(z − zs), (2.19) which is the governing equation for assessing the potential distribution in a medium due to a radially emitted current from a point source. For a fictitious homogeneous half-space, where the equipotential surfaces are hemispheres, the analytical solution for the electric potential at a distance r in the medium from a single point current electrode is given by [99] ρI Φ = . (2.20) 2πr In actual field surveys, a four-electrode arrangement is preferred, separating current injection (with a positive and negative current electrode C1 and C2) from voltage measurement (with potential electrodes P1 and P2) to minimize the influence of contact resistance. Thus, the potential distribution in the medium is a superposition of potentials produced from both of the current electrodes. From the measured potential difference U = ∆Φ between electrodes P1 and P2, an apparent resistivity ρa can be calculated from the applied current and for a given electrode arrangement by [99] U ρa = K , (2.21) I 16 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 2.2: Common electrode arrangements in resistivity surveys and their configuration factors K. In the figure, a refers to the dipole separation length, n is the dipole separation factor, C1 and C2 are the current injection electrodes, and P1 and P2 measure the potential. Slika 2.2: Tipične geometrijske razporeditve elektrod in ustrezni konfiguracijski faktorji K pri geoelektri- čnih meritvah. Na sliki parameter a označuje razdaljo med dipoloma, n je faktor razdalje med diploma, C1 in C2 sta tokovni elektrodi, P1 in P2 pa merita napetost. where 1 1 1 1 −1 K = 2π − − + (2.22) rC1P 1 rC2P 1 rC1P 2 rC2P 2 is a configuration factor that depends on the electrode arrangement and rij is the distance between electrodes i and j. The most commonly used electrode configurations and their corre- sponding configuration factors are presented in Fig. 2.2. The term apparent resistivity is used to relate to the resistivity of a homogeneous and isotropic half-space that would produce the measured relationship between the potential difference and the current for a given electrode array [99]. In other words, the apparent resistivity relates to the average resistivity of an inhomogeneous material over the total current path length. To obtain the true resistivity distribution of the investigated inhomogeneous medium, an inversion problem has to be solved. Solving this problem mathematically means adapting a resistivity distribution model iteratively until the calculated apparent resistivities match the measured ones within acceptable limits. To perform this, a general least-square optimization method can be used. However, beside the used optimization method, the calculated model parameters depend on the way the model is discretized. For 2D and 3D arbitrary problems, the most versatile methods are the finite-difference and finite-element methods [99]. 2.3.3 Electrode configurations In practice, the most commonly used electrode configurations for 2D imaging survey are the Wenner, dipole-dipole, Wenner-Schlumberger, pole-pole and pole-dipole (Fig. 2.2). They all have advantages and disadvantages and differ in the depth penetration capabilities, the sensitivity to vertical and horizontal changes in the resistivity distribution, the horizontal data coverage and the signal strength. Especially for noisy environments, the Wenner array has shown to be a good choice due to its high signal strength and good vertical resolution [99]. To enable a good horizontal resolution and data coverage, the dipole-dipole array is more suitable. However, this electrode configuration has very low signal strength for large dipole-dipole distances and is Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 17 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. relatively insensitive to vertical changes in the resistivity distribution. The Wenner-Schlumberger array is a suitable choice when both good horizontal and vertical resolution are needed and is therefore a compromise between the Wenner and dipole-dipole arrays. In Fig. 2.3 a sequence of measurements used to build up a pseudo-section (distribution of apparent resistivities in a section) with the Wenner array is shown. Figure 2.3: The arrangement of electrodes for a 2D resistivity survey of a brickwork specimen and the sequence of measurements used to build up a pseudo-section with the Wenner array (modified after Loke, 2011: 26). Slika 2.3: Razporeditev elektrod pri geometrijski razporeditvi Wenner za 2D geoelektrično meritev ope- čnatega preizkušanca ter sosledje meritev, ki ustvari psevdo prerez (prirejeno po Loke, 2011: 26). 2.4 Infrared thermography The first IRT application for NDT dates back to 1935, when Nichols used a radiometer to check the uniformity of heated steel slabs after stell-rolling [103]. However, the first commercially available IR camera for military applications appeared in the 1960s [104] and later found its way also in NDT. IRT is based on the recording of the emitted IR radiation from a specimen’s surface through the assessment of surface temperatures, with the aim to resolve the underlying structure [104]. For NDT, both passive and active IRT can be performed. However, active IRT in general enables the detection of smaller thermal contrasts in the medium since it employs an external energy source brought to the specimen. Hence, it is the most desired way of applying IRT in NDT. Several testing techniques can be used for active IRT: • Pulsed thermography (PT), which is based on the application of a short pulse (for a few milliseconds) to the specimen and later recording of the surface temperature decay [105, 106]. • Square pulse thermography (SPT), which has been proposed as an extension to PT. The difference from PT is that here, longer square pulses (for a few seconds to a few minutes) are employed. However, since the principle of PT and SPT is the same, only the term PT 18 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. is often used. • Lock-in thermography (LT), which is based on the generation of thermal waves in the specimen by using modulated heating. The thermal waves amplitudes and phases are measured relative to the launched modulation signal [107]. • Step heating thermography (SHT), where the increase in surface temperature is monitored during the application of a relatively long uniform heating [108]. • Ultrasound thermography (UT), also known as vibrothermography or thermosonics, where a modulated ultrasonic (or sonic) pulse is used for internal heat generation. UT is based on the principle that externally induced mechanical vibrations generate thermal waves inside the specimen by direct conversion between mechanical and thermal energy. Heat is released by friction at locations where abnormalities appear, which enables direct defect detection [104, 109]. • Inductive thermography (IT), where eddy currents are used for internal heat generation [109]. From the above techniques, PT, SPT, LT and SHT can be classified as optical techniques, where heat is delivered to the specimen by either reflection or transmission of the incident radiation [109]. The principle of all optical IRT techniques in reflection mode is presented in Fig. 2.4. In particular, the different waveforms used for heat generation are presented. The thesis specifically deals with SPT in reflection mode, being the technique most often used in NDT-CE. However, since SPT employs the experimental and data processing principles of PT, and most literature on NDT-CE uses the term PT, the term PT will be used hereinafter. Figure 2.4: The principle of optical IRT techniques in reflection mode – pulsed thermography (PT), square pulse thermography (SPT), lock-in thermography (LT) and step heating thermography (SHT). Particularly, the different used waveforms for heat generation are presented. Slika 2.4: Princip delovanja tehnik optične IRT v odbojnem načinu merjenja – pulzne termografije (PT), termografije s kvadratnim pulzom (SPT), termografije z odzivom na periodično motnjo (LT) in termografije s stopničastim pulzom (SHT). Za vsako tehniko so prikazani različni načini ogrevanja. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 19 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 2.4.1 Interaction of incident radiation with matter When an incident radiation strikes the surface of a medium, part of the total incident heat flux density (ji) is reflected (jr), part absorbed (ja) and transmitted (jt), contrary to a perfect blackbody, where all radiation is absorbed. In thermal equilibrium, the energy conservation law yields the total flux exchange ji = jr + ja + jt. (2.23) The individual heat flux fractions may be expressed by the material’s radiation characteristics: reflectance r, absorbance a and transmittance t. In general, these quantities depend on the radiation wavelength, orientation of the incident flux and/or reflected flux, as well as on the temperature and quality of the surface [104]. For IRT in practice, values are assumed to be con- stants for a particular temperature range, i.e. total-hemispheric quantities, in accordance with the spectral5 (independence on the wavelengths) and hemispheric conditions (uniform reflection and transmission over the total hemispheric space), so that Eq. (2.23) rewrites to r + a + t = 1. (2.24) When examining building structures with relatively large thickness, the transmitted energy is considered to be minimal, with t ≈ 0. An object that absorbs heat well also radiates well. This is expressed by the Kirchhoff’s law of thermal radiation e = a, (2.25) where e is the emissivity, defined as the ratio between the emitted radiation from a surface of an real object and the radiation that would be emitted from an ideally blackbody surface. 2.4.2 Thermal emission IRT is based on the recording of IR radiation, emitted from a body. For NDT applications, particularly the middle IR range (1.5–20 µm) is of importance [104]. The total thermal emission from a surface of temperature T and with the total-hemispheric emissivity e can be expressed by the radiant intensity j∗(T ) (also termed radiant exitance) as [110] j∗(T ) = e σsbT 4, (2.26) where σsb is the Stefan-Boltzmann constant. The above equation describes that the thermal emission from an object that will strike the IR camera detector depends on the object’s surface temperature and emissivity. Thus, a high emissivity value is desired, which holds for most building materials. For concrete, brick and plaster, emissivity ranges from 0.9 to 0.95 [104]. In 5An object whose material radiation characteristics are in general independent on the radiation wavelength is called a gray body. 20 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. general, emissivity depends on the quality and roughness of the surface, as well as on the spectral and geometrical distribution of the radiation. However, for IRT applications in NDT-CE, the total-hemispheric emissivity is a good approximation. The physics of thermal emission from bodies can in general be understood by Planck’s law, which describes the spectral distribution of EM radiation from a blackbody and writes [110]: dj∗ 2πhc2 1 b (λ, T ) = · , (2.27) dλ λ5 ehc/λkBT − 1 where dj∗/dλ (λ, T ) is the spectral radiancy and describes the fraction of radiant intensity b j∗(λ, T ) in a range of wavelengths from λ to λ + dλ. In Eq. (2.27), h is the Planck constant, c b is the velocity of EM waves in free space, kB is the Boltzmann constant, T refers to the temper- ature of the blackbody and subscript b relates to the blackbody. As can be seen from Fig. 2.5, a blackbody at room temperature radiates the most in the IR region of the EM spectrum. This phenomenon is exploited by IRT for NDT applications. The wavelength where the spectral radiancy in Eq. (2.27) is maximal for a given temperature is given by Wien displacement law and writes λmaxT = 2898 µm K. (2.28) Figure 2.5: The spectral distribution of EM radiation from a blackbody at different temperatures as described by Planck’s law. The arrows mark the wavelength where the blackbody at a particular temperature radiates the most. From the inset (y-axis is logarithmic), it can be seen that a blackbody at room temperature (300 K) radiates much less than bodies of some 1000 K even in the IR region. Slika 2.5: Spektralna porazdelitev sevanja črnega telesa pri različnih temperaturah telesa kakor ga opisuje Planckov zakon. Navpične puščice označujejo valovno dolžino, pri kateri telo z izbrano temperaturo seva največ energije. Iz vložene slike (skala na ordinatni osi je logaritemska) je razvidno, da telesa s sobno temperaturo (300 K) tudi v IR območju sevajo precej manj kot telesa s temperaturo nekaj tisoč Kelvinov. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 21 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. By integrating the spectral radiancy described by Planck’s law over all wavelengths, the radiant intensity of a blackbody for a given temperature j∗(T ) is obtained as given by Stefan-Boltzmann b law. Eq. (2.26) is a generalization of Stefan-Boltzmann law to describe the radiant intensity from real bodies with respect to the their surface emissivity. 2.4.3 Conduction heat transfer As can be seen from Eq. (2.26), the detected radiation depends on the object’s surface temper- ature that changes with respect to the heat transfer processes. According to the first law of thermodynamics, which is the law of energy conservation, part of the total absorbed incident energy will be stored in the system, whereas another part will be lost by convection and radiation heat transfer. Radiation losses are given by Eq. (2.26), whereas convection losses are related to the heat transferred from an object’s surface by a moving fluid so that the heat flux density j may be expressed by Newton’s law of cooling j = ¯ h (TS − TF ) , (2.29) where ¯ h is the average convective heat transfer coefficient, TS is the object’s surface temperature and TF is the temperature of the moving fluid. The energy stored in the system will undergo an unsteady conduction heat transfer due to the variable temperature distribution within the body. Conduction refers to the heat transfer within a solid that is not in thermal equilibrium, i.e. it is a result of a temperature gradient. On the microscopic level, the energy is transferred by physical interactions between neighbouring constituents [104]. It is worth noticing that conduction is of paramount importance for the IRT concept used in NDT, since it depends on the material thermal properties. Hence, the presence of a particular abnormality with different thermal properties within the specimen will be directly reflected on the detected surface temperature distribution as an abnormal temperature pattern. The basic equation for the conduction mechanism is the Fourier law of thermal conduction, which is a result of experimental evidence. It describes that the heat flux density j is proportional to the material’s thermal conductivity k and the negative temperature gradient ∇T , according to j = −k∇T. (2.30) In many engineering problems where the surface of the investigated structure is very large com- pared to its thickness, a unidimensional heat flux approximation can be assumed. For a semi- infinite slab, the unidimensional heat flux in the x-direction writes dT jx = −k . (2.31) dx Eq. (2.30) or (2.31) only holds for stationary heat conduction. However, for IRT, this approximation cannot be assumed valid. In fact, the unsteady distribution of the surface temperature is also the phenomenon that is exploited for defect characterization in IRT for NDT. Hence, to 22 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. obtain the temperature distribution, the Fourier diffusion equation should be taken into account, which for a generalized dimension writes as [111] ∂T = α ∇2T, (2.32) ∂t where α = k/cpρm is the thermal diffusivity, ρm is the mass density, cp is the specific heat at constant pressure and ∂2 ∂2 ∂2 ∇2 = + + ∂x2 ∂y2 ∂z2 is the Laplace operator. Thermal properties of most building materials are given in Table 2.2. Table 2.2: Thermal properties (mass density ρm, thermal conductivity k, specific heat cp, thermal diffusivity α) of most common building materials (Brink, 2005: 119). Preglednica 2.2: Termične lastnosti (gostota ρm, toplotna prevodnost k, specifična toplota cp, termična difuzivnost α) najpogostejših gradbenih materialov (Brink, 2005: 119). Material ρm (kg/m3) k (W/m K) cp (J/kg K) α (m2/s) Air at 30 ◦C 1.149 0.026 1007 2.28 ·10−5 Water 1000 0.600 4180 1.40 ·10−7 Concrete 2400 2.100 1000 8.75 ·10−7 Brick 2000 1.050 900 5.83 ·10−7 Polystyrene 30 15 0.030 1500 1.33 ·10−6 Lime mortar 1700 0.850 1050 4.60 ·10−7 Cement mortar 2000 1.200 1050 6.67 ·10−7 Gypsum mortar 1200 0.350 850 3.43 ·10−7 Mortar 1900 0.930 800 6.12 ·10−7 Assuming rather simple conditions such as a semi-infinite isotropic plate with finite thickness L, neglected thermal losses and an instantaneous heat pulse (with assumed Dirac delta function shape) applied to the plate, the solution for the surface temperature reduces to [112, 113] " # Q ∞ X n2π2α t T (t) = 1 + 2 (−1)n exp − , (2.33) cpρmL L2 n=1 where Q is the amount of heat absorbed per unit area through the sample face. However, for the inspection of defects in CE, complex geometries have to be taken into account, as well as a finite heat pulse and thermal losses. Hence, the problem is in most cases solved with numerical methods such as finite-difference or finite-element approaches. 2.4.4 Quantitative data analysis in pulsed thermography In IRT for NDT, data is in most cases collected for a given time period in forms of 2D IR images called thermograms, which present the surface temperature distribution. As a result, 3D data sets are obtained, with two spatial dimensions and the time. It should be noted that the recording as well as heating times for the inspection of building materials range for some minutes, contrary Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 23 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. to IRT applied in aerospace, where heat pulse durations range from a few milliseconds to a few seconds [114]. Contrary to GPR, CR and ultrasonic, thermal data do not directly provide the estimation of the defect depth, which, in general, is of utmost importance in NDE. Nonetheless, quantitative data analysis procedures have been proposed and will be briefly described in the sequel. 2.4.4.1 Thermal contrast After the application of a heat pulse in PT, the generated thermal front penetrates into the specimen by thermal diffusion. The presence of a defect within the structure alters the diffusion rate once the thermal front reaches the defect, which is reflected on the surface temperature distribution. Thus, the time t when a defect appears with highest contrast directly relates to the defect’s depth. It has been shown that as a rough estimate, the time t and the thermal contrast C relate to the defect depth zdef by [115, 116] z2 t ∝ def , (2.34a) α 1 C ∝ , (2.34b) z3def where the thermal contrast is given by [104] C(t) = Tdef (t) − Tsound(t), (2.35) where Tdef relates to the surface temperature above the defect and Tsound to the sound area. Due to the above relations, a qualitative estimation of the defect depths may be performed by presenting a contrast image, where each pixel is associated with the value of its maximal thermal contrast Cmax = max C(t). Similarly, timegrams – images referring to times when Cmax appear, tCmax – may be used [117]. This data processing approach is also known as thermal tomography [104]. Definition of the thermal contrast according to Eq. (2.35) is for a specimen with defects given in Fig. 2.6. The maximal thermal contrast Cmax and its corresponding time tCmax are presented as well. However, according to Balageas et al. [118], a quantitative estimation of defect depths was proposed from the inspection of carbon-epoxy composites. It relates zdef to Cmax and tCmax by the following expression √ z n def = A tCmax Cmax m K−n s−1/2, (2.36) where A and n are dimensionless coefficients. The above equation is an inverse problem as coefficients A and n have to be previously determined from calibration curves obtained on the same material and under the same heating conditions. An additional disadvantage of this approach is that the position of the non-defect area has to 24 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 2.6: The definition of the thermal contrast C(t) as the difference in surface temperatures above the defect Tdef and sound area Tsound. Here, Cmax refers to the maximal thermal contrast and tCmax is its corresponding time. Slika 2.6: Določitev temperaturnega kontrasta C(t) iz razlike med površinsko temperaturo nad anomalijo Tdef in površinsko temperaturo nad homogenim področjem Tsound. Pri tem Cmax označuje maksimalni temperaturni kontrast, tCmax pa čas nastopa le-tega. be known a priori in order to calculate the thermal contrast [119]. Finally, the thermal contrast between the defective vs. non-defective area has shown to be smaller due to the blurring effect of all frequencies composed in the pulse compared to the more advanced techniques based on phase characteristics [104, 120], as the technique described below. 2.4.4.2 Pulsed phase thermography Pulsed phase thermography (PPT) [114] is a data analysis technique for PT, but employs the data analysis principle of lock-in thermography (LT), as well as the principle of the thermal wave theory. Contrary to LT, where the specimen is excited with one particular frequency, pulsed heating can be considered as inspection with multiple frequencies simultaneously since mathematically, a pulse of amplitude Ap and finite duration tp can be decomposed into several sinusoidal waves [121] with the frequency distribution following (see also Fig. 2.7) A F (ω) = A ptp sin(ωtp/2) ptp sinc(ωtp/2) = . (2.37) ωtp/2 Figure 2.7: The Fourier transform of a pulse with amplitude Ap and duration tp. Slika 2.7: Fourierjeva transformacija pulza z amplitudo Ap in dolžine tp. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 25 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. In PPT the extraction of various frequency content is performed by a discrete 1D Fourier trans- form (FT) of the time evolution T (t) for each pixel of the data set of N thermograms by the formula6 [122] N 1 −1 X Fn = T (k)e−2πikn/N = ReF N n + i ImFn, (2.38) k=0 where t = k∆t and ∆t is the time interval between individually collected thermogrames. The frequency fn = ωn/2π is given by fn = n/N ∆t, where n is the frequency increment. Due to the symmetry of FT7 and the Nyquist criterion [122], the maximal frequency that can be obtained is fmax = 1/2∆t [123]. It is worth noticing that due to Euler’s formula ei ωt = cos(ωt)+i sin(ωt) and the Fourier series analogy, the real component ReFn can be regarded as the cosine contribution, whereas the imaginary component ImFn may be regarded as the sine contribution for a given angular frequency ωn since the following holds8 ∞ X ∞ X ∞ X T (t) = Fn eiωnt = [ReFn cos(ωnt) − ImFn sin(ωnt)] = An cos(ωnt −φn), (2.39) n=0 n=0 n=0 where amplitudes An and phases φn are then computed by q A 2 2 n = |Fn| = ReFn + ImFn , (2.40a) φn = arg(Fn). (2.40b) By applying the above procedure for each pixel of the thermogram sequence, ampligram and phasegram sequences are obtained for a range of frequencies as displayed in Fig. 2.8. The interpretation of the amplitude and phase images is coupled with the thermal wave theory. Strongly attenuated and dispersive thermal waves are generated within the specimen after a heat pulse is applied. For a semi-infinite slab, onto which a sinusoidal heating is applied with angular frequency ω (as in LT), the resulting temperature distribution is given by [124] T (z, t) = T0 e−z/µ cos (ωt − φ(z)) , (2.41) where µ is the thermal diffusion length and φ is the phase of thermal waves given by r2α λ µ = = , (2.42a) ω 2π 2πz z φ(z) = = . (2.42b) λ µ 6Due to the lengthy computation of discrete FT, Eq. (2.38) is for practical computations replaced by the Fast FT (FFT) algorithm, commonly available in commercial software packages. 7The produced real part of F (f ) is an even (symmetrical) function, whereas the imaginary part is an odd (antisymmetrical) function. This property is reflected on the amplitude A(f ) and phase φ(f ) profiles, which are even and odd with respect to zero frequency [120]. 8For a proper amplitude T (t), the expression for T (t) has to be multiplied with ∆t. 26 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 2.8: (a) The principle of pulsed phase thermography (PPT) and (b) presentation of the phase contrast curve with the frequency of the maximal phase contrast fch. Slika 2.8: (a) Princip delovanja pulzno-fazne termografije (PPT) in (b) prikaz krivulje faznega kontrasta in frekvence, pri kateri nastopi maksimalni fazni kontrast fch. Eq. (2.42a) directly implies that higher modulation frequencies are restricted to the near-surface region. Hence, for PT in CE applications, more energy should be concentrated at lower fre- quencies, which is provided by a longer heat pulse (see Eq. (2.37) and Fig. 2.7). In addition, by comparing Eq. (2.41) to (2.39) it is evident that both amplitudes as well as phases carry the information about the thermal waves diffusion length. However, it has been shown that amplitude images exhibit a low-pass filter behaviour – resolve the internal structure up to a particular depth, whereas phase images exhibit the band-pass filter behaviour – resolve the structure within a particular depth range [104]. Thus, phase images are less sensitive to non-uniform heating, variable surface emissivity [121] and probe approximately twice deeper than amplitude images [125], for which the maximal observation depth is restricted to approximately µ [126]. However, espe- cially at high frequencies, phase images are sensitive to noise [104], but as low frequencies are of interest in CE, phase images are of particular importance. For quantitative inversion of PPT data, methods such as neural networks, wavelets, as well as statistical methods have been proposed [104]. Recently, an approach based on the phase contrast calculation has been given by Ibarra-Castanedo [120, 127] (the phase contrast is defined in analogy with the thermal contrast given in (2.35), ∆φ(f ) = φdef (f ) − φsound(f) – see also Fig. 2.8). It states that defects may be detected up to a depth-dependent, so called blind frequency fb. According to the relation given in (2.42a), the value is smaller for deeper defects. By combining Eq. (2.42a) and (2.42b), Ibarra-Castanedo proposed the following equation for the estimation of defects depth r α zdef = C1 = C πf 1µ, (2.43) b Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 27 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. as well as an alternative approach for badly sampled data [127]. Here, α is the diffusivity of the material (sound area) and C1 is a correlation coefficient. C1 = 1 directly holds for amplitude images due to the low-pass filter principle, whereas for phase images, the reported values range up to 2 [128]. Nonetheless, both approaches failed in CE [129] and hence, Arndt9 introduced an alternative equation, based on the frequency of the maximal phase contrast fch [131, 123] (see Fig. 2.8), r α zdef = kc , (2.44) fch with the correlation factor kc that has to be assessed for various conditions. However, for both amplitude and phase images it has been shown that kc is almost independent of the heating time and may therefore be roughly set to 1 [130]. 2.5 Ultrasonic The ultrasonic pulse method (shortly ultrasonic) is a well established method in medicine and for the investigation of metals. However, as late as in the 1990s, ultrasonic was first applied in CE for thickness measurements of structural elements [18] and localisation of reinforcement and tendon ducts [54], as well as for the characterization of surface cracks [132] in concrete structures. The method is based on the generation, propagation and recording of either reflected or trans- mitted sound waves in the ultrasonic region, i.e. at frequencies above 20 kHz. For NDT-CE applications, the typical frequency range is 50–200 kHz [12]. Contrary to other acoustic methods use in NDT-CE (see section 2.1), ultrasonic uses piezoelectric transducers for both sound wave generation and recording. The features which distinguish ultrasonic from GPR are that ultra- sonic waves travel about 100,000 times slower than EMW and that they can easily penetrate materials such as metals [133]. On the other hand, the GPR signal is almost totally reflected by a metal. Ultrasonic can be performed by single point-contact measurements that in general suffer from being slow and require a good coupling provided by a coupling agent [11]. However, for the investigation of concrete structures using the ultrasonic echo technique (records the reflected ultrasonic waves), a multi-sensor ultrasonic echo array system has been used [22, 134]. Here, the system consists of several dry point-contact transducers which alternately act as a pulser or receiver. The collected data is then processed using the SAFT (Synthetic Aperture Focusing Technique) algorithm [135], which gives a 2D or 3D reconstructed image of the aperture covered by the array. 2.5.1 Acoustic wave theory Contrary to EMW, sound waves are mechanical waves that require a medium to propagate. Here, the sound energy is transferred by the deformation of matter, more exactly the oscillation 9The adaptation of PPT proposed by Arndt for qualitative and quantitative data analysis in CE was later termed square pulse thermography (SPT) in the frequency domain [130]. 28 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. of constituents close to its equilibrium place, which generates pressure changes. For an isotropic, homogeneous, linear, and dispersion-less material, the propagation of sound waves is governed by the acoustic wave equation which for the pressure p writes10 [133] ∂2p = v2 · ∇2p, (2.45) ∂t2 and has the general solution in the form of p(z, t) = p0 ei(ωt−βz), (2.46) where ω is the angular frequency and β is the phase constant. The dependence between the pressure p and the speed of sound v is given by ∂p v2 = , (2.47) ∂ρm where ρm is the mass density. The particles in solids can oscillate both in the direction of wave propagation as well as per- pendicularly to the direction, which results in longitudinal (or volumetric for 3D solids) and shear deformations, respectively. The quantity that describes the tendency of the material to deform under pressure is the modulus of elasticity. Particularly, the Young’s modulus E relates to longitudinal deformations, the shear modulus G to shear deformations and the bulk modulus K to volumetric deformations. With respect to the deformations induced in solids, longitudi- nal (compression) and transverse (shear) waves exist. In a 3D homogeneous solid, their sound velocities are given by the following equations [136] s s K + 4/ E (1 − µ) v 3 G l = = , (2.48a) ρm ρm (1 + µ)(1 − 2µ) s s G E vt = = , (2.48b) ρm 2ρm(1 + µ) where µ is the Poisson’s ratio11 which describes the ratio between the transverse and axial strain and ranges from 0 to 0.5 [137]. Considering this, vl is always larger than vt, according √ to vl ≥ 2 vt. The right side of the above equations are obtained by applying the following relations [133]: E K = , (2.49a) 3(1 − 2µ) E G = . (2.49b) 2(1 + µ) From Eq. (2.48a) and (2.48b) it is evident that in materials that cannot exhibit shear deformations 10The acoustic wave equation may be equivalently written for both the pressure p and the particle velocity u. 11Note that in this section, µ refers to the Poisson’s ratio and should not be related to magnetic permeability. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 29 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. with G = 0 (like liquids and gases), only longitudinal waves can be generated. The longitudinal wave velocities for some building materials are given in Table 2.3. Table 2.3: The typical range of longitudinal sound wave velocity vl of some building materials (Wendrich, 2009: 11). Preglednica 2.3: Okvirne vrednosti hitrosti longitudinalnih zvočnih valov vl v nekaterih gradbenih ma- terialih (Wendrich, 2009: 11). Material vl (m/s) Air 344 Concrete 3000–4830 Brickwork masonry 2000–3500 Sandstone 2300–2800 General stone 2000–4800 2.5.2 Reflection and refraction of acoustic waves Upon the propagation in a material, acoustic waves exhibit reflection and refraction from subsur- face discontinuities, similarly to EMW. The fraction of sound energy being reflected and trans- mitted is governed by the acoustic impedance of the medium. It is an intrinsic material property related to the mass density and speed of sound in the medium by the following relation [136] Z = ρmv. (2.50) For the ultrasonic measurement, differences in the acoustic impedance of the medium and discon- tinuities are of importance since this enables defect detection. For an ultrasonic wave incident on a planar boundary, the reflection coefficient r and the transmission coefficient t are given by [136] Z r = 2 − Z1 , (2.51a) Z2 + Z1 2Z t = 2 , (2.51b) Z2 + Z1 where Z1 and Z2 are the acoustic impedances of medium 1 and 2. 2.5.3 Acoustic attenuation and dispersion The acoustic wave theory introduced in section 2.5.1 is based on the assumption of an isotropic, homogeneous, linear, loss-less and dispersion-less material. As for the EM properties of building materials (see section 2.2.1), the first three conditions are for simplification in most cases assumed valid, whereas the acoustic attenuation and dispersion have to be taken into account. The attenuation of acoustic waves can be introduced in Eq. (2.46) in the same analogy as for EMW with the complex propagation factor k, such that ik = α + iβ, where here, α refers to the acoustic attenuation coefficient [133]. The pressure solution is then rewritten in the form given for the electric field in Eq. (2.4). The acoustic attenuation reflects on the sound velocity (2.50) by the introduction of a complex acoustic impedance, similarly as for the electrical impedance (2.13). 30 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. In general, the acoustic attenuation has several sources: the heat release due to deformation of the material caused by the travelling wave (also referred to as thermoelastic loss), the phonon viscosity phenomenon, as well as the Rayleigh scattering loss from aggregates, pores and impu- rities [133]. All terms are frequency-dependent, referring to the acoustic dispersion. Since the phonon viscosity phenomenon becomes important only at very high frequencies, it’s contribution is not considered for ultrasonic applications in NDT [133]. For low frequencies used in NDT-CE, the principal component is the Rayleigh scattering from objects with dimensions close to the sound wavelength. Here, the attenuation is strongly frequency-dependent (α ∝ ω4) [133]. It is interesting to notice that in concrete and masonry, the acoustic wavelengths are smaller than EM wavelengths for the used frequency range (the acoustic wavelengths are thus closer to the size of aggregates, pores and impurities). Therefore, for ultrasonic, the Rayleigh scattering is more pronounced than it is for GPR. Water molecules embedded in the material result in even stronger scattering and can greatly reduce the signal intensity. The effect of water on the scattering intensity was the scope of a study using GPR, CR and ultrasonic on brickwork masonry. The methods performance with respect to the moisture content of the masonry was taken under consideration. The results are presented in the article given in Appx. C. 2.5.4 Piezoelectric transducers For ultrasonic, piezoelectric transducers are used for both wave generation and receiving. Piezo- electricity refers to the phenomenon that when a piezoelectric material undergoes stress, both strain as well as a potential difference are produced between the opposite faces of the material producing an electrical signal [133]. As such, the conversion of electrical signals into mechanical vibrations is employed in the transmitting mode, and the conversion of mechanical vibrations into electrical signals in the receiving mode. The physics of the piezoelectric effect can be un- derstood in the way that when stress is applied to the material, a molecular dipole moment is induced (due to the asymmetric distribution of positive and negative charges). This generates a net polarization, which results in a potential difference [133]. Prior to the 1950s, piezoelectric crystals made from quartz were used as piezoelectric transducers. Nowadays, most piezoelectric ceramic materials used in NDT are based on the lead zirconate titanate (PZT) family. PZT exhibits good piezoelectric properties, is easy to manufacture in various shapes and sizes, can operate at low voltage and is usable up to 300 ◦C [138]. However, beside the piezoelectric element, additional components build the piezoelectric transducer in order to provide effective damping and enough mechanical energy. For the scope of this thesis, ultrasonic tomography was performed on masonry by single point- contact measurements (see Appx. C). Here, a coupling agent is required to provide the required contact. As noted in [139], several agents can be used, such as glycerine, vaseline and even water for horizontal inspections. Based on the results given by Wendrich when using ultrasonic on masonry [139], vaseline was chosen in our experiment since it yielded the strongest signal. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 31 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 3 NDT DATA FUSION WITH CLUSTERING 3.1 Introduction to data fusion According to Gros [62], NDT data fusion can be thought as multi-sensor fusion, i.e. synergistic use of information from multiple sensors/sources in order to facilitate data interpretation and increase the probability of defect detection. Its application in NDE emerged from many fields, including military applications (target tracking, airborne surveillance), robotics, remote sens- ing, pattern recognition and medical imaging [140]. Although it has been almost two decades since data fusion was introduced in NDE [63, 64] and an outstanding progress has been made in aerospace and nuclear industry, NDT data fusion remains a challenging task. Due to the ongoing development of new sensors and/or techniques, as well as an increasing number of po- tential applications of NDT, there is no general data fusion model, which could handle numerous applications. Moreover, due to the increasing requirements on the accuracy of NDT and NDE on one hand, as well as on the cost and time efficient inspections (resulting in incomplete data with limited resolution) on the other hand, the development of efficient fusion models is of major importance. In addition, a data fusion model for NDE-CE should account for the extremely heterogeneous material to be investigated and for an even larger variety of possible applications on-site. 3.2 A data fusion framework Data fusion can in general be performed on data sets of different dimensions – signals, 2D images or on the whole volume. Moreover, several modes of a single NDT method can be considered (e.g. multiple polarizations or frequencies), or several NDT methods. This thesis focuses on image fusion, as well as on data fusion from multiple NDT methods. Taking into account that different methods respond both complementarily as well as by over- lapping performance, a good data fusion model should integrate the detected features, where methods respond complementarily, but also discriminate different features detected by several sources [53]. To fulfil this task, several fusion algorithms have been proposed based on opti- mization methods, multi-resolution approaches, heuristic methods and probabilistic methods, which can be further classified to Bayesian inference, Dempster-Shafer (D-S) evidence and fuzzy logic. For image fusion, these methods can be introduced into a data fusion framework at three different levels [53]: • pixel-level fusion, • feature-level fusion, • symbol-level fusion. At pixel level, images are fused at each pixel individually without using pre-processing units. The information from fusion is obtained after image classification using optimization, multi- resolution or heuristic methods, as well as cluster analysis. On the other hand, feature and symbol level fusion are applied on a higher level, using the information obtained in the pre- processing step. Feature-level fusion tends to fuse features extracted by segmentation in the 32 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 3.1: Data fusion schemes at pixel-level fusion, feature-level fusion and symbol-level fusion. Slika 3.1: Združevanje podatkov na nivoju slikovne točke, z uporabo značilnic in na simbolnem nivoju. pre-processing step using e.g. clustering or neural networks, whereas symbol level fusion is in most cases based on sensor modelling. At both levels, the pre-processing units are fused using averaging or probabilistic methods. The concept of various data fusion schemes is illustrated in Fig. 3.1. One of the main restrictions in using optimization methods such as the LMMSE (linear minimum mean square error) approach is proper definition of the input LMMSE filter. Similarly, also the implementation of the alternative Kalman filter remains a complex problem for NDT applications. Multi-resolution approaches using particularly wavelets require an appropriate wavelet filter with definition of the decomposition level to be used. Similarly, the heuristic neural networks require a good training data set, which is in most cases hard to accomplish, especially for NDT-CE. For higher-level fusion schemes (feature- and symbol-level), which employ probabilistic methods, a challenging task remains to correctly assign the conditional and a priori probabilities, as well as mass functions, which govern the evaluation. Although many studies have proposed different approaches for its calculation [73, 71, 141, 74], none of the methods can work fully unsupervised. On the contrary, clustering methods require some sensitivity measures to be set, which, however, does not affect the analysis significantly as evident throughout our work, but they do not need prior knowledge on the statistical data distribution. This is especially promising in the field of NDT-CE, where knowledge about sample composition is limited due to the large diversity of specimens. The thesis focuses on clustering-based unsupervised image fusion. Throughout the work, clus- tering methods were introduced into various data fusion models both at a low level, as well as at a higher feature level. For the investigation of stone masonry walls and for the multi-sensor data set collected by the BetoScan robot system, clustering techniques were utilized for image segmentation at the pixel level. Here, this concept of image fusion was found more appropriate since the physical data contained in the images from various NDT methods were not linked to the Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 33 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. same spatial position (all methods did not give a depth dependant data visualization). Moreover, since the severity of damage was of interest rather than a complete visualization of the internal structure, an image fusion model based on segmentation was derived for defect detection. On the other hand, data obtained from several sources for the investigation of the laboratory concrete specimens were linked to a common spatial position. Thus, feature-level fusion was employed, where membership information from fuzzy clustering methods served as pre-processing units. In the case of the concrete specimens, a comparison with supervised symbol-level fusion was additionally performed as presented in the article given in Appx. A. Moreover, a feature- level based fusion model was also proposed for the derivation of smooth depth profiles of sodium content in the cement, where clustering methods were applied for the classification of spatially acquired LIBS spectra. Before introducing the concept of cluster analysis and presenting different clustering methods, a note to a complete data fusion framework will be given. The architecture of a generalized system for NDT data fusion can be divided into 3 main stages: data registration, data fusion and expert system reasoning [142]. Data registration is a prior step before data fusion and refers to the association of data from multiple sources to physical points on the specimen [53]. Data alignment to a common region as well as adjustment to a common spatial resolution are performed. In addition, signal/image processing steps may be employed at this level, such as signal enhancement, filtering and migration of GPR data, analysis of IR thermal images in the frequency domain, the inversion procedure of complex resistivity data, to name just a few. At the next step, a data fusion algorithm as presented above is applied. The last stage introduces the reasoning of an intelligent system, which refers to situation assessment and failure analysis [142]. The architecture of a generalized system for NDT data fusion is shown in Fig. 3.2. Figure 3.2: Schematic diagram of the architecture of a generalized system for NDT data fusion (modified after Gros et al., 1995: 2). Slika 3.2: Shematični prikaz arhitekture splošnega sistema za združevanje podatkov (prirejeno po Gros et al., 1995: 2). Among the three stages of the presented model, the last reasoning stage is particularly application dependant and should be applied at the final stage of a data fusion model. Thus, the development 34 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. of a data fusion framework within the thesis was restricted to the application of the first two steps – data registration and data fusion. The specific image processing steps of particular NDT methods are presented in chapter 4, as well as throughout the specific applications given in the appendices. The different data fusion algorithms are explained in detail in the appendices as well. However, their common basis, i.e. cluster analysis, is introduced in the following sections of this chapter, where different clustering methods are also presented. 3.3 Cluster analysis Clustering or cluster analysis means grouping a set of data (observations) into subsets (clus- ters, groups or classes). It can be done hierarchically (subsequent division into clusters and sub-clusters) or partitionally (determination of all clusters at once) according to the following criteria [143]: • Homogeneity of clusters, i.e. the members of a cluster should be as similar as possible. • Heterogeneity among clusters, i.e. members belonging to different clusters should be as dissimilar as possible. 3.3.1 Distance and similarity The term similarity, according to which clustering is performed, refers to differences in observa- tional properties (variables). A common measure of similarity between two observations x1, x2 is the Euclidean norm/distance d, which can be for M variables written as1 M X d2(x1, x2) = (x1i − x2i)2 = kx1 − x2k2 = (x1 − x2)T (x1 − x2). (3.1) i=1 The Euclidean distance, introduced in most clustering algorithms, implies a view of the data clusters with a hyper-spherical geometry. To avoid this constraint, a generalization has been proposed by the Mahalanobis distance, which induces hyper-ellipsoidal clusters with [144] d2(x1, x2) = kx1 − x2k2A = (x1 − x2)T A(x1 − x2), (3.2) where A refers to a norm-inducing matrix [145]. The different distance functions employed in fuzzy clustering algorithms are presented in Fig. 3.3. Note that the difference between the diagonal norm and the Mahalonobis norm is that the diagonal norm restricts the axes of the hyper-ellipsoid to the coordinate axes, whereas in the Mahalonobis norm, they are arbitrary. 3.3.2 Objective function-based clustering – a general note Partitional clustering builds clusters of data sets on the basis of the minimization of a certain objective function (thus the term objective function-based clustering ) and can therefore be treated 1A list of common distance functions can be found in Table 1.1 in [144]. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 35 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 3.3: The distance norms used in fuzzy clustering algorithms (Babuška, 2009: 70). Slika 3.3: Norme uporabljene v algoritmih za mehko združevanje (Babuška, 2009: 70). as an optimization technique. For a given data set X of N observations, where each consist of n measured variables2   x1,1 x1,2 · · · x1,N   x2,1 x2,2 · · · x2,N  X =    . . . .  , (3.3)  .. .. . . ..  xn,1 xn,2 · · · xn,N partitioning into c number of clusters is performed by iteratively assigning labels uij to column vectors in X by partition matrix U = [uij]c×N , so that the observations are moved in each step to the cluster, the centre of which is closest to them, until a convergence limit is reached. A generalized objective function (functional) J to be minimized in partitioning the data can be defined as the sum of the distances of N observations from their respective cluster centre v c X N X Jm(U, V; X) = um ij d2 ij , (3.4) i=1 j=1 where dij = kxj − vikA. Note that for the Euclidean distance, A is identity matrix. A graphical representation of partitioning a data set of two variables into three clusters is given in Fig. 3.4. With respect to the partition matrix U, the resulting partitions (and also clustering methods presented in section 3.3.3) may be classified as hard (or crisp), fuzzy probabilistic or fuzzy possibilistic [146]. Hard and fuzzy probabilistic partition matrices have to satisfy the conditions that each cluster is nontrivial (does not include all patterns and is nonempty), as well as that the total membership degree of each observation sums to 1. We state this by c X uij = 1, ∀j ∈ {1,...,N}, (3.5a) i=1 N X 0 < uij < N, ∀i ∈ {1, . . . , c}. (3.5b) j=1 2For X, the matrix notation may be replaced by a column vector notation for each observation xj = [x n 1j , x2j , . . . , xnj ]T , xj ∈ R so that the data set may be denoted by X = {x1, x2, . . . , xN}. 36 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 3.4: Representation of partitioning a data set of two variables into three clusters. The assignment of observations (data points) to a specific cluster (here, red, green or blue) is associated by the colour. For a specific observation xj, the distances to the three cluster centres v1, v2, v3 are marked as well. Slika 3.4: Prikaz gručenja množice podatkov dveh spremenljivk v tri razrede. Razporeditev posameznega podatka v ustrezen razred je na sliki ponazorjena z barvo razreda (rdeči, zeleni ali modri). Za podatek xj so prikazane razdalje do središč razredov v1, v2, v3. The main difference in hard vs. fuzzy partition, is that for hard partition, uij ∈ {0, 1}, (3.6) whereas for fuzzy, it follows that uij ∈ [0, 1], (3.7) i.e., for hard clustering, each observation belongs to a single cluster, whereas for fuzzy cluster- ing, observations may be associated to several clusters with a given membership degree (thus “probabilistic clustering”). Eq. (3.7) holds for both fuzzy probabilistic and possibilistic clus- tering. However, the main difference between the two is that possibilistic clustering drops the constraint that the memberships of each observation have to make a total of 1 over all clusters as in Eq. (3.5a). The functioning of hard vs. fuzzy clustering is illustrated in Fig. 3.5. 3.3.3 Objective function-based clustering algorithms 3.3.3.1 Hard clustering A very common hard clustering algorithm is the k-means algorithm [82], which uses the Eu- clidean distance given in (3.1) and satisfies the restrictions for the hard partition matrix given P P by (3.5a),(3.5b),(3.6), which yields simplified equation (3.4), J (V; X) = c N d2 . For the i=1 j=1 ij k-means, the cluster centres correspond to the mean value of their observations, which can be Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 37 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 3.5: The principle of hard and fuzzy clustering. Slika 3.5: Princip delovanja trdega in mehkega gručenja. for the i-th cluster centre defined as Ni P xk v k=1 i = (3.8) Ni where xk is an observation classified to the i-th cluster and Ni is the number of observations in the i-th cluster. 3.3.3.2 Fuzzy clustering In this chapter, first, the standard fuzzy c-means (FCM) algorithm will be introduced, followed by its extensions – the Gustafson-Kessel (GK) algorithm, which uses an adaptive distance measure, as well as noise and possibilistic clustering, which account for outliers. Fuzzy c-means clustering The FCM algorithm [83] with the FCM functional and the Euclidean distance defined by (3.1) and (3.4) can be regarded as the conventional fuzzy clustering algorithm since it forms the basis for other fuzzing algorithms. It is a probabilistic clustering algorithm, where uij describes the membership of observation xj to the i-th cluster. The “fuzzing” exponent or “fuzzifier” m introduced in (3.4) is a positive real number and determines the fuzziness of the resulting partition. It is recommended to range from 1.2 to 2 [143]. In our version of the FCM (and its extensions), m was fixed to 2, being the value most recommended by other authors [144, 145]. 3 3For the effect of the fuzzifier values on the membership function, refer to Figure 1.9 in [144]. 38 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. To minimize Jm, which represents a nonlinear optimization problem,4 it can be shown that uij and vi should be updated in each iteration by 1 uij = , (3.9a) c X(dij/dkj)2/(m−1) k=1 N X umij · xj j=1 vi = . (3.9b) N X umij j=1 Gustafson-Kessel clustering Several algorithms which employ an adaptive distance norm to overcome the restriction of hyper- spherical clusters of the FCM algorithm, so called “fuzzy c varieties”, have been proposed [143]. However, in the thesis, we refer to the GK algorithm [85], a probabilistic extension of the FCM algorithm, which uses the Mahalanobis norm (3.2) and the same functional as the FCM (3.4). 5 Here, each cluster has its own norm-inducing matrix. However, for a feasible solution, Ai of the i-th cluster should be calculated on the basis of the cluster volume ρi and the fuzzy covariance matrix Fi,6 A −1 i = [ρi det(Fi)]1/n · Fi , (3.10a) N X umij (xj − vi)(xj − vi)T j=1 Fi = . (3.10b) N X umij j=1 Since Gustafson and Kessel proposed the original GK algorithm, some extensions have been proposed, such as the adaptive fuzzy clustering (AFC) algorithm [148] or the GK algorithm with adaptive cluster volumes (GKAρ) [149]. However, since the AFC and GKAρ algorithms were found to produce poor partitions if little is known about the initial determination of the cluster volumes, we used the general GK algorithm with cluster volumes fixed to ρi = 1 [149]. To improve the numerical performance of the original GK algorithm (to account for the singularity problems 4The stacionary points of the FCM functional (3.4) are found with Picard iteration through the first-order conditions by adjoining the constraint (3.5a) to Jm by means of Lagrange multipliers and by setting the gradients of Jm with respect to U, V and λ to zero [146]. 5The Gath-Geva algorithm [147] is also a well-known extension of the FCM with adaptive distance measure. Here, the distance norm involves an exponential part and therefore decreases faster than the GK algorithm. However, since it enables the possibility to detect clusters of varying shapes and volumes, it requires good initial- ization. As the initialization with the FCM algorithm did not produce a proper solution, its performance is not considered in the thesis. 6For a feasible solution, matrix Ai should vary in the optimization process with its determinant fixed (corre- sponding to the optimization of the cluster shape with a fixed volume ρi). Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 39 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. of F), we used the algorithm proposed by Babuška et al. [150], where two additional parameters (β and γ) are introduced. Parameter β limits the condition number of F, and γ ∈ [0, 1] is the tuning parameter which constraints the clusters to have a more or less equal shape. The GK algorithm cannot be used in a fully automatic mode as some preliminary information about the cluster size needs to be given. Kruse et al. [146] recommended using the final FCM partition for the initialization of GK partitioning. This was also used in our version of the algorithm. Noise clustering The sensitivity of the FCM to outliers can be overcome by using noise clustering [84], where an additional noise cluster is introduced. For the noise centre vc, the distance of the observation xj to vc is dcj = δ for each observation. Therefore, the distances in Eq. (3.4) and (3.9a) should be replaced according to dij = kxj − vik for 1 ≤ j ≤ N, 1 ≤ i ≤ c − 1, (3.11a) dij = δ for 1 ≤ j ≤ N, i = c, Pc−1 PN d2 δ2 = λ · i=1 j=1 ij . (3.11b) N (c − 1) For the calculation of δ, parameter λ was estimated by using the standard deviation of the inter- point distances [84]. However, to achieve the best partition, we further multiplied this parameter by factor L, using 1 as the default value (the values taken into account for this parameter will be discussed within the results). Similarly as for the FCM, noise clustering is applicable to the GK or the k-means algorithms. Possibilistic clustering Apart from noise clustering described above, fuzzy possibilistic clustering algorithms are able to deal with noisy data. Since they drop the constraint that the memberships of each observation have to sum to 1 over all clusters (see section 3.3.2), memberships should rather be interpreted as the typicality of a data point to a specific cluster [146]. The original possibilistic c-means (PCM) algorithm by Krishnapuram and Keller [151] helps to identify outliers, but at the same time is also sensitive to initialization and very often generates coincident clusters. Therefore, we refer to two modifications of the PCM. In 1997, Pal et al. proposed a mixed c-means approach called the fuzzy-possibilistic c-means (FPCM) algorithm [152]. The model is based on generating both membership and typicality values. Typicalities to a cluster are constrained to sum to 1 over all data points, which however was found to produce unrealistic results for large data sets. To avoid the problems of the FPCM, the same authors proposed a new model, possibilistic-fuzzy c-means (PFCM) [87]. Since NDT data sets are relatively large, only the improved PFCM algorithm will be taken into account, 40 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. where the following objective function should be minimized c X N X c X N X Jm,η(U, T, V; X, γ) = a um ij + b tη γ (1 ij · d2ij + i − tij)η , (3.12) i=1 j=1 i=1 j=1 where a > 0, b > 0, m > 1 and η > 1 are user-defined parameters and γi should be calculated by [151] N P umij · d2ij j=1 γi = K · , K > 0. (3.13) N P umij j=1 To minimize Jm,η, the necessary conditions for memberships uij, typicalities tij and cluster centres are 1 uij = , (3.14a) c X (dij/dkj)2/(m−1) k=1 1 tij = , (3.14b) 1/(η−1) 1 + b · d2 γi ij N X a um ij + b tη ij · xj j=1 vi = . (3.14c) N X a um ij + b tη ij j=1 In our algorithm we used the final FCM partition for initialization as for the GK. The default parameter values were set to a = 1, b = 1, m = 2 and η = 2. While optimizing these parameters, m remained fixed to its default value since this was the most suitable choice made by the authors of the FCM. Besides using the mixed c-means approach, we employed a PCM variant using a repulsion term proposed by Timm et al. [86]. The objective function of PCM with repulsion is given by the following equation c X N X c X N X c X c X Jm(T, V; X, γ, ω) = tm ij · d2 ij + γi (1 − tij)m + ω · 1/d2(vi, vk) , i=1 j=1 i=1 j=1 i=1 k=1,k6=i (3.15) Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 41 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. where ω is a user-defined parameter. Here, the necessary conditions are given by 1 tij = , (3.16a) 1/(m−1) 1 + 1 · d2 γi ij N X c X tij · xj − ω · vk/d2(vi, vk) j=1 k=1,k6=i vi = . (3.16b) N X c X tij − ω · 1/d2(vi, vk) j=1 k=1,k6=i As for the PFCM, the final FCM partition was used for initialization. 3.3.4 Validity measures All clustering algorithms described above require the number of clusters c to be determined in advance. The selection of the optimal cluster number, which yields the best partition is a user- defined parameter. However, to perform partitioning without subjective judgement, one can use validity measures to assess the suitability of a partition for a different c, taking into account both compactness and separation between clusters [83]. Compactness should be thought of as a measure of data scattering within a cluster and therefore the optimum c should minimize the compactness, but maximize the separation between clusters [78]. Several quality validity indices have been introduced (see e.g. Abonyi and Feil [153]). In our case, we used three of them that account for both compactness and separation of clusters, as well as fuzziness: 1. The partition index SC(c), describing the ratio of the sum of compactness and separation of the clusters, where normalization is done by the fuzzy cardinality of each cluster Ni [78]: N X umijkxj − vik2 c X j=1 SC(c) = . (3.17) c X i=1 Ni kvi − vkk2 k=1 2. The separation index S(c), which uses the same nominator as SC, but the minimal separation between clusters as the denominator [78]: c X N X umijkxj − vik2 i=1 j=1 S(c) = . (3.18) N · mini,k,i6=k kvi − vkk2 42 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 3. Xie and Beni’s index XB(c), which uses the same nominator as the SC and S indices, but the minimal distance of data points to cluster centres as the denominator [154]: c X N X umijkxj − vik2 i=1 j=1 XB(c) = . (3.19) N · mini,j kxj − vik2 As fuzzy cardinality normalization of the SC index makes it insensitive to cluster size, the S and XB indices were taken into account when searching for the optimal number of clusters, whereas the SC index was used to estimate the quality of different partitions for a specific number of clusters. A good partition should minimize all three validity indices. Although the above validity indices have been proposed to automatically define the optimal number of clusters, they are all to some extent affected to monotonicity and can derive conflicting results [144]. In the case of the results of most studied NDT data sets, they did not deliver a proper interpretation of the results. Here, the number of clusters was rather defined according to the suspected severity of the damage and the heterogeneity of the inspected material (refer to the results). 3.3.5 Objective function-based clustering algorithm scheme A generalized objective function-based clustering algorithm scheme starts with a random group- ing of observations into clusters based on the initialized partition matrix (the minimum number of clusters to be chosen is 2 and all have to be non-empty). In the following steps, the observa- tions are moved to the cluster whose centre is closer to them, resulting in a change of the overall association and movement of cluster centres. The iterative computation (update) of the cluster centres and the partition matrix is performed until no further movement occur, i.e. when a con- vergence is reached. Since clustering algorithms converge to a local minima of the functional, the result is dependent on the initial random partition. Thus clustering should be repeated several times. For the initialization of the partition matrix, a simple approach is a random selection of the cluster centres and the computation of the corresponding partition matrix [145]. This approach was also used in our algorithms. The termination criterion is reached when the norm of the difference between two successive partition matrices Uiter+1 and Uiter does not exceed a certain termination tolerance ε. The choice of ε is application oriented and was for our purposes fixed to 10−4 [144], whereas the maximum allowed iterations were set to 200. The algorithm scheme for a generalized clustering algorithm (based on the calculation of cluster centres and the partition matrix) is presented in Fig. 3.6. The version of the FCM and GK algorithms we employed is based on the algorithms proposed by Balasko et al. [155], whereas the noise and possibilistic clustering algorithms were written by ourselves. The clustering algorithms as well as the complete data fusion frameworks were implemented in Matlab R2012a. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 43 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 3.6: Flowchart of a generalized clustering algorithm. Slika 3.6: Diagram poteka splošnega algoritma za gručenje podatkov. 44 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 45 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 4 NON-DESTRUCTIVE TESTING OF LABORATORY CONCRETE SPECI- MENS WITH INBUILT DEFECTS AND INCLUSIONS In this chapter, a sensitivity evaluation of GPR and IRT is performed based on the results from laboratory concrete specimens. The outcomes of this study served for further symbol level fusion of GPR and IRT data as presented in the article in Appx. A. Although specimen description and methods are given in the corresponding article, they are reviewed herein for better understanding of the presented results. It should be mentioned that for the task of this study, concrete specimens were preferred over masonry since they have a more homogeneous structure, which enables a more controlled study of the effects material properties have on the detection of defects. As a prerequisite for data fusion of GPR and IRT data along the depth, an analysis of the pos- sibilities of data characterization using IRT was performed. Techniques based on the thermal contrast and PPT as presented in section 2.4.4, were used. Apart from the search for a quanti- tative depth retrieval, the thermal contrast was used to study the effects of typical defects and inclusions in building structures on the performance of IRT. 4.1 Experimental 4.1.1 Specimen description For this study, four laboratory concrete specimens were constructed. All have a size of 50×50×15 cm3 and were made from the same concrete. Due to their complex internal structure, self- consolidating concrete was used with a maximum aggregate size of 4/8 mm (crushed limestone was used as aggregate). A cement type CEM I 42.5R and a superplastificator Sika Viscocrete 5 NPL were used. As can be seen in Fig. 4.1, all specimens contain inbuilt anomalies varying in material (polystyrene, air and water) and depth below the surface. Defect specifications are given in Table 4.1. The anomalies in specimen S0 served to study the methods’ sensitivity with respect to different defects depth. Anomalies in specimens S1, S2 and S3 feature some of the most typical defects and inclusions in building structures, such as localized water and the presence of voids (S1), reinforcement (S2), as well as pipes and delamination (S3). The inbuilt cuboids in specimens S0 and S2 have a size of 8 × 8 × 6 cm3, whereas the size of voids in S1 is 8 × 8 cm2 with varying depth (the voids were from one side open in order to fill them with water). The reinforcement in S2 is of two densities (horizontal distance between bars is 5 and 10 cm, respectively) with a concrete cover of 2.5 cm. The polystyrene plates that simulate delamination in specimen S3 have a thickness of 1 cm (a) and 2 cm (b), and the diameters of the plastic pipes are 1, 1.5 and 2.5 cm (from left to right in Fig. 4.1, S3). All defects in specimen S3 have a concrete cover of 1.5–2 cm. It is worth mentioning that artificial voids of various shapes and sizes particularly serve to study honeycombs in concrete structures, as well as air voids in masonry structures. Honeycombs are quite common in practice, but are very difficult to create in a laboratory during casting the concrete. Therefore, they are often simulated by polystyrene voids or rubber balls [8]. 46 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.1: Sketches of laboratory concrete test specimens with inbuilt defects. Slika 4.1: Skice laboratorijskih betonskih preizkušancev z vgrajenimi anomalijami. Table 4.1: Material and concrete cover of inbuilt defects in specimens S0–S3. Preglednica 4.1: Material in krovni sloj betona vgrajenih anomalij v preizkušancih S0–S3. Defect Defect material Concrete cover (cm) S0 a polystyrene 3/6 S0 b polystyrene 1.5/7.5 S0 c,d polystyrene 4.5 S1 a air 6 S1 b air 3 S1 c water 6 S1 d water 3 S2 a,b polystyrene 6.6 S2 c,d polystyrene 4.6 S3 a,b polystyrene 1.5–2 S3 pipes plastic 1.5–2 4.1.2 Methods For IRT, two IR heaters (1.2 kW each) were used for heating the specimens surface at a distance of 45 cm. Within the optimization of the heating time, the specimens were heated for 5, 15, 30 and 45 min. Thermographic data was acquired with a FLIR A320 IR camera (Fig. 4.2) (for camera speci- fications, see [156]). The camera detects IR radiation in a spectral range of 7.5–13 µm with a bolometer-type focal plane array having a resolution of 320×240 pixels and a thermal sensitivity Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 47 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. of 50 mK (at 30 ◦C). Data can be acquired at a frequency of 30 Hz. The cooling down behaviour was monitored for 45 min (for the PPT technique, 150 min as a longer observation time is required to obtain a smaller frequency increment in FT – see sec- tion 2.4.4). At a frequency of 30 Hz such long observation time would produce a data file of some 10 GB. Thus, the frequency of image registration was reduced to 0.2 Hz, which gave a data file of approximately 1 GB. For controlling the camera and for setting the parameter values of a thermographic measurement, FLIR ResearchIR Max 3.2 was used. Data processing was further performed with Matlab R2012a. Figure 4.2: The IR camera FLIR A320 used for the thermographic measurements. Slika 4.2: Termokamera FLIR A320, ki smo jo uporabili pri raziskavah. The GPR data was obtained with the equipment from MALÅ Geoscience, using a 1.6 GHz monostatic shielded antenna (Fig. 4.3). A calibrated survey wheel was used and radargrams were collected in both directions with a line spacing of 5 cm and a point distance of 0.5 cm. Figure 4.3: The radar system used for the GPR measurements. Slika 4.3: Georadarski sistem MALÅ Geoscience, ki smo ga uporabili pri raziskavah. The GPR data was collected in forms of a 3D data set. Data processing was performed with the Reflexw commercial software, Ver. 6.0.5 [157]. First, a proper profile coordinate assignment was determined. Next, header gain removal, DC-shift and static correction (correction of the maximal phase and start time movement) were applied. As the detection of very shallow defects was of interest, the true zero position of A-scans was determined carefully. Based on the results from many previous test measurements, the 1st minimum was selected as the true zero position. A detailed description on the selection of the true zero position can be found in [158]. 48 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 4.2 Results 4.2.1 Characterization of defects and inclusions using infrared thermography The surface temperature distribution of specimen S0 immediately after heating, as well as after 15, 30 and 45 min of cooling is presented in Fig. 4.4. It can be seen that the time of defect detection is directly related to defects depth. The shallowest defects a and b (at a depth of 1.5 cm and 3 cm) appear immediately after heating, whereas the more deeper defects c and d (at a depth of 4.5 cm) appear after 15 min of cooling. Figure 4.4: Thermograms of specimen S0 at different cooling down times. Each image has a different colour scaling. Slika 4.4: Termogrami za preizkušanec S0 pri različnih časih ohlajanja. Vsaka slika ima svojo tempera- turno skalo. 4.2.1.1 Thermal contrast The time dependent thermal contrast C(t) is for the S0 defects at different depths presented in Fig. 4.5 after 30 min of heating. It can be seen that with increasing defects depth, the thermal contrast decreases, whereas the time of the maximal thermal contrast tCmax increases. Moreover, from the graph it is evident that defects up till a depth of 6 cm could be detected (the defect at a depth of 7.5 cm has a negative Cmax and thus was not detected). Figure 4.5: The time dependent thermal contrast C(t) for defects in specimen S0 after 30 min of heating. Slika 4.5: Potek časovno odvisnega temperaturnega kontrasta C(t) za različno globoke anomalije v preizkušancu S0 po 30 min segrevanju. Besides defects, localized water or higher moisture content as well as reinforcement may be present in building structures, which could affect the detection of defects with an IR camera. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 49 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Therefore, for specimens S1 and S2 the effect of both localized water and reinforcement of two densities was analysed. In Fig. 4.6a it can be seen that for a water anomaly, the thermal contrast represents only half of the thermal contrast obtained for the air defect. Hence, the detection of water is greatly reduced compared to air, whereas on the other hand, the change in tCmax √ between water and air is negligible. The latter is in accordance with Eq. (2.34a), z ∝ α · tCmax , which relates tCmax for a given defect depth only to the thermal diffusivity of a sound area, here concrete. However, the observed dependence of Cmax on the thermal properties of defects is p contrary to Eq. (2.34b), z ∝ 3 1/Cmax . Nevertheless, a decrease in Cmax could be explained by higher thermal conductivity of water compared to air, and thus by a smaller difference in thermal conductivity between the defect and sound area (thermal conductivity of air is 0.026 W/m K, of water 0.6 W/m K and of concrete 2.1 W/m K). Similar conclusions can be made for results in Fig. 4.6b. Here, it is evident that reinforcement with a horizontal distance of 10 cm between bars affects the detection of a polystyrene defect at a depth of 4.6 cm only slightly. However, the thermal contrast is reduced by approximately 30 % when a dense reinforcement with a horizontal distance of 5 cm between bars is present. The reduction in thermal contrast when reinforcement is present is in accordance with higher thermal conduction of iron compared to concrete. (a) (b) Figure 4.6: The time dependent thermal contrast C(t) for defects in specimens S1 and S2 after 30 min of heating: (a) for a water and an air defect at a depth of 3 cm, and (b) for a polystyrene defect at a depth of 4.6 cm with present reinforcement of two densities (5 and 10 cm). Slika 4.6: Potek časovno odvisnega temperaturnega kontrasta C(t) za anomalije v preizkušancih S1 in S2 po 30 min segrevanju: (a) za vodno in zračno anomalijo na globini 3 cm ter (b) za anomalijo iz stiropora na globini 4,6 cm in ob prisotni armaturi rastra 5 oz. 10 cm. The above results indicate that the thermal contrast may be greatly affected by the type of anomaly, whereas tCmax is mostly affected by the depth of defects. Thus, the interpretation of both Cmax and tCmax could qualitatively characterize the defect, i.e. determine the material properties of the defects as well as their depths. This can be achieved by contrast images and timegrams introduced in section 2.4.4.1. Here, each pixel of the image is associated with the value of Cmax and tCmax, respectively. Contrast images and timegrams of all specimens are shown in Fig. 4.7–4.10. From the timegram in Fig. 4.7b it is clear that the defects in S0 appear at different depths, whereas the defects in the timegrams in Fig. 4.8 and 4.9 relate to the same depth. Here, contrast images reveal different material properties of the defects. In addition, 50 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. (a) (b) Figure 4.7: (a) A contrast image and (b) a timegram of specimen S0 after 30 min of heating. Slika 4.7: (a) Kontrastna slika in (b) časovna slika za preizkušanec S0 po 30 min segrevanju. (a) (b) Figure 4.8: (a) A contrast image and (b) a timegram of specimen S1 after 30 min of heating. Slika 4.8: (a) Kontrastna slika in (b) časovna slika za preizkušanec S1 po 30 min segrevanju. (a) (b) Figure 4.9: (a) A contrast image and (b) a timegram of specimen S2 after 30 min of heating. Slika 4.9: (a) Kontrastna slika in (b) časovna slika za preizkušanec S2 po 30 min segrevanju. (a) (b) Figure 4.10: (a) A contrast image and (b) a timegram of specimen S3 after 30 min of heating. Slika 4.10: (a) Kontrastna slika in (b) časovna slika za preizkušanec S3 po 30 min segrevanju. the contrast image in Fig. 4.8a slightly detects even the air defect at a depth of 6 cm (top left defect), which could not be observed in the thermograms. However, the greatest advantage of Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 51 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. this data processing technique is that a qualitative characterization of defects may be performed with only two images, rather than with the whole sequence of thermograms. Therefore, data evaluation is greatly simplified and the time needed for data analysis can be reduced. Both is of high importance for the application of thermography in practice. Regardless of the specimen and defects inspected, the detection of defects is also dependent on the heating time. The results of Cmax and tCmax with respect to defect depths and different heating times are presented in Fig. 4.11. It can be seen that for all defects, the optimal heating time is 30 min, since it yields the highest thermal contrast and the deepest penetration (up till 6 cm). It is interesting to notice that both shorter (15 min) and longer (45 min) heating times gave smaller thermal contrast and penetration (up till 4.5 cm). These results are not in accordance with the results from concrete specimens as reported in [41], where defects at a depth of more than 4 cm were taken into account. Here, higher thermal contrast was obtained by increasing the heating time. An explanation for lower values of Cmax at 45 min of heating time could be that above the more shallow defects, more heat started to accumulate also in the surrounding. This could reflect in the changed behaviour of the reference profile. (a) (b) Figure 4.11: Dependence of (a) the maximal thermal contrast Cmax and (b) its corresponding time tCmax on the depth of defects for different heating times (15, 30 and 45 min). The dotted lines are guides to the eye. Slika 4.11: Odvisnost (a) maksimalnega temperaturnega kontrasta Cmax in (b) časa njegovega nastopa tCmax od globine anomalije za različne čase gretja (15, 30 in 45 min). Črta le povezuje merske točke za lažje spremljanje rezultatov. As can be seen in Fig. 4.11, Cmax and tCmax exhibit a particular dependence on the depth of defects. This was tried to be modelled by Eq. (2.36) proposed by Balageas et al. from the testing of carbon-epoxy composites. The fitting results for the optimal heating time of 30 min are presented in Fig. 4.12. It is evident that the model cannot properly describe the dependence of Cmax and tCmax on the depth of defects (the coefficient of determination is R2 = 0.25). For this, several explanations are possible. The heating time was much longer compared to the heating times used for the inspection of composite structures. Therefore, the temperature increased inside the specimen already during heating. Moreover, in our experiment, different boundary conditions were present (convective and radiating losses occurred only at three edges of the specimen), which could particularly affect the surface temperature evolution of the areas 52 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. above defects. In addition, non-ideal heating, as well as inaccurate estimation of Cmax and tCmax at larger depths (see Fig. 4.5), could contribute. It is worth mentioning that also slight tilting of the defects during concreting could occur, which reflects in a small deviation of defect depths from the reference given in Table 4.1. However, the problem of fitting experimental values to this model was also stressed by Maldague [104] for the results obtained on metal and composite structures. As a more accurate parameter than tCmax, Maldague suggests using tC1/2max, when half of Cmax is obtained. Moreover, the use of the inverse technique based on a numeric model is recommended. However, particularly for CE structures, the latter concept could be hardly brought into practice. In most cases, thermographic measurements are not performed under well-defined conditions. The heating process can hardly be modelled properly and the thermal properties of structures may differ to a large extent. Moreover, regardless of the data analysis technique used, the poor depth penetration of the thermal contrast remains. Therefore, the use of PPT, which was found a powerful technique both in aerospace applications [128] as well as in CE [130], was examined. Figure 4.12: The fitting result for Cmax and tCmax using the model proposed by Balageas et al. (2.36). Experimental results for defects in specimen S0 obtained at a heating time of 30 min were used. Slika 4.12: Rezultat prilagajanja eksperimentalnih podatkov za Cmax in tCmax z modelom po enačbi (2.36). Vključeni so rezultati za anomalije v preizkušancu S0 za čas gretja 30 min. 4.2.1.2 Pulsed phase thermography When using PPT, amplitudes and phases are obtained from the thermogram sequence, as ex- plained in section 2.4.4.2. The phase contrast ∆φ of the area above defects at a depth of 1.5 cm and 3 cm in specimen S0 is presented in Fig. 4.13. It can be seen that for deeper defects, the max- imal phase contrast appears at smaller frequencies fch, which is in accordance with the definition of the thermal diffusion length µ and the phase of thermal waves as given by Eq. (2.42a) and (2.42b). Similar behaviour was already observed by Castanedo when testing a carbon fibre rein- forced plastic (CFRP) specimen [128] and by Arndt when testing concrete specimens [129, 130]. The results of fch for all defects in specimen S0 are given in Table 4.2. It should be noted that for a better estimation of fch, zero-padding [122] up till 213 points was used before applying FFT. For zero-padding, the last recorded temperature of each pixel was used. Although relatively many “artificial” points were added to the thermogram sequence, this only slightly affected the outcome of FFT as the surface temperature was almost stabilized after 150 min of monitored Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 53 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. cooling. On the other hand, a much smaller frequency increment of ∆f = 2.4 · 10−5 Hz could be obtained. (a) (b) Figure 4.13: The frequency dependent phase contrast ∆φ for (a) the defect at a depth of 1.5 cm and (b) the defect at a depth of 3 cm in specimen S0 after 45 min of heating and 150 min of cooling. Slika 4.13: Potek frekvenčno odvisnega faznega kontrasta ∆φ za (a) anomalijo na globini 1,5 cm in (b) za anomalijo na globini 3 cm v preizkušancu S0 po 45 min segrevanju in merjenju ohlajanja za 150 min. Table 4.2: The frequency of the maximal phase contrast fch for defects in specimen S0. Preglednica 4.2: Frekvenca pri maksimalnem faznem kontrastu fch za anomalije v preizkušancu S0. zref (± 0.5 cm) f ch (Hz) 1.5 1.60 ·10−3 3 4.3 ·10−4 4.5 1.9 ·10−4 6 1.4 ·10−4 7.5 1.0 ·10−4 Fig. 4.14 presents the results obtained when fitting experimental data for fch in Table 4.2 to Eq. (2.44). It can be seen that the data fits the model with high accuracy (the coefficient of determination is R2 = 0.99). Here, the value of the correlation factor kc from Eq. (2.44) is kc = 1.11. According to Arndt [129], this parameter is only slightly dependent on the heating time and in addition, for a rough estimate of the defect depths, kc = 1 is a good approximation. Moreover, as can be seen in Fig. 4.14, even the deepest defect (at a depth of 7.5 cm) can be observed using PPT. The fact that Eq. (2.44) can properly describe the dependence of fch on defect depths and that kc = 1 is a good approximation makes it possible to consider phase images at given frequencies as depth slices. This concept was used for the fusion of GPR and IR phase images as described in the article given in Appx. A. At this point it is also worth mentioning the good performance of phase images or rather phase contrast images, which were used in the data fusion process. Here, each pixel is associated with the value of the maximal phase contrast, similarly as contrast images refer to the values of the maximal thermal contrast. In Fig. 4.15, the thermogram obtained directly after heating (a), the contrast image (b) and the phase contrast image at a frequency of 3.7 · 10−3 Hz (c) are presented 54 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.14: The fitting result for fch using the model proposed by Arndt et al. (2.44). Experimental results for defects in specimen S0, obtained after 45 min of heating and 150 min of cooling, were used. Slika 4.14: Rezultat prilagajanja eksperimentalnih podatkov za fch z modelom po enačbi (2.44). Vključeni so rezultati za anomalije v preizkušancu S0, dobljeni po 45 min segrevanju in merjenju ohlajanja za 150 min. for specimen S3. Here, all defects appear at a depth of 1.5–2 cm. It can be seen that the signal- to-noise ratio of the thermogram is too low to detect the plastic pipes. However, the thermogram reveals that part of specimen surface above the polystyrene plates is warmer, which could be referred to as a slight tilting of plates during concreting. On the other hand, the contrast image has higher signal-to-noise ratio than the thermogram, but exhibits poorer spatial resolution since it does not detect the tilted plates. The phase contrast image exhibits both high signal-to-noise ratio as well as extremely good spatial resolution. Both the plastic pipes and tilted plates can be visualized well. Figure 4.15: IR images obtained for specimen S3: (a) the thermogram obtained directly after heating, (b) the contrast image and (c) the phase contrast image at a frequency of 3.7 · 10−3 Hz. Slika 4.15: IR slike za preizkušanec S3: (a) termogram ob začetku snemanja, (b) kontrastna slika in (c) fazno-kontrastna slika pri frekvenci 3, 7 · 10−3 Hz. The good performance of phase contrast images can also be seen in Fig. 4.16 for specimen S1 (a) and S2 (b). The images also show that the frequency fch is independent (or only slightly dependent) on the thermal properties of defects since both air- and water defects can be observed at the same frequency (Fig. 4.16a). This is in accordance with Eq. (2.44) and (2.34a), relating fch and tCmax only to the defect depths. Moreover, it is evident that the water defect reduces the Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 55 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. phase contrast compared to air, and the same is true for the detection of the defect below dense reinforcement. Hence, similar as the thermal contrast, the phase contrast shows dependence on the thermal conductivity of defects. This is also in accordance with Eq. (2.42b), relating the phase to thermal conductivity. Figure 4.16: The phase contrast images at (a) a frequency of 6.2 · 10−4 Hz for specimen S1 and (b) at a frequency of 3.0 · 10−4 Hz for specimen S2. In Fig. 4.16a, the upper defect refers to air (b), the lower to water (d). In Fig. 4.16b, the right part of the image refers to the part of the specimen with dense reinforcement. The red squares mark the defects position, labelled according to Table 4.1. Slika 4.16: Fazno-kontrastna slika pri (a) frekvenci 6, 2 · 10−4 Hz za preizkušanec S1 in pri (b) frekvenci 3, 0 ·10−4 Hz za preizkušanec S2. Na sliki 4.16a je zračna anomalija zgoraj (b), vodna anomalija pa spodaj (d). Na sliki 4.16b je gostejša armatura na desni strani. Rdeči kvadrati označujejo območje anomalij (za oznake anomalij glej preglednico 4.1). 4.2.1.3 Numerical simulation The conclusions drawn from the analysis of experimental data using the thermal contrast and PPT were further evaluated using numerical modelling. The work was performed with the help of Dr. Dejan Kolarič (UL FGG) using Mathematica. In particular, a 2D model was analysed (Fig. 4.17), simulating two cross-sections of a concrete specimen measuring 50 × 50 × 15 cm3 with inbuilt polystyrene defects. However, note that the arrangement of defects is not the same as in specimen S0. Here, defects appear at depths 1.5, 3.5, 6 and 7.5 cm. Modelling specifications For the simulation, finite differences were used and the second order Crank-Nicolson method with uniform partition in space and time. The space increment (the difference between two consecutive space points) along the x-axis was ∆x = 0.9375 mm and along the y-axis ∆y = 1.6892 mm, whereas the time increment was set to ∆t = 0.5 s. Apart from heat conduction governed by Eq. (2.32), radiation (2.26) and convection (2.29) losses were taken into account at all edges. For the average convective heat transfer coefficient ¯ h = 10 W/m2 K was used and the surface emissivity was set to 0.91 (for concrete). A heat flux density of j = 600 W/m2 on the specimen surface was determined based on experimental results. The temperature of the surrounding was set to 293.15 K. For concrete, a thermal diffusivity of α = 8.86 · 10−7 m2/s (ρm = 2371 kg/m3, cp = 1000 J/kg K, k = 2.1 W/m K) was used, whereas for polystyrene α = 1.0 · 10−6 m2/s (ρm = 25 kg/m3, cp = 1400 J/kg K, k = 0.035 W/m K) was used. The heating time was set to 30 min, whereas the cooling down was monitored for 120 min. 56 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.17: A 2D numerical model simulating two cross-sections of a concrete specimen with inbuilt polystyrene defects (white colour). The points on the surface where temperature was monitored are marked with red dots. Slika 4.17: 2D numerični model, ki ponazarja dva prereza betonskega preizkušanca z vgrajenimi anoma- lijami iz stiropora (belo območje). Točke na sprednji strani modela, kjer smo spremljali temperaturo, so označene z rdečimi pikami. Modelling results During the simulation, the temperature evolution was monitored at selected points on the sur- face of the model (marked with red dots in Fig. 4.17). Surface points above defect areas, as well as reference surface points were taken into account. The simulated data were evaluated using the thermal contrast and the PPT technique in the same way as for the evaluation of the results of the experiment (see the preceding sections). The results for Cmax, tCmax and fch are summarized in Table 4.3. It can be seen that all results obtained from simulation match the experimental values very well (compare to Fig. 4.11 and Table 4.2). However, the simulation yields slightly higher values of the thermal contrast and as a result, enables the detection of the deepest defect. This could mostly be related to inaccurate modelling of the material as well as of the heating process. Moreover, the neglect of additional effects present in the experiment could have a contribution. However, since the simulation yields also smaller values of tCmax, the sound material (concrete), was probably modelled with higher thermal conductivity than the thermal conductivity of the investigated specimens. The fact that an increase in thermal conduc- tivity of the sound material results in a smaller tCmax is in accordance with Eq. (2.34a), where √ z ∝ α · tCmax . Since the thermal front propagates faster, it reaches the defect earlier. This dependence of tCmax on the thermal conductivity of concrete was also noticed by Maierhofer et al. [45]. The discrepancy in the thermal conductivity of the modelled concrete compared to the concrete of the measured specimens is also in accordance with the higher values of fch obtained Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 57 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Table 4.3: The simulation results of Cmax, tCmax and fch for defects at depths of 1.5, 3.5, 6 and 7.5 cm (see Fig. 4.17). Preglednica 4.3: Rezultat simulacije za Cmax, tCmax in fch za anomalije na globinah 1,5, 3,5, 6 in 7,5 cm (glej sliko 4.17). zref (cm) Cmax (K) tCmax (s) f ch (Hz) 1.5 9.28 40 1.74 ·10−3 3.5 2.21 430 4.6 ·10−4 6 0.65 1220 1.7 ·10−4 7.5 0.12 1550 1.5 ·10−4 p during simulation. According to Eq. (2.44), where z ∝ α/fch , higher thermal diffusivity (and thus thermal conductivity) results in a higher fch for a given depth. The time dependent thermal contrast C(t) from the simulation is presented in Fig. 4.18a for all defects. Compared to the evolution of the thermal contrast for defects in specimen S0, the simulated thermal contrast exhibits faster decrease in time. This could be again attributed to different thermal conductivity of the modelled concrete or, even more likely, to higher thermal conductivity of modelled polystyrene. However, it is not easy to derive certain conclusions, since for the evolution of the thermal contrast in time, also material density and specific heat (or rather thermal diffusivity) play an important role. The frequency evolution of the simulated phase contrast in Fig. 4.18b resembles the one obtained from the experimental data (Fig. 4.13). The simulated results additionally reveal that contrary to the maximal thermal contrast, the maximal phase contrast decreases only slightly with increasing defect depth. This contributes to deeper penetration of phase images. (a) (b) Figure 4.18: (a) The time dependent thermal contrast C(t) and (b) the frequency dependent phase contrast ∆φ for defects at depths of 1.5, 3.5, 6 and 7.5 cm after 30 min of simulated heating and 120 min of simulated cooling. The inset in Fig. 4.18a presents the time evolution of thermal contrast for the whole cooling period of 120 min. Slika 4.18: (a) Potek časovno odvisnega temperaturnega kontrasta C(t) in (b) potek frekvenčno odvisnega faznega kontrasta ∆φ za anomalije na globinah 1,5, 3,5, 6 and 7,5 cm po 30 min simuliranem segrevanju in 120 min simuliranem ohlajanju. Vložena slika na sliki 4.18a prikazuje potek temperaturnega kontrasta za čas celotnega ohlajanja (120 min). 58 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Finally, the simulated data was fitted to the models given by Eq. (2.36) and (2.44). The results presented in Fig. 4.19 confirm the outcome of the measurements. Therefore, we propose that a longer heating time, compared to the heating times used for the inspection of composite structures, has the largest contribution to the deviation of Cmax and tCmax from the model proposed by Balageas et al. (a) (b) Figure 4.19: (a) The fitting results for Cmax and tCmax using the model proposed by Balageas et al. (2.36) and (b) the fitting results for fch using the model proposed by Arndt et al. (2.44). Slika 4.19: (a) Rezultat prilagajanja podatkov za Cmax in tCmax z modelom po enačbi (2.36) in (b) rezultat prilagajanja podatkov za fch z modelom po enačbi (2.44). The results from both the experiment and the simulation have proven good performance of PPT in defect depth retrieval. In addition, PPT yielded high signal-to-noise ratio and spatial resolution of phase images. However, it should be stressed that this technique requires relatively long monitoring of the cooling behaviour. As an alternative to the required long monitoring, zero- padding can be applied before FFT. This was also used in our study. Nevertheless, the thermal contrast is a useful tool for the thermal analysis when neither quantitative characterization of defects nor high spatial resolution are required. 4.2.2 Sensitivity evaluation of ground penetrating radar and infrared thermography For the sensitivity evaluation and later data fusion of IRT and GPR, data sets were further normalized. For GPR data, a global normalization was performed according to the maximal amplitude present in the data set. For IRT data, normalized phase contrasts were taken into account. Data alignment was performed and a common spatial reference of 5 mm was established for both data sets. The performance of both NDT methods with respect to different defect materials and depths was quantitatively evaluated based on a ground truth reference from the known position of inbuilt defects. For the GPR C-scans and IR phase contrast images, sensitivity and specificity values [159] were calculated from the number of true positive (T P ), true negative (T N ), false Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 59 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.20: A schematic presentation of true positive (T P ), true negative (T N ), false positive (F P ) and false negative (F N ) calls for the detection of a defect. Slika 4.20: Shematični prikaz kazalcev za vrednotenje zaznavanja anomalij: pravilno pozitivni (T P ), pravilno negativni (T N ), napačno pozitivni (F P ) in napačno negativni (F N ). positive (F P ) and false negative (F N ) calls (see Fig. 4.20) by using T P sens = , (4.1a) T P + F N T N spec = , (4.1b) T N + F P where positive and negative refer to an identified and a rejected defect, respectively. Thus, a true positive call means that a pixel was correctly identified as a defect. A combined sensitivity measure c_sens was further calculated by a · sens + b · spec c_sens = , (4.2) a + b where weights a and b refer to the specimen’s corresponding defect and non-defect area, respec- tively. As a prerequisite for the sensitivity calculation, a threshold value was determined for both GPR and IRT data, which defines weather a defect was detected or not (the limit for positive and negative calls). For the threshold value, the mean value of all data values corresponding to the defects area was chosen. The results from sensitivity calculation are presented in Fig 4.21. The results for polystyrene defects (blue marks) show that the performance of GPR (full marks) is largely decreased at the very near-surface region, where a sensitivity of c_sens = 84.5 % is obtained. However, the sensitivity increases rapidly already at the depth of 3 cm and varies only to a small extent up till the depth of 7.5 cm. The poor performance of GPR very close to the surface is expected due to the antenna coupling/induction effects in the near-surface region. Contrary to GPR, the sensitivity of IRT (empty marks) decreases along the depth. It has to be noted that IRT exhibits such good depth penetration particularly due to the use of phase images. Nevertheless, it is expected that the methods capability would drop remarkably at greater depths. The results in Fig. 4.21 further show that the performance of IRT is more affected by the material properties of the defects than it is in the case of GPR. This was particularly expressed in the case of a water defect at a depth of 6 cm, where a sensitivity of c_sens = 56.7 % could only be detected for IRT (this value is not presented in Fig. 4.21). On the other hand, GPR 60 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Figure 4.21: The results of sensitivity evaluation in terms of c_sens (in %) for GPR (full marks) and IRT (empty marks) with respect to the material and depth of the defects in specimens S0–S2. The dotted lines are guides to the eye. Slika 4.21: Rezultat občutljivostne analize za georadar (polni znaki) in IRT (prazni znaki) v odvisnosti od materiala in globine anomalij v preizkušancih S0–S2. Rezultati so podani v % parametra c_sens. Pikčasti črti povezujeta merske točke za lažje spremljanje rezultatov. responded to the detection of water defects by increased sensitivity, which is in accordance with the greater difference in the dielectric constant between water and concrete than it is between air and concrete. Moreover, as expected, the detection of defects below dense reinforcement (orange marks) is more reduced in the case of GPR since much of the EM signal is reflected at the reinforcement. For specimen S3, the GPR depth slice at 1.5 cm and the corresponding IR phase contrast image at a frequency of 3.7 · 10−4 Hz are shown in Fig 4.22. It can be seen that especially in the region of plastic pipes, GPR exhibits poor contrast. This was further justified by the sensitivity evaluation of the right polystyrene plate (b), which yielded 75.4 % and 90.5 % for GPR and IRT, respectively (the left plate seems to have tilted during concreting and was thus not assessed). By correlating these results with the ones in Fig. 4.21, it can be concluded that the performance of GPR decreases greatly for the detection of near-surface defects. Figure 4.22: The imaging results of specimen S3 with marked polystyrene plates according to Table 4.1: (a) a GPR depth slice at a depth of 1.5 cm and (b) an IR phase contrast image at a frequency of 3.7 · 10−4 Hz. Slika 4.22: Grafični prikaz rezultatov za preizkušanec S3 z označenimi ploščami iz stiropora (pregled- nica 4.1): (a) georadarski grafični prikaz prereza na globini 1,5 cm in (b) IR fazno-kontrastna slika pri frekvenci 3, 7 · 10−4 Hz. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 61 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 5 CONCLUSION AND OUTLOOK The central aim of the thesis was to develop an image fusion methodology based on clustering techniques, which would enable more efficient and reliable evaluation of multi-sensor NDT data in CE applications. The thesis presents the application of clustering techniques for data fusion of multi-sensor NDT of laboratory concrete and masonry specimens, as well as their implementation into two data fusion frameworks for practical application. Most data fusion applications on metal and composite structures in the aerospace industry deal with methods of validated performance and focus on the detection of well-defined defects such as notches. Hence, the research on data fusion generally confronts the reduction of noise. However, to fully characterize the structure under inspection and increase the accuracy of the evaluation in CE, data fusion has to account for the complementary and overlapping performance of NDT- CE methods. Besides, several not yet standardized applications need to be addressed. The thesis in particular focused on the detection of near-surface defects in concrete and masonry structures using GPR and IRT – an issue that is rarely tackled in NDT-CE. Before data fusion, the methods’ performance with respect to various testing problems was validated. In particular, laboratory concrete specimens aimed at studying the effect of different material properties under well-defined conditions. The sensitivity of GPR and IRT on concrete was evaluated in terms of GPR depth slices and IR phase contrast images, the latter resulting from using the pulsed phase thermography technique, which enables depth characterization of defects. The results show that whilst the sensitivity of GPR increases with increasing depth, for IRT it decreases. However, already at a depth of 3–4.5 cm, GPR reaches the performance which remains constant up till 7.5 cm. This directly reflects that the functioning of GPR is largely affected by the reactive near-field region (for the used antenna, it extends up till a depth of 1 cm [96]), while the close radiating near-field region has a minor effect. IRT has an extremely high probability of defect detection, which decreases only slowly up till 7.5 cm. Its good behaviour in particular resulted from using IR phase contrast images. These exhibited much deeper penetration than thermograms and contrast images and in addition yielded high signal-to-noise ratio as well as extremely good spatial resolution, enabling the detection of thin defects such as pipes. As expected, the detection of defects below dense reinforcement was greatly decreased when GPR was used compared to IRT since much of the EM signal is reflected at the reinforcement. However, it was observed that the contrast in the dielectric constant and thermal conductivity between defects and sound area governs the material characterization of defects. Hence, GPR more accurately detects a water anomaly than an air anomaly, and the opposite is true for IRT. The well-known position of defects in the concrete specimens enabled a quantitative evaluation of the data fusion results. Here, data fusion was performed on a feature level since a complete visualization of the structure along the depth was of interest rather than only the detection of defects. Fuzzy clustering was employed to derive classifier units in terms of memberships for each pixel, which were later fused using simple weighted averaging. Among the tested fuzzy probabilistic clustering algorithms (FCM, GK, NCFCM), as well as fuzzy possibilistic clustering 62 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. algorithms (PFCM, PCM with repulsion), the PCM with repulsion in general produces coincident clusters as the original PCM algorithm, whereas the NCFCM classified signals of very high amplitude (detected defects) as noise. For the hard k-means algorithm it was observed already from the results of the spectroscopic spatial data on concrete that the algorithm is very susceptible to initialization and was therefore not taken into account. In general, the FCM, PFCM and GK algorithms exhibited good functioning, where particularly the performance of PFCM resembled the one of FCM. This suggests that noise could have a minor effect on the resulting partition in the case of inspection of most typical defects in concrete structures, which is contrary to the inspection of structures in the aerospace industry [67, 69, 65]. However, among all algorithms, the GK with the parameter γ set to γ = 0.1 was found the most powerful since it also provided the best visualization of plastic pipes. As the relevance of only IRT was of importance for the detection of pipes, elongated clusters were produced. This restricted their detection by FCM and PFCM, which are sensitive to spherical clusters. A comparison of clustering-based feature level fusion with symbol level fusion confirmed the robust performance of clustering in situations with very limited knowledge about the material properties and depth of defects. Clustering in general increased the reliability of the NDT inspection in cases where the methods had an overlapping penetration capability and sensitivity to different physical properties. Moreover, in cases where one method greatly dominated over the other (complementary behaviour), they worsened its performance only to a small extent. On the other hand, supervised fusion using weighted averaging and D-S deteriorated the level of detection if weights were not assigned according to the reliability of the respective NDT method. A complementary use of GPR and IRT proved also beneficial for the inspection of plastered stone walls. Apart from the detection of the walls’ texture and morphology with the multi-leaf structure, seismic related damage of various extents could be detected. The plaster delamination could be particularly well resolved using IRT, as an air gap as small as 2 mm could be resolved, whereas the minimal detectable air gap with GPR measured 8 mm. This indicates that the first limit state of plaster could be resolved with high reliability when using IRT, whereas only the extent of damage referred to the third limit state could be detected by the 1.6 GHz GPR. Hence, IRT can be regarded as a potential method for defining the performance limit state of the plaster on multi-leaf masonry walls in practice. Particularly in the case of masonry walls in cultural heritage structures, the use of a non-contact method such as IRT plays an important role for defining the characteristic limit points of the plaster. Here, the walls have very often valuable frescoes, stuccoes or mosaics. IRT proved additionally capable of detecting surface and subsurface crack patterns below the 2– 3.5 cm thick plaster. However, we show that phase images at higher frequencies should be used in order to resolve small thermal contrasts resulting from cracking compared to the lowest frequency, which is better suited for imaging the underlying texture and the delamination pattern. This implies that the frequency, at which a defect was detected, could not be correlated with the defect’s depth according to the equation proposed by Arndt [131] and which performed well on concrete. Two effects could contribute to this – the extremely heterogeneous structure of masonry Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 63 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. as well as the very shallow depth. Therefore, for masonry in practice, where delamination can occur at different depths, a synthesis of GPR and IRT results would be crucial to characterize the defects. We show that by employing the GK clustering algorithm for the segmentation of GPR and IR data, both delamination as well as cracks could be characterized. In this context it should be mentioned that the thesis aimed in deriving a validation of the methods’ performance for the detection of internal cracks with respect to their size. Unfortunately, IRT revealed the subsurface crack pattern from only one tested wall, which restricted the validation process. Therefore, this could even not exclude the possibility that the detected “cracks” refer to the contours of a detected delamination. The effect of moisture on the detection of voids in masonry was tackled through GPR, ultrasonic and CR tomographies. In general, ultrasonic data showed higher sensitivity to moisture and greater scattering than GPR data. This phenomenon could be explained by both a greater effect of ambient vibrations on the ultrasonic data, and a stronger difference in the acoustic properties of masonry materials (brick, mortar) compared to the change in the electromagnetic properties. Albeit all techniques proved capable of detecting the large air void, their efficiency differed significantly. GPR tomography is by far the most reliable technique for assessing voids in both dry and wet masonry. On the other hand, ultrasonic tomography requires longer measuring times, and a reliable tomographic image can be achieved solely by a carefully performed automatic picking of travel times. CR tomography turned out to be a reliable technique even at low humidity levels, although it is limited by lower depth resolution than the other two methods and suffers from poor electrode coupling, which might even largely affect the interpretation of the reconstructed resistivities. As already debated in [101], the interpretation of complex resistivities on masonry is limited to their magnitudes. To explore the performance of the analysed methods with respect to the moisture level in more detail, further research should focus on more humidity conditions, as well as different tomography configurations. Along with the defined goals of the thesis, clustering was implemented into two NDT-data fusion frameworks for practical application. Their use for NDE of data from the multi-sensor BetoScan robot system provided a characterization of the damage. By applying clustering on data from potential mapping, eddy current and microwave moisture sensors, a reinforced concrete floor affected by corrosion could be segmented into damage zones. Moreover, the potential sources of the corrosion process, such as carbonation and high chloride concentration, could be identified. In addition, the application of clustering for cement identification of spectroscopic (LIBS) spatial data from concrete proved particularly beneficial. As both noise and spatial resolution affect the spectra, the noise handling clustering algorithms NCFCM and PFCM provided the best separa- tion of spectra dominated by cement or aggregate content. The resulting sodium depth profiles gave an almost ideal fit to the reference profile and outperformed the standard classification based on the O/Ca criterion to a large extent. The proposed parameter optimization model enables the algorithms to work fully unsupervised. The greatly improved data evaluation process of the LIBS technique will play an important role in the implementation of the in situ mobile system, which is currently under construction and will bring LIBS into a productive environment. 64 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. For an NDT multi-sensor system such as the BetoScan, image fusion with clustering is of par- ticular importance. Its main advantage is that it can work almost fully unsupervised. This is of particular interest in implementing NDT in practice in terms of automated application by trained engineers and technicians rather than scientists. Applied on the low pixel level fusion, it provides a segmented image, which may characterize the defects present in the structure as well as the extent of damage. Delivering only one reliable image, which takes into account the correlation between the various NDT methods, rather than multiple ones to the end-user is of the utmost importance. However, when applying clustering at the higher feature level fusion, the whole internal structure may be visualized. This approach is especially promising for NDT on concrete, where a complete visualization of structures is generally favourable. Particularly in the cases of large and/or hardly accessible inspection areas, a multi-sensor robot or scanning system comprising GPR, ultrasonic, IE and other methods is needed. For a reliable, fast and an efficient inspection, data should be acquired at different polarizations, frequencies and spatial resolutions of the measurement system. It is believed that clustering could handle the data fusion of various testing problems. Nonetheless, further research should also be devoted to the potential symbol level fusion. With respect to this, a more detailed sensor modelling should be performed for various NDT methods. Further research that goes beyond the frontiers of this thesis will be devoted to the sensitivity evaluation of the methods’ penetration capabilities. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 65 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. SUMMARY Non-destructive testing (NDT) methods, such as ground penetrating radar (GPR), ultrasonic, impact-echo (IE) and infrared thermography (IRT), are due to the non-invasive effect they have on the structure more and more often used as an alternative for visual inspection in combination with destructive tests on samples extracted from the structure. However, with respect to the composition and condition of the inspected structure, all methods exhibit different penetration capability and sensitivity to the material physical properties. Hence, an extensive sensitivity evaluation should be carried out with respect to several not yet standardized applications. It is evident that regardless of the knowledge about the methods’ performance, multi-sensor testing is required for a complete inspection of the structure. This also rises the need for an efficient and reliable data processing technique. Contrary to most data fusion methods, clustering does not require the knowledge about the structural composition and can thus work unsupervised. In the thesis their functioning was tested on the results from the inspection of concrete and masonry structures. Hard and fuzzy clustering algorithms were analysed. As a hard clustering algorithm, the k-means was taken into account, whereas for the fuzzy ones, the conventional fuzzy c-means (FCM) and its variants were used. These are either better suited for handling noisy data (noise handling fuzzy c-means (NCFCM), possibilistic c-means (PCM) with repulsion, possibilistic-fuzzy c-means (PFCM)) or use an adaptive distance norm (Gustafson-Kessel (GK)). The main research on the possibilites of defect detection in the near-surface region was performed on four laboratory concrete specimens using GPR and IRT. In the analysis, the material prop- erties, type, shape and size of anomalies were taken into account. Particularly, voids, localized water, delamination, plastic pipes and reinforcement were simulated. Special consideration was put on the data evaluation of the results using pulsed thermography. Techniques based on the thermal contrast and pulsed phase thermography (PPT) were employed. The phase contrast images enabled accurate depth characterization of anomalies and deeper probing. In addition, they exhibited high contrast and good spatial resolution. Thus, for the sensitivity evaluation of GPR and IRT, phase contrast images were used together with GPR C-scans. The evaluation revealed that the sensitivity of GPR is lower than the one of IRT up till a depth of 3–4.5 cm. For IRT, the sensitivity decreases only slowly up till 7.5 cm, which particularly resulted from using phase contrast images. For both methods it could be observed that the material characterization of defects is predominantly governed by the contrast in dielectric constant and thermal conduc- tivity between defects and sound area. Hence, GPR more accurately detects a water anomaly than an air anomaly, and the opposite is true for IRT. For the synthesis of GPR C-scans and IR phase contrast images, feature level fusion was em- ployed, where classifier units were obtained using clustering. Among all tested clustering algo- rithms, the GK with parameter γ set to γ = 0.1 was found the most powerful as it exhibited both robust performance and enabled the best detection of plastic pipes. The detection of pipes was possible since the GK algorithm is not limited to the detection of spherical clusters as are the FCM and PFCM algorithms. These two algorithms however revealed similar behaviour, 66 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. which indicates that noise has only a modest effect on the detection of most typical defects in concrete structures. The robust performance of clustering-based data fusion in situations with very limited knowledge of the structure was further confirmed using symbol level fusion. The complementary use of GPR and IRT also enabled a complete structural characterization of plastered multi-leaf stone walls. In this case, GPR and IRT were particularly selected since they enable inspection with minimal interference on the structure. This is of paramount importance in the case of inspection of masonry walls in cultural heritage buildings. Apart from the detection of the walls’ morphology and texture, plaster delamination could be detected as a result of simulated seismic activity using a cyclic shear test. A comparison with the results from the 3D digital image correlation technique revealed that a delamination corresponding to the first limit state of the plaster (≈ 2 mm air gap) could be resolved using IRT, whereas only the extent of damage referred to the third limit state (≈ 8 mm air gap) could be detected by GPR. By using IR phase images at higher frequencies, surface and subsurface cracks could be detected, whereas the delamination and texture were detected when using phase images at the lowest frequency. For IRT, the extremely heterogeneous masonry structure prevented a quantitative evaluation of thermal data, which could be performed on concrete. Hence, we proposed a data fusion performed at the pixel level for the segmentation of data. This enabled the detection of both plaster delamination and cracks. The performance of NDT methods with respect to the moisture of the material was studied on a laboratory brickwork specimen. In particular, the detection of a larger void was considered using GPR, ultrasonic and complex resistivity (CR) tomographies. The measurements revealed that ultrasonic is more sensitive to changes in the moisture content than GPR and in addition, exhibits also greater scattering of data. GPR proved to be the most reliable technique for the detection of voids in both dry and wet masonry. To enable good void detection when using ultrasonic tomography, longer measuring times would be required, as well as an accurate selection of the first travel time. Also when using CR tomography, the void could be detected at both moisture conditions of the masonry. However, the method exhibited poor depth resolution compared to GPR and ultrasonic. In addition, efficient imaging of the resistivity distribution was restricted by poor electrode coupling. Clustering methods were successfully implemented into two NDT systems – for data fusion of a multi-sensor BetoScan robot system and for classification of spectroscopic spatial data from concrete. Clustering methods were applied on the pixel level for the segmentation of data. For the identification of the cement matrix in concrete specimens using spectroscopic (LIBS) data, the NCFCM and PFCM algorithms exhibited the best performance. For these algorithms, the proposed parameter optimization model enabled a fully automated classification of data. This is of great significance for the implementation of LIBS into a productive environment. In the case of the multi-sensor data sets from the BetoScan system, clustering provided the segmentation of a reinforced concrete floor affected by corrosion into damage zones. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 67 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 6 RAZŠIRJENI POVZETEK V SLOVENSKEM JEZIKU (EXTENDED AB- STRACT IN SLOVENE) 6.1 Uvod Pregled ožjega znanstvenega področja in opredelitev znanstvenega problema V gradbeništvu se prve neporušne metode za preiskave tlačne trdnosti strjenega betona pojavi- jo okoli leta 1940. Večina izmed teh metod (metoda z udarno-povratnim kladivom, preizkus s sondo Windsor, preizkus izvlečne sile, ultrazvočna metoda) je danes standardiziranih [2, 3]. Šele v zadnjih nekaj letih usmerjamo intenzivno raziskovanje v možnost uporabe neporušnih metod za oceno stanja obstoječih gradbenih konstrukcij, ki bodisi vsebujejo konstrukcijske napake bodisi je na njih prisotno propadanje materiala zaradi zunanjih vplivov na konstrukcijo. Na velikem deležu betonskih konstrukcij (predvsem mostovi, garažne hiše in predori) je propadanje materiala posledica vdora topnih soli v pore cementne matrike [4, 7]. Le-to v kombinaciji z neustrezno materialno sestavo in majhnim krovnim slojem betona vodi do korozije armature, razpokanja in odstopanja krovne plasti betona [7]. Zidane konstrukcije, od katerih je velik delež stavb kulturne dediščine, imajo poleg prisotnih topnih soli pogosto povečano vlažnost in slabšo povezanost zidovja. V mediteranskem območju so zidane konstrukcije še dodatno prizadete zaradi večje potresne aktivnosti [9]. Trenutna ocena stanja obstoječih konstrukcij temelji na vizualni oceni v kombinaciji s porušnimi preiskavami odvzetih vzorcev. Tak pristop ne omogoča zanesljive ocene stanja in obenem pred- stavlja velik poseg na konstrukcijo. Zato so bile kot alternativa predlagane metoda georadarja, ultrazvočna metoda, metoda udarec-odmev, infrardeča termografija (IRT) idr. [10, 11, 12]. Vse te metode imajo zaradi heterogene strukture gradbenih materialov in množice prisotnih anomalij v konstrukcijskih elementih spremenljivo penetracijsko sposobnost in občutljivost na fizikalne la- stnosti materialov. Zato je le malo primerov uporabe teh metod v praksi standardiziranih [1, 3, 5]. To zahteva obsežno analizo občutljivosti metod glede na področja njihove uporabe v praksi [6]. Metoda georadarja se je v raziskavah tako betonskih kot tudi zidanih konstrukcijskih elementov pokazala kot učinkovita metoda za zaznavanje armature in za oceno vlažnosti ter koncentracije soli v materialu [13, 14, 16]. Za oceno debeline betonskih konstrukcijskih elementov, zaznavanje mehke armature in zaščitnih cevi za prednapetje so raziskovalci uspešno uporabili ultrazvočno metodo [18, 19, 20]. Nedavne raziskave poročajo tudi o uporabi ultrazvočne metode in metode udarec-odmev za preučevanje napak ob injektiranju zaščitnih cevi za prednapetje [22, 23, 27]. Sadri [28] uporabi metodo udarec-odmev tudi za ugotavljanje povezanosti kamnitega zidu ter jo predlaga za zaznavanje večjih votlin in za kontrolo uspešnosti injektiranja. Za zaznavanje večslojnosti in povezanosti zidov, odstopanj ometa, prisotnosti večjih votlin ter vlažnosti zidovja večina raziskav poroča o uspešni uporabi georadarja, metode s kladivom in IRT [9, 29, 30, 31, 32, 33]. Predvsem IRT ima zaradi nekontaktne narave veliko prednost pri preiskovanju zidanih objektov kulturne dediščine. Uporabili so jo za vizualizacijo podometne teksture in za zaznavanje odstopanja posameznih plasti [9, 31, 34, 35]. Na objektih kulturne dediščine so IRT in ultrazvočno metodo uporabili še za določitev globine površinskih razpok [48] in njihove orientacije [49]. Za 68 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. raziskave razpokanosti betonskih konstrukcijskih elementov zaradi delovanja obtežbe predlagajo metodo akustične emisije [46, 47]. Razvidno je, da je za celostno karakterizacijo preiskovane strukture potrebna uporaba več neporušnih metod, tj. večsenzorska meritev [52, 53]. Številni avtorji so v svojih delih že primerjali rezultate različnih neporušnih metod na betonskih [18, 54, 55, 56, 57, 58, 27] in zidanih [9, 59, 60, 61] konstrukcijskih elementih, vendar so obdelavo podatkov izvedli neodvisno za posamezno metodo. V praksi, kjer je največkrat potrebna obdelava velikega števila podatkov, je tak pristop časovno potraten in drag. Poleg tega je tudi podvržen subjektivni oceni rezultatov. Sintezo podatkov večsenzorskih preiskav, ki bi omogočila avtomatsko obdelavo podatkov, so v gradbeništvu oprav- ili na prednapetih mostnih nosilcih. Združeni so bili različni georadarski podatki (pridobljeni pri različnih frekvencah in v obeh polarizacijskih smereh) kot tudi georadarski in ultrazvočni podatki [14, 76, 77]. Avtorji teh del so za sintezo rezultatov uporabili preproste matematične algoritme. Naprednejši algoritmi, ki omogočajo bolj natančno združevanje podatkov neporušnih preiskav, so bili predlagani za preiskave materialov v letalski in jedrski industriji. Uporabili so algoritme, ki temeljijo na nevronskih mrežah, valovni analizi, Bayesovi analizi in Dempster- Shaferjevi (D-S) teoriji evidenc [69, 65, 73, 67]. Glavni cilji in vsebina doktorske disertacije Glavni namen disertacije je bil razvoj ustreznega postopka za združevanje slik, ki bi temeljil na metodah gručenja in bi omogočil učinkovito in zanesljivo obdelavo podatkov večsenzorskih neporušnih preiskav gradbenih konstrukcijskih elementov. Metode gručenja so že uporabili pri združevanju slik. Pri medicinskem slikanju so jih uporabili za razpoznavanje vzorcev, kot so tumorne celice [78, 79], v geofiziki pa za določitev geološke slojevitosti [80, 81]. Glavna prednost metod gručenja je, da lahko delujejo avtomatizirano. To omogoča obdelovanje podatkov neporušnih preiskav brez subjektivnega vpliva in izkušenosti raziskovalca, kar je bistvenega po- mena za prenos metod v inženirsko prakso. V disertaciji smo za združevanje slik večsenzorskih neporušnih preiskav uporabili algoritme trdega in mehkega gručenja, in sicer trdo metodo k-srednjih vrednosti [82], osnovno mehko metodo c-srednjih vrednosti [83] ter njene različice. Te bodisi bolje obvladujejo močno zašumljene po- datke [84, 86, 87] bodisi uporabljajo prilagojeno normo [85]. Poleg razvoja ustrezne tehnike združevanja slik smo v okviru dela opravili tudi analizo občut- ljivosti več neporušnih metod. Analizo smo opravili na rezultatih preiskav betonskih in zidanih laboratorijskih preizkušancev. Obravnavali smo tako različne konstrukcijske anomalije kot tudi različne materialne lastnosti preiskovane strukture. Osnovne raziskave možnosti zaznavanja anomalij v območju blizu površja smo opravili z georadar- jem in IRT na štirih laboratorijskih betonskih preizkušancih. Vključili smo različne anomalije: votline, lokalizirano vodo, odstopanje plasti, plastične cevi in armaturo. Preiskave smo opravili z georadarjem in IRT, ker smo ju ocenili kot komplementarni metodi za zaznavanje v območju blizu površja (do globine 10 cm). Obe metodi smo uporabili tudi za neporušne preiskave labo- Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 69 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. ratorijskih večslojnih kamnitih zidov. Preučili smo možnost zaznavanja teksture in morfologije zidov ter odstopanja ometa in nastanka razpok zaradi delovanja strižne obremenitve. Namen preiskave je bil raziskati možnosti uporabe izbranih metod za preiskave zidov v objektih kul- turne dediščine. Tu so neporušne preiskave še posebej pomembne, saj zahtevamo čim manjši poseg v konstrukcijo. Na inštitutu BAM smo na opečnatem zidovju preverili občutljivost neporušnih metod na pri- sotnost vlage v zidovju. Uporabili smo georadarsko, ultrazvočno in geoelektrično (kompleksno- uporovno) tomografijo. Ker tomografska tehnika omogoča 3D vizualizacijo, smo posebej preučili zaznavanje večje votline glede na pogoje vlažnosti zidovja. V okviru doktorskega dela smo metode gručenja uvedli v dvoje aplikativnih ogrodij združevanja podatkov na inštitutu BAM: za združevanje podatkov večsenzorskega robotnega sistema Be- toScan kot tudi za popolnoma avtomatizirano klasifikacijo spektroskopskih podatkov betonskih preizkušancev. 6.2 Osnove izbranih neporušnih metod Neporušne metode v gradbeništvu v grobem razvrščamo v dve glavni skupini – zvočne in ele- ktromagnetne. Zvočne metode temeljijo na širjenju mehanskega zvočnega valovanja in so zato občutljive na elastične lastnosti materialov, medtem ko elektromagnetne (EM) metode temeljijo na merjenju EM lastnosti materialov kot posledica širjenja elektromagnetnega valovanja (EMV). Med zvočne metode sodijo mikroseizmične metode, metoda s kladivom, metoda udarec-odmev, ultrazvočna metoda in metoda akustične emisije. Izmed EM metod so najbolj poznane geo- radar, metoda z vrtinčnimi tokovi, elektro-uporovne metode ter EM potencialne metode. Poleg omenjenih metod v gradbeništvu uporabljamo še IRT in radiografijo. Slednja temelji na mer- jenju absorpcije visokofrekvenčnega EMV (rentgenskih ali gama žarkov) ali toka nevtronov v odvisnosti od gostote materiala. Georadar Georadar temelji na generiranju in širjenju EM pulzov v radiofrekvenčnem območju znotraj preiskovanega materiala ter detekciji bodisi odbitega bodisi prepuščenega valovanja z uporabo antene [94]. Za preiskovanje gradbenih konstrukcijskih elementov je najpogostejša uporaba an- ten s središčno frekvenco valovanja med 300 MHz in 2,5 GHz [76]. EMV se pri širjenju skozi material sipa, njegova jakost slabi, obenem se tudi odbija in lomi na mejnih ploskvah anomalij z drugačnimi dielektričnimi konstantami, kot je dielektrična konstanta preiskovanega medija. Za georadarske meritve v odbojnem načinu je za zaznavanje anomalij ključna razlika v dielektričnih konstantah, saj je amplituda odbitega valovanja, ki prispe do antene, določena z zvezo [95] √ √ ε ε r = r2 − r1 √ √ , (6.1) εr2 + εr1 kjer sta εr1 in εr2 relativni dielektrični konstanti dveh različnih medijev. Njihova razlika se odraža v spremembi amplitude in faze zaznanega odbitega valovanja. 70 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Pri merjenju v odbojnem načinu globino anomalije z določimo na podlagi povprečne hitrosti EMV v snovi v in časa potovanja EMV od oddajne do sprejemne antene t v skladu z enačbo vt ct z = = √ , (6.2) 2 2 εr kjer je c hitrost EMV v vakuumu in εr relativna dielektrična konstanta preiskovanega mate- riala. Seveda pa smo pri zaznavanju anomalij omejeni z ločljivostjo antene in s penetracijsko sposobnostjo valovanja, na katero najbolj vplivata uporabljena frekvenca valovanja in električna prevodnost snovi. Pri georadarski meritvi zberemo več točkovnih meritev (refleksijskih sledi) v georadarski profil, kjer prikažemo časovno odvisnost amplitude odbitega valovanja. Princip georadarske meritve z zaznavanjem odbojev EMV od mej materialov z različnimi dielektričnimi konstantami je razviden iz slike 2.1. Prikazani so tudi različni načini grafičnega prikazovanja rezultatov: grafični prikaz refleksijske sledi (A-scan), radargram georadarskega profila (B-scan) in grafični prikaz prereza (C-scan). Kompleksno-uporovna metoda Vse elektro-uporovne metode (tudi geoelektrične metode) določajo specifično upornost snovi na podlagi meritve napetosti med dvema elektrodama kot posledica električnega toka v skladu z Ohmovim zakonom. Kompleksno-uporovna metoda za razliko od navadne uporovne metode uporablja izmenični tok visokih frekvenc in je zato bolj občutljiva na meritve specifične upornosti [99]. Pri meritvi specifične upornosti materialov največkrat uporabljamo štiri elektrode na površini preizkušanca, da zmanjšamo vpliv kontaktnih napetosti. Pri tem sta dve tokovni elektrodi, dve pa merita napetost. Če območje preizkušanca opišemo s polprostorom, lahko za znan električni tok I in ob meritvi napetosti U med elektrodama izračunamo navidezno specifično upornost kot [99] U ρa = K , (6.3) I kjer je K konfiguracijski faktor in je odvisen od geometrijske razporeditve elektrod. Nekaj najbolj tipičnih geometrijskih razporeditev elektrod je prikazanih na sliki 2.2. Izraz navidezna specifična upornost poudarja, da gre za povprečno specifično upornost merjene snovi za območje med elektrodama. Prave specifične upornosti nehomogene strukture dobimo šele z inverznim računom, kjer po korakih popravljamo razporeditev specifične upornosti numeričnega modela, dokler se izračunane navidezne specifične upornosti dovolj dobro približajo merjenim. Infrardeča termografija IRT nam da podatke o površinski temperaturi preizkušanca. Temelji na zaznavanju IR sevanja, ki ga telo pri določeni temperaturi oddaja. Za neporušne preiskave je najpogostejša uporaba aktivne IRT, kjer z zunanjim grelcem v preizkušancu povzročimo nestacionaren toplotni tok. Glede na Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 71 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. način ogrevanja ločimo tri tehnike aktivne IRT [104, 109]: pulzna termografija 1, termografija z odzivom na periodično motnjo in termografija s stopničastim pulzom. V disertaciji smo uporabili pulzno termografijo, ki je v gradbeništvu najbolj pogosta. Gostoto energijskega toka, ki ga telo s temperaturo T izseva, opiše enačba [104] j∗ = e σsbT 4, (6.4) kjer je σsb Stefan-Boltzmannova konstanta in e emisivnost površine preizkušanca. Za večino gradbenih materialov (beton, malto, opeko) je emisivnost med 0,9 in 0,95 [104]. Sevanje teles lahko razlagamo s sevanjem črnega telesa, za katerega velja Planckov zakon. Ta opiše spek- tralno porazdelitev izsevane energije, ki je za črno telo pri različnih temperaturah prikazana na sliki 2.5. Poleg sevanja je za IRT ključnega pomena tudi poznavanje procesov prenosa toplote v snovi, ki vplivajo na časovni potek površinske temperature. Odstopanje v porazdelitvi površinske temperature od pričakovane nakazuje na pojav anomalije v preizkušancu. V primerjavi z metodami, kot so npr. georadar, ultrazvočna metoda ali geoelektrične metode, je IRT hitrejša, relativno poceni in ne potrebuje fizičnega stika med preizkušancem in kamero. Slabost metode pa je majhen globinski doseg in zahtevna karakterizacija anomalije, predvsem njene globine. Za oceno globine anomalije sta bila na podlagi rezultatov preiskav materialov v strojništvu predlagana dva glavna pristopa k obdelavi podatkov. Prvi temelji na tempera- turnem kontrastu, drugi, pulzno-fazna termografija, na obdelavi podatkov v frekvenčni domeni. Pri tehniki temperaturnega kontrasta je za oceno globine anomalije predlagan izraz, ki temelji na maksimalnem temperaturnem kontrastu in času njegovega nastopa [118], medtem ko je za pulzno-fazno termografijo predlagan izraz, ki uporablja frekvenco pri maksimalnem faznem kon- trastu [131, 123]. Ultrazvočna metoda Ultrazvočna metoda temelji na generiranju, širjenju in zaznavanju bodisi odbitega bodisi pre- puščenega zvočnega valovanja v ultrazvočnem območju, tj. pri frekvencah nad 20 kHz. Za preiskave v gradbeništvu najpogosteje uporabljamo frekvence v območju 50–200 kHz [12]. Za ra- zliko od ostalih zvočnih metod, uporabljenih za neporušne preiskave v gradbeništvu, ultrazvočna metoda uporablja piezoelektrični senzor tako za oddajnik kot za sprejemnik valovanja. Za širjenje zvočnega valovanja je za razliko od EMV potreben medij, saj se valovanje širi z deformiranjem snovi oz. natančneje z nihanjem delcev snovi okoli ravnovesnih leg. V trdni snovi lahko delci nihajo vzdolž smeri širjenja valovanja, kar povzroča longitudinalne deformacije, ali pa prečno na smer valovanja, kar povzroča transverzalne deformacije. Glede na to, katera vrsta deformacij je v snovi povzročena, ločimo longitudinalno (tlačno) in transverzalno (strižno) valovanje. Hitrost longitudinalnega valovanja je večja od hitrosti transverzalnega, obe pa sta odvisni od gostote snovi, elastičnega modula in Poissonovega števila [133]. 1V pulzno termografijo uvrščamo tudi termografijo s kvadratnim pulzom, kjer preizkušanec segrevamo dlje časa (običajno nekaj minut). 72 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Pri merjenju v odbojnem načinu globino anomalije določimo podobno kot pri georadarju z enačbo z = vt/2 (6.2), pri čemer je v hitrost longitudinalnega oz. transverzalnega valovanja. Za za- znavanje anomalij je potrebna dovolj velika razlika v zvočni impedanci, saj je delež odbitega valovanja določen z [136] Z r = 2 − Z1 , (6.5) Z2 + Z1 kjer sta Z1 in Z2 zvočni impedanci dveh različnih medijev in Z = ρmv [136], kjer je ρm gostota snovi. Ultrazvočne preiskave lahko opravimo kot posamezne točkovne meritve. Slabost teh je, da so razmeroma dolgotrajne, rabimo pa tudi sredstvo (npr. glicerin, vazelin ali vodo), ki zagotovi dober stik senzorja s preizkušancem [11]. Za preiskovanje betonskih konstrukcijskih elementov v odbojnem načinu se zato raje uporablja sistem več ultrazvočnih senzorjev [22, 134]. 6.3 Združevanje podatkov z gručenjem Model združevanja podatkov Združevanje podatkov neporušnih preiskav lahko razumemo kot sintezo večsenzorskih podatkov [62]. Združevanje lahko poteka na nivoju posameznih signalov, 2D slik ali celotnega volumna, združujemo pa lahko tudi rezultate ene same neporušne metode (npr. georadarski podatki, pri- dobljeni pri različnih frekvencah ali polarizacijah antene) ali rezultate večih metod. V disertaciji obravnavamo združevanje slik več neporušnih metod. Za združevanje podatkov neporušnih preiskav so raziskovalci predlagali različne algoritme, ki temeljijo na metodah optimizacije, večločljivostni analizi, hevrističnih metodah in statističnih metodah (Bayesova analiza, Dempster-Shaferjeva (D-S) teorija evidenc, mehka logika) [53]. Ne glede na uporabljeni algoritem lahko združevanje slik poteka na treh nivojih: na nivoju posamezne slikovne točke, z uporabo značilnic in na simbolnem nivoju [53]. Pri slednjih dveh uporabimo informacije, ki jih pridobimo v koraku predhodne obdelave podatkov. Značilnice pri- dobimo s predhodno segmentacijo slik (npr. s pomočjo gručenja ali nevronskih mrež), medtem ko za združevanje podatkov na simbolnem nivoju uporabimo podatke senzorja. Glavna prednost metod gručenja pred ostalimi zgoraj omenjenimi metodami združevanja po- datkov je, da ne potrebujemo v naprej poznane statistične porazdelitve podatkov. To ima bistven pomen pri združevanju podatkov neporušnih preiskav v gradbeništvu, saj je vedenje o sestavi preizkušancev še posebno slabo zaradi množice različnih anomalij v praksi. Metode gručenja Gručenje pomeni razvrstitev množice podatkov v podmnožice (razrede, skupine) na način, da so podatki, ki pripadajo istemu razredu čim bolj podobni, podatki različnih razredov pa naj se čim bolj razlikujejo [143]. Pri gručenju z namensko funkcijo je tipična mera, ki pove, kako sta si dva podatka podobna, Evklidska norma. Gručenje množice podatkov pričnemo z naključno Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 73 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. razporeditvijo v naprej določeno število razredov c. Razporeditev nato korakoma popravljamo tako, da je vsak podatek razvrščen v tisti razred, da je njegova razdalja do središča razreda najmanjša. To je značilnost trdega gručenja. Pri mehkem gručenju lahko podatek razporedimo v več razredov hkrati z določeno mero pripadnosti u. Za poljuben podatek j in razred i tako velja, da je mera pripadnosti pri trdem gručenju uij ∈ {0, 1}, pri mehkem pa uij ∈ [0, 1]. Mehko gručenje naprej delimo na verjetnostno in možnostno gručenje. Pri verjetnostnem zahtevamo, da je vsota vseh verjetnosti nahajanja določenega podatka enaka 1, pri možnostnem pa tega pogoja ni. Razporeditev N podatkov v število razredov c poteka ob pogoju minimizacije namenske funkcije, ki jo opišemo kot c X N X Jm(U, V; X) = um ij d2 ij , (6.6) i=1 j=1 kjer je dij = kxj − vikA razdalja podatka xj do središča razreda vi. Matrika A pri tem definira vrsto norme. Minimizacija namenske funkcije predstavlja nelinearen optimizacijski problem, ki zahteva, da vi in uij v vsakem koraku znova izračunamo. Enačba (6.6) posebej opisuje verjetnostno gručenje. Najbolj osnovna metoda verjetnostnega gručenja je mehka metoda c-srednjih vrednosti (FCM) [83], ki uporablja Evklidsko normo, kjer je A enotska matrika. Zaradi tega predpostavlja sferično obliko razredov. Dodatna slabost te metode je še, da je razmeroma občutljiva na podatke, kjer je prisotno veliko šuma. Za opis razre- dov različnih geometrij (ne le sferične oblike) lahko uporabimo druge algoritme, ki temeljijo na verjetnostnem gručenju, in sicer: Gustafson-Kessel (GK) [85], algoritem prilagodljivega mehkega gručenja (AFC) [148] ter algoritem GK s prilagodljivimi volumni razredov (GKAρ) [149]. Vsi trije uporabljajo normo, kjer je matrika A odvisna od volumna posameznega razreda in mehke kovariančne matrike. Za opis močno zašumljenih podatkov lahko uporabimo algoritme verjetnostnega gručenja, ki uvedejo dodaten razred za najbolj zašumljene podatke [84], ter algoritme, ki temeljijo na mož- nostnem gručenju. Poleg osnovnega mehkega možnostnega gručenja (PCM) [151] sodijo v to skupino algoritmov še FPCM [152], PFCM [87] in PCM s kazenskim členom [86]. Vsi algoritmi poleg matrike verjetnosti nahajanja podatkov U = {uij} uporabljajo še matriko tipičnosti T = {tij}, za katero velja c X 0 < tij ≤ 1, ∀j ∈ {1, . . . , N}. (6.7) i=1 Zato moramo pri postopku minimizacije namenske funckije v vsakem koraku preračunati vi, uij in tij. Omeniti velja, da je pred začetkom gručenja treba določiti število razredov c. Za avtomatsko določitev optimalnega števila razredov lahko uporabimo različne kazalce [153], vendar lahko vsi v določenih primerih privedejo do napačnih rezultatov. Tudi v večini naših primerov uporabe 74 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. gručenja niso dali ustreznih rezultatov. Za podatke neporušnih preiskav se je kot boljša izbira števila razredov izkazala stopnja ocenjene poškodovanosti strukture oz. heterogenosti merjenega materiala. 6.4 Neporušne preiskave laboratorijskih betonskih preizkušancev z vgrajenimi ano- malijami 6.4.1 Eksperimentalne metode in materiali Za preiskave smo pripravili štiri betonske preizkušance dimenzij 50×50×15 cm3. Kot je razvidno iz slike 4.1, imajo preizkušanci vgrajene različne anomalije, ki smo jih opisali v preglednici 4.1. Z anomalijami iz stiropora v preizkušancu S0 smo raziskovali vpliv različne globine enake anomalije na uspešnost zaznavanja z georadarjem in s termokamero. Vključki v preizkušancih S1–S3 pa simulirajo tipične napake in vključke v gradbenih konstrukcijskih elementih, kot so lokalizirana voda in zračne votline (S1), armatura (S2), instalacijske cevi in delaminacija (S3). Za preiskave z IRT smo uporabili termokamero FLIR A320 (slika 4.2), ki s sistemom bolometrov zaznava IR svetlobo z valovno dolžino med 7,5 in 13 µm, njena maksimalna občutljivost pa je 50 mK (pri 30 ◦C). Za gretje preizkušancev smo uporabili IR grelca z močjo po 1,2 kW, postavljena 45 cm od površine preizkušanca, da je bilo zagotovljeno približno enakomerno segrevanje. V sklopu optimizacije časa segrevanja smo preizkušance greli 5, 15, 30 in 45 min. Temperaturno ohlajanje smo spremljali 45 min (za potrebe pulzno-fazne termografije 150 min) pri frekvenci zajemanja slik 0,2 Hz. Georadarske meritve smo opravili z georadarskim sistemom MALÅ Geoscience. Uporabili smo zaščiteno oddajno-sprejemno anteno s središčno frekvenco 1,6 GHz (slika 4.3). Georadarske pro- file smo zajemali s pomočjo merilnega koleščka v obeh smereh pri medsebojnem razmiku med vzporednimi georadskimi profili 5 cm in pri razmiku med posameznimi točkovnimi meritvami 0,5 cm (razmik med georadarskimi sledmi). Za združevanje georadarskih podatkov z rezultati IRT smo uporabili grafične prikaze prereza. Te smo pridobili s koraki predhodne obdelave, in sicer z odstranitvijo ojačitve signala, premikom signala in popravkom začetnega časa. 6.4.2 Rezultati Karakterizacija anomalij z infrardečo termografijo Karakterizacijo anomalij z IRT smo opravili s pomočjo temperaturnega kontrasta in tehnike pulzno-fazne termografije. Iz slike 4.6 je razvidno, da lahko zračno in vodno anomalijo ločimo iz razlike v temperaturnem kontrastu. Za to je v največji meri odgovorna razlika v toplotni prevodnosti med anomalijo in preiskovanim materialom, v našem primeru betonom. Znižanje temperaturnega kontrasta za anomalijo pod armaturno mrežo rastra 5 cm prav tako vpliva na slabše zaznavanje anomalij, medtem ko armaturna mreža rastra 10 cm ne vpliva na potek tem- peraturnega kontrasta. Rezultati na slikah 4.5 in 4.6 kažejo, da je temperaturni kontrast močno odvisen od vrste anomalije, medtem ko je čas tCmax, tj. čas nastopa maksimalnega temperaturnega kontrasta (Cmax), od te skoraj neodvisen in določen le z njeno globino. To omogoča, da z Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 75 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. opazovanjem obeh parametrov, Cmax in tCmax, kvalitativno določimo vrsto anomalije in njeno globino. Ugotovitev potrjujejo kontrastne in časovne slike vseh preizkušancev na slikah 4.7–4.10. Na sliki 4.11 vidimo, da je pri vseh globinah (1,5–7,5 cm) optimalni čas segrevanja 30 min. Tam namreč doseže Cmax največje vrednosti in obenem najgloblji doseg – do 6 cm. Odvisnost Cmax in tCmax od globine anomalije smo poskušali opisati z modelom po enačbi (2.36). Rezultati prilagajanja (slika 4.12) kažejo, da model ne more opisati eksperimentalnih rezultatov (vre- dnost determinacijskega koeficienta znaša R2 = 0, 25). V nasprotju s tem pa lahko model po enačbi (2.44), ki temelji na pulzno-fazni termografiji in frekvenci največjega faznega kontrasta fch, z veliko natančnostjo opiše eksperimentalne rezultate (vrednost determinacijskega koeficienta znaša R2 = 0, 99) – glej sliko 4.14. Ugotovitve eksperimentalnih rezultatov potrjujejo tudi rezultati numerične simulacije (sliki 4.18 in 4.19). Ugotovitev, da lahko fch natančno koreliramo z globino anomalije, omogoča, da fazne slike pri določenih frekvencah interpretiramo kot slike na ustrezni globini. Dodatna prednost fazno- kontrastnih slik je, da različne termične lastnosti anomalij vplivajo le na velikost faznega kon- trasta, ne pa na fch (slika 4.16). Fazno-kontrastne slike imajo obenem velik kontrast in zelo dobro prostorsko ločljivost (slika 4.15). Za združevanje z georadarskimi grafičnimi prikazi prereza smo zato uporabili fazno-kontrastne slike. Analiza občutljivosti georadarja in infrardeče termografije Občutljivost georadarske metode in IRT na zaznavanje anomalij pri različnih globinah smo kvan- titativno ocenili s predlaganim parametrom c_sens, kot ga podaja enačba (4.2). Rezultati zbrani na sliki 4.21 kažejo, da ima georadar nizko vrednost c_sens na globini 1,5 cm (c_sens = 84, 5 %), kar potrjujejo tudi slike preizkušanca S3 na sliki 4.22. Kljub temu se njegova vrednost že na globini 3 cm naglo poveča. V nasprotju z georadarjem občutljivost IRT z globino pada, obenem pa je metoda tudi bolj občutljiva na spremembe materialnih lastnosti anomalij. Združevanje podatkov s kvantitativno validacijo Georadarske in IR slike smo združili z uporabo značilnic po enačbi (8) v prilogi A (v nadaljevanju poglavja 6.4.2 sklicevanje na članek v prilogi A opuščam, saj se vsi nadaljnji rezultati navezujejo nanj). Kot značilnice so služile mere pripadnosti podatkov uij, ki smo jih predhodno dobili z gručenjem. Rezultate združevanja z gručenjem smo kvantitativno ovrednotili s parametrom c_sens ter primerjali z rezultati simbolnega združevanja podatkov. Za simbolno združevanje smo uporabili uteženo povprečje in D-S teorijo evidenc, za uteži oz. verjetnostne mase pa uporabili rezultate občutljivostne analize. Kvalitativni rezultati združevanja so prikazani na slikah 2–4. Za preizkušanec S1, kjer sta bili vrednosti c_sens za georadar in IRT zelo podobni, je razvidno, da dobimo s simbolnim združe- vanjem podobne rezultate kot z združevanjem z uporabo značilnic. Na sliki preizkušanca S2 pa se pokaže, da združevanje z uporabo značilnic bistveno izboljša zaznavanje anomalije pod gostejšo armaturno mrežo, obenem pa zmanjša šum zaradi zaznane armaturne mreže na georadarski sliki. Tudi za preizkušanec S3 (slika 4) dobimo z združevanjem z gručenjem večji kontrast slike kot s 76 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. simbolnim združevanjem, saj lahko jasneje zaznamo cevi v preizkušancu. Kvantitativni rezultati združevanja so zbrani v preglednici 2 (pri združevanju z uporabo značilnic prikazujemo le rezultate, ki smo jih dobili z uporabo metode FCM). Pokaže se, da združevanje georadarskih in IR slik z uporabo značilnic v splošnem poveča zanesljivost neporušne preiskave. Celo v primeru preizkušanca S3, kjer je bila občutljivosti IRT bistveno večja od občutljivosti georadarja, je to združevanje le malenkost poslabšalo občutljivost IRT. Simbolno združevanje z uporabo uteženega povprečja in D-S teorije evidenc da zelo podobne rezultate. V primerih, ko je bila večja utež dodeljena neporušni metodi z boljšo sposobnostjo zaznavanja, sta obe metodi navkljub pričakovanjem le malenkostno povečali občutljivost preiskave. V primerih, ko utež ni bila v skladu z občutljivostjo georadarja in IRT (preizkušanec S2 z armaturo), pa sta metodi bistveno poslabšali sposobnost zaznavanja, kar kaže na veliko odvisnost simbolnega združevanja od izbire uteži. Primerjava delovanja različnih algoritmov mehkega gručenja je podana na sliki 5 in v preglednici 3. Algoritem PFCM da zelo podobne rezultate kot osnovni mehki algoritem FCM, kar kaže na dejstvo, da šum ne vpliva na zaznavanje obravnavanih anomalij. To je v nasprotju z združevanjem rezultatov neporušnih preiskav materialov v strojništvu, kjer je vpliv šuma izrazit [67]. Za delovanje algoritma GK se izkaže, da je parameter γ najbolje omejiti na γ = 0, 1. Pri tem doseže algoritem GK podobno obnašanje kot FCM, obenem pa tudi bolje kot FCM in PFCM zazna cevi. Algoritem GK izkazuje glede na robustnost metode in sposobnost zaznavanja majhnih sprememb v strukturi najboljše delovanje. 6.5 Neporušne preiskave laboratorijskih ometanih večslojnih kamnitih zidov 6.5.1 Eksperimentalne metode in materiali Neporušne preiskave smo v okviru evropskega projekta PERPETUATE (www.perpetuate.eu) opravili na štirih laboratorijskih ometanih trislojnih kamnitih zidovih dimenzije 100 × 150 × 40 cm3. Za gradnjo smo uporabili pol obdelan kamen iz apnenca ter apneno malto. Dva zidova sta imela v vsaki drugi vrsti povezovalne kamnite bloke, ki so po prerezu potekali čez vse tri sloje (vezniška zveza) – glej sliko 1 v prilogi B (v nadaljevanju poglavja 6.5 sklicevanje opuščam). Georadarske in IRT preiskave smo opravili z isto merilno opremo kot za neporušne preiskave labo- ratorijskih betonskih preizkušancev, ki sem jih opisala v poglavju 6.4.1. Za IRT smo preizkušance greli približno 30 min z IR grelci na oddaljenosti 10 cm, njihovo ohlajanje pa merili 45 min pri frekvenci zajemanja slik 0,2 Hz. Postopek georadarske meritve ustreza opisu v poglavju 6.4.1, prav tako tudi koraki predhodne obdelave georadarskih in IR podatkov. Z neporušnimi preiskavami smo preverili možnost zaznavanja teksture in morfologije zidov. Prek simulirane seizmične obremenitve s cikličnim strižnim testom (glej sliki 2 in 3) smo preverili še možnost zaznavanja odstopanja ometa in nastanka razpok. Poleg neobremenjenega zidu smo spremljali dve stanji obremenitve. Prvo stanje ustreza začetnemu odstopanju ometa in pojavu prvih razpok (prvo in drugo mejno stanje ometa iz preglednice 1), drugo stanje pa ustreza močno poškodovanemu oz. odstopanemu ometu (tretje mejno stanje ometa iz preglednice 1). Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 77 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Za validacijo učinkovitosti zaznavanja odstopanja ometa smo uporabili tako vizualen pregled kot tudi 3D fotogrametrično tehniko. 6.5.2 Rezultati Vizualizacija morfologije zidov Morfologijo trislojnih zidov smo jasno zaznali z georadarskih meritev, kot je prikazano na sliki 4. Iz georadarskega profila in grafičnega prikaza prereza je razvidno, da se odboji pojavijo v vsaki drugi vrsti, kjer ni povezovalnih kamnitih blokov. Sklepamo lahko, da je razlika v dielektrični konstanti med malto in kamnom dovolj velika, da lahko zaznamo odbiti signal. Rezultati na sliki 4 še kažejo, da smo z veliko natančnostjo lahko določili debelino zidov. Vizualizacija teksture zidov Pri zaznavanju anomalij zelo blizu površja smo z georadarjem omejeni z vertikalno ločljivostjo antene in tudi z reaktivno cono, kjer je možna induktivna sklopitev odbitega valovanja s poljem antene. Kljub temu smo podometno teksturo zidu lahko zaznali z grafičnega prikaza prereza na globini 1,3 cm (slika 5a). Teksturo bistveno slabše zaznamo, če je pri meritvi medsebojni razmik med vzporednimi georadskimi profili 10 cm (slika 5b). Iz slike 5 je še razvidno, da dobimo pri IRT s faznimi slikami veliko boljšo ločljivost, kot je to možno s termogrami (slikami površinskih temperatur). Zaznavanje odstopanja ometa in širjenja razpok z metodo gručenja Na sliki 6 so prikazani rezultati georadarske in IRT meritve pri posameznih stanjih obremenitve zidu kot tudi rezultati fotogrametrične preiskave. Tako georadarski kot termografski rezultati kažejo, da se z večanjem strižne sile površina odstopanega ometa veča – pri georadarju to vidimo iz večjih amplitud, pri faznih slikah pa iz večjih faz. Kljub temu pa se natančnost zaznavanja med metodama bistveno razlikuje. Iz primerjave z rezultati fotogrametrične preiskave vidimo, da lahko z IRT zaznamo 2 mm zračno rego že v prvem stanju obremenitve zidu, medtem ko georadar zazna odstopanje v velikosti 8 mm šele v drugem stanju obremenitve. Razpoke smo zaznali na faznih slikah pri višjih frekvencah, saj so te občutljive na manjše tempe- raturne kontraste, medtem ko smo odstopanje ometa vselej zaznali pri najnižji frekvenci. Primer zaznane površinske razpoke v drugem stanju obremenitve zidu je prikazan za IR fazno sliko pri višji frekvenci (2, 22 · 10−3 Hz) na sliki 8. Slika 8 še prikazuje odstopanje ometa, ki smo ga zaznali na georadarskem grafičnem prikazu prereza in IR fazni sliki pri osnovni frekvenci (3, 7 · 10−4 Hz). Kot je prikazano na sliki 9 smo z IR faznimi slikami pri višjih frekvencah lahko zaznali tudi podometne razpoke. Zaradi kompleksne strukture samega zidovja anomalij nismo mogli kvantitativno karakterizirati z uporabo pulzno-fazne termografije. V praksi bi za zaznavanje anomalij v večslojnih zidovih tako bilo potrebno združevanje georadarskih in termografskih rezultatov, vendar ne po globini. V našem primeru smo združevanje z gručenjem zato izvedli na nivoju posamezne slikovne točke za 78 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. segmentacijo združenih georadarskih in IR slik. Rezultat na sliki 8 pokaže, da lahko z gručenjem (z algoritmom GK) zaznamo tako odstopan omet kot tudi površinsko razpoko. Z uteženim povprečjem razpoke ne moremo zaznati. 6.6 Neporušne preiskave laboratorijskega opečnatega zidu pod vplivom različne vlažnosti 6.6.1 Eksperimentalne metode in materiali Neporušne tomografske preiskave smo izvedli na laboratorijskem opečnatem zidu dimenzije 139× 99 × 74 cm3. Preizkušanec vsebuje votline različnih velikosti. Tomografske preiskave smo izvedli za prerez zidu čez največjo votlino dimenzije 27 × 27 × 27 cm3 – glej sliko 1 v prilogi C (v nadaljevanju poglavja 6.6 sklicevanje opuščam). Raziskavo vpliva vlažnosti na zaznavanje votline z georadarjem, ultrazvočno metodo in geoelektrično (kompleksno-uporovno) metodo smo izvedli za dve vlažnosti (za 3 in 8 masnih %). Preizkušanec pri 3 masnih % vlažnosti imenujemo suh, pri 8 masnih % pa vlažen. Georadarske meritve smo opravili z georadarskim sistemom SIRveyor SIR-20 podjetja GSSI. Uporabili smo ločeni oddajni in sprejemni anteni s središčno frekvenco 1,5 GHz. Točkovne meritve smo z vseh strani preizkušanca zajemali na razmiku 13 cm na suhem zidovju in 5 cm na vlažnem (glej sliki 2 in 3). Ultrazvočno tomografijo smo opravili z longitudinalnimi piezoelektričnimi senzorji tipa Acsys SO202, ki oddajajo zvočno valovanje s frekvenco 25 kHz. Uporabili smo 40 µs dolg kvadratni pulz s frekvenco ponavljanja 20 Hz. Dober stik senzorja s preizkušancem smo zagotovili z uporabo vazelina. Postopek meritve je bil enak kot pri georadarski tomografiji. Geoelektrične meritve smo izvedli z dvema merilnima opremama – SIP256c in LGM 4-Point- Light (prva za razliko od druge uporablja višje napetosti in deluje v širšem frekvenčnem ob- močju). Uporabili smo tri geometrijske razporeditve elektrod: dipol-dipol, Wenner in Wenner- Schlumberger (slika 2.2 v poglavju 2). Meritve smo zajemali s pomočjo EKG elektrod na razmiku 5 cm. 6.6.2 Rezultati Rezultati georadarske in ultrazvočne tomografije Za potrebe tomografskega računa smo iz vseh točkovnih georadarskih in ultrazvočnih meritev določili čas prvega prispelega signala. Ker je izbira začetnega časa lahko podvržena vplivu različnih napak, smo izvedli kalibracijske meritve valovanja v zraku. Opravili smo tudi kontrolo hitrosti valovanja in recipročnosti posamezne meritve. Hitrost valovanja smo izračunali iz znane razdalje med oddajnikom in sprejemnikom ter izmerjenega časa, s kontrolo recipročnosti pa smo preverili razliko v potovalnem času za nasprotno postavljena pretvornika (slika 4). Iz slike 6 je razvidno, da imajo hitrosti, izračunane na podlagi večjih razdalj, večji raztros vrednosti zaradi večjega števila sipanj in lomov valovanja. Večji raztros izračunanih hitrosti izkazuje tudi vlažni Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 79 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. preizkušanec v primerjavi s suhim. To potrjujejo tudi rezultati kontrole recipročnosti (slika 7), ki tudi izključujejo vpliv večjih sistematičnih napak na meritev. Preglednica 1 kaže, da imajo ultrazvočni rezultati na suhem in vlažnem zidovju večji raztros vrednosti kot rezultati georadarske preiskave. Vidimo tudi, da je ultrazvočna metoda bolj občutljiva na spremembe vlažnosti zidovja kot georadar. Tomografske slike, ki kažejo porazdelitev hitrosti EM in ultrazvočnega valovanja v obravna- vanem prerezu preizkušanca, so prikazane na sliki 8. Rezultati na slikah potrjujejo zgornje ugo- tovitve, da večje razlike nastopijo v porazdelitvi hitrosti zvočnega valovanja. Slabše delovanje ultrazvočne metode napram georadarju pri zaznavanju votline lahko razumemo kot posledico ra- zličnega načina potovanja valovanja (zvočno valovanje praktično ne potuje skozi zračno votlino, medtem ko EMV potuje). Preglednica 2 kaže, da so hitrosti obeh valovanj v vlažnem zidovju manjše od hitrosti v suhem zidovju (povprečna hitrost EMV v suhem zidovju je 0,13 m/ns, v vlažnem 0,10 m/ns, povprečna hitrost zvočnega valovanja v suhem zidovju pa je 3100 m/s, v vlažnem 2800 m/s). Iz slike 8 še vidimo, da ima vlažnost zidovja pomembnejšo vlogo na način zaznavanja votline kot sam inverzni postopek. Način zaznavanja votline v odvisnosti od vlažnosti zidovja z obema metodama smo kvantitativno ovrednotili z enačbama (7) in (8). Preglednica 2 kaže, da se sposobnost zaznavanja v vlažnem zidovju za georadarsko tomografijo zmanjša za 9 %, medtem ko je ta padec v primeru ultrazvočne tomografije 14 %. Rezultati geoelektrične tomografije Meritve specifične upornosti so bile v splošnem precej zašumljene, najverjetneje zaradi slabega stika elektrod s preizkušancem in zaradi majhnih tokov pri meritvi. To se je še posebej pokazalo za merilno opremo SIP256c, zato v članku prikazujemo le rezultate meritev z merilno opremo LGM 4-Point-Light. Na sliki 10 so za merjeni prerez prikazane amplitude specifične upornosti za Wennerjevo razporeditev elektrod, ki se je v preiskavi večjega števila geometrijskih razporeditev elektrod izkazala kot najustreznejša. Faz kompleksne specifične upornosti nismo mogli upoštevati kot merodajen rezultat zaradi majhnega razmerja signala proti šumu. Iz slike 10 je razvidno, da so vrednosti amplitud specifične upornosti suhega zidovja med 400 in 700 Ωm, za vlažno zidovje pa se vrednosti amplitud gibljejo med 12 in 24 Ωm. Razvidno je, da spodnjega robu votline z geoelektrično tomografijo ne moremo natančno določiti. To je posledica slabše globinske ločljivosti geoelektrične metode glede na georadar in ultrazvočno metodo. Kljub temu lahko votlino zaznamo že pri suhem zidovju (z 20 % zanesljivostjo), na vlažnem zidovju pa se sposobnost zaznavanja enkrat poveča. 6.7 Neporušne preiskave z večsenzorskim robotnim sistemom BetoScan Večsenzorski robotni sistem BetoScan [160] je bil razvit v okviru projekta BetoScan (www.beto- scan.de). Na inštitutu BAM sistem uporabljajo za neporušne preiskave večjih betonskih površin (v garažnih hišah in na mostovih), ki so poškodovane zaradi korozije armature. Glavni namen preiskav je določitev con poškodovanosti z večjim številom senzorjev, kot prikazuje slika 7 v 80 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. prilogi A (sklicevanje na prilogo v nadaljevanju poglavja 6.7 opuščam). Metode gručenja smo zato uporabili na nivoju posamezne slikovne točke za segmentacijo združenih slik EM potencialne metode, metode z vrtinčnimi tokovi in dveh mikrovalovnih merilcev vlage. 6.7.1 Eksperimentalne metode in materiali Analizirali smo rezultate meritev armirane plošče določene garažne hiše v Nemčiji. Površina preiskovanega območja je merila 5 × 7 m2. Razmik med sledmi vozila je znašal 20 cm, med posameznimi točkovnimi meritvami pa 1 cm. Za EM potencialno metodo, ki meri elektrokemijski potencial armature glede na referenčno elektrodo, smo uporabili napravo Canin+ podjetja Proceq v kombinaciji z referenčno elektrodo Cu/CuSO4. Za metodo z vrtinčnimi tokovi smo uporabili napravo Profometer 5+ podjetja Proceq, ki na podlagi elektromagnetne indukcije omogoča tako lokalizacijo armature kot tudi določitev debeline krovnega sloja betona. Za meritev relativne vlažnosti betona smo uporabili dva mikrovalovna merilca podjetja HF-Sensor – Moist RP (globina penetracije je 4 cm) in Moist PP (globina penetracije je 20 cm). Za validacijo rezultatov metod iz sistema BetoScan smo uporabili rezultate radar-magnetne metode [161]. S to metodo lahko določimo realno in imaginarno komponento dielektrične kon- stante preiskovanega materiala. 6.7.2 Rezultati Na sliki 8a območja z vrednostjo elektrokemijskega potenciala pod -300 mV nakazujejo na visoko verjetnost korozije armature v plošči. Glede na to, da sta za pojav korozije armature odgo- vorna tako karbonatizacija betona (kot posledica majhnega krovnega sloja betona) kot tudi velika koncentracija kloridnih ionov, smo rezultate EM potencialne metode skušali razložiti z rezultati metode z vrtinčnimi tokovi in z rezultati meritve vlažnosti. S prvo metodo dobljene meritve krovnega sloja betona kažejo, da je za pojav korozije na določenih mestih odgovorna karbonatizacija betona (slika 8b), medtem ko je rezultate meritve vlažnosti težje interpretirati (sliki 8c in 8d). Boljšo korelacijo s koncentracijo kloridnih ionov v betonu da meritev z radar- magnetno metodo, in sicer rezultati imaginarne komponente dielektrične konstante (slika 8f). Razvidno je, da stopnje poškodovanosti armirane plošče zaradi korozije ne moremo oceniti na podlagi ločene interpretacije rezultatov posamezne metode. Za različne kombinacije neporušnih metod smo zato z gručenjem opravili segmentacijo merjenega območja na cone poškodovanosti. Rezultate prikazujemo za algoritem GK s parametrom γ = 0, 1, ki se je že na rezultatih laborato- rijskih betonskih preizkušancev izkazal kot najustreznejši. Kot je razvidno iz slike 9, lahko s celo- tnim večsenzorskim sistemom BetoScan z veliko natančnostjo določimo tri cone poškodovanosti armirane plošče zaradi korozije. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 81 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 6.8 Uporaba metod gručenja za avtomatsko klasifikacijo spektroskopskih podatkov na betonskih preizkušancih Spektroskopijo lasersko inducirane plazme (LIBS) na inštitutu BAM uporabljajo za določitev koncentracije in porazdelitve klora, natrija in žvepla v vzorcih z betonskih konstrukcij. Pri tem ima ključno vlogo določitev spektrov, ki pripadajo le cementni matriki. Za ta namen smo preverili uporabo metod gručenja. 6.8.1 Eksperimentalne metode in materiali LIBS, ki ga uporablja inštitut BAM, je shematično predstavljen na sliki 1 v prilogi D (v nadalje- vanju poglavja 6.8 sklicevanje na prilogo opuščam). Za izvor svetlobe služi laser Nd:YAG (Innolas Spitlight 600) v kombinaciji s sistemom leč. Za odstranitev prašnatih delcev in ojačenje signala uporabljajo helij. Zaznavanje svetlobe omogoča spektrometer tipa Shamrock 303i z detektorjem CCD. Premikanje vzorca omogoča translacijska mizica. Uporabo gručenja za klasifikacijo spektroskopskih podatkov smo preverili na večjem številu vzorcev. V delu podrobneje prikazujemo rezultate z laboratorijskega preizkušanca, na katerem so bili opravljeni testi alkalno-silikatne reakcije v 3,6 % raztopini NaCl pri 40 ◦C. Globinski prerez preizkušanca je prikazan na sliki 2. Za pripravo preizkušanca smo uporabili agregat iz kre- menovega peščenjaka ter cement tipa CEM I. Spektroskopsko meritev smo opravili na površini 94, 5 × 95 mm2, pri ločljivosti 0,5 oz. 1 mm. 6.8.2 Rezultati Za gručenje smo obravnavali naslednje spektralne črte: Ca (849,80 nm), O (844,64 nm), Na (818,32 nm), Cl (837,59 nm), C (833,52 nm) in Ti (846,85 nm). Normalizirane intenzitete posame- zne spektralne črte prikazujemo na sliki 4. Razvidno je, da je stopnja heterogenosti preizkušanca visoka. Zato smo za referenčno klasifikacijo vzorca na območje agregata in cementne matrike (slika 6) uporabili tako rezultate standardizirane metode klasifikacije, ki uporablja spektralni črti Ca in O, kot tudi natančen vizualni pregled vzorca. Na podlagi dveh kazalcev (XB in S) [153] smo določili, da je optimalno število razredov za gručenje 5. Rezultati gručenja so predstavljeni na sliki 7. Razvidno je, da algoritem z doda- tnim razredom za najbolj zašumljene podatke (NCFCM) pravilno zbere točke, ki imajo visoko intenziteto natrijeve spektralne črte (slika 7d). Iz slike 7 še opazimo koincidenčni problem algo- ritma PCM s kazenskim členom (slika 7e). Delovanje vseh algoritmov je strnjeno v preglednici 1. Razvidno je, da sta za segmentacijo vzorca najuspešnejša algoritma NCFCM in PFCM. V praksi se spektroskopska meritev vzorcev izvaja pri ločljivosti 1 oz. 2 mm. Zato smo za to ločljivost preverili klasifikacijo spektroskopskih podatkov za določitev koncentracije natrijevih ionov kot posledica alkalno-silikatne reakcije. Globinski profil intenzitete natrijeve spektralne črte na sliki 8 kaže, da je klasifikacija podatkov z vsemi obravnavanimi metodami mehkega gručenja natančna. Klasifikacija s standardizirano metodo, ki uporablja spektralni črti Ca in O, močno odstopa od referenčne klasifikacije. 82 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Za popolno avtomatizacijo postopka klasifikacije spektroskopskih podatkov z algoritmoma NC- FCM in PFCM predlagamo uporabo kazalca XB. Za PFCM je merodajen njegov globalni minimum, medtem ko je za NCFCM vrednost kazalca potrebno omejiti okoli 1. 6.9 Zaključki in obeti Glavni namen disertacije je bil razviti ustrezen postopek za združevanje slik, ki bi temeljil na metodah gručenja in bi omogočil učinkovito in zanesljivo obdelavo podatkov večsenzorskih neporušnih preiskav gradbenih konstrukcijskih elementov. V delu sem opisala uporabo gručenja za združevanje podatkov večsenzorskih neporušnih preiskav laboratorijskih betonskih in zidanih preizkušancev. Metode gručenja smo uporabili tudi v dveh aplikativnih ogrodjih združevanja po- datkov: za združevanje podatkov večsenzorskega robotnega sistema BetoScan ter za popolnoma avtomatizirano klasifikacijo spektroskopskih podatkov betonskih preizkušancev. Osrednje področje naloge je bila uporaba metod združevanja podatkov na rezultatih georadarskih in IRT meritev betonskih in zidanih preizkušancev. Namen združevanja podatkov je bil izboljšati zaznavanje različnih anomalij v območju blizu površja. V sklopu preiskav smo izvedli tudi obsežno analizo občutljivosti obeh metod. Na betonskih preizkušancih analiza pokaže, da je občutljivost georadarja do 3–4,5 cm globine manjša, kot je občutljivost IRT. IRT v splošnem izkazuje počasno slabšanje sposobnosti zaznavanja do 7,5 cm, kar je predvsem posledica uporabe fazno-kontrastnih slik. Za obe metodi ugotovimo, da ima ključno vlogo pri materialni kara- kterizaciji anomalij razlika v dielektrični konstanti oz. toplotni prevodnosti med anomalijo in preiskovanim materialom. Zaradi tega georadar natančneje zazna vodno anomalijo kot zračno, obratno pa velja za IRT. Uporaba metod gručenja za združevanje podatkov na nivoju značilnic kaže, da najboljše deluje algoritem GK s parametrom γ = 0, 1. S tem algoritmom in parametrom smo hkrati dosegli robustno delovanje metode, obenem pa tudi najbolje zaznali plastične cevi v betonu. Z GK smo cevi lahko zaznali, ker algoritem prepoznava razrede poljubne oblike in ne le sferične kot FCM in PFCM. Slednja algoritma sta v splošnem pokazala zelo podobno delovanje, kar pomeni, da ima pri preiskovanju tipičnih anomalij v betonskih konstrukcijskih elementih šum manjšo vlogo. Primerjava z rezultati simbolnega združevanja potrjuje, da metode gručenja delujejo ro- bustno pri združevanju podatkov v primerih s še posebno omejenim vedenjem o sestavi strukture preizkušanca. Hkratna uporaba georadarja in IRT je omogočila tudi celostno karakterizacijo strukture ometanih kamnitih zidov. Poleg morfologije in teksture zidov smo lahko zaznali tudi odstopanje ometa. Izkaže se, da lahko IRT zazna odstopanje, ki ustreza prvemu mejnemu stanju ometa (≈ 2 mm rega), medtem ko georadar zazna šele velikost odstopanja za tretje mejno stanje (≈ 8 mm rega). Pri IRT smo s faznimi slikami pri višjih frekvencah lahko zaznali tudi površinske in podometne razpoke, medtem ko smo odstopanje ometa in teksturo zidu zaznali na faznih slikah pri najnižji frekvenci. V primeru preiskovanja zidovja se izkaže, da zaradi njegove heterogene strukture ne moremo uporabiti kvantitativnih tehnik obdelave termografskih podatkov, ki so uporabne na betonu. Zato predlagamo združevanje georadarskih in IR podatkov na nivoju posamezne slikovne Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 83 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. točke, in sicer za segmentacijo slik. Tako lahko uspešno zaznamo odstopan omet in razpoke. Meritev vpliva vlažnosti zidovja na zaznavanje votline z georadarsko, ultrazvočno in geoelektri- čno tomografijo pokaže, da je ultrazvočna metoda bolj občutljiva kot georadar na spremembe vlažnosti. Ultrazvočni podatki imajo obenem tudi večji raztros vrednosti kot georadarski po- datki. Najbolj zanesljiva neporušna metoda za zaznavanje votline v suhem in vlažnem zidovju je georadar. Za ultrazvočno tomografijo opazimo, da je za uspešno zaznavanje votline potreben daljši čas merjenja in natančna določitev časa prvega prispelega signala. Tudi z geoelektrično tomografijo lahko zaznamo votlino pri obeh vlažnostih zidovja, vendar ima metoda slabšo glo- binsko ločljivost v primerjavi z georadarjem in ultrazvokom. Dodatne težave povzroča slab stik med elektrodami in preizkušancem. Metode gručenja smo uspešno uporabili v dveh sistemih za segmentacijo podatkov neporušnih preiskav. Za identifikacijo cementne matrike v betonskih preizkušancih na podlagi spektroskop- skih (LIBS) podatkov najboljše delovanje izkazujeta algoritma NCFCM in PFCM. S predlaganim optimizacijskim postopkom dosežemo popolnoma avtomatizirano klasifikacijo podatkov, kar ima ključen pomen za prenos tehnike LIBS v prakso. Na podatkih večsenzorskega robotnega sis- tema BetoScan metode gručenja omogočijo določitev con poškodovanosti armirane plošče zaradi korozije. Na podlagi zgornjih ugotovitev lahko zaključimo, da so metode gručenja dovolj robustne, da jih lahko implementiramo v različne večsenzorske sisteme za neporušne preiskave v gradbeništvu. Njihova prednost je, da jih lahko v celoti avtomatiziramo, kar ima ključen pomen pri prenosu neporušnih preiskav v prakso v obliki uporabniku preproste, a vendar zanesljive interpretacije podatkov. 84 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 85 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. REFERENCES [1] Büyüköztürk, O. (ed.), Taşdemir, M. A. (ed.), Güneş, O. (ed.), Akkaya, Y. (ed.). 2013. Nondestructive Testing of Materials and Structures, RILEM Bookseries, Vol. 6. Springer Netherlands: 1278 p. [2] Whiting, D., Todres, A., Nagi, M., Yu, T., Peshkin, D., Darter, M., Holm, J., Andersen, M., Geiker, M. 1993. Synthesis of Current and Projected Concrete Highway Technology. Strategic Highway Research Program. Washington, DC, National Research Council: 286 f. [3] Rhazi, J. 2001. NDT in civil engineering: the case of concrete bridge decks. e-Journal of Nondestructive Testing & Ultrasonics 6, 5. http://www.ndt.net/article/v06n05/rhazi/ rhazi.htm (Accessed 20 December 2013). [4] Banville, M. H. 2008. Assessment and repair of concrete structures. In: RCI’s 23rd Int. Convention & Trade Show, Phoenix, Arizona, Feb 28 – Mar 4, 2008. pp. 1–9. http://www.rci-online.org/interface/2008-07-banville.pdf (Accessed 14 February 2014). [5] Verma, S. K., Bhadauria, S. S., Akhtar, S. 2013. Review of nondestructive testing methods for condition monitoring of concrete structures. Journal of Construction Engineering 2013: 1–11. [6] Wiggenhauser, H. 2009. Advanced NDT methods for quality assurance of concrete struc- tures. In: Abraham, O. (ed.), Derobert, X. (ed.). Proc. of the 7th Int. Symp. on NDT in Civil Engineering (NDTCE’09), Nantes, France, Jun 30 – Jul 3, 2009. Nantes, LCPC: pp. 1–12. http://www.ndt.net/article/ndtce2009/papers/1002.pdf (Accessed 19 February 2012). [7] Schutter, G. D. 2012. Damage to Concrete Structures. Boca Raton, FL, CRC Press: 209 p. [8] Wiggenhauser, H., Taffe, A. 2012. NDT in civil engineering: research, application, valida- tion and training. In: Manfred, J. (ed.). Proc. of the 18th World Conf. on Nondestructive Testing, Durban, South Africa, Apr 16–20, 2012. SAINT: pp. 1–13. http://www.ndt.net/article/wcndt2012/papers/612_wcndtfinal00611.pdf (Accessed 14 February 2014). [9] Binda, L., Saisi, A. 2009. Application of NDTs to the diagnosis of historic structures. In: Abraham, O. (ed.), Derobert, X. (ed.). Proc. of the 7th Int. Symp. on NDT in Civil Engineering (NDTCE’09), Nantes, France, Jun 30 – Jul 3, 2009. Nantes, LCPC: pp. 1–28. http://ndt.net/article/ndtce2009/papers/1005.pdf (Accessed 19 February 2012). [10] Breysse, D. (ed.). 2012. Non-Destructive Assessment of Concrete Structures: Reliability and Limits of Single and Combined Techniques, State-of-the-Art Report of the RILEM Technical Committee 207-INR, RILEM State-of-the-Art Reports, Vol. 1. Springer Nether- lands: 374 p. [11] McCann, D. M., Forde, M. C. 2001. Review of NDT methods in the assessment of concrete and masonry structures. NDT&E Int. 34, 2: 71–84. [12] Wiggenhauser, H. 2009. Automated NDE of PT concrete structures. In: NDT presenta- tions, ACI, Seville, Spain, Oct 15, 2009. http://www.germann.org/Publications/Sevilla/Automated%20NDE%20of%20PT% 20concrete%20structures,%20Wiggenhauser.pdf (Accessed 19 February 2012). 86 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [13] Maierhofer, C., Wöstmann, J. 1998. Investigation of dielectric properties of brick materials as a function of moisture and salt content using a microwave impulse technique at very high frequencies. NDT&E Int. 31, 4: 259–263. [14] Maierhofer, C., Zacher, G., Kohl, C., Wöstmann, J. 2008. Evaluation of radar and com- plementary echo methods for NDT of concrete elements. J. Nondestruct. Eval. 27, 1–3: 47–57. [15] Tsui, F., Matthews, S. L. 1997. Analytical modelling of the dielectric properties of concrete for subsurface radar applications. Constr. Build. Mater. 11, 3: 149–161. [16] Maierhofer, C., Kind, T. 2003. Application of georadar for the determination of moisture content and distribution of masonry. In: Binda, L. (ed.), de Vekey, R. C. (ed.). Proc of the Int. RILEM Workshop on On Site Control and Non Destructive Evaluation of Masonry Structures and Materials, Mantova, Italy, Nov 12–14, 2003. Bagneux, RILEM: pp. 227–238. [17] Maierhofer, C., Leipold, S. 2001. Radar investigation of masonry structures. NDT&E Int. 34, 2: 139–147. [18] Krause, M., Maierhofer, C., Wiggenhauser, H. 1995. Thickness measurement of concrete elements using radar and ultrasonic impulse echo techniques. In: Forde, M. C. (ed.). Proc. of the 6th Int. Conf. on Structural Faults and Repair, London, U.K., Jul 3, 1995, Vol. 1. Edinburgh, Engineering Technics Press: pp. 17–24. [19] Afshari, A., Frazer, D., Creese, R. 1996. Ultrasonic techniques for the bonding of rebar in concrete structure. In: Hartbower, P. E. (ed.), Stolarski, P. J. (ed.). Structural Materials Technology – An NDT Conference, San Diego, U.S.A., Feb 20–23, 1996. Lancaster, Basel, Technomic Publishing Co., Inc.: pp. 3–8. [20] Krause, M., Mielentz, F., Milmann, B., Streicher, D., Müller, W. 2003. Ultrasonic imaging of concrete elements: state of the art using 2d synthetic aperture. In: Proc. of the Int. Symp. Non-Destructive Testing in Civil Engineering (NDTCE 2003), Berlin, Germany, Sep 16–19, 2003. Berlin, DGZfP: V51, BB 85-CD. [21] Hévin, G., Abraham, O., Pedersen, H. A., Campillo, M. 1998. Characterization of surface cracks with rayleigh waves: a numerical model. NDT& E Int. 31, 4: 289–297. [22] Krause, M., Milmann, B., Mielentz, F., Streicher, D., Redmer, B., Mayer, K., Langenberg, K.-J., Schickert, M. 2008. Ultrasonic imaging methods for investigation of post-tensioned concrete structures: a study of interfaces at artificial grouting faults and its verification. J. Nondestruct. Eval. 27, 1–3: 67–82. [23] Im, S. B., Hurlebaus, S., Trejo, D. 2010. Inspection of voids in external tendons of post- tensioned bridges. Transport. Res. Rec. 2172: 115–122. [24] Krause, M., Mayer, K., Friese, M., Milmann, B., Mielentz, F., Ballier, G. 2011. Progress in ultrasonic tendon duct imaging. Eur. J. Environ. Civ. En. 15, 4: 461–485. [25] Tinkey, Y., Olson, L. D., Wiggenhauser, H. 2005. Impact echo scanning for discontinuity detection and imaging in posttensioned concrete bridges and other structures. Mater. Eval. 63, 1: 64–69. [26] Ata, N., Mihara, S., Ohtsu, M. 2007. Imaging of ungrouted tendon ducts in prestressed concrete by improved SIBIE. NDT&E Int. 40, 3: 258–264. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 87 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [27] Beutel, R., Reinhardt, H. W., Grosse, C. U., Glaubitt, A., Krause, M., Maierhofer, C., Algernon, D., Wiggenhauser, H., Schickert, M. 2008. Comparative performance tests and validation of NDT methods for concrete testing. J. Nondestruct. Eval. 27, 1–3: 59–65. [28] Sadri, A. 2003. Application of impact-echo technique in diagnoses and repair of stone masonry structures. NDT&E Int. 36, 4: 195–202. [29] Cantini, L. 2011. Research on the construction roles of ancient buildings: use of appropriate diagnostic non-destructive techniques. Doctoral Dissertation. Milano, (L. Cantini): 323 f. [30] Uranjek, M. 2011. Propadanje in trajnostna obnova ovoja stavbne dediščine (Degradation and sustainable renovation of heritage buildings envelope). Doktorska disertacija (Doctoral Dissertation). Ljubljana, (M. Uranjek): 261 f. [31] Maierhofer, C., Röllig, M., Arndt, R. 2009. Non-destructive testing of concrete and masonry structures using active thermography – influence of surface and bulk properties. Mater. Test. 51, 9: 587–592. [32] Bosiljkov, V., Maierhofer, C., Köpp, C., Wöstmann, J. 2010. Assessment of structure through non-destructive tests (NDT) and minor destructive tests (MDT) investigation: Case study of the church at carthusian monastery at Zice (Slovenia). Int. J. Archit. Herit. 4, 1: 1–15. [33] Bosiljkov, V., Uranjek, M., Žarnić, R., Bokan-Bosiljkov, V. 2010. An integrated diagnostic approach for the assessment of historic masonry structures. J. Cult. Herit. 11, 3: 239–249. [34] Arndt, R., Maierhofer, C., Röllig, M., Weritz, F., Wiggenhauser, H. 2004. Structural investigation of concrete and masonry structures behind plaster by means of pulse phase thermography. In: Proc. of the 7th Int. Conf. on Quantitative Infrared Thermography (QIRT), Rhode-St.-Genese, Belgium, Jul 5–8, 2004. pp. 1–6. [35] Maierhofer, C., Röllig, M. 2009. Active thermography for the characterization of surfaces and interfaces of historic masonry structures. In: Abraham, O. (ed.), Derobert, X. (ed.). Proc. of the 7th Int. Symp. on NDT in Civil Engineering (NDTCE’09), Nantes, France, Jun 30 – Jul 3, 2009. Nantes, LCPC: pp. 1–6. http://www.ndt.net/article/ndtce2009/papers/17.pdf (Accessed 20 February 2012). [36] Avdelidis, N. P., Moropoulou, A. 2004. Applications of infrared thermography for the investigation of historic structures. J. Cult. Herit. 5, 1: 119–127. [37] Avdelidis, N. P., Koui, M., Ibarra-Castanedo, C., Maldague, X. 2007. Thermographic studies of plastered mosaics. Infrared Phys. & Techn. 49, 3: 254–256. [38] Theodorakeas, P., Ibarra-Castanedo, C., Sfarra, S., Avdelidis, N. P., Koui, M., Maldague, X., Paoletti, D., Ambrosini, D. 2012. NDT inspection of plastered mosaics by means of transient thermography and holographic interferometry. NDT&E Int. 47: 150–156. [39] Wild, W., Buscher, K., Wiggenhauser, H. 1998. Amplitude sensitive modulation- thermography to measure moisture in building materials. In: Snell, J. (ed.), Wurzbach, R. (ed.). Thermosense XX, Vol. 3361, Proc. of SPIE, Orlando, FL, U.S.A., Apr 13, 1998. SPIE: pp. 156–163. [40] Maierhofer, C., Krankenhagen, R., Röllig, M., Schlichting, J., Schiller, M., Seidl, T., Mecke, R., Kalisch, U., Hennen, C., Meinhardt, J. 2010. Investigating historic masonry structures at different depths with active thermography. In: Maldague, X. (ed.). Proc. of the 10th Int. Conf. on Quantitative Infrared Thermography (QIRT10), Québec, Canada, Jul 27–30, 2010. Les Éboulements, É. du CAO: pp. 1–7. 88 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [41] Maierhofer, C., Brink, A., Röllig, M., Wiggenhauser, H. 2002. Transient thermography for structural investigation of concrete and composites in the near surface region. Infrared Phys. Techn. 43, 3–5: 271–278. [42] Maierhofer, C., Wiggenhauser, H., Brink, A., Röllig, M. 2004. Quantitative numerical analysis of transient IR-experiments on buildings. Infrared Phys. Techn. 46, 1–2: 173–180. [43] Maierhofer, C., Arndt, R., Röllig, M., Rieck, C., Walther, A., Scheel, H., Hillemeier, B. 2006. Application of impulse-thermography for non-destructive assessment of concrete structures. Cement Concrete Comp. 28, 4: 393–401. [44] Weritz, F., Arndt, R., Röllig, M., Maierhofer, C., Wiggenhauser, H. 2005. Investigation of concrete structures with pulse phase thermography. Mater. Struct. 38, 283: 843–849. [45] Maierhofer, C., Arndt, R., Röllig, M. 2007. Influence of concrete properties on the detection of voids with impulse-thermography. Infrared Phys. Techn. 49, 3: 213–217. [46] Shokouhi, P., Zoega, A., Wiggenhauser, H. 2010. Nondestructive investigation of stress- induced damage in concrete. Advances in Civil Engineering 2010: 9. [47] Saenger, E. H., Kocur, G. K., Jud, R., Torrilhon, M. 2011. Application of time reverse modeling on ultrasonic non-destructive testing of concrete. Appl. Math. Model 35, 2: 807–816. [48] Akevren, S. 2010. Non-destructive examination of stone masonry historic structures – quantitative IR thermography and ultrasonic testing. M. Sc. Thesis. Ankara, (S. Akevren): 126 f. [49] Maierhofer, C., Krankenhagen, R., Myrach, P., Meinhardt, J., Kalisch, U., Hennen, C., Mecke, R., Seidl, T., Schiller, M. 2013. Monitoring of cracks in historic concrete structures using optical, thermal and acoustical methods. In: Proc. of Built Heritage 2013, Moni- toring Conservation Management, Milan, Italy, Nov 18–20, 2013. Milano, Politechnico di Milano: pp. 1–8. [50] Tung, S. H., Shih, M. H., Sung, W. P. 2008. Development of digital image correlation method to analyse crack variations of masonry wall. Sadhana 33, 6: 767–779. [51] Salmanpour, A. H., Mojsilović, N. 2013. Application of digital image correlation for strain measurements of large masonry walls. In: Proc. of the APCOM & ISCM, Singapore, Dec 11–14, 2013. APACM, IACM: pp. 1–7. [52] Hall, D. L., Llinas, J. 1997. An introduction to multisensor data fusion. In: Proc. of the IEEE, Vol. 85, 1. IEEE: pp. 6–23. [53] Liu, Z., Forsyth, D. S., Komorowski, J. P., Hanasaki, K., Kirubarajan, T. 2007. Survey: state of the art in NDE data fusion techniques. IEEE T. Instrum. Meas. 56, 6: 2435–2451. [54] Krause, M., Bärmann, M., Frielinghaus, R., Kretzschmar, F., Kroggel, O., Langenberg, K., Maierhofer, C., Müller, W., Neisecke, J., Schickert, M., Schmitz, V., Wiggenhauser, H., Wollbold, F. 1997. Comparison of pulse-echo methods for testing concrete. NDT&E Int. 30, 4: 195–204. [55] Pöpel, M., Flohrer, C. 1995. Combination of a covermeter with a radar system – an improvement of radar application in civil engineering. In: Schickert, G. (ed.), Wiggen- hauser, H. (ed.). Proc. of the Int. Symp. on Non-Destructive Testing in Civil Engineering (NDT-CE), Berlin, Germany, Sep 26–28, 1995. Berlin, DGZfP: pp. 737–743. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 89 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [56] Dérobert, X., Aubagnac, C., Abraham, O. 2002. Comparison of NDT techniques on a post-tensioned beam before its autopsy. NDT&E Int. 35, 8: 541–548. [57] Naumann, J., Haardt, P. 2003. NDT methods for the inspection of highway structures. In: Proc. of the Int. Symp. Non-Destructive Testing in Civil Engineering (NDTCE 2003), Berlin, Germany, Sep 16–19, 2003. Berlin, DGZfP: BB 85-CD. [58] Goueygou, M., Abraham, O., Lataste, J. F. 2008. A comparative study of two non- destructive testing methods to assess near-surface mechanical damage in concrete struc- tures. NDT&E Int. 41, 6: 448–456. [59] Wendrich, A., Trela, C., Krause, M., Maierhofer, C., Effner, U., Wöstmann, J. 2006. Loca- tion of voids in masonry structures by using radar and ultrasonic traveltime tomography. In: Proc. of the 9th Eur. NDT Conf. (ECNDT 2006), Berlin, Germany, Sep 25–29, 2006, Vol. 11, 11. Berlin, DGZfP: pp. 1–11 (CD-ROM). [60] Binda, L., Bosiljkov, V., Saisi, A., Zanzi, L. 2006. Guidelines for the diagnostic investigation of historic buildings. In: Proc. of the 7th Int. Masonry Conf., London, UK, Oct 30 – Nov 1, 2006. London, British Masonry Society: pp. 1–10. [61] Kruschwitz, S., Niederleithinger, E., Trela, C., Wöstmann, J. 2010. Complex resistivity and radar investigation of building material: first results of field scale measurements. Acta Geophys. 58, 1: 96–113. [62] Gros, X. E. (ed.). 2001. Application of NDT Data Fusion. Boston, Dordrecht, London, Kluwer Academic Publishers: 277 p. [63] Georgel, B., Lavayssière, B. 1994. Fusion de données: un nouveau concept en cnd. In: Proc. of the 6th Eur. Conf. on NDT, Nice, France, Oct 24–28, 1994, Vol. 1. pp. 31–35. [64] Gros, X. E., Strachan, P., Lowden, D. W., Edwards, I. 1994. NDT data fusion. In: Proc. of the 6th Eur. Conf. on NDT, Nice, France, Oct 24–28, 1994, Vol. 1. pp. 355–359. [65] Gros, X. E., Liu, Z., Tsukada, K., Hanasaki, K. 2000. Experimenting with pixel-level NDT data fusion techniques. IEEE T. Instrum. Meas. 49, 5: 1083–1090. [66] Dong, C. Y., Yuan, Q., Wang, Q. 2010. A combined wavelet analysis-fuzzy adaptive algorithm for radar/infrared data fusion. Expert. Syst. Appl. 37, 3: 2563–2570. [67] Balakrishnan, S., Cacciola, M., Udpa, L., Rao, B. P., Jayakumar, T., Raj, B. 2012. De- velopment of image fusion methodology using discrete wavelet transform for eddy current images. NDT&E Int. 51: 51–57. [68] Zhang, Z. L., Wang, Q., Sun, S. H. 1999. A new fuzzy neural network architecture for multisensor data fusion in non-destructive testing. In: Proc. of IEEE Int. Fuzzy Syst. Conf. (FUZZ-IEEE ’99), Seoul, South Korea, Aug 22–25, 1999, Vol. 3. IEEE: pp. 1661– 1665. [69] Gros, X. E., Bousigue, J., Takahashi, K. 1999. NDT data fusion at pixel level. NDT&E Int. 32, 5: 283–292. [70] Horn, D., Mayo, W. R. 2000. NDE reliability gains from combining eddy-current and ultrasonic testing. NDT&E Int. 33, 6: 351–362. [71] Kaftandjian, V., Zhu, Y. M., Dupuis, O., Babot, D. 2005. The combined use of the evidence theory and fuzzy logic for improving multimodal nondestructive testing systems. IEEE T. Instrum. Meas. 54, 5: 1968–1977. 90 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [72] Ploix, M.-A., Garnier, V., Breysse, D., Moysan, J. 2011. NDE data fusion to improve the evaluation of concrete structures. NDT&E Int. 44, 5: 442–448. [73] Liu, Z., Forsyth, D. S., Safizadeh, M.-S., Fahr, A. 2008. A data-fusion scheme for quan- titative image analysis by using locally weighted regression and Dempster-Shafer theory. IEEE T. Instrum. Meas. 57, 11: 2554–2560. [74] Li, G., Huang, P., Chen, P., Hou, D., Zhang, G., Zhou, Z. 2011. Application of multi- sensor data fusion in defects evaluation based on Dempster-Shafer theory. In: Proc. of the 2011 IEEE Instrumentation and Measurement Technology Conference (I2MTC), Binjiang, China, May 10–12, 2011. IEEE: pp. 1–5. http://www.ndt.net/article/aspnde2009/papers/15.pdf (Accessed 13 February 2012). [75] Reza, H. P. R., Rezaie, A. H., Sadeghi, S. H. H., Moradi, M. H., Ahmadi, M. 2007. A density-based fuzzy clustering technique for non-destructive detection of defects in mate- rials. NDT&E Int. 40, 4: 337–346. [76] Kohl, C., Krause, M., Maierhofer, C., Wöstmann, J. 2005. 2D-and 3D-visualisation of NDT-data using data fusion technique. Mater. Struct. 38, 9: 817–826. [77] Cui, J., Huston, D. R., Arndt, R., Jalinoos, F. 2010. Data fusion techniques of multiple sensors nondestructive evaluation of a concrete bridge deck. In: Proc. of NDE/NDT for Highway Bridges: Structural Materials Technology (SMT) 2010 Conference, New York City, NY, USA, Aug 16–20, 2010. [78] Bensaid, A. M., Hall, L. O., Bezdek, J. C., Clarke, L. P., Silbiger, M. L., Arrington, J. A., Murtagh, R. F. 1996. Validity-guided (re)clustering with applications to image segmentation. IEEE T. Fuzzy Syst. 4, 2: 112–123. [79] Santoro, M., Prevete, R., Cavallo, L., Catanzariti, E. 2006. Mass detection in mammograms using gabor filters and fuzzy clustering. In: Bloch, I. (ed.), Petrosino, A. (ed.), Tettamanzi, A. G. B. (ed.). Fuzzy Logic and Applications, Revised Selected Papers on the 6th Int. Workshop, WILF 2005, Crema, Italy, Sep 15–17, 2005. Berlin Heidelberg, Springer: pp. 334–343. [80] Paasche, H., Tronicke, J., Holliger, K., Green, A. G., Maurer, H. 2006. Integration of diverse physical-property models: subsurface zonation and petrophysical parameter estimation based on fuzzy c-means cluster analyses. Geophysics 71, 3: 33–44. [81] Paasche, H., Tronicke, J. 2007. Cooperative inversion of 2d geophysical data sets: a zonal approach based on fuzzy c-means cluster analysis. Geophysics 72, 3: 35–39. [82] MacQueen, J. B. 1967. Some methods for classification and analysis of multivariate obser- vations. In: Cam, L. M. L. (ed.), Neyman, J. (ed.). Proc. of the 5th Berkeley Symp. on Math. Statist. and Prob., Berkeley, U.S.A., Jun 21 – Jul 18, 1965, Dec 27 –Jan 7, 1966, Vol. 1. Berkeley, University of California Press: pp. 281–297. [83] Bezdek, J. Z. 1981. Pattern Recognition with Fuzzy Objective Function Algorithms. New York, Plenum Press: 256 p. [84] Dave, R. N. 1991. Characterization and detection of noise in clustering. Pattern. Recogn. Lett. 12, 11: 657–664. [85] Gustafson, D., Kessel, W. 1979. Fuzzy clustering with a fuzzy covariance matrix. In: 1978 IEEE Conf. on Decision and Control including the 17th Symp. on Adaptive Processes. IEEE: pp. 761–766. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 91 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [86] Timm, H., Borgelt, C., Kruse, R. 2001. Fuzzy cluster analysis with cluster repulsion. In: Proc. of the Eur. Symp. on Intelligent Technologies, Hybrid Systems and their implemen- tation on Smart Adaptive Systems, Tenerife, Spain, Dec 13–14, 2001. EUNITE: pp. 1–7 (CD-ROM). [87] Pal, N. R., Pal, K., Keller, J. M., Bezdek, J. C. 2005. A possibilistic fuzzy c-means clustering algorithm. IEEE T. Fuzzy Syst. 13, 4: 517–530. [88] Maierhofer, C., Köpp, C., Binda, L., Zanzi, L., Santiago, J. R., Knupfer, B., Johansson, B., Modena, C., Porto, F., Marchisio, M., Gravina, F., Falci, M., Ruiz, J. C., Tomaževič, M., Bosiljkov, V., Hennen, C. 2006. Project Report EUR 21696 EN – Onsiteformasonry project – On-site investigation techniques for the structural evaluation of historic masonry buildings, Deliverable D7.1–D7.4. Luxembourg, European Comission, Office for Official Publications of the European Communities: 139 f. [89] Sansalone, M. J., Streett, W. B. 1998. The impact-echo method. NDTnet 3, 2: 1–8. http://www.ndt.net/article/0298/streett/streett.htm (Accessed 20 December 2013). [90] Huang, M., Jiang, L., Liaw, P. K., Brooks, C. R., Seeley, R., Klarstrom, D. L. 1998. Using acoustic emission in fatigue and fracture materials research. JOM 50, 11. http://www.tms.org/pubs/journals/JOM/9811/Huang/Huang-9811.html (Accessed 18 December 2013). [91] Ward, S. H. 1990. Resistivity and induced polarization methods. In: Ward, S. H. (ed.). Geotechnical and Environmental Geophysics, Vol. 1. Society of Exploration Geophysicists: pp. 147–190. [92] Stoppel, M. 2011. Differenzpotentialfeldmessung in der automatisierten Prufung von Stahlbetonbauteilen. Doctoral Dissertation. Berlin, (M. Stoppel): 171 f. [93] Hülsenbeck & Co. 1926. German Pat. No. 489434. [94] Neal, A. 2004. Ground-penetrating radar and its use in sedimentology: principles, problems and progress. Earth-Science Reviews 66: 261–330. [95] Daniels, J. D. 2004. Ground-penetrating radar, 2nd ed. London, The Institution of Elec- trical Engineers: 726 p. [96] Jol, H. M. (ed.). 2009. Ground Penetrating Radar: Theory and Applications. Amsterdam, Oxford, Elsevier Science: 544 p. [97] Powers, M. 1997. Modeling frequency-dependent GPR. The Leading Edge 16, 11: 1657– 1662. [98] Schlumberger, C. 1920. Étude sur la prospection électrique du sous-sol. Paris, Gaultier- Villars et Cie.: 96 p. [99] Loke, M. H. 2001. Tutorial: 2-D and 3-D electrical imaging surveys. [100] Marshall, D., Madden, T. 1959. Induced polarization, a study of its causes. Geophysics 24, 4: 790–816. [101] Kruschwitz, S. 2008. Assessment of the complex resistivity behavior of salt affected building materials. Doctoral Dissertation. Berlin, (S. Kruschwitz): 165 f. [102] Dey, A., Morrison, H. F. 1979. Resistivity modelling for arbitrary shaped two-dimensional structures. Geophys. Prospect. 27: 106–136. 92 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [103] Nichols, J. T. 1935. Temperature measuring, U.S. Pat. No. 2008793. [104] Maldague, X. P. V. 2001. Theory and Practice of Infrared Technology for Nondestructive Testing. New York, Chichester, John Wiley & Sons, Inc.: 684 p. [105] Cielo, P. 1984. Pulsed photothermal evaluation of layered materials. J. Appl. Phys. 56, 1: 230–234. [106] Almond, D. P., Lau, S. K. 1994. Defect sizing by transient thermography. I. An analytical treatment. J Phys D: Appl Phys 27, 5: 1063–1070. [107] Busse, G., Wu, D., Karpen, W. 1992. Thermal wave imaging with phase sensitive modu- lated thermography. J. Appl. Phys. 71, 8: 3962–3965. [108] Spicer, J. W. M., Kerns, W. D., Aamodt, L. C., Murphy, J. C. 1992. Source patterning in time-resolved infrared radiometry of composite structures. In: Eklund, J. K. (ed.). Thermosense XIV, Vol. 1682, Proc. of SPIE – An Int. Conf. on Thermal Sensing and Imaging Diagnostic Applications, Orlando, FL, U.S.A., Apr 22–24, 1992. SPIE: pp. 248– 257. [109] Ibarra-Castanedo, C., Tarpani, J. R., Maldague, X. P. V. 2013. Nondestructive testing with thermography. Eur. J. Phys. 34: S91–S109. [110] Halliday, D., Resnick, R., Krane, K. S. 1992. Physics, Vol. 2, 4th ed. John Wiley & Sons, Inc.: 688 p. [111] Arpaci, V. S. 1966. Conduction Heat Transfer. Addison-Wesley Pub. Co.: 550 p. [112] Parker, W. J., Jenkins, R. J., Butler, C. P., Abbott, G. L. 1961. Flash method of deter- mining thermal diffusivity, heat capacity, and thermal conductivity. J. Appl. Phys. 32, 9: 1679–1684. [113] Chen, L., Clarke, D. R. 2009. A numerical solution based parameter estimation method for flash thermal diffusivity measurements. Comp. Mater. Sci. 45, 2: 342–348. [114] Maldague, X. P., Marinetti, S. 1996. Pulse phase infrared thermography. J. Appl. Phys. 79, 5: 2694–2698. [115] Cielo, P., Maldague, X., Deom, A. A., Lewak, R. 1987. Thermographic nondestructive evaluation of industrial materials and structures. Mater. Eval. 45, 6: 452–460. [116] Allport, J., McHugh, J. 1988. Quantitative evaluation of transient video thermography. In: Thompson, D. O. (ed.), Chimenti D. E. (ed.). Review of Progress in Qunatitative Nondestructive Evaluation, Vol. 8A. New York, Plenum Press: pp. 253–262. [117] Krapez, J.-C., Maldague, X., Cielo, P. 1991. Thermographic nondestructive evaluation: data inversion procedures. Res. Nondestruct. Eval. 3, 2: 101–124. [118] Balageas, D. L., Deom, A. A., Boscher, D. M. 1987. Characterization and nondestructive testing of carbon-epoxy composites by a pulsed photothermal method. Mater. Eval. 45, 4: 456–465. [119] Grinzato, E., Vavilov, V., Bison, P. G., Marineti, S., Bressan, C. 1995. Methodology of processing experimental data in transient thermal nondestructive testing (NDT). In: Semanovich, S. A. (ed.). Thermosense XVII, Vol. 2473, Proc. of SPIE, Bellingham, WA, U.S.A., Mar 28, 1995. SPIE: pp. 167–178. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 93 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [120] Ibarra-Castanedo, C., Maldague, X. 2004. Pulsed phase thermography reviewed. Quant. Infrared Thermogr. J. 1, 1: 47–70. [121] Maldague, X., Galmiche, F., Ziadi, A. 2002. Advances in pulsed phase thermography. Infrared Phys. Techn. 43, 3–5: 175–181. [122] Press, W. H., Teukolsky, S. A., Vetterling, W. T., Flannery, B. P. 2007. Numerical Recipes 3rd ed.: The Art of Scientific Computing. New York, Cambridge University Press: 1256 p. [123] Arndt, R. 2007. Rechteckimpuls-Thermografie im Frequenzbereich, Adaption der Puls- Phasen-Thermografie für die qualitative und quantitative zerstörungsfreie Prüfung ober- flächennaher Strukturen im Bauwesen. Doctoral Dissertation. Berlin, (R. Arndt): 178 f. [124] Favro, L. D., Han, X. 1998. Thermal wave material characterization and thermal wave imaging, in sensing for materials characterization, processing, and manufacturing. In: Birnbaum, G. (ed.), Auld, B. A. (ed.). Sensing for Materials Characterization, Processing and Manufacturing, Vol. 1. ASNT TONES: pp. 399–415. [125] Vavilov, V., Marinetti, S. 1999. Thermal methods pulsed phase thermography and fourier- analysis thermal tomograph. Russian Journal of Nondestructive Testing 35, 2: 134–145. [126] Busse, G., Rosencwaig, A. 1980. Subsurface imaging with photoacoustics. Appl. Phys. Lett. 36, 10: 815–816. [127] Ibarra-Castanedo, C. 2005. Quantitative subsurface defect evaluation by pulsed phase thermography: depth retrieval with the phase. Doctoral Dissertation. Laval, (C. Ibarra- Castanedo). . [128] Ibarra-Castanedo, C., Maldague, X. P. V. 2005. Interactive methodology for optimized defect characterization by quantitative pulsed phase thermography. Res. Nondestruct. Eval. 16: 175–193. [129] Arndt, R., Maierhofer, C., Röllig, M. 2006. Quantitative pulse-phase-thermography for masonry and concrete structures. In: Proc. of the 9th Eur. NDT Conf. (EC-NDT 2006), Berlin, Germany, Sep 25–29, 2006. pp. 1–8. http://www.ndt.net/article/ecndt2006/doc/P149.pdf (Accessed 16 April 2012). [130] Arndt, R. W. 2010. Square pulse thermography in frequency domain as adaptation of pulsed phase thermography for qualitative and quantitative applications in cultural heritage and civil engineering. Infrared Phys. Techn. 53, 4: 246–253. [131] Arndt, R. 2008. Square pulse thermography in frequency domain. In: Vavilov, V. P. (ed.), Burleigh, D. D. (ed.). Thermosense XXX, Vol. 6939, Proc. of SPIE, Orlando, FL, U.S.A., Mar 16, 2008. SPIE: pp. 1–12. http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=834339 (Accessed 13 September 2012). [132] Kuo, M. K., Lin, T. R., Liu, P. L., Wu, T. T. 1998. Locating the crack tip of a surface- breaking crack, Part I. Line crack. Ultrasonics 36, 7: 803–811. [133] Cheeke, J. D. N. 2002. Fundamentals and applications of ultrasonic waves. Boca Raton, Florida, CRC Press LLC: 480 p. [134] Shokouhi, P., Wiggenhauser, H. 2011. Multi-probe ultrasonic testing for detection of de- lamination in concrete bridge decks. Transportation Research Board (TRB) 91st Annual Meeting, Washington, DC, U.S.A., Jan 22-26, 2012. http://amonline.trb.org/1sl7d2/1 (Accessed 10 December 2012). 94 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [135] Schickert, M., Krause, M., Müller, W. 2003. Ultrasonic imaging of concrete elements using reconstruction by synthetic aperture focusing technique. J. Mater. Civ. Eng. 15, 3: 235–246. [136] Maierhofer, C. (ed.), Reinhardt, H. W. (ed.), Dobmann, G. (ed.). 2010. Non-destructive Evaluation of Reinforced Concrete Structures, Vol. 2: Non-destructive Testing Methods. Oxford, Cambridge, New Delhi, Woodhead Publishing: 624 p. [137] Lago, S., Brignolo, S., Cuccaro, R., Musacchio, C., Albo, P. G., Tarizzo, P. 2014. Applica- tion of acoustic methods for a non-destructive evaluation of the elastic properties of several typologies of materials. Appl. Acoust. 75: 10–16. [138] NDT Education Resource Center. 2001–2012. Piezoelectric Transducers. http://www.ndt-ed.org/EducationResources/CommunityCollege/Ultrasonics/ EquipmentTrans/piezotransducers.htm (Accessed 10 January 2014). [139] Wendrich, A. 2009. Zerstorungsfreie Ortung von Anomalien in historischem Mauerwerk mit Radar und Ultraschall – Moglichkeiten und Grenzen. Doctoral Dissertation. Berlin, (A. Wendrich): 185 f. [140] Gros, X. E. 1997. NDT Data Fusion. London, Arnold: 205 p. [141] Bogaert, P., Fasbender, D. 2007. Bayesian data fusion in a spatial prediction context: a general formulation. Stoch. Environ. Res. Risk Assess. 21, 6: 695–709. [142] Gros, X. E., Strachan, P., Lowden, D. W. 1995. Fusion of multiprobe NDT data for ROV inspection. In: Conf. Proc. OCEANS’95 MTS/IEEE, Challenges of Our Changing Global Environment, San Diego, CA, U.S.A., Oct 9–12, 1995, Vol. 3. IEEE: pp. 2046–2050. [143] Höppner, F., Klawonn, F., Kruse, R., Runkler, T. 1999. Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition. Chichester, John Wiley & Sons Ltd.: 300 p. [144] Pedrycz, W. 2005. Clustering and fuzzy clustering. In: Knowledge-Based Clustering: From Data to Information Granules. Hoboken, NJ, U.S.A., John Wiley & Sons, Inc.: pp. 1–27. [145] Babuška, R. 2009. Fuzzy clustering. In: Fuzzy and Neural Control DISC Course Lectures Notes (November 2009). Delft, Netherlands, Delft University of Technology: pp. 59–75. http://www.dcsc.tudelft.nl/~disc_fnc/transp/fncontrol.pdf (Accessed 12 December 2013). [146] Kruse, R., Doring, C., Lesot, M. J. 2007. Fundamentals of fuzzy clustering. In: Oliveira, J. V. (ed.), Pedrycz, W. (ed.). Advances in Fuzzy Clustering and its Applications. Chichester, England, John Wiley & Sons Ltd.: pp. 3–30. [147] Gath, I., Geva, A. B. 1989. Unsupervised optimal fuzzy clustering. IEEE Trans. Pattern Anal. Mach. Intell. 11, 7: 773–780. [148] Dave, R. N. 1990. Use of the adaptive fuzzy clustering algorithm to detect lines in digital images. In: Casasent, D. P. (ed.). Proc. of SPIE, Intelligent Robots and Computer Vision VIII: Algorithms and Techniques, Vol. 1192. SPIE: pp. 600–611. [149] Krishnapuram, R., Kim, J. 1999. A note on the Gustafson–Kessel and adaptive fuzzy clustering algorithms. IEEE T. Fuzzy Syst. 7, 4: 453–461. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 95 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. [150] Babuška, R., van der Veen, P. J., Kaymak, U. 2002. Improved covariance estimation for Gustafson-Kessel clustering. In: Proc. of the 2002 IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE’02), Vol. 2, Honolulu, Hawaii, May 12–17, 2002. IEEE: pp. 1081–1085. [151] Krishnapuram, R., Keller, J. M. 1996. The possibilistic c-means algorithm: insights and recommendations. IEEE Trans. Fuzzy Syst. 4, 3: 385–393. [152] Pal, N. R., Pal, K., Bezdek, J. C. 1997. A mixed c-means clustering model. In: Proc. of the 6th IEEE Int. Conf. on Fuzzy Systems, Barcelona, Spain, Jul 1–5, 1997, Vol. 1. IEEE: pp. 11-21. [153] Abonyi, J., Feil, B. 2007. Aggregation and visualization of fuzzy clusters based on fuzzy similarity measures. In: Oliveira, J. V. (ed.), Pedrycz, W. (ed.). Advances in Fuzzy Clustering and its Applications. Chichester, England, John Wiley & Sons Ltd.: pp. 95– 121. [154] Xie, X. L., Beni, G. 1991. A validity measure for fuzzy clustering. IEEE T. Pattern Anal. 13, 8: 841–847. [155] Balasko, B., Abonyi, J., Feil, B. 2011. Fuzzy Clustering and Data Analysis Toolbox. Veszprem, Hungary, University of Veszprem, Department of Process Engineering: 77 p. www.fmt.vein.hu/softcomp (Accessed 22 July 2011). [156] FLIR A320 Technical Specifications. http://www.shopflir.com/crm_uploads/flir-a320-specifications.pdf (Accessed 28 February 2012). [157] Sandmeier, K. J. 2004. Reflexw, instruction manual for processing of seismic, acoustic or electromagnetic reflection, refraction and transmission data, Ver. 3.5. [158] Yelf, R., Yelf, D. 2006. Where is true time zero? Electromagnetic Phenomena 7, 1: 158–163. [159] Altman, D. G., Bland, J. M. 1994. Statistics notes: Diagnostic tests 1: sensitivity and specificity. BMJ 308: 1552. [160] Stoppel, M., Taffe, A., Wiggenhauser, H., Kurz, J. H., Boller, C. 2011. Automated multi- sensor systems in civil engineering for condition assessment of concrete structures. In: Grantham, M. (ed.). Proc. of the 4th Int. Conf. on Concrete Repair, Dresden, Germany, Sep 26–28, 2011. Boca Raton, FL, CRC Press: pp. 397-403. [161] Krause, H. J., Rath, E., Sawade, G., Dumat, F. 2007. Radar-Magnet-Betontest: Eine neue Methode zur Bestimmung der Feuchte und des Chloridgehalts von Brückenfahrbahnplatten aus Beton. Beton- und Stahlbetonbau 102, 12: 825–834. 96 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 97 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. LIST OF PUBLICATIONS [1] Cotič, P., Niederleithinger, E., Wilsch, G. 2014. Unsupervised clustering of laser-induced breakdown spectroscopy data from concrete. JAAS (Sent for publication). [2] Cotič, P., Jagličić, Z., Niederleithinger, E., Bosiljkov, V. 2014. Image fusion for improved detection of near-surface defects in NDT-CE using unsupervised clustering methods. J. Nondestruct. Eval. doi:10.1007/s10921-014-0232-1. [3] Heidari, S., Safaei, E., Wojtczak, A., Cotič, P. 2013. Oxalate-bridged binuclear iron(III) complexes of two pyridine-based aminophenol ligands. Inorg. Chim. Acta. 405: 134–139. [4] Počuča-Nešić, M., Marinković Stanojević, Z., Branković, Z., Cotič, P., Bernik, S., Sousa Góes, M., Marinković, B., Varela, J., Branković, G. 2013. Mechanochemical syn- thesis of yttrium manganite. J. Alloy. Compd. 552: 451–456. [5] Cotič, P., Jagličić, Z., Bosiljkov, V. 2013. Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value. J. Cult. Herit. doi:http://dx.doi.org/10.1016/ j.culher.2013.11.004. [6] Karimpour, T., Safaei, E., Wojtczak, A., Cotič, P. 2013. Synthesis and characterization of oxalate-bridged binuclear iron(III) complex: [(N)2(O)2Fe(µ-ox)Fe(N)2(O)2]. J. Mol. Struct. 1038: 230–234. [7] Cotic, P., Jaglicic, Z., Niederleithinger, E., Effner, U., Kruschwitz, S., Trela, C., Bosiljkov, V. 2013. Effect of moisture on the reliability of void detection in brickwork masonry using radar, ultrasonic and complex resistivity tomography. Mater. Struct. 46, 10: 1723–1735. [8] Heidari, S., Safaei, E., Wojtczak, A., Cotič, P., Kozakiewicz, A. 2013. Iron(III) complexes of pyridine-based tetradentate aminophenol ligands as structural model complexes for the catechol-bound intermediate of catechol dioxygenases. Polyhedron 55: 109–116. [9] Cotič, P., Jagličić, Z., Bosiljkov, V., Niederleithinger, E. 2013. GPR and IR thermography for near-surface defect detection in building structures. In: Grum, J. (ed.), Kek, T. (ed.). Proc. of the 12th Int. Conf. of the Slovenian Society for Non-Destructive Testing titled Application of contemporary non-destructive testing in engineering, Portorož, Slovenia, Sep 4–6, 2013. Ljubljana, Slovenian Society for Non-Destructive Testing: pp. 225–232. [10] Cotič, P., Niederleithinger, E., Bosiljkov, V., Jagličić, Z. 2013. NDT data fusion for the enhancement of defect visualization in concrete. In: Damage assessment of structures X: selected, peer reviewed papers from the 10th Int. Conf. on Damage Assessment of Structures (DAMAS 2013), Dublin, Ireland, Jul 8-10, 2013. Basu, B. (ed.) Key Engineering Materials 569–570: 175–182. [11] Saberikia, I., Safaei, E., Rafiee, M., Cotič, P., Bruno, G., Rudbari, H. A. 2012. A biolog- ically relevant iron(III) phenoxyl radical complex: A thermodynamic investigation on the structure-radical stability relationship. J. Mol. Struct. 1022: 109–116. [12] Saberikia, I., Safaei, E., Kowsari, M. H., Lee, Y.-I., Cotic, P., Bruno, G., Rudbari, H. A. 2012. A new iron(III) complex of glycine derivative of amine-chloro substituted phenol ligand: Synthesis, characterization and catechol dioxygenase activity. J. Mol. Struct. 1029: 60–67. [13] Bosiljkov, V., Cotič, P., Uranjek, M., Kržan, M. 2012. Uncertanties in the assessment of the seismic vulnerability of stone masonry buildings. In: Roman, H. R. (ed.), Parsekian, G. 98 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. A. (ed.). Proc. of the 15th Int. Brick and Block Masonry Conference, Florianópolis, Santa Caterina, Brazil, Jun 3–6, 2012. Florianópolis, Federal University of Santa Catarina: pp. 1–10. [14] Kržan, M., Cotič, P., Bosiljkov, V. 2012. Laboratorijske preiskave troslojnih kamnitih zidov = laboratory experimental tests on three-leaf stone masonry walls. In: Lopatič, J. (ed.), Markelj, V. (ed.), Saje, F. (ed.). Zbornik 34. zborovanja gradbenih konstruktorjev Slovenije, Bled, Hotel Golf, Slovenia, Oct 11–12, 2012. Ljubljana, Slovensko društvo gradbenih kon- struktorjev: pp. 203–210. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. 99 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Appendices 100 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A1 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. APPENDIX A: ARTICLE 1 Image fusion for improved detection of near-surface defects in NDT-CE using unsupervised clustering methods P. Cotič, Z. Jagličić, E. Niederleithinger, M. Stoppel & V. Bosiljkov J Nondestruct Eval (2014) DOI 10.1007/s10921-014-0232-1 A2 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval DOI 10.1007/s10921-014-0232-1 Image Fusion for Improved Detection of Near-Surface Defects in NDT-CE Using Unsupervised Clustering Methods Patricia Cotič · Zvonko Jagličić · Ernst Niederleithinger · Markus Stoppel · Vlatko Bosiljkov Received: 28 August 2013 / Accepted: 24 January 2014 © Springer Science+Business Media New York 2014 Abstract The capabilities of non-destructive testing (NDT) 1 Introduction methods for defect detection in civil engineering are charac- terized by their different penetration depth, resolution and In civil engineering (CE), several non-destructive testing sensitivity to material properties. Therefore, in many cases (NDT) methods such as ground penetrating radar (GPR), multi-sensor NDT has to be performed, producing large data ultrasonic, impact echo and infrared (IR) thermography are sets that require an efficient data evaluation framework. In used to image the internal structure and to detect defects such this work an image fusion methodology is proposed based on as delamination, spalling, surface and subsurface cracks, unsupervised clustering methods. Their performance is eval- reinforcement corrosion, the presence of small air voids, uated on ground penetrating radar and infrared thermography and moisture distribution [1–5]. However, these methods are data from laboratory concrete specimens with different sim- characterized by different penetration depths, as well as dif- ulated near-surface defects. It is shown that clustering could ferent sensitivity to the physical properties of the material effectively partition the data for further feature level-based [6–9]. Therefore, in order to fully characterize the objects data fusion by improving the detectability of defects simulat- under inspection and to increase the accuracy, several NDT ing delamination, voids and localized water. A comparison techniques should be applied simultaneously. Consequently, with supervised symbol level fusion shows that clustering- often a huge amount of data has to be processed and inter- based fusion outperforms this, especially in situations with preted efficiently. Currently data processing and interpreta- very limited knowledge about the material properties and tion of the results are performed for each NDT technique depths of the defects. Additionally, clustering is successfully individually. This is both costly and time consuming and applied in a case study where a multi-sensor NDT data set does not provide a reliable and straightforward evaluation of was automatically collected by a self-navigating mobile robot the structural condition. To overcome these constraints, data system. fusion techniques can be used. To improve imaging results and defect characterization, Keywords Non-destructive testing · Concrete · Defect image fusion has already been proposed for non-destructive detection · Data fusion · Cluster analysis evaluation (NDE). Methods from Bayesian inference, the Dempster–Shafer (D–S) theory of evidence, and wavelet P. Cotič (B) · Z. Jagličić multi-resolution analysis were used on eddy current and IR Institute of Mathematics, Physics and Mechanics, Jadranska 19, thermography images from composite and steel structures 1000 Ljubljana, Slovenia [10–13]. In NDT-CE applications, image fusion was per- e-mail: patricia.cotic@imfm.si formed on GPR data acquired at different frequencies and P. Cotič · Z. Jagličić · V. Bosiljkov polarizations, and on GPR and ultrasonic data on prestressed Faculty of Civil and Geodetic Engineering, University of Ljubljana, concrete bridge beams and test specimens [8,14,15]. How- Jamova 2, 1000 Ljubljana, Slovenia ever, the applied fusion techniques were restricted to simple arithmetical methods. P. Cotič · E. Niederleithinger · M. Stoppel BAM Federal Institute for Materials Research and Testing, The present work deals with clustering-based image Division VIII.2, Unter den Eichen 87, 12205 Berlin, Germany fusion of NDT data acquired on concrete with defects in the 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A3 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval near-surface region. For image fusion, clustering methods which, however, do not affect the analysis significantly, but have already been applied in medical imaging for pattern they do not need a priori knowledge of the statistical data dis- recognition (e.g. for the detection of cancerous cells from tribution. This is especially promising in the field of NDT, magnetic resonance images [16] and mammograms [17]), where knowledge about sample composition is limited due as well as on spectroscopic spatial measurement results from to the large diversity of specimens. concrete for cement identification [18]. Their main advantage Clustering refers to the task of grouping a set of data is that they can work fully unsupervised. This is of partic- (observations) into subsets (clusters, groups and classes) in ular interest in implementing NDT in practice in terms of such a way that the members of a cluster are as similar as automated application by trained engineers and technicians possible, whereas members belonging to different clusters rather than scientists. are as dissimilar as possible [24]. For objective-based clus- Data fusion should both improve the results where NDT tering, starting with a random grouping of observations into methods respond complementarily and put more weight upon clusters, the observations are iteratively moved to the cluster the method with better performance. Therefore, to assess whose centre is the nearest to them until a convergence limit the effectiveness of mining data for the detection of near- is reached. A common measure of similarity between two surface defects in concrete structures, laboratory concrete observations x 1, x 2 is the Euclidian distance d, which for M specimens with simulated defects were studied by GPR and variables can be written as IR thermography, regarded as complementary methods in the M 1 / 2 near-surface region. Defects in the form of variations in the d = (x 1 i − x 2 i) 2 = x1 − x2 . (1) material, shape and depth below the surface were taken into i =1 account. Some preliminary results on data fusion from this data set have been presented in [19]. However, this paper In partitioning an unlabelled data set X = {x1 , x2 , . . . , x N } develops a complete data fusion framework based on a quan- into c clusters by iteratively assigning labels ui j to vectors titative evaluation with respect to the sensitivity calculation. in X by the partition matrix U c× N = ui j , the target is to Based on the outcome of the fusion methodology derived for minimize an objective function J , which is the sum of the dis- GPR and IR thermography data on concrete, we further apply tances of N observations to their respective cluster centre v: clustering methods to a case study where a multi-sensor NDT c N 2 data set was automatically collected by a self-navigating Jm(U , V; X ) = um x . (2) i j j − v i mobile robot system. i =1 j=1 With respect to the partition matrix, clustering algorithms 2 Image Fusion with Clustering can be classified as hard (or crisp), fuzzy probabilistic or fuzzy possibilistic [24]. In contrast to hard clustering where Image fusion is applied in many fields, including medical u imaging, microscopic imaging, remote sensing, computer i j possess values of only 0 or 1 (ui j ∈ {0 , 1} ), fuzzy prob- abilistic clustering is defined as soft partitioning, allowing vision and robotics. It can be performed at different levels, i.e. observations to belong to more than one cluster with a calcu- at pixel, feature and symbol level. At pixel level, images are lated probability, i.e. u fused at each pixel individually without using pre-processing i j may possess any number between 0 and 1 (u units. Feature level fusion on the other hand tends to fuse fea- i j ∈ [0 , 1] ). This is especially promising in the case of NDT data, where u tures extracted by segmentation in the pre-processing step i j could be regarded as the reliability of the NDT system. Moreover, since the hard k-means algo- using clustering, neural networks (NN) and wavelet meth- rithm [25] was found to be particularly sensitive to initializa- ods, whereas symbol level fusion is in most cases based on tion [18], hard clustering was not considered in this paper. sensor modelling [20]. Regardless of the level used, statisti- The best known fuzzy probabilistic algorithm is the fuzzy cal averaging, Bayesian, D–S, as well as fuzzy logic can be c-means (FCM) algorithm [26]. Here u used for the fusion algorithm. One of the main restrictions i j describes the mem- bership of observation x in using probabilistic methods for symbol level fusion is the j in the i th cluster, which can be interpreted as probability (hence “probabilistic clustering”). correct assignment of the conditional and a priori probabil- The “fuzzing” exponent or “fuzzifier” m, introduced in Eq. ities as well as mass functions which govern the evaluation. 2, is a positive real number and is recommended to range Although many studies have proposed different approaches from 1.2 to 2 (in our calculation m was fixed to 2, the value for its calculation [12,21–23], none of the methods can work most often recommended by other authors). To optimize J , fully unsupervised. For feature level fusion on the other hand, u wavelet analysis and NN require an appropriate wavelet fil- i j and v i have to be updated in each iteration by ter with definition of the decomposition level to be used and u , (3) good training data set, respectively [13]. In contrast, clus- i j = 1 c (d tering methods require some sensitivity measures to be set, i j /dk j ) 2 /(m−1 ) k=1 123 A4 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval N where a > 0, b > 0, m > 1 and η > 1 are user-defined um · x i j j j =1 parameters and γi should be calculated as proposed in [33]. v i = , (4) N To minimize J , the necessary conditions for memberships um i j ui j , typicalities ti j and cluster centres have to be adapted as j =1 well. In our version of the PFCM algorithm we used the final FCM partition for initialization, as well as fixed parameter with d i j = x j − v i , 1 ≤ i ≤ c, 1 ≤ j ≤ N . values a = 1.4, b = 1.8, m = 2 and η = 1.8, which produced the Due to the fact that the Euclidian distance is used, the FCM best partition among several other parameter combinations. method is based on the assumption of spherical clusters and The chosen combination of parameters also worked well in is sensitive to outliers and disturbances. To account for the our previous studies on spectroscopic spatial clustering [18]. first assumption, the Gustafson-Kessel (GK) algorithm [27], It should be noted that the number of clusters is a user- the adaptive fuzzy clustering (AFC) algorithm [28] or the defined parameter. Although several validity indices have GK algorithm with adaptive cluster volumes (GKA ρ) [29] been proposed to automatically define the optimal number of could be used to detect clusters of different shapes in one data clusters, it was observed that this procedure does not deliver set. All three algorithms employ an adaptive distance norm, a proper interpretation of the results in our case. As evident which adapts the FCM functional as throughout our study, the number of clusters should rather be defined according to the suspected severity of the damage c N and the heterogeneity of the inspected material. Jm(U , V; X ) = um (x i j j − v i ) TAi (x j − v i ) , (5) i =1 j=1 3 Development of an Image Fusion Framework Based on Clustering where A i is the norm-inducing matrix of the i th cluster that should be calculated on the basis of the cluster volume ρ For the development of an image fusion framework based on i and the fuzzy covariance matrix F [30]. clustering, GPR and IR thermography measurements were Since the AFC and GKA ρ algorithms were found to carried out on laboratory concrete specimens with artificial produce poor partitions if little is known about the initial defects. The numerous factors that govern the performance determination of the cluster volumes, we used the general of these two methods such as material properties, type, shape GK algorithm with cluster volumes fixed to ρ and the size of defects were taken into account. i = 1 [29]. To improve the numerical performance of the GK algorithm (to account for the singularity problems of F), we used the algo- 3.1 Specimen Description rithm proposed by Babuška et al. [30], where two additional parameters ( β and γ ) are introduced. As suggested by the All concrete specimens have a size of 50 × 50 × 15 cm3 authors, parameter β (the parameter that limits the condition and were made from the same concrete. The specimens con- number of F) was fixed to β = 1015, whereas parameter γ ∈ tain inbuilt anomalies varying in material (polystyrene, air [0,1] (i.e. the tuning parameter which constraints the clusters and water) and depth below the surface (Fig. 1). The inbuilt to have a more or less equal shape) was varied as discussed cuboids in specimen S2 have a size of 8 × 8 × 6 cm3, while in Sect. 3.3. Additionally, we followed Kruse et al. [31] in the size of S1 cuboids is 8 × 8 cm2 with varying thickness. using the final FCM partition for the initialization of the GK Specimen S2 features reinforcement with different densities algorithm. (horizontal distance between bars 5 or 10 cm), but a uniform The sensitivity of the FCM algorithm to outliers can be concrete cover of 2.5 cm. The polystyrene plates (simulating overcome either by using noise clustering [32], where an delamination) in specimen S3 have a thickness of 1 cm (a) additional noise cluster is introduced, or by fuzzy possibilis- and 2 cm (b), and the diameters of the plastic pipes are 1, tic clustering algorithms (for some variants, refer to [33–36]). 1.5 and 2.5 cm (from left to right in Fig. 1, S3). The con- Among the latter, the possibilistic-fuzzy c-means (PFCM) crete cover over all the defects in specimen S3 is between algorithm [36] worked particularly well on our NDT data, 1.5 and 2 cm. Defect specifications for all the specimens are where the following objective function should be minimized: summarized in Table 1. c N 3.2 Methods η J 2 m,η(U , T , V; X , γ ) = (aum + bt ) · x i j i j j − v i i =1 j=1 3.2.1 Ground Penetrating Radar c N + γ GPR is based on the propagation and reflection of high fre- i ( 1 − tij)η, (6) i =1 j =1 quency electromagnetic impulses produced by an antenna 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A5 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 1 Sketches of laboratory concrete test specimens with marked inbuilt defects Table 1 Material and concrete cover of simulated defects for specimens raphy technique was originally proposed by Maldague and S1–S3 Marinetti [40] and further adapted for the special require- Defect Defect material Concrete cover (cm) ments of NDT-CE by Arndt [41] in terms of square pulse thermography (SPT) in the frequency domain. Here, the S1a Air 6 specimen is square pulse heated [42] and analysis of the S1b Air 3 recorded temperature evolution versus time of each pixel is S1c Water 6 performed in the frequency domain by fast Fourier transfor- S1d Water 3 mation, deriving amplitude and phase images. Phase images S2a,b Polystyrene 6.6 in particular exhibit higher depth penetration, higher resolu- S2c,d Polystyrene 4.6 tion and are less sensitive to non-uniform heating than ther- S3a,b Polystyrene 1.5–2 mal and amplitude images [43]. Arndt [41] additionally pro- S3pipes Plastic 1.5–2 posed a quantitative method where the depth z of a defect can be expressed by the frequency fch of the maximum phase or amplitude contrast between the defect and the sound area system, where reflections occur at the interfaces between through the following equation materials with different dielectric properties. In the near- surface region, the analysis and interpretation of GPR data α z = kc · , (7) is often complicated by the influence of near-field antenna fch coupling/induction effects, which according to [37] range to a depth of 1 . 5 · λ, where λ is the wavelength of the elec- where kc is a correction factor and α is the thermal diffusivity tromagnetic waves. For concrete with a dielectric constant of the investigated material. of around 9 and an antenna with a central frequency of 1.6 The IR thermography data used for this study was acquired GHz, the wavelength is around 6.3 cm and the near-field with a FLIR A320 IR camera, having a thermal sensitivity region extends up to a depth of almost 10 cm. Apart from of 50 mK (at 30 ◦C), a spatial resolution of 1.36 mrad and a the near-field effects the performance of GPR is also depen- focal plane array with a resolution of 320 × 240 pixels and a dent on the resolution. For calculation of the depth resolution spectral range of 7.5–13 µm. For heating the specimens, two (defined as the ability to differentiate two adjacent signals in IR heaters (1.2 kW each) were moved parallel to the surface at time), 0 . 25 · λ has been proposed [38]. a distance of about 45 cm. All thermal images were recorded The GPR data used for this study was obtained with equip- at a frame rate of 0.2 Hz. The heating and cooling down times ment from MALÅ Geoscience, using a 1.6 GHz monostatic were adjusted with regard to the problem. shielded antenna. A calibrated survey wheel was used and radargrams were collected in both directions with a line spac- 3.2.3 Image Fusion and Validation ing of 5 cm and a point distance of 0.5 cm. GPR and IR thermography images were combined on a fea- ture level-based data fusion using fuzzy clustering methods 3.2.2 Square Pulse Thermography in the Frequency for data segmentation and simple weighted averaging to fuse Domain the derived memberships for each pixel according to IR thermography is based on recording the IR radiation from u R,i xR + uT, j xT x , (8) the previously heated surface of the specimen and allows U S(i, j) = u R,i + uT, j detection of the presence of defects to within a depth of approximately 10 cm [39]. In order to enhance the detectabil- where xR and xT refer to a specific pixel of GPR and IR ity of thermal images, the use of the pulse phase thermog- images and u R and uT are the associated membership val- 123 A6 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval ues. Index i marks that observation x, which is segmented to range. For the fusion algorithm, weighted averaging and the cluster i , should be weighted with its corresponding mem- D–S rule of combination were used (we preferred the D–S to bership to that cluster. Different indices ( i, j) particularly Bayesian theory approaches, since the D–S does not assign refer to independent segmentation of GPR compared to IR belief or disbelief to the fused image if there is insufficient thermography. For GPR C-scans were used, whereas for IR knowledge). For our data sets, weighted averaging for each thermography phase images at particular frequencies served pixel was calculated according to as input data. Since the concrete test validation specimens contained defects at known depths, the quantitative approach wRxR + wT xT xw = , (12) of SPT in the frequency domain with respect to the mater- wR + wT ial properties of concrete and the defects was applied. As a where w result, the phase contrast images could be regarded as depth R and wT are the weights associated with the prob- ability of defect detection of the GPR and IR thermography slices, enabling a direct correlation with the GPR C-scans. measurement system and x Before fusion, data alignment was performed and a common R and xT refer to a specific pixel of GPR and IR images. Following Gros et al. [10] for the spatial reference was established. In addition, both data sets application of the D–S [46], we first introduce a frame of were globally normalized; amplitudes for GPR and phase discernment as a set of mutually exclusive events a and b, contrasts for IR thermography. In this paper we present the where = { a- GPR is true, b- thermography is true}. A power results obtained using the FCM, GK and PFCM clustering set P () = {∅ , a , b , } is defined and for each element of methods performed for each depth separately. Noise cluster- P () masses, associated with the probability of defect detec- ing, though normally extremely powerful in handling noisy tion of the GPR and IR thermography measurement system, data [18], failed in our case as signals of very high amplitude are assigned. With the mass assignment, evidential intervals (detected defect) were classified as noise. could be determined for GPR and thermography images by Based on ground truth reference, the results from fusion on applying the D–S rule of combination [46], which can be concrete were assessed both qualitatively and quantitatively. mathematically expressed as The quality of the resulting images was estimated by visual inspection, whereas quantitative evaluation was based on the m R(x)mT (y) known area of the defects. Sensitivity and specificity values x∩ y= z m R,T (z) = m R ⊕ mT (z) = , [44] were calculated from the number of true positive, true 1 − m R(x)mT (y) x∩ y=0 negative, false positive and false negative calls by using (13) sens = T P , (9) where m T P + F N R and m T are mass functions of GPR and IR ther- mography, respectively. In our case, m R,T (a) refers to the spec = T N , (10) evidence that the information on the GPR image is correct, T N + F P and the same is true for m R,T (b) referring to the evidence where positive and negative refer to an identified and a for the IR image. To obtain a fused image, derived evidences rejected defect, respectively. Thus, a true positive call means were combined through weighted averaging that a pixel was correctly identified as a defect. A combined sensitivity measure c _ sens was further calculated by xDS = m R,T (a) xR + m R,T (b) xT . (14) m R,T (a) + m R,T (b) c _ sens = a · sens + b · spec , (11) a + b 3.3 Results and Discussion where weights a and b refer to the specimen’s corresponding The results of GPR and IR thermographic inspections are defect and non-defect area, respectively. shown for the specimens S1 at the depth of 3 cm (Fig. 2), The clustering results were additionally compared with S2 at the depth of 4.6 cm (Fig. 3) and S3 at the depth of 1.5 the results obtained after applying supervised symbol level cm (Fig. 4) (results obtained at other depths are not taken fusion, based on the assignment of weights referring to into account due to the suspected movement of defects dur- the probability of defect detection of the measurement sys- ing concreting). Red rectangles mark the known position of tem. Based on results from previous experiments on con- the defects, which served as a reference to evaluate the effec- crete with the same measurement system, a recommenda- tiveness of image fusion. For a qualitative evaluation, images tion for weights of both NDT methods was given in accor- resulting from both feature (based on the FCM clustering dance with the depth of defects through sensitivity curves algorithm) and symbol (based on the supervised assignment [45]. In this study we refer to the sensitivity curve calculated of weights) level fusion using weighted averaging are shown for polystyrene defects since it covers the investigated depth next to images used as input data. For symbol level fusion, 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A7 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 2 Results from specimen S1 at the depth of 3 cm with marked 6.2 × 10−4 Hz (b), results from weighted averaging of images (a) and defects according to Table 1 [the upper defect refers to air (b), the lower (b) using unsupervised feature level fusion (c) and supervised symbol to water (d)]: GPR depth slice (a), IR phase contrast image at frequency level fusion (d) Fig. 3 Results from specimen S2 at the depth of 4.6 cm with marked IR phase contrast image at frequency 3.0 × 10−4 Hz (b), results from defects according to Table 1 (the right part of the images refers to the weighted averaging of images (a) and (b) using unsupervised feature part of the specimen with dense reinforcement): GPR depth slice (a), level fusion (c) and supervised symbol level fusion (d) the weights used for the GPR/IR images were 0.88/0.89 for quently, the shape of the plastic pipes, that were more accu- specimen S1 at the depth of 3 cm, 0.89/0.89 for specimen rately detected by IR thermography, could be well visualized S2 at the depth of 4.6 cm and 0.75/0.91 for specimen S3 when using both supervised averaging and the D–S rule of at the depth of 1.5 cm [45]. In Fig. 2 the lower rectangle combination, where a slightly higher contrast refers to D–S refers to the water (d) and the upper to the air (b) defect. It (Fig. 4e). However, it can be seen that the fusion with the can be seen that due to the overlapping performance of the FCM algorithm yielded an image with even higher contrast, two NDT methods both unsupervised feature level (Fig. 2c) where both plates and pipes could be well resolved (Fig. 4c). as well as supervised symbol level fusion (Fig. 2d) yielded To account for the drawback of the subjective visual evalu- very similar results. However, in the case of specimen S2, ation of image fusion we further evaluated the results quanti- clustering-based image fusion helped particularly in identi- tatively. To perform the sensitivity calculation (in accordance fying the defect below the more dense reinforcement as well with Eq. 11) in regard to whether a defect was detected or as in eliminating the reinforcement pattern detected by GPR not, images were partitioned into two clusters. To provide (Fig. 3c). Although it was hoped that clustering would dif- an objective comparison of the results and thus to avoid the ferentiate between the air and water defect when employing assignment of a threshold value for the classification, the higher cluster numbers, this was not the case since GPR could standard FCM algorithm was employed, where the threshold not derive any information about the phase at the depth of 3 value is assigned automatically with respect to the condition cm. for minimizing the functional J in Eq. 2 (we preferred the In contrast to specimens S1 and S2, the weights assigned FCM to the hard k-means algorithm to avoid the k-means in the case of specimen S3 at the depth of 1.5 cm tended initialization problems as noted in Sect. 2). To obtain a hard to assign much more knowledge to IR thermography (0.91 association of the most probable cluster, only the cluster with for IR thermography compared to 0.75 for GPR). Conse- the highest membership degree was taken into account. Table 123 A8 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 4 Results from specimen S3 at the depth of 1.5 cm with marked averaging of images (a) and (b) using unsupervised feature level fusion polystyrene plates according to Table 1: GPR depth slice (a), IR phase (c) and supervised symbol level fusion (d), and results from D–S of contrast image at frequency 3.7 × 10−3 Hz (b), results from weighted images (a) and (b) using supervised symbol level fusion (e) Table 2 Results (in %) of the combined sensitivity ( c _ sens) for the methods, where the dielectric constant and the thermal con- detection of different defects by GPR and IR thermography, as well as ductivity govern the behaviour of GPR and IR thermogra- by data fusion using the unsupervised averaging with fuzzy c-means phy, respectively. The greater difference in dielectric con- (FCM) algorithm, and the supervised averaging and Dempster–Shafer (D–S) stant between water and concrete compared to air and con- crete (for air, water and concrete the dielectric constants are Defect GPR IR thermo. Unsup. average Sup. average Sup. D–S approximately 1, 81 and 9, respectively) results in a higher S1b 89.8 90.1 90.5 90.4 90.3 amplitude of the reflected radar signal and thus better image S1d 92.9 91.9 93.7 93.7 93.7 contrast. For IR thermography, the opposite relation exists S2d 54.4 89.0 89.7 76.0 77.2 for the difference in the thermal conductivity between water S3b 74.9 92.3 90.6 89.7 90.8 and concrete compared to air and concrete (for air, water and concrete the thermal conductivities are approximately 0.024, For the defect specifications refer to Table 1 0.58 and 2.1 W/mK, respectively). It can be seen that for both the air and water defect, where the sensitivities of the two 2 presents the results of c _ sens when only the GPR or IR NDT methods did not differ greatly, clustering-based fusion image was taken into account in comparison to the results with the FCM further improved defect detection. However, obtained with weighted averaging using the FCM algorithm in the case of specimen S2, the calculated sensitivity of GPR as well as with weighted averaging and D–S using sensor at the part of the specimen with dense reinforcement is much information. It can be seen that at the depth of 3 cm IR ther- smaller than the sensitivity of IR thermography (see results mography has a slightly higher probability of detecting an for defect S2d). It was thus expected that the fused image air void (defect S1b) than GPR, whereas a water void at would not outperform the IR image. Nevertheless, the results the same depth (defect S1d) was better detected by GPR. show that the FCM even further improved the detectability of This is consistent with the expected performance of the two the defect below dense reinforcement. In the case of specimen 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A9 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval S3, quantitative evaluation is presented only for the right (b) limiting IR thermography to the smallest values. This means polystyrene plate (the left plate seems to have tilted during that the GK is able to classify data points with respect to the concreting and the exact position of the plastic pipes could more relevant (IR thermography) results while treating GPR not be assessed due to small movements during concreting). data as more noisy. That is in accordance with Fig. 4a, where As already observed from visual inspection, the results show especially in the region of the plastic pipes, GPR data exhibit that IR thermography detects the plate more accurately. Here, poor contrast. the FCM could not improve localization, but at the same time To quantitatively prove the clustering results above, Table worsened the sensitivity of IR thermography only to a small 3 summarizes the outcome from sensitivity calculation for extent. all the defects taken into account. The results show that the Regarding the outcome from supervised averaging and behaviour of PFCM in general resembles the one of FCM. D–S in Table 2, it is evident that in general both D–S and This suggests that noise does not appreciably restrict the weighted averaging performed similarly. Moreover, where detection of defects in the case of our NDT data. However, remarkably higher weights were assigned to the system with this is contrary to the results of an eddy current inspection of better performance, it was suspected that both supervised stainless steel cladding tubes [13], or spectroscopic measure- methods would outperform clustering-based fusion. On the ment results on concrete [18] where noise played an impor- contrary, only negligible improvement was obtained with tant role. In addition, it can be seen that the GK performs D–S (see results for defect S3b). On the other hand, where differently with regards to the FCM when parameter γ does weights were not assigned according to the reliability of the not additionally restrict the shape of clusters (for γ = 0). two NDT methods (in the case of dense reinforcement), the Although the GK with γ = 0 could best visualize the plas- performance of both averaging and D–S deteriorated to a tic pipes, its poor performance stands out particularly in the considerable extent (see results for defect S2d). This proves case of defect S2d, where its localization deteriorated by that the operation of both methods depends greatly upon the more than 10 % compared to the FCM. An explanation for a priori selected weights, which has already been argued in this could be derived from the GPR results in Fig. 3a. Due to [10]. In contrast, clustering methods work unsupervised and the detection of reinforcement, the cluster centres are close can handle various data sets without knowledge about the to each other and the clusters are extremely long in one direc- data. tion, which has been reported to produce poor models [30]. To derive an optimal clustering-based data fusion frame- In contrast, by only slightly restricting the algorithm with γ work, we further compared the performance of different = 0.1, the GK performed similarly to the FCM while being fuzzy clustering algorithms for segmentation of the data. at the same time also sensitive to the detection of elongated A qualitative comparison of the resulting classification of defects. To summarize, by taking into account both the robust specimen S3 is given in Fig. 5, where the cluster colour performance of the methods as well as the sensitivity to small is displayed according to the hard association of the most variations, the modified GK (restricted by β and γ ) can be probable cluster (the cluster with the highest membership regarded as the most recommendable clustering algorithm degree), whereas the factor of transparency takes into account for the investigated NDT data sets. the membership degree (e.g. ui j = 1: opaque, ui j = 0.5: 50 % transparency). The results show that the GK algorithm (with parameter γ = 0) outperforms the FCM for the detec- 4 Image Fusion Applied to a Case Study tion of pipes, whereas for the PFCM no improvement from the FCM can be seen. The good performance of the GK is As already mentioned in Sect. 1, one of the main tasks in understandable since the method is able to detect clusters deriving a generalized data fusion framework for NDE-CE of various shapes, whereas the FCM and PFCM algorithms applications is to promote fast, efficient and reliable evalu- are restricted to clusters of equal shape. However, it should ation of large data sets. Examples of large area inspection be noted that clustering methods treat each data point (pixel include parking garages and bridge decks suffering from value) independently. Thus, the shape of clusters should not severe corrosion of the reinforcement, which is reflected in refer to the geometrical shapes detected from the GPR and IR cracking, spalling and losses of the concrete cross section. thermography images, but rather to the geometrical shapes In order to evaluate the extent of degradation adequately, obtained from data plots correlating the GPR and IR ther- investigation of surfaces of some thousand square metres mography data. For specimen S3, correlation plots between with a dense grid is necessary. To solve this problem, a self- GPR and IR thermography data are shown in Fig. 6 using the navigating mobile robot system has been developed within FCM and GK (for γ = 0) algorithms. It can be seen that in the BetoScan project (www.betoscan.de). The key task of contrast to the FCM, the GK yields an elongated black clus- the BetoScan system is to divide the investigated structure ter (corresponding to the black (non-defect) data points in into zones with defined damage classes. Therefore, potential Fig. 5c) with GPR extending over the whole dimension and maps together with other parameters such as the distribution 123 A10 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 5 Results from clustering of images Fig. 4a, b using the fuzzy to detected defects, black to non-defect area with the factor of trans- c-means (FCM) (a), possibilistic-fuzzy c-means (PFCM) (b) and parency referring to the membership degree Gustafson-Kessel (GK) with γ = 0 (c) algorithms. White cluster refers Fig. 6 Correlation plots between GPR and IR thermography normal- points refer to the white and black cluster in Fig. 5, whereas the red ized data from specimen S3 using the fuzzy c-means (FCM) (a) and point marks the corresponding cluster centre Gustafson-Kessel (GK for γ = 0) (b) algorithms. White and black data Table 3 Results (in %) of the combined sensitivity ( c _ sens) for the – eddy current + GPR (to assess the concrete cover above detection of different defects with data fusion using the fuzzy c- the reinforcement layer), means (FCM), possibilistic-fuzzy c-means (PFCM) and Gustafson- – ultrasonic (to assess the thickness of the investigated Kessel (GK) algorithms structure, the presence of voids and cracks), Defect FCM PFCM GK ( γ = 0) GK ( γ = 0.1) – microwave (to assess the humidity distribution). S1b 90.5 90.5 90.6 90.5 S1d 93.7 93.7 94.4 93.7 Except for ultrasonic, all the sensors can collect data S2d 89.7 89.3 80.3 87.0 contactlessly, enabling a driving speed from 0.1 up to S3b 90.6 90.6 91.5 90.8 0.5 m/s (depending on the methods used). As well as the methods given above, additional methods and devices For the defect specifications refer to Table 1 with a digital interface may be included in the system. of concrete cover and moisture have to be assessed simultane- A more detailed description of a current multi-sensor sys- ously. With the BetoScan system this is achieved by an auto- tem is given in [47]. Although the BetoScan system has mated multi-sensor system comprising the following NDT a fully automated data acquisition system, data analy- methods (Fig. 7): sis is currently performed manually by direct comparison of the results. The development of an efficient and reli- – potential mapping (to assess the probability of active cor- able data fusion framework is therefore of major inter- rosion), est. 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A11 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval radar-magnetic method—a relatively new technique still in the state of development [48]. Here, the moisture and the salinity of concrete are estimated by the real and imagi- nary parts of the dielectric constant of the concrete covering the reinforcement. Firstly, the static magnetic field of pre- magnetized rebars is measured. The depth of the rebars is obtained through a knowledge of the magnetic field com- ponents orthogonal to the rebar direction. The mathematical approximation was described by Krause et al. in [48]. Sec- ondly, the real ( ε ) ) r and imaginary part ( ε r of the dielectric constant are calculated using the travel time tr and reflec- tion amplitude rtot measured by GPR, as well as the concrete cover previously assessed by the magnetic method. Measurements with the radar-magnetic method were per- formed with a system introduced by the same authors, apply- ing the GPR RAMAC system from MALÅ Geoscience with a 1 GHz monostatic antenna and a set of four GMR magne- tometers. The acquired data was aligned in accordance with Fig. 7 The BetoScan robotic system with NDT sensors the scanned area by the BetoScan. 4.2 Results and Discussion 4.1 Methods From the surface plot of the electrochemical potentials of the We tested our clustering-based data fusion methodology for concrete floor investigated, critical areas with a high prob- data segmentation on a BetoScan data set acquired from the ability of corrosion could be identified by potentials lower reinforced concrete floor of a parking garage in Germany. than –300 mV (Fig. 8a). The main effects that induce corro- The area inspected measured approximately 5 × 7 m2. The sion in concrete structures are carbonation and high chloride spacing between tracks was around 20 cm, with about 1 cm concentration [49]. In general, the probability of carbonation between data points of each track. Before data fusion, nor- increases with decreasing concrete cover above the reinforce- malization and alignment of the data was performed and a ment layer. The results show that at some parts of the surface common spatial reference with data interpolation was estab- where the concrete cover was even less than 30 mm, the cor- lished. rosion process was very intensive (compare the right side of Potential mapping, eddy current, as well as microwave Fig. 8a and Fig. 8b). However, it is hard to draw conclusions moisture sensors were used simultaneously. Potential map- from the moisture measurements alone. The data acquired by ping is based on measurement of the electrochemical poten- the Moist RP sensor (with a penetration capability of up to tials of the reinforcement on the concrete surface by the use 4 cm depth) suggest a variable humidity distribution of the of a reference electrode known as a half-cell and a high- scanned area that cannot be correlated with either the poten- impedance voltmeter. A potential lower than –300 mV indi- tial or concrete cover results (Fig. 8c). On the other hand, the cates a high probability of active corrosion. For the BetoScan humidity distribution assessed by the Moist PP sensor sug- system, a Canin+ instrument from Proceq was used in com- gests a relative humidity of more than 3 vol. % in parts with bination with a copper/copper sulphate reference wheel elec- low concrete cover (Fig. 8d). Therefore, the striking corro- trode. For eddy current, which is based on electromagnetic sion on the right side of the floor is probably due to both pulse induction to detect flaws in conductive materials, a carbonation and a high salinity concentration (by assuming Profometer 5+ from Proceq was used, which combines rebar a correlation between moisture and salinity concentration). detection and measurement of concrete cover. To measure the To support our assumption, we additionally present results relative moisture distribution of the concrete, the microwave- by the radar-magnetic method. As shown from the results of based devices Moist RP (4 cm penetration) and Moist PP (20 the Moist RP sensor, the real part of the dielectric constant cm penetration) from HF-Sensor were used. The microwave also suggests a variable humidity distribution (Fig. 8e). In method is based on the fact that with increase in moisture contrast, the plot of the imaginary part of the dielectric con- content of the material, microwaves lose more energy due to stant ε r is in good agreement with the results from potential induced oscillations of the electric field during transmission. mapping and eddy current (Fig. 8f). Since ε r refers to energy To support the results obtained by the BetoScan system, losses of the electromagnetic wave due to free charges in the we additionally took into account the data obtained by a material, a high value of ε r refers to concrete with higher 123 A12 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 8 Results from the BetoScan measurement of a concrete parking (f) part of the dielectric constant (note that due to the chosen colour deck using potential mapping (a), eddy current (b), microwaves with a bar in Fig. 8f, the white area corresponds to values up till 0.35 of the Moist RP sensor (c) and microwaves with a Moist PP sensor (d). Results imaginary part of the dielectric constant) from the radar-magnetic measurement with the real (e) and imaginary salinity. This demonstrates the information from salinity con- ever, the resulting partitions show that for the combination centration to be better correlated to corrosion than moisture. of potential mapping, eddy current and microwaves with the The BetoScan data, as well as the data from the radar- Moist PP sensor, zones with a higher probability of dam- magnetic measurement, show that estimation of the condition age can also be clearly detected (Fig. 9b). Here, the red of the structure is not straightforward. Each of the applied cluster could be identified with the zone of severe corro- NDT methods suffers from limited reliability in identify- sion, the black cluster with the area of either small concrete ing certain parameters. The efficiency of using clustering cover or high salinity content (or both) where the corro- methods to estimate zones with a different degree of degra- sion process is not yet observable, whereas the white area dation is demonstrated in Fig. 9. Only the results obtained could be characterized as undamaged (or minimally dam- using the GK algorithm with γ = 0.1 are presented when aged) concrete as a result of higher concrete cover and good using three clusters. FCM produced an almost similar result concrete quality. To prove the robustness of the clustering for this data set. Regarding the correlation between data, methods, we further evaluated the whole of the BetoScan the optimal solution would be to combine potential map- data. In Fig. 9c it can be seen that although the data acquired ping, eddy current and radar-magnetic measurement data by microwave with the Moist RP sensor worsened the final with the imaginary part of the dielectric constant (Fig. 9a) partition, the most important damage zone can still be well (for the correlation plot for this data set see Fig. 10). How- recognized. 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A13 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval Fig. 9 Results from image fusion using the Gustafson-Kessel (GK) severe corrosion, the black one with the area of either small concrete clustering algorithm while taking into account different data sets: poten- cover or high salinity content (or both) where the corrosion process is tial mapping + eddy current + radar-magnetic method (Im ε) (a), poten- not yet observable, whereas the white area could be characterized as tial mapping + eddy current + microwaves with sensor Moist PP (b) undamaged (or minimally damaged). The factor of transparency refers and potential mapping + eddy current + microwaves with sensors Moist to the cluster membership degree PP and Moist RP (c). Red cluster could be identified with the area of both defect localization as well as characterization of the defects. The use of fuzzy probabilistic clustering (employing the FCM and GK algorithms), as well as fuzzy possibilistic clus- tering (with the PFCM algorithm), was explored on con- crete validation specimens with GPR and IR thermography data. Results from the sensitivity evaluation showed that the PFCM in general performed similarly as the FCM, which suggests that noise has only a minor effect on the resulting partition. On the contrary, the use of the modified version of the GK method helped particularly to detect clusters of arbi- trary shape, since the algorithm is not restricted to spherical clusters as are the FCM and PFCM. Elongated clusters were produced when the method took into account the relevance of only one method, such as for the detection of plastic pipes and cracks. For best performance of the GK it is suggested that the parameter γ be set to 0.1. Fig. 10 Correlation plots between potential mapping, eddy current and radar-magnetic method (Im ε) normalized data using the Gustafson- The efficiency of using clustering for data classification Kessel (GK) clustering algorithm. Cluster colours refer to the colours in feature level fusion was evaluated through comparison in Fig. 9a, whereas the yellow points mark the corresponding cluster with supervised symbol level fusion using weighted averag- centre ing and D–S. The results showed that in the case where more knowledge was assigned to the NDT system with better per- 5 Conclusions formance, only a negligible improvement was obtained with supervised fusion using D–S compared to clustering-based The use of clustering methods for image classification fusion. However, if weights were not assigned according to of fused NDT-CE data sets was studied. The cluster- the reliability of the respective NDT methods, the perfor- ing’s direct classification outcome as well as their use in mance of both averaging and D–S deteriorated to a large feature-level fusion were taken into account. The meth- extent. The unsupervised clustering methods, on the other ods proved particularly helpful in detecting near-surface hand, showed robust performance. defects simulating delamination, voids and localized water The use of clustering techniques to combine data from as well as in partitioning a reinforced concrete floor from GPR and IR thermography in general increased the reliability a parking garage into damage zones, and thus provided of NDT inspection in cases where the methods had a com- 123 A14 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval plementary penetration capability and sensitivity to different 9. Wiggenhauser, H.: Advanced NDT methods for quality assur- physical properties. Moreover, in cases where one method ance of concrete structures. In: Proceedings NDTCE’09, Non- destructive testing in civil engineering. LCPC, Paris (2009) greatly dominated over the other, they worsened its perfor- 10. Gros, X.E., Bousigue, J., Takahashi, K.: NDT data fusion at pixel mance only to a small extent. This shows clustering methods level. NDT E Int. 32, 283–292 (1999) to be robust in situations with very limited knowledge about 11. Gros, X.E., Liu, Z., Tsukada, K., Hanasaki, K.: Experimenting with the material properties and depths of the defects, which was pixel-level NDT data fusion techniques. IEEE T Instrum. Meas. 49(5), 1083–1090 (2000) additionally proven by the BetoScan data set. Here, the use of 12. Liu, Z., Forsyth, D.S., Safizadeh, M.S., Fahr, A.: A data-fusion clustering also enabled characterization of the defects. More- scheme for quantitative image analysis by using locally weighted over, it should be emphasized that for a multi-sensor system regression and Dempster–Shafer theory. IEEE T Instrum. Meas. such as the BetoScan, image fusion with clustering is of par- 57(11), 2554–2560 (2008) 13. Balakrishnan, S., Cacciola, M., Udpa, L., Rao, B.P., Jayakumar, T., ticular importance since it delivers one final image that takes Raj, B.: Development of image fusion methodology using discrete into account the correlation between the various NDT meth- wavelet transform for eddy current images. NDT E Int. 51, 51–57 ods. In practice, displaying only one reliable image rather (2012) than multiple ones to the end-user is of the utmost impor- 14. Kohl, C., Krause, M., Maierhofer, C., Wöstmann, J.: 2D-and 3D- visualisation of NDT-data using data fusion technique. Mater. tance. Struct. 38, 817–826 (2005) 15. Cui, J., Huston, D.R., Arndt, R., Jalinoos, F.: Data fusion techniques Acknowledgments The first author acknowledges the financial sup- of multiple sensors nondestructive evaluation of a concrete bridge port of the Slovenian Research Agency through grant 1000-10-310156 deck. TRB Annual Meeting Online (AMOnline). http://amonline. and the Slovene Human Resources and Scholarship Fund through grant trb.org/2vdh1b/2vdh1b/1 (2010). Accessed 11 February 2012 11012-13/2012. The authors wish to thank Dr. Hans-Joachim Krause 16. Bensaid, A.M., Hall, L.O., Bezdek, J.C., Clarke, L.P., Silbiger, and colleagues for the radar-magnetic data set. Contributions and sup- M.L., Arrington, J.A., et al.: Validity-guided (re)clustering with port from Assoc. Prof. Violeta Bokan Bosiljkov, Primož Murn and Dr. applications to image segmentation. IEEE T Fuzzy Syst. 4(2), 112– Parisa Shokouhi are acknowledged. 123 (1996) 17. Santoro, M., Prevete, R., Cavallo, L., Catanzariti, E.: Mass detec- tion in mammograms using gabor filters and fuzzy clustering. In: Bloch, I., Petrosino, A., Tettamanzi, A.G.B. (eds.) Fuzzy logic and applications. WILF 2005: Proceedings of the 6th international workshop, pp. 334–343. Springer, Berlin (2006) References 18. Cotič, P., Niederleithinger, E., Wilsch, G.: Unsupervised clustering of spatially acquired laser-induced breakdown spectroscopy data 1. McCann, D.M., Forde, M.C.: Review of NDT methods in the from concrete. Cem. Concr. Res. (submitted) assessment of concrete and masonry structures. NDT E Int. 34, 19. Cotič, P., Niederleithinger, E., Bosiljkov, V., Jagličić, Z.: NDT data 71–84 (2001) fusion for the enhancement of defect visualization in concrete. Key 2. Beutel, R., Reinhardt, H.W., Grosse, C.U., Glaubitt, A., Krause, Eng. Mater. 569–570, 175–182 (2013) M., Maierhofer, C., et al.: Comparative performance tests and val- 20. Liu, Z., Forsyth, D.S., Komorowski, J.P., Hanasaki, K., Kirubara- idation of NDT methods for concrete testing. J. Nondestruct. Eval. jan, T.: Survey: state of the art in NDE data fusion techniques. IEEE 27, 59–65 (2008) T Instrum. Meas. 56, 2435–2451 (2007) 3. Shokouhi, P., Wiggenhauser, H.: Multi-probe ultrasonic test- 21. Kaftandjian, V., Zhu, Y.M., Dupuis, O., Babot, D.: The combined ing for detection of delamination in concrete bridge decks. use of the evidence theory and fuzzy logic for improving mul- TRB Annual Meeting Online (AMOnline). http://amonline.trb.org/ timodal nondestructive testing systems. IEEE T Instrum. Meas. 1sl7d2/1 (2011). Accessed 5 January 2013 54(5), 1968–1977 (2005) 4. Maierhofer, C., Arndt, R., Röllig, M., Rieck, C., Walther, A., 22. Bogaerta, P., Fasbender, D.: Bayesian data fusion in a spatial pre- Scheel, H., et al.: Application of impulse-thermography for non- diction context: a general formulation. Stoch Environ. Res. Risk destructive assessment of concrete structures. Cem. Concr. Com- Assess. 21, 695–709 (2007) pos. 28, 393–401 (2006) 23. Li, G., Huang, P., Chen, P., Hou, D., Zhang, G., Zhou, Z. Appli- 5. Cotič, P., Jagličić Z, Bosiljkov, V.: Validation of non-destructive cation of multi-sensor data fusion in defects evaluation based on characterization of the structure and seismic damage propagation Dempster–Shafer theory. IEEE Xplore Digital Library. http://ieee of plaster and texture in multi-leaf stone masonry walls of cultural- xplore.ieee.org/xpl/articleDetails.jsp?arnumber=5944335 (2011). artistic value. J. Cult. Herit. (2013). doi:10.1016/j.culher.2013.11. Accessed 13 February 2012 004 24. Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy cluster 6. Cotic, P., Jaglicic, Z., Niederleithinger, E., Effner, U., Kruschwitz, analysis: methods for classification, data analysis and image recog- S., Trela, C., et al.: Effect of moisture on the reliability of void nition, 1st edn. Wiley, Chichester (1999) detection in brickwork masonry using radar, ultrasonic and com- 25. MacQueen, J.B.: Some methods for classification and analysis of plex resistivity tomography. Mater. Struct. 46, 1723–1735 (2013). multivariate observations. Proceedings of the 5th Berkeley sympo- doi:10.1617/s11527-012-0011-3 sium on mathematical statistics and probability, vol 1: Statistics, 7. Maierhofer, C., Wöstmann, J.: Investigation of dielectric properties pp. 281–297. University of California Press, Berkeley (1967) of brick materials as a function of moisture and salt content using 26. Bezdek, J.Z.: Pattern recognition with fuzzy objective function a microwave impulse technique at very high frequencies. NDT E algorithms. Plenum Press, New York (1981) Int. 31(4), 259–263 (1998) 27. Gustafson, D., Kessel, W.: Fuzzy clustering with a fuzzy covariance 8. Maierhofer, C., Zacher, G., Kohl, C., Wöstmann, J.: Evaluation matrix. Proceedings of the 1978 IEEE conference on decision and of radar and complementary echo methods for NDT of concrete control including the 17th Symposium on adaptive processes, pp. elements. J. Nondestruct. Eval. 27, 47–57 (2008) 761–766. IEEE Control Systems Society, San Diego (1979) 123 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. A15 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. J Nondestruct Eval 28. Dave, R.N.: Use of the adaptive fuzzy clustering algorithm to detect 40. Maldague, X.P., Marinetti, S.: Pulse phase infrared thermography. lines in digital images. In: Casasent, D.P. (ed.) Intelligent robots J. Appl. Phys. 79, 2694–2698 (1996) and computer vision VIII: algorithms and techniques, vol 1192. 41. Arndt, R.: Square pulse thermography in frequency domain. SPIE, pp. 600–611 (1990) SPIE Digital Library. http://proceedings.spiedigitallibrary.org/ 29. Krishnapuram, R., Kim, J.: A note on the Gustafson-Kessel and proceeding.aspx?articleid=834339 (2008). Accessed 13 Septem- adaptive fuzzy clustering algorithms. IEEE T Fuzzy Syst. 7(4), ber 2012 453–461 (1999) 42. Vavilov, V., Burleigh, D.: Heat transfer. In: Maldague, X., Moore, 30. Babuška, R., van der Veen, P.J., Kaymak, U.: Improved covariance P.O. (eds.) Nondestructive handbook, infrared and thermal testing, estimation for Gustafson-Kessel clustering. In: Proceedings of the vol. 3, 3rd edn. ASNT Press, Columbus (2001) 2002 IEEE international conference on fuzzy systems, vol 2. IEEE, 43. Weritz, F., Arndt, R., Röllig, M., Maierhofer, C., Wiggenhauser, H.: pp. 1081–1085 (2002) Investigation of concrete structures with pulse phase thermography. 31. Kruse, R., Döring, C., Lesot, M.J.: Fundamentals of fuzzy clus- Mater. Struct. 38, 843–849 (2005) tering. In: Oliveira, J.V., Pedrycz, W. (eds.) Advances in fuzzy 44. Altman, D.G., Bland, J.M.: Statistics notes: diagnostic tests 1: sen- clustering and its applications, pp. 3–30. Wiley, Chichester (2007) sitivity and specificity. BMJ 308, 1552 (1994) 32. Dave, R.N.: Characterization and detection of noise in clustering. 45. Cotič, P., Jagličić, Z., Bosiljkov, V., Niederleithinger, E.: GPR and Pattern Recogn. Lett. 12(11), 657–664 (1991) IR thermography for near-surface defect detection in building struc- 33. Krishnapuram, R., Keller, J.M.: The possibilistic c-means algo- tures. In: Grum, J. (ed.) Proceedings of the 12th international con- rithm: insights and recommendations. IEEE Trans. Fuzzy Syst. ference of the Slovenian society for non-destructive testing, Appli- 4(3), 385–393 (1996) cation of Contemporary Non-Destructive Testing in Engineering. 34. Pal, N.R., Pal, K., Bezdek, J.C.: A mixed c-means clustering model. Slovenian Society for Nondestructive testing, Ljubljana (2013) In: Proceedings of the 6th IEEE international conference on fuzzy 46. Shafer, G.: A mathematical theory of evidence. Princeton Univer- systems, vol 1. IEEE, pp. 11–21 (1997) sity Press, Princeton (1976) 35. Timm, H., Borgelt, C., Kruse, R.: Fuzzy cluster analysis with 47. Stoppel, M., Taffe, A., Wiggenhauser, H., Kurz, J.H., Boller, C.: cluster repulsion. CiteSeer X β . http://citeseerx.ist.psu.edu/viewdoc/ Automated multi-sensor systems in civil engineering for condition summary?doi=10.1.1.100.9787 (2001). Accessed 16 November assessment of concrete structures. In: Grantham, M. (ed.) Proceed- 2012 ings of the 4th International Conference on Concrete Repair, pp. 36. Pal, N.R., Pal, K., Keller, J.M., Bezdek, J.C.: A possibilistic fuzzy 397–403. CRC Press, Boca Raton (2011) c-means clustering algorithm. IEEE T Fuzzy Syst. 13(4), 517–530 48. Krause, H.J., Rath, E., Sawade, G., Dumat, F.: Radar-Magnet- (2005) Betontest: Eine neue Methode zur Bestimmung der Feuchte und 37. Daniels, J.D. (ed.): Ground-penetrating radar, 2nd edn. The Insti- des Chloridgehalts von Brückenfahrbahnplatten aus Beton. Beton- tution of Electrical Engineers, London (2004) und Stahlbetonbau 102(12), 825–834 (2007) 38. Jol, H. (ed.): Ground penetrating radar: theory and applications, 1st 49. Reichling, K., Raupach, M., Wiggenhauser, H., Stoppel, M., Dob- edn. Elsevier Science, Amsterdam, Oxford (2009) mann, G., Kurz, J.: BETOSCAN: an instrumented mobile robot 39. Maierhofer, C., Röllig, M.: Application of active thermography to system for the diagnosis of reinforced concrete floors. Restor. the detection of safety relevant defects in civil engineering struc- Build. Monum. 15(4), 277–286 (2009) tures. In: Proceedings OPTO 2009 & IRS2 2009. AMA, pp. 215– 220 (2009) 123 A16 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. B1 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. APPENDIX B: ARTICLE 2 Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi- leaf stone masonry walls of cultural-artistic value P. Cotič, Z. Jagličić & V. Bosiljkov J Cult Herit (2013) DOI http://dx.doi.org/10.1016/j.culher.2013.11.004 B2 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS Journal of Cultural Heritage xxx (2013) xxx–xxx Available online at ScienceDirect www.sciencedirect.com Original article Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value Patricia Cotič a,b,∗, Zvonko Jagliči ć a,b,1, Vlatko Bosiljkov b,2 a Institute of Mathematics, Physics and Mechanics, Jadranska 19, 1000 Ljubljana, Slovenia b Faculty of Civil and Geodetic Engineering, University of Ljubljana, Jamova 2, 1000 Ljubljana, Slovenia a r t i c l e i n f o a b s t r a c t Article history: Assessment of multi-leaf stone masonry in earthquake-prone areas is mostly related to the evaluation Received 2 July 2013 of its texture, morphology, leaf detachment and structural cracking due to previous seismic activity, as Accepted 7 November 2013 well as disintegration due to material deterioration. For the plastered masonry with heritage or artistic Available online xxx value (paintings, frescoes etc.), both the type of structure and the extent of damage should be charac- terized with minimal interference to the structure, which could be overcome solely by non-destructive Keywords: testing (NDT). However, due to the complexity of plastered multi-leaf masonry structure, the perfor- Multi-leaf stone masonry mance of well-known NDT methods could be significantly worsened. Therefore, as a prerequisite for Seismic damage propagation applying NDT on multi-leaf stone masonry in practice, a validation process should be carried out. In this Non-destructive validation Ground penetrating radar study, complementary ground penetrating radar (GPR) and infrared (IR) thermography measurements Infrared thermography on plastered laboratory three-leaf stone masonry walls were performed. Apart from assessing the wall Data fusion texture and morphology with the type of connection between the leaves, detection of gradual plaster delamination and crack propagation while subjecting the walls to an in-plane cyclic shear test was taken into account. The results showed that GPR could successfully visualize header stones passing through the whole depth of the specimen. The masonry texture behind the plaster could be well resolved by both methods, although GPR near-field effects worsened its localisation. For the detection of plaster delamination, IR thermography outperformed GPR by detecting delamination as small as 2 mm as well as structural crack patterns, whereas GPR only detected delamination larger than 8 mm. It was shown that the performance of both methods for defect detection could be further improved by image fusion based on unsupervised clustering methods. © 2013 Elsevier Masson SAS. All rights reserved. 1. Research aims 2. Introduction The aim of this study was to evaluate the complementary per- A large proportion of existing stone masonry buildings in Euro- formance of ground penetrating radar and infrared thermography pean Mediterranean countries, as well as elsewhere, represent in non-destructively assessing the type of structure (texture and cultural heritage assets. Very often, the stone masonry consists of morphology with type of connection) and seismic related damage several leaves, of which three-leaf masonry is one of the most char- (plaster detachment and crack propagation) of plastered multi-leaf acteristics [1]. Here, the outer leaves consist of stones arranged stone masonry. The application of NDT methods to such masonry in various masonry bonds (hereinafter referred to as texture), with artistic or cultural-heritage value is of particular importance whereas the inner core is a mixture of stone rubble and loose adhe- in evaluating the structural condition with minimal interference. sive material. The leaves may be connected with so-called header stones, i.e. connecting stones passing through the whole depth of the specimen [2]. In most cases, this type of building is vulnerable to material ∗ Corresponding author. Institute of Mathematics, Physics and Mechanics, Jadran- deterioration and seismic action, resulting in leaf detachment and ska 19, 1000 Ljubljana, Slovenia. Tel./fax: +386 1 426 71 78. cracks which can be further emphasized due to the presence of E-mail addresses: patricia.cotic@imfm.si (P. Cotič), zvonko.jaglicic@imfm.si internal voids. For effective repair of the masonry, an accurate (Z. Jagliči ć), vlatko.bosiljkov@fgg.uni-lj.si (V. Bosiljkov). 1 Tel.: +386 1 426 71 78; fax: +386 1 426 71 78. diagnosis of the defects present and visualization of the structural 2 Tel.: +386 1 425 06 81; fax: +386 1 425 06 81. model defining the behaviour of the structure should be carried 1296-2074/$ – see front matter © 2013 Elsevier Masson SAS. All rights reserved. http://dx.doi.org/10.1016/j.culher.2013.11.004 Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. B3 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS 2 P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx out. For that, the structure’s geometry, texture and morphology of a complex structure of multi-leaf stone masonry could further (the multi-leaf structure and connections between the leaves), as worsen the performance of well-known NDT methods, the methods well as the material properties of the masonry and its constituents should be validated as a precondition for their practical application (stones/bricks and mortar) need to be determined [3]. on-site. Taking into account that for cultural heritage buildings with Within the framework of the EU research project PERPETUATE artistic value the structural condition should be assessed with the (www.perpetuate.eu), the complementary use of GPR and IR ther- minimal possible intervention, the application of non-destructive mography was studied for the structural assessment of plastered testing (NDT) methods for both structural visualization and defect laboratory three-leaf stone masonry walls. Additionally, the per- detection play an essential role. Several NDT methods have formance of both methods was evaluated for defect detection such been proposed for structural investigation of building elements as gradual plaster delamination and crack propagation while sub- [4]. Contact methods, such as ground penetrating radar (GPR), jecting the walls to an in-plane cyclic shear test monitored by a ultrasonic, sonic and impact echo are especially promising in imag- three dimensional (3D) digital image correlation (DIC) technique. ing the inner structure of buildings at depths of more than 5 The well-defined conditions enabled performance of a complete to 10 cm, depending on the specific method [5]. On masonry, validation process of the efficiency of the methods applied for such methods have been applied for detection of the morphol- the detection of seismic related damage, this being of the utmost ogy, localization of voids and metal inclusions, determination of importance for bringing NDT into practice for on-site investiga- the moisture distribution, detection of severe delamination, as tions. well as to control the effectiveness of repair by injection tech- niques [6–10]. On the other hand, non-contact active infrared 3. Experimental (IR) thermography (hereinafter we omit “active”) has been shown to be a powerful technique for imaging the structure very 3.1. Specimen description close to the surface, such as for visualization of the masonry texture, surface cracks, moisture distribution and plaster delam- The performance of NDT methods was studied on four ination [1,11–13], as well as for the investigation of mosaics plastered laboratory three-leaf masonry walls measuring [14,15]. 100 × 150 × 40 cm3. For the two layers of plaster (both coarse All the above methods are characterized by a certain optimal and fine), traditional recipes based on lime mortar were used. The depth resolution (i.e. the ability to detect defects at a certain depth external leaves of the masonry were constructed from regular with high probability), penetration capability, as well as sensitiv- coursed squared ashlar rough tooled limestone, while the internal ity to the different physical properties of the masonry [16–18]. core was filled with stone rubble and lime mortar. Two of the Moreover, due to the inhomogeneity of the masonry material, sev- specimens had header stones in every second row passing through eral methods have been used complementarily to fully characterize the whole depth of the specimens (Fig. 1a), whereas the other two the structure [1,3,19]. Taking into account that the inhomogeneity had no connecting stones (Fig. 1b). Fig. 1. The morphology of the connected (a) and unconnected (b) walls with two layers of plaster applied. Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 B4 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx 3 3.2. Methods Table 1 Limit states (LS) for the wall structure and plaster subjected to seismic loading with respect to the displacement performance of the structure and the performance of 3.2.1. Ground penetrating radar the plaster [2]. GPR is based on the propagation and reflection of high frequency electromagnetic impulses produced by an antenna system, where LS for the wall structure LS of Damage Limitation (DL) Displacement where the first crack on reflections occur at the interfaces between materials with differ- the structure appears ent dielectric properties. In complex specimens, the analysis and LS of Significant Damage (SD) Displacement where the maximum interpretation of GPR data is often complicated by the influence strength of the structure is obtained of near-field antenna coupling/induction effects, which according LS of Near Collapse (NC) Maximal displacement of the structure to Daniels [20] range up to a depth of 1.5 , where is the wave- reached length of the electromagnetic waves. For masonry with a dielectric LS for the plaster constant of around 9 and an antenna with a central frequency of First LS First detachment of the plaster Second LS First visible crack on the plaster 1.6 GHz, the wavelength is around 6.3 cm and the near-field region Third LS Plaster largely detached but still extends to a depth of almost 10 cm. Apart from the near-field repairable or significantly damaged effects, the performance of GPR is also dependent on the resolu- Fourth LS Partial or full collapse of the plaster tion. For calculation of the depth resolution (defined as the ability to differentiate two adjacent signals in time), a value of 0.25 has been proposed [21]. The IR thermography data used for the study was acquired with The GPR data used for this study was obtained with equip- type FLIR A320 IR camera, having a thermal sensitivity of 50 mK ment from MALÅ Geoscience, using a 1.6 GHz monostatic shielded (at 30 ◦C), a spatial resolution of 1.36 mrad and a focal plane array antenna. A calibrated survey wheel was used and radargrams were with an IR resolution of 320 × 240 pixels and a spectral range of collected in both directions with a line spacing of 5 cm and a point 7.5–13 ␮m. For homogeneous heating of the specimens, two IR distance of 0.5 cm. heaters (1.2 kW each) were moved parallel to the surface at a dis- tance of about 10 cm. The heating time was approximately 30 min, 3.2.2. Square pulse thermography in the frequency domain while cooling down was recorded for 45 min at a frame rate of IR thermography is based on recording IR radiation from the 0.2 Hz at approximately 3 m from the specimen’s surface. Dur- previously heated specimen’s surface and allows detection of the ing heating, the maximal reached temperature of the surface was presence of defects within a depth of approximately 10 cm [22]. In 45 ◦C. order to enhance the detectability of thermal images, we applied the pulse phase thermography (PPT) technique originally proposed 3.2.3. Experimental procedure for the detection of plaster by Maldague and Marinetti [23], and further adapted for the spe- delamination and crack propagation cial requirements of NDT applications in civil engineering by Arndt The walls were subjected to an in-plane cyclic shear test, which [24] in terms of square pulse thermography (SPT) in the frequency was originally performed in order to assess the seismic behaviour domain. Here, the specimen is square pulse heated [25] and analysis of the wall structure and plaster. Their state of damage was evalu- of the recorded temperature evolution versus time of each pixel is ated through the limit states defined in Table 1 with respect to the performed in the frequency domain by Fast Fourier Transformation displacement performance of the structure and the performance of (FFT), deriving the amplitude and phase images. The phase images the plaster, respectively [2]. The shear test set-up and the principle in particular exhibit higher depth penetration, higher resolution of the test are presented in Fig. 2. However, in our case, the shear and are less sensitive to non-uniform heating than the thermal and test also enabled us to study the application of NDT for assessing amplitude images [26]. gradual plaster delamination, as well as gradual formation of the Fig. 2. Shear test set-up and the principle of testing single and both fixed panels. Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. B5 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS 4 P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx 4.1. Visualization of masonry morphology Due to the limited depth resolution of IR thermography, struc- tural visualization of the type of connection was performed only by GPR since the position of the reflections from the inner stones in rows lacking a full connection was expected at a depth of more than 15 cm. The results revealed that the maximum reflections of the radar signal appeared at depths from 16 cm to 20 cm, which was found to be very close to the measured position of the inner stones. Fig. 4 shows the structure detected on a vertical B-scan (b) and on the corresponding depth slice at the depth of 19 cm (c) for wall W3. In the B-scan, reflections appear at every second row of stones lacking a full connection (see full arrows) and can be clearly distin- guished from connecting rows with header stones. This suggests that the difference in the dielectric constants of mortar and stone was high enough to be detected at the resolution of the 1.6 GHz antenna. It should be noted that in Fig. 4c, the reflections do not appear at all the expected positions due to the roughness of the stone surface which consequently did not produce a reflection at the same depth. Additionally, Fig. 4b shows that the wall’s 40 cm thickness could be clearly detected (see dotted arrow) at the depth of approximately 41 cm, thus with an error being less than the antenna’s vertical resolution of around 1.6 cm. 4.2. Visualization of masonry texture Examination of the stone-mortar texture behind the plaster was performed by GPR and IR thermography. Due to the uneven- ness of the masonry surface, the plaster thickness varied between 2–3.5 cm. Taking into account both the GPR near-field effects and Fig. 3. GPR and IR thermography test set-up. its vertical resolution, relatively poor localization and resolution was expected for the visualization of the underlying texture using GPR. However, for wall W1 at the depth of 1.3 cm, the amplitudes crack patterns produced due to the induced shear force through of the reflected radar signal yielded a clear reconstruction of the cyclic stepwise loading. NDT measurements were performed at the texture at a line spacing of 5 cm (Fig. 5a). Since most applica- unloaded (reference) state of the specimen, as well as at two load- tions on-site generally restrict the line spacing to 10 cm (for a time ing steps (hereinafter referred to the 1st and 2nd loading state). The efficient measurement), the corresponding reconstruction is pre- 1st loading state corresponded to the lateral displacement induced sented in Fig. 5b, where the resolution is notably decreased. The where both the first detachment of plaster and the first visible sur- IR thermal and phase images are shown in Fig. 5c and d, respec- face cracks appeared (corresponding to the first and second limit tively. The thermal image presented was obtained after 7 min of state of the plaster according to Table 1). The 2nd loading state cooling, whereas the phase image corresponds to the lowest fre- was defined by largely detached or significantly damaged plaster, quency of 3.7 × 10−4 Hz (calculated as fmin = fs/NFFT, where fs is the where a complete collapse of the plaster was assumed in the fol- sampling frequency and NFFT the number of frames). Compared lowing loading step (corresponding to the third limit state of the to GPR, IR thermography does not appreciably contribute to the plaster according to Table 1). NDT measurements had to be per- detection of the wall’s texture. However, in contrast to the ther- formed while keeping the induced lateral displacement constant, mal image, the phase image exhibits higher contrast, but on the thus preventing their application on more loading steps. For the other hand requires a longer observation time in order to derive experimental set-up of the NDT, see Fig. 3. information from a certain depth region. Nevertheless, since uni- To assess the wall’s deformation shape, a 3D DIC technique form heating cannot typically be achieved on-site and a varying which yields a 3D displacement field was applied at each loading plaster thickness can be expected (both of which greatly affect step. Following Tung et al. [27], where DIC was used for moni- the thermal images), the use of IR phase images along with GPR toring crack propagation during the testing of masonry, we used is recommended for the detection of the underlying masonry tex- DIC out-of-plane displacements together with visual inspection as ture. a reference estimation of the gradual plaster delamination. The ref- erence served to assess the complementary performance of the 4.3. Detection of plaster delamination and crack propagation applied NDT techniques both qualitatively and quantitatively. The detection of gradual plaster delamination during shear 4. Results and discussion loading was analysed with both NDT methods. For GPR, the amplitudes of the reflected radar signal were taken into account as NDT measurements were performed in order to assess the per- above. For IR thermography, we refer only to the IR phase images, formance of GPR and IR thermography for the visualization of the since the thermal images could neither visualize the delamination wall’s structure (i.e. the wall’s texture and morphology with the limit nor detect cracks. As already mentioned, DIC together with type of connection between the leaves), and for the detection of visual inspection served as a reference to evaluate the efficiency gradual plaster delamination and crack propagation. With respect of detection for both methods. For wall W1, DIC out-of-plane to the problem investigated, results are referred to the specific wall displacements for the loading states taken into account are shown (W1–W4) and loading state. in Fig. 6. It should be noted that information is derived by DIC from Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 B6 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx 5 Fig. 4. A photo of specimen W3 with a grid for the GPR measurement and marked B-scan (a); a sketch of the wall morphology and a vertical B-scan (b) and the GPR depth slice at 19 cm (c). the overall movements of the wall. Thus, both plaster delamination increased amplitude of the reflected signal and larger phases (due as well as leaf separation should contribute to the resulting out-of- to higher thermal contrast), respectively. The increased phases can plane displacements. Nevertheless, although the results presented be understood by observing the temperature variation T(t) for a refer to the unconnected wall (outer leaves not connected with pixel at the non-delaminated (P1) vs. the delaminated (P2) area header stones), visual leaf separation was not observed at either (see Fig. 6e and Fig. 7). It can be seen that the presence of an air gap of the analysed states. Its negligible contribution to the total slows down the cooling process due to its lower thermal conduc- out-of-plane displacement was also justified by the LVDT data, tivity than that of masonry. For a rough estimate, the temperature which showed it to be approximately 0.12 mm at the 2nd loading variation follows an exponential decay f(t) = e−˛·t and the corre- state. Moreover, the maximum air gap produced by the plaster sponding phase of the Fourier Transform is ph(F(ω)) = atan(–ω/˛). detached at the top right edge was visually estimated to be around Since pixel P2 has a smaller ˛ than P1, the larger phases refer to P2, 3.5 mm and more than 12 mm for the 1st and 2nd loading state, i.e. to the delaminated part. respectively. This is in a good agreement with the results in Fig. 6 However, the reliability of GPR and IR thermography for delam- and therefore proved DIC to be a reliable method for quantitative ination detection differs considerably. By comparing the IR phase validation of GPR and IR thermography data. image at the frequency of 3.7 × 10−4 Hz to the DIC results and visual The results from both NDT methods in Fig. 6 show that the inspection for the specific wall at the 1st loading state (see Fig. 6d, delaminated area detected increases with increasing lateral force. e), it is evident that an air gap larger than only 2 mm is detected GPR and IR thermography respond to the detected air gap by an using IR thermography. The latter can also be seen from the results Fig. 5. GPR depth slice at 1.3 cm for 5 cm (a) and 10 cm (b) line spacing; IR thermal image (24.5–29 ◦C) after 7 min of cooling (c) and IR phase image at the frequency of 3.7 × 10−4 Hz (d) for specimen W1. Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. B7 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS 6 P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx Fig. 6. Detection of plaster delamination by DIC, IR thermography and GPR for specimen W1: DIC out-of-plane displacements (a, d, g), IR phase images at frequency 3.7 × 10−4 Hz (b, e, h) and GPR depth slices at 1.3 cm (c, f, i). Images from the first, second and third rows refer to the reference state, first and second loading states, respectively. The position of the delamination limit is marked with arrows. in Fig. 6h, obtained at the 2nd loading state, which suggests a contrasts. On the other hand, lower frequencies carry informa- delamination over the whole right edge of the specimen. In tion from the whole depth profile, reflecting the average thermal contrast, at the 1st loading state, GPR yields a very unclear delam- behaviour of the specimen. Phase images at the lowest frequency ination pattern. According to Fig. 6i, at the 2nd loading state, a (3.7 × 10−4 Hz) were therefore used to visualize the underlying tex- delamination larger than approximately 8 mm was only detected. ture and the delamination patterns, whereas higher frequencies Although the results commented here are from only one tested were used for the detection of cracks. An example of a detected ver- wall, the limits presented for delamination detection were obtained tical surface crack at the 2nd loading state of specimen W2 is shown from the overall analysis taking into account the results from all in Fig. 8c for the IR phase image at a frequency of 2.22 × 10−3 Hz. The four walls. underlying texture and the delamination patterns are visualized by Next, the detection of cracks was considered. As expected, this the corresponding phase image at 3.7 × 10−4 Hz (Fig. 8b), together problem could not be handled using GPR [3], whereas IR thermo- with the GPR depth slice at the depth of 2 cm (Fig. 8a). In Fig. 9a, the graphy was found capable of detecting both surface and subsurface subsurface shear crack pattern at the 2nd loading state of specimen crack patterns. The latter was achieved by using IR phase images at W3 is visualized by the IR phase image at 1.85 × 10−3 Hz before the higher frequencies, which enable the detection of small thermal fall of the plaster. The corresponding photograph of the specimen Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 B8 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx 7 on the right side most probably results from both the subsurface cracks as well as plaster delamination. However, it should be noted that the method’s good performance for the detection of the sub- surface cracks resulted from having masonry with joints filled with mortar. In case of partly unfilled joints, a comparison between IR phase images at different frequencies should be carried out. Although most of the defects could be detected by using IR thermography, estimation of the depth of defects was not possi- ble. Despite the fact that use of quantitative SPT in the frequency domain has been proposed for depth interpretation of laboratory concrete specimens [28], the complexity of the masonry structure, together with the presence of different defects, restricted its use. However, in practice, multi-leaf masonry walls delamination can occur between the plaster layer and the outer leaf, or between different leaves. Cracks together with small air voids may occur anywhere in the masonry structure. Therefore, for correct local- ization of defects, the fusion of images obtained by GPR and IR thermography (image fusion) should be used to increase the reli- Fig. 7. Temperature variation of pixels P1 and P2 marked in Fig. 6e. ability of an NDT inspection on-site. To improve the imaging results and defect characterization, sev- (Fig. 9b) proved that at the time when the IR data was collected, eral algorithms have been proposed for fusion of NDT data. Bayesian only tiny surface cracks were apparent at the right side of the wall’s inference, the Dempster-Shafer (D-S) theory of evidence, wavelet surface (marked on Fig. 9b). After the plaster fell off, (i.e. following and multi-resolution analysis have been used on eddy current the 2nd loading state), a particular crack pattern could be detected and IR thermography images from composite and steel structures (Fig. 9c). By comparing the latter to the IR phase image (Fig. 9d), [29–32]. In NDT applications in civil engineering, image fusion was it could be suggested that the detected IR pattern on the left side performed on GPR data acquired at different frequencies and pola- results from the subsurface crack pattern, whereas the IR pattern rizations, and on GPR and ultrasonic data on prestressed concrete Fig. 8. Detection of plaster delamination and surface cracks by GPR and IR thermography for specimen W2 at the second loading state: GPR depth slice at 2 cm (a), IR phase images at frequencies 3.7 × 10−4 Hz (b) and 2.22 × 10−3 Hz (c), results from image fusion of images (a)–(c) using the Gustafson-Kessel (GK) algorithm (d) and weighted averaging (e). The position of the delamination limit and surface crack is marked with arrows. Fig. 9. IR phase image at frequency 1.85 × 10−3 Hz (a) and a corresponding photo of the surface cracks at the second loading state of specimen W3 (b); a photo of the subsurface crack pattern after fall of the plaster (c); overlapped Fig. (a) and (c), (d). Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. B9 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS 8 P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx bridge beams and test specimens [33–35]. Here, the applied fusion the first limit state could be resolved by IR thermography with high techniques were restricted to simple arithmetical methods. In our reliability. Moreover, since it yielded the extent of damage from the case, GPR and IR thermography images were fused on a pixel-by- whole area and not only the visible parts, it could be regarded as pixel basis using clustering methods to derive a partitioned image, a potential method for defining the performance limit state of the where the defects of different patterns could be localized and char- plaster. The use of a non-contact method for defining the character- acterized. As an image fusion technique, we have already applied istic limit points of the plaster on masonry walls in cultural heritage clustering methods to spectroscopic spatial measurement results structures plays an important role since very often such walls have on concrete for cement identification [36], as well as to laboratory valuable frescoes, stuccos or mosaics. In contrast, only the extent concrete specimens with simulated defects for defect detection of damage referred to the third limit state could be detected by [37]. There, various fuzzy clustering algorithms were assessed and 1.6 GHz GPR. Taking into account that applying GPR on a plaster of the results showed the Gustafson-Kessel (GK) algorithm [38] to the third limit state could cause even more damage, GPR could not be the most efficient for defect detection. The target of cluster- be regarded as a useful technique for defining the plaster’s perfor- ing techniques is to group a set of data (observations) into groups mance limit state. However, a complementary use of both methods (or clusters) by minimizing an objective function [39]. Their main would still be acknowledged where accurate localization of minor advantage is that they can work fully unsupervised, i.e. they do not delamination is of importance. need a priori knowledge of the statistical distribution of data. IR thermography proved additionally capable of detecting sur- In the example of the 2nd loading state of wall W2, a vertical face and subsurface crack patterns. We showed that phase images surface crack was detected in the IR phase image at a frequency of higher frequencies should be used in order to resolve small ther- of 2.22 × 10−3 Hz, whereas the underlying texture and delamina- mal contrasts resulting from cracking, while phase images at the tion pattern were detected by GPR and the IR phase image at lowest frequency are better suited for imaging the underlying tex- 3.7 × 10−4 Hz (Fig. 8). The results suggest that for a supervised ture and delamination pattern. However, it should be noted that fusion algorithm more weight should be given to IR thermography, for masonry with partly unfilled joints, additional post-processing being more effective in the detection of delamination and crack of IR phase images should be performed. patterns than GPR. However, an unsupervised clustering algorithm To conclude, the results from this study showed that by comple- should provide an image where the superior thermal data prevails. mentary use of GPR and IR thermography the structure and extent Following the outcome of our previous experiments with clustering of damage of laboratory three-leaf walls could be fully assessed. methods, the GPR and IR thermography data were fused using the It is believed that the outcome should contribute appreciably in GK algorithm, which is able to detect clusters of different shapes understanding the challenges of both methods for their application in one data set. The results are shown in Fig. 8d. For comparison, to multi-leaf walls in general. However, for further characterisa- the results of image fusion using supervised weighted averaging tion of variable masonry in practice, we showed that an even more are presented in Fig. 8e. Here, the weights were roughly estimated robust performance of both methods could be achieved by image from previous experiments on concrete specimens, i.e. 0.85/0.90 fusion based on clustering methods. Since the methods work unsu- for GPR/IR images at the depth of 1.5 cm [40]. It can be seen that pervised, they could also be used in situations with very limited both the delamination as well as the crack pattern could be distin- knowledge about the masonry and the defects. Moreover, they guished using the GK algorithm, whereas the vertical crack could could shorten the NDT evaluation process and contribute to the not be resolved by averaging. It should be noted that the use of the decision process regarding the condition of the masonry. GK algorithm helped particularly in identifying the vertical crack. Acknowledgements 5. Conclusions Part of the results were obtained through the project PERPET- The challenge to assess non-destructively the type of structure UATE (www.perpetuate.eu), funded by the European Commission and seismic damage of plastered multi-leaf stone masonry was in the 7th Framework Programme (FP7/2007-2013), under grant tackled through the complementary use of GPR and IR thermogra- agreement no 244229. The first author acknowledges the financial phy. To validate the methods’ performance for assessment of the support of the Slovenian Research Agency through grant 1000-10- type of the structure, laboratory three-leaf walls with well-known 310156. The authors wish to thank the Jožef Stefan International geometry and structure were investigated. Moreover, the methods’ Postgraduate School for the use of their radar system, the company performance in assessing seismic damage of various extents was Modri Planet d.o.o. for performing DIC, as well as Damjan Špeglič, investigated by subjecting the walls to an in-plane cyclic shear test Meta Kržan and Franci Čepon for extensive measurement support. which produced gradual plaster delamination and crack propaga- tion monitored by a 3D DIC. The DIC out-of-plane displacements References proved to be in direct correlation to the plaster delamination and therefore served for quantitative validation of GPR and IR thermo- [1] L. Binda, A. Saisi, Application of NDTs to the diagnosis of historic structures, graphy data. in: Proceedings of NDTCE’09 7th International Symposium on Non-Destructive As expected, GPR proved especially powerful in probing the Testing in Civil Engineering, Nantes, June 30–July 3, 2009. [2] M. Kržan, V. Bosiljkov, Experimental evaluation of the seismic behaviour of deeper inner structure, where inner stones in rows lacking a full multi-leaf stone masonry walls, in: Proceedings of 12th Canadian Masonry connection produced a strong and accurately positioned reflection Symposium, Vancouver, BC, June 2–5, Masonry Institute of BC, Vancouver, BC, of the radar signal and could be therefore clearly distinguished from 2013. [3] L. Binda, V. Bosiljkov, A. Saisi, L. Zanzi, Guidelines for the diagnostic inves- connecting rows with header stones. Moreover, in spite of the GPR tigation of historic buildings, in: Proceedings of 7th International Masonry near-field effects, the underlying stone-mortar texture, appearing Conference, London, October 30–November 1, British Masonry Society, London, at the depth of 2–3.5 cm, could be well visualized although its local- 2006. [4] D.M. McCann, M.C. Forde, Review of NDT methods in the assessment of concrete isation was slightly worsened. IR thermography, on the other hand, and masonry structures, NDT E Int. 34 (2001) 71–84. did not greatly contribute to the detection of the texture, but out- [5] Ch. Maierhofer, Ch. Köpp, L. Binda, L. Zanzi, J.R. Santiago, B. Knupfer, B. performed GPR in detecting an air gap produced by the detachment Johansson, C. Modena, F. Porto, M. Marchisio, F. Gravina, M. Falci, J.C. of plaster as small as 2 mm, compared to the minimum 8 mm air gap Ruiz, M. Tomazevic, V. Bosiljkov, C.H. Hennen, Project Report EUR 21696 EN – Onsiteformasonry project – On-site investigation techniques for the struc- detected by GPR. By taking into consideration the performance limit tural evaluation of historic masonry buildings, European Commission, Office for states of the plaster as defined in Table 1, it can be concluded that Official Publications of the European Communities, Brussels, 2006. Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 B10 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. G Model CULHER-2813; No. of Pages 9 ARTICLE IN PRESS P. Cotič et al. / Journal of Cultural Heritage xxx (2013) xxx–xxx 9 [6] D. Ranalli, M. Scozzafava, M. Tallini, Ground penetrating radar investigations for [24] R. Arndt, Square pulse thermography in frequency domain, in: V.P. Vavilov, D.D. the restoration of historic buildings: the case study of the Collemaggio Basilica Burleigh (Eds.), Thermosense XXX, Proceedings of SPIE, Orlando, FL, March 16, (L’Aquila, Italy), J. Cult. Herit. 5 (2004) 91–99. Society of Photo Optical, 2008, pp. 69390X–69390X-12. [7] C.H. Maierhofer, J. Wöstmann, Non-destructive investigation of complex his- [25] V. Vavilov, D. Burleigh, Heat Transfer, in: X.P. Maldague, P.O. Moore (Eds.), Non- toric masonry structures with impulse radar, in: Proceedings of NDT-CE 2003, destructive Handbook, Infrared and Thermal Testing, vol. 3, third ed., ASNT International Symposium on Non-Destructive Testing in Civil Engineering, Press, Columbus, Ohio, 2001. Berlin, September 16–19, DGZfP, Berlin, 2003. [26] F. Weritz, R. Arndt, M. Röllig, Ch. Maierhofer, H. Wiggenhauser, Investigation [8] A. Anzani, L. Binda, A. Carpinteri, S. Invernizzi, G. Lacidogna, A multilevel of concrete structures with pulse phase thermography, Mater Struct. 38 (2005) approach for the damage assessment of historic masonry towers, J. Cult. Herit. 843–849. 11 (2010) 459–470. [27] S.H. Tung, M.H. Shih, W.P. Sung, Development of digital image correlation [9] A. Sadri, Application of impact-echo technique in diagnoses and repair of stone method to analyse crack variations of masonry wall, Sadhana 33 (2008) masonry structures, NDT E Int. 36 (2003) 195–202. 767–779. [10] L. Binda, M. Lualdi, A. Saisi, L. Zanzi, Radar investigation as a complementary [28] R. Arndt, Ch. Maierhofer, M. Röllig, Quantitative pulse-phase-thermography tool for the diagnosis of historic masonry buildings, Int. J. Mater Struct. Integr. for masonry and concrete structures, in: Proceedings of 9th European NDT 5 (2011) 1–25. Conference (ECNDT), Berlin, September 25–29, DGZfP, Berlin, 2006. [11] R. Arndt, Ch. Maierhofer, M. Röllig, F. Weritz, H. Wiggenhauser, Structural inves- [29] X.E. Gros, J. Bousigue, K. Takahashi, NDT data fusion at pixel level, NDT E Int. 32 tigation of concrete and masonry structures behind plaster by means of pulse (1999) 283–292. phase thermography, in: Proceedings of 7th International Conference on Quan- [30] X.E. Gros, Z. Liu, K. Tsukada, K. Hanasaki, Experimenting with pixel-level NDT titative Infrared Thermography (QIRT), Rhode-St-Genese, July 5–8, 2004. data fusion techniques, IEEE T Instrum. Meas. 49 (2000) 1083–1090. [12] N.P. Avdelidis, A. Moropoulou, Applications of infrared thermography for the [31] Z. Liu, D.S. Forsyth, M.S. Safizadeh, A. Fahr, A data-fusion scheme for quantita- investigation of historic structures, J. Cult. Herit. 5 (2004) 119–127. tive image analysis by using locally weighted regression and Dempster–Shafer [13] Ch. Maierhofer, M. Röllig, Active thermography for the characterization of theory, IEEE T Instrum. Meas. 57 (2008) 2554–2560. surfaces and interfaces of historic masonry structures, in: Proceedings of [32] S. Balakrishnan, M. Cacciola, L. Udpa, B.P. Rao, T. Jayakumar, B. Raj, Development NDTCE’09, 7th International Symposium on Non-Destructive Testing in Civil of image fusion methodology using discrete wavelet transform for eddy current Engineering, Nantes, June 30–July 3, 2009. images, NDT E Int. 51 (2012) 51–57. [14] P. Theodorakeas, C. Ibarra-Castanedo, S. Sfarra, N.P. Avdelidis, M. Koui, X. [33] C. Kohl, M. Krause, Ch. Maierhofer, J. Wöstmann, 2D-and 3D-visualisation Maldague, D. Paoletti, D. Ambrosini, NDT inspection of plastered mosaics by of NDT-data using data fusion technique, Mater Struct. 38 (2005) means of transient thermography and holographic interferometry, NDT E Int. 817–826. 47 (2012) 150–156. [34] Ch. Maierhofer, G. Zacher, C. Kohl, J. Wöstmann, Evaluation of radar and com- [15] N.P. Avdelidis, M. Koui, C. Ibarra-Castanedo, X. Maldague, Thermographic stud- plementary echo methods for NDT of concrete elements, J, Nondestruct. Eval. ies of plastered mosaics, Infrared Phys. Techn. 49 (2007) 254–256. 27 (2008) 47–57. [16] Ch. Maierhofer, S. Leipold, Radar investigation of masonry structures, NDT E [35] J. Cui, D.R. Huston, R. Arndt, F. Jalinoos, Data fusion techniques of multiple sen- Int. 34 (2001) 139–147. sors non-destructive evaluation of a concrete bridge deck, in: Proceedings of [17] S. Kruschwitz, E. Niederleithinger, C.H. Trela, J. Wöstmann, Complex resistivity NDE/NDT for Highway Bridges: Structural Materials Technology (SMT) Confer- and radar investigation of building material: first results of field scale measure- ence, New York City, NY, August 16–20, 2010. ments, Acta Geophys. 58 (2010) 96–113. [36] P. Cotič, E. Niederleithinger, G. Wilsch, Unsupervised clustering of laser- [18] P. Cotic, Z. Jaglicic, E. Niederleithinger, U. Effner, S. Kruschwitz, C. Trela, V. induced breakdown spectroscopy data from concrete. Cement Concrete Res Bosiljkov, Effect of moisture on the reliability of void detection in brickwork (submitted for publication). masonry using radar, ultrasonic and complex resistivity tomography, Mater [37] P. Cotič, Z. Jagliči ć, E. Niederleithinger, M. Stoppel, V. Bosiljkov, Image fusion Struct 46 (2013) 1723–1735, http://dx.doi.org/10.1617/s11527-012-0011-3. for improved detection of near-surface defects in NDT-CE using unsupervised [19] L. Binda, L. Zanzi, M. Lualdi, P. Condoleo, Complementarity of non-destructive clustering methods. J Nondestruct Eval (submitted for publication). techniques in the diagnosis of damaged historic structures, in: Proceedings of [38] D. Gustafson, W. Kessel, Fuzzy clustering with a fuzzy covariance matrix, in: 10th International Conference on Structural Faults and Repair, London, July Proceedings of the 1978 IEEE Conference on Decision and Control including the 1–3, Engineering Tech Press, London, 2003. 17th Symposium on Adaptive Processes, San Diego, CA, January 10–12, IEEE [20] J.D. Daniels (Ed.), Ground-penetrating Radar, second ed., The Institution of Elec- Control Systems Society, San Diego, CA, 1979, pp. 761–766. trical Engineers, London, 2004. [39] F. Höppner, F. Klawonn, R. Kruse, T. Runkler, Fuzzy Cluster Analysis: Methods [21] H. Jol (Ed.), Ground Penetrating Radar: Theory and Applications, first ed., Else- for Classification, Data Analysis and Image Recognition, John Wiley & Sons Ltd, vier Science, Amsterdam, Oxford, 2009. Chichester, 1999. [22] C.H. Maierhofer, M. Röllig, Application of active thermography to the detec- [40] P. Cotič, Z. Jagliči ć, V. Bosiljkov, E. Niederleithinger, GPR and IR thermography tion of safety relevant defects in civil engineering structures, in: Proceedings for near-surface defect detection in building structures, in: Proceedings of the of OPTO 2009 & IRS2 2009, SENSOR+TEST Conference, Nürnberg, May 26–28, 12th International Conference of the Slovenian Society for Non-Destructive 2009. Testing, “Application of Contemporary Non-Destructive Testing in Engineer- [23] X.P. Maldague, S. Marinetti, Pulse phase infrared thermography, J. Appl. Phys. ing”, Ljubljana, September 4–6, 2013. 79 (1996) 2694–2698. Please cite this article in press as: P. Cotič, et al., Validation of non-destructive characterization of the structure and seismic damage propagation of plaster and texture in multi-leaf stone masonry walls of cultural-artistic value, Journal of Cultural Heritage (2013), http://dx.doi.org/10.1016/j.culher.2013.11.004 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C1 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. APPENDIX C: ARTICLE 3 Effect of moisture on the reliability of void detection in brickwork masonry using radar, ultrasonic and complex resistivity tomog- raphy P. Cotič, Z. Jagličić, E. Niederleithinger, U. Effner, S. Kruschwitz, C. Trela & V. Bosiljkov Mater Stuct (2013) 46(10):1723–1735 DOI 10.1617/s11527-012-0011-3 C2 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 DOI 10.1617/s11527-012-0011-3 O R I G I N A L A R T I C L E Effect of moisture on the reliability of void detection in brickwork masonry using radar, ultrasonic and complex resistivity tomography Patricia Cotic • Zvonko Jaglicic • Ernst Niederleithinger • Ute Effner • Sabine Kruschwitz • Christiane Trela • Vlatko Bosiljkov Received: 8 June 2012 / Accepted: 21 December 2012 / Published online: 10 January 2013 Ó RILEM 2013 Abstract The influence of moisture on the reliability Keywords Radar, ultrasonic and complex of detection of larger voids in brickwork masonry was resistivity tomography Masonry Moisture investigated using three non-destructive techniques: Void detection radar, ultrasonic and complex resistivity (CR). Radar and ultrasonic travel time tomography, as well as CR tomography, were performed over a specific cross 1 Introduction section of a specimen containing a large void at a known position to determine the influence of different For some time, the non-destructive testing (NDT) levels of moisture content in the brickwork on the community has worked on the application of several wave velocities and the CR magnitude. We defined a techniques on brick and stone masonry structures. numerical estimator to quantitatively determine the Contrary to the original application of NDT techniques void detection efficiency from the images obtained on homogeneous materials (e.g. steel), its application to when exposing the specimen to moisture. The results masonry needs extensive calibration to ensure deeper showed radar to be the most reliable technique for void understanding of the effect of typical problems in detection in both dry and wet masonry, while CR masonry structures. Besides, complementary use of performed much better in detecting larger air voids in different non-destructive techniques may better resolve wet masonry. the structural inhomogeneity of masonry structures under different conditions, rather than a single method [1, 3, 5]. Typical problems related to masonry structures are the detection of multiple leaf walls, detachment of layers, the presence of layers of different materials, P. Cotic (&) Z. Jaglicic Institute of Mathematics, Physics and Mechanics, grouting faults, inclusions, larger voids and deteriora- Jadranska 19, 1000 Ljubljana, Slovenia tion due to moisture [1, 18]. While recent studies in e-mail: patricia.cotic@imfm.si infrared thermography show increasing success in detecting near-surface features in masonry [1, 19], P. Cotic Z. Jaglicic V. Bosiljkov Faculty of Civil and Geodetic Engineering, University radar and sonic techniques exploit the capability of of Ljubljana, Jamova 2, 1000 Ljubljana, Slovenia surveying the deeper inner structure [1, 2, 6, 18, 20]. Among the previously stated problems, the presence E. Niederleithinger U. Effner S. Kruschwitz C. Trela of larger voids and inclusions especially may indicate BAM Federal Institute for Materials Research and Testing, Unter den Eichen 87, 12205 Berlin, Germany lack of connection and decrease in resistance of load Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C3 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1724 Materials and Structures (2013) 46:1723–1735 bearing walls and pillars in masonry buildings. For additional typical problem in masonry buildings can spatial imaging of inclusions and larger voids, tomog- affect both the mechanical characteristics of masonry, raphy should be performed. Although it cannot be as well as the plaster and rendering and therefore has to regarded as a standard technique for the investigation be identified in combination with other structural of masonry structures due to its complexity, tomog- problems. The influence of moisture content on radar raphy may resolve some ambiguities encountered in and resistivity measurements has been discussed in echo configuration measurements [14]. Kruschwitz et al. [14] and Maierhofer and Colla [17]. Beside a carefully conducted tomographic mea- Within the scope of this work, firstly physical surement, the inversion procedure (imaging) also effects of material properties on the propagation of plays an important role. Different inversion techniques electromagnetic and acoustic waves and CR properties for acoustic and electromagnetic waves have been are briefly presented, followed by a field scale developed. In addition to conventional travel time experiment and tomographic results. Analysis of input tomography (TT) and amplitude tomography (AT), data for radar TT is discussed as well. Camplani et al. [7] as well as Valle et al. [27] explored the benefit of migration tomography (MIG) as well as diffraction tomography (DT). Both have been high- 2 Material properties lighted as capable of a remarkable improvement in resolution, especially in edge detection. However, in 2.1 Electromagnetic properties our study, only TT was performed for ultrasonic and radar data, as it proved to be sufficient to resolve larger The radar method is based on high frequency electro- voids in the central area of the measured specimen. Up magnetic wave propagation. The signal is affected by to now, TT on masonry has been explored by only a both the dielectric constant and the electrical conduc- few authors. Kong and By [13] applied both TT and tivity of the various materials within the structure. AT to stone pillars using frequency stepped radar, Since the relative magnetic permeability lr can be whereas Binda et al. [4] applied acoustic TT among taken as 1 for most building materials, the velocity of with attenuation tomography in frequency domain to radar wave propagation is given by historic masonry pillars to evaluate masonry quality c and homogeneity. A complementary survey of stone v ¼ pffiffiffiffi ð1Þ er masonry bridges with radar and sonic TT, as well as conductivity tomography was performed by Colla where c is the speed of light in a vacuum and er the et al. [8]. Schuller et al. [22] proposed acoustic TT on relative permittivity or dielectric constant of the masonry wall specimens for the identification of material. damage patterns following in-plane seismic loading The wave experiences attenuation due to absorption and the effectiveness of grouting injection. Flint et al. of its electromagnetic energy. The change in the wave [10] demonstrated the response of seismic, radar and amplitude can be expressed by the attenuation coef- electrical resistivity tomography to changes in the ficient, which is basically a function of the electro- condition of the masonry, such as the ingress of water. magnetic wave frequency and the material conductivity Wendrich et al. [28] applied radar and ultrasonic TT and can be expressed by Jol [12] for the detection of voids in brick masonry structures. " rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi !#1= There, a comparison between inversion results of pffiffiffiffiffi 2 1 r2 a le experimental data versus numerical modelling was ¼ x 1 þ 1 ð2Þ 2 x2e2 presented. The aim of this study was to present the capabilities where x = 2p f is the angular frequency and f is the of a complementary survey using radar and ultrasonic frequency of the electromagnetic wave; l is the mag- TT, as well as complex resistivity (CR) tomography, netic permeability of the material, defined by l = lrl0, on masonry for the detection of a larger void while where l0 is the free-space magnetic permeability; e is subjecting masonry to different humidity conditions. the permittivity of the material, defined by e = ere0, The effect of moisture on the reliability of detection where e0 is the free-space permittivity; and r is the was explored. It is well known that moisture as an electrical conductivity of the material. C4 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1725 The higher the frequency and conductivity, the influence of contact resistances. Taking into account higher is the attenuation, and consequently the lower is the polarization phenomenon of porous building the penetration depth. Moisture increases both the materials, the CR magnitude |q| as well as the CR dielectric constant and the conductivity of the material phase shift U can be measured in response to a and thus reduces the penetration depth. Moreover, due stimulus with an alternating current [14]. The CR q00 is to the increased moisture content the propagation given by velocity decreases (Eq. 1) and affects the radar q00 ¼ jqjeiU: ð6Þ resolution. For TT tomography the following expres- sion has been proposed for its calculation [23] It should be noted that the electrical resistivity of rffiffiffiffiffiffiffi materials is sensitive to their porosity, the morphology vx D 0 tomo ð3Þ of the pore space and the resistivity of the pore fluid fc [10]. Therefore, the moisture content should be deter- where x mined with respect to the amount and type of porosity. 0 is the half distance between transducers (as anomaly is in the middle) and fc is the central frequency of the radar antenna signal. As the propagation velocity decreases the radar resolution improves. However, the 3 Experimental procedure ray coverage, accuracy of travel time measurement etc. may further limit the resolution. For the investigation of larger voids in masonry buildings, the ‘‘Asterix I’’ brick masonry test specimen 2.2 Acoustic properties [28] was chosen. The specimen height is 1.39 m, length 0.99 m and width 0.74 m and it has inbuilt voids In solids, both longitudinal p-waves and transversal of different dimensions (simulated by omitting bricks s-waves are present. For the investigation of masonry, in certain layers during construction). Tomographic p-waves are of importance as they are the fastest and measurements were performed only at cross section thus easy to detect and pick in TT. The sound velocity A–A over the largest void of size 0.27 9 0.27 9 0.62 of p-waves is governed by the equation m3 (see Fig. 1). The specimen was constructed from sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi solid clay bricks with completely filled mortar joints. Eð1 lÞ Thus it enables the investigation of inbuilt anomalies v ¼ ð4Þ qð1 þ lÞð1 2lÞ and moisture under ideal conditions. To study the influence of the moisture level in the where E is the modulus of elasticity, q is the den- masonry on void detection by radar, ultrasonic and CR sity and l is the Poisson ratio of the material. In techniques, two sets of tomographic measurements engineering materials, these properties are related to were performed: first on a relatively dry specimen with material strength and moisture content [10]. an average moisture content of 3 mass% (hereinafter referred to as the ‘‘dry specimen’’); second on a wet 2.3 Complex resistivity properties specimen with an average moisture content of 8 mass% (hereinafter referred to as the ‘‘wet specimen’’) over The fundamental physical basis for electrical methods cross section A–A. The humidity level was determined is Ohm’s Law. By measuring the potential difference by the classic Darr test by boring and simultaneously U between adjacent electrodes for a fixed electric obtaining powder every 0.05 m along the depth of the current I, the apparent or average resistivity qa of the borehole. The average moisture content was deter- investigated inhomogeneous material can be deter- mined as the mean value from four boreholes that were mined using a configuration factor K, which depends made along the longer size of the specimen with larger on the electrode and specimen geometry depth to the void (top edge of cross section A–A in Fig. 1). After performing the first set of measurements U q ¼ on the dry specimen, the specimen was immersed in a a K : ð5Þ I steel water bath for 8 days. The water level reached the Four-electrode array, separating current injection from top of the 3rd layer of bricks. After immersion, all voltage measurement, is preferred to minimize the tomographic measurements were repeated. Altogether, Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C5 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1726 Materials and Structures (2013) 46:1723–1735 - Fig. 1 View 1 and 2 of the View 2 View 1 ‘‘Asterix I’’ masonry test 1 specimen with the position 16 and the size of voids, and 15 14 cross section A–A over the 13 largest void 12 11 10 Cross Section A-A 9 139 cm 8 1 4R 7 A A 6 4R 5 4 4R 3 27 2 1 1 27 99 cm 74 cm 2 2 we acquired six sets of NDT data: radar-, ultrasonic- possible combinations (Fig. 3). For each transmitter and CR-tomographies of the dry and the wet specimen. position, the electromagnetic signals were recorded It should be mentioned that the radar and ultrasonic for all moving receiver positions from all four sides. tomographic data of the dry state were collected much earlier [28] which results in different number of data 3.2 Ultrasonic equipment collected for the tomography on the dry and wet specimen. Taking into account that the drying and For ultrasonic tomographic measurements we used wetting process of the specimen could not be ensured Acsys SO202 longitudinal transducers with a fre- totally homogeneously the Darr test results cannot be quency of 25 kHz, an arbitrary function generator extrapolated to assess the moisture distribution over including an amplifier for signal generation and a pre- the whole analysed cross section. Therefore, the amplifier on the receiver side. Data were recorded by notations dry and wet should not be linked to the an industry standard analog/digital converter and actual moisture content of the specimen but rather to stored on a PC. For our purposes, a rectangular pulse the state before and after immersion. with a duration of 40 ls and a repetition frequency of 20 Hz was used. Vaseline served as the coupling 3.1 Radar equipment agent. The ultrasonic measurement for both dry and wet state of the specimen was performed in the same For radar tomographic measurements, the SIRveyor geometry as in radar tomography. SIR-20 radar system from Geophysical Survey Sys- tems Inc. (GSSI; Salem, USA) was used with bow tie 3.3 Complex resistivity equipment bistatic antennas with a central frequency of 1.5 GHz. The tomographic measurement of the dry specimen Among electrical methods, the CR method was chosen was performed in a point measurement mode using a due to its higher sensitivity compared to normal spacing of 13 cm between measurement points, resistivity methods. Using alternating currents of high whereas the tomography of the wet specimen was frequencies, materials of very low conductivity can be performed in a continuous mode using a calibrated detected. For CR tomography, two devices were used; survey wheel and a spacing of 5 cm between mea- a multichannel SIP256c device (Radic Research), as surement points. Altogether, around 500 data was well as an LGM 4-Point-Light (Lippmann) device collected on the dry and around 1680 on the wet with an ActEle extension (active electrodes for CR specimen. Ray paths that were collected by radar tomography) (Fig. 3). Both measure CR magnitude tomography on the dry specimen are displayed in and phase shift across the specimen relative to a shunt Fig. 2. In the radar TT measurements, the transmitter resistor with a high input impedance. In comparison and the receiver antenna, placed on different sides, with the LGM, the SIP 256c uses a higher applied were moved along the scanned section in a series of voltage (up to 230 V) and measurements can be C6 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1727 Fig. 2 Ray paths that were collected by radar tomography on the dry specimen Fig. 3 Radar tomographic measurement over cross section A–A and CR tomographic measurement over several cross sections using the LGM device acquired over a broader frequency range from 1 mHz each longer side and 15 electrodes on each shorter side to 1 kHz. of the specimen. The measurement was continued The following three electrode configurations were until resistivities were determined for all necessary applied: dipole–dipole, Wenner and Schlumberger combinations of electrode positions. ECG- (Electro- arrays [15]. For a complete four-sided measurement, cardiography) electrodes were positioned around the 68 electrodes were used with 19 electrodes placed on specimen separated by a distance of 0.05 m. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C7 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1728 Materials and Structures (2013) 46:1723–1735 4 Measurement results and the wet specimen, whereas in Fig. 7 travel time reciprocity results are displayed. As expected, a linear 4.1 Radar and ultrasonic results dependence between the travel times and the ray lengths can be observed from Fig. 6. However, For the tomographic inversion of radar and ultrasonic velocities belonging to longer distances are slightly travel times, initially the first arrival times were picked more scattered due to more refractions and scatterings from the recorded data. For radar waves this was done on mortar joints. Nonetheless, this phenomenon is less automatically with the Reflexw commercial software pronounced than stated in Schuller et al. [22], where [21]. However, as the ultrasonic data exhibited higher acoustic waves produced by an instrumental hammer scattering due to noise, an automatic picking algo- were used. rithm based on the Akaike information criterion (AIC) Moreover, R-squared values of the fitted lines was used, where the first arrival time was defined by reveal that both radar and ultrasonic data obtained on the minimum of the AIC function [16]. For our the more humid specimen exhibit higher scattering inversion routine, only transmitted waves needed to be compared to the data acquired on the dry specimen. taken into account and distinguished from refracted This is also proved by the normalized reciprocity and reflected waves. In the case of a large incident travel time residuals (normalized on the corresponding angle, refracted waves could arrive at the receiver measured travel time), on comparing the results of faster than transmitted waves. It should be noticed that the choice of the first arrival time, which influences the imaging, can be remarkably affected by various sources of distortions, which comprise variable ray coverage, systematically incorrect coordinate position, dislocations of anten- nas/sensors and the influence of high-frequency uncorrelated noise that decreases the signal to noise ratio [9, 24, 25]. Air calibration measurements and data quality checks of the picked travel times prior to the inversion process may identify possible errors and contribute to more reliable tomographic images [25]. In our study, the latter was taken into account but is not discussed here as it is not the subject of the paper. However, as the first arrival times contain valuable Fig. 4 Representation of the reciprocity check information about the physical effects of material properties on the propagation of electromagnetic and acoustic waves, analysis of the picked travel times can contribute to the interpretation of the tomographic velocity images. Thus, velocity and reciprocity check- ing of travel times after the removal of outliers is presented. For the velocity check, the velocity of each ray was calculated according to its direct distance and the travel time of the ray. The aim of the reciprocity check was to determine the travel time difference between opposite transmitter and receiver positions, as schematically displayed in Fig. 4. For the velocity check, only rays not passing the void area were used (Fig. 5), whereas for the reciprocity check, the whole data set was investigated. In Fig. 6 travel time versus ray length dependence Fig. 5 Part of the region of rays taken into account for the is displayed for radar and ultrasonic data on the dry velocity check C8 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1729 Fig. 6 Velocity check of radar travel times of the dry (a) and the wet specimen (b), and velocity check of ultrasonic travel times of the dry (c) and the wet specimen (d) Fig. 7a–c vs. Fig. 7d–f and Fig. 7g–i vs. Fig. 7j–l. residuals were in the majority of the order of 5 % Additionally, the normal distribution of residuals which is at short distances almost the limit of accuracy around zero (Fig. 7a, d, g, j) excludes the possibility of both methods. Small antenna dislocation, incorrect of apparent systematic errors (only a small deviation transducer positioning, unsmooth surface of the spec- from zero can be observed in Fig. 7g). Contrary to imen and poor coupling of ultrasonic transducers results obtained from other studies [25], our results do might all be possible reasons for small systematic not exhibit any obvious dependency of normalized travel time residuals. travel time residuals on incidence angle either for the For a qualitative comparison of the quality of the radar or the ultrasonic data. In case 7b, e higher radar and ultrasonic data, standard deviations (STDV) residuals can be observed on short distances, which of normalized reciprocity travel time residuals are means that the reliability of data collected on short ray displayed in Table 1. As already shown (Figs. 6, 7), the lengths is poor. However, as this data contains ultrasonic data exhibit higher scattering than the radar information from the corners of the specimen, the data, where according to the STDV up to 16 % higher data should not be discarded before the inversion but scattering was obtained on the dry and around 27 % on rather should the reconstructed velocities at the the wet specimen. Two explanations of this effect are corners be taken as less reliable. Moreover, for both possible. According to the first, the ultrasonic data radar and ultrasonic data the highest normalized could be more affected by ambient vibrations at the test Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C9 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1730 Materials and Structures (2013) 46:1723–1735 Fig. 7 Reciprocity check of radar travel times of the dry (a)–(c) and the wet specimen (d)–(f), and reciprocity check of ultrasonic travel times of the dry (g)–(i) and the wet specimen (j)–(l) site, resulting in a low signal to noise ratio and in acoustic properties of masonry materials compared consequently larger picking errors in comparison to the to the variation in electromagnetic properties. Accord- radar data. The second possibility could be that the high ing to Flint et al. [10], this gives rise to larger angles of scattering of ultrasonic data is due to stronger variation refraction at various interfaces within masonry (brick C10 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1731 Table 1 Standard deviation of normalized reciprocity travel From the recorded travel times, tomographic images time residuals for radar and ultrasonic data of the radar and ultrasonic velocity distribution were obtained for the investigated section (Fig. 8). Pre- STDV (%) sented images were obtained after the removal of Radar dry 1.48 outliers detected from the data quality check. For Radar wet 1.81 inversion, the commercial software Geo-Tom3D [26] US dry 1.72 was used. Due to the great difference in the material US wet 2.30 properties of the investigated section, curved ray inversion was used after a certain number of straight and mortar joints) and therefore the data appear more ray iterations. This took into account refractions from scattered. Besides, by comparing the differences in internal interfaces and ensured a convergence limit. As STDV between the wet and the dry specimen for the expected, higher velocity variations appear at the radar and ultrasonic data, the latter seem to be corners of the specimen due to poor data accuracy at significantly more sensitive to moisture. One explana- short distances. According to the conclusions from the tion for this could be that moisture produces an even data quality check, higher variations appear on sound higher variation in acoustic properties of various velocity images. Besides, ultrasonic velocity images masonry materials, while leaving the difference in reveal the inhomogeneous moisture distribution of the electromagnetic properties relatively unchanged. analysed cross section, that is especially noticeable However, it should be pointed out that also the from the red coloured area of the wet specimen. It has to different data density acquired on the dry and wet be mentioned that the reconstructed velocity values for specimen could contribute to variations in scattering. the void area cannot be assigned to air neither for radar Fig. 8 Velocity images for radar data of the dry (a) and the wet specimen (b), and ultrasonic data of the dry (c) and the wet specimen (d) where the rectangle marks the exact position of the void Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C11 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1732 Materials and Structures (2013) 46:1723–1735 Table 2 Tomographic results for radar and ultrasonic data of the dry and the wet specimen Average Void detection Level of surrounding efficiency (%) scattering (%) velocity Radar dry 0.13 m/ns 89 10 Radar wet 0.10 m/ns 81 11 US dry 3100 m/s 43 4.4 US wet 2800 m/s 37 15 The coefficient of variation (COV) of all average surrounding velocities ranges between 10 and 15 % nor for ultrasonic results. The main reason for this is the restriction of the inversion process. Unfortunately, this drawback limits the possibility of material character- Fig. 9 Parameters for computation of the efficiency of void detection E ization of the void but nonetheless, a clear identifica- void and the level of scattering Escatter in the border area outside the void tion of an anomaly can be given. Another weakness of the inversion process is the reconstructed shape of the void, where high material contrast between the different moisture levels, an estimator had to be chosen surrounding and the void limits the boundary detection. carefully. To assess the efficiency of void detection, we However, according to the tomographic images, the define two estimators: Evoid and Escatter. Besides deter- different moisture conditions of the specimen affect the mining the void detection efficiency directly with reliability of void detection far more than the inversion Evoid, the level of scattering (defined by Escatter) in a process. Note also the much lower resolution at the defined border area outside the void should also be taken void area between radar and ultrasonic. The reason for into account. Firstly, a threshold value was determined this is the difference in wave propagation between for both estimators. This means for radar and ultrasonic electromagnetic and sound waves, where sound waves tomographic images a velocity threshold value and for by avoiding travelling through the air void result in CR tomographic images a resistivity threshold value. much poorer ray coverage over the void. On trying different values, the mean value between the From Table 2 it is evident that the average velocity in average velocity/resistivity of the surrounding area and the area surrounding the void decreases as a function of the maximum velocity/resistivity of the void area was moisture increase for both radar and ultrasonic data. For found to be the most suitable one. In determining the radar a decrement of 25 % in average velocity can be efficiency of void detection for radar data this means that observed with increase in moisture from 3 to 8 mass%, the void was detected on a particular pixel if the velocity while for ultrasonic only a drop of 10 %. It can be pointed of the pixel was greater than or equal to the threshold out that moisture remarkably increases the dielectric value. The two estimators can be given by the following constant of masonry (already reported by Colla et al. [8]), equations: whereas it has a smaller effect on the elastic properties of Ain E ð7Þ masonry. This is, however, contrary to the influence on void ¼ Avoid scattering discussed previously and thus means that on Aout one hand moisture gives rise to a change in acoustic Escatter ¼ ð8Þ A properties of masonry materials, while on the other it has border little influence on the average acoustic properties that where Ain is the area belonging to the detected signal determine the velocity of propagation of sound waves. in the void area; Avoid the area of the void; Aout the area The limits of void detection can hardly be determined belonging to the detected signal in the border area; and effectively by visually examining velocity images Aborder is the area of a 0.10 m border around the void. (Fig. 8). Therefore, to give a reliable evaluation of the Parameters for the computation of the efficiency of change in efficiency of void detection with the three void detection, as well as the level of scattering in the techniques used, while subjecting masonry to two border area outside the void, are displayed in Fig. 9. In C12 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1733 an ideal case, Evoid should equal 1 and Escatter should The measured CR magnitudes exhibited relatively be 0. According to the efficiency values obtained for high error values, probably due to poor electrode void detection (Table 2), a drop of 9 % in radar data coupling and low currents in the measurement equip- can be observed due to the moisture increase, whereas ment. However, in the inversion routine only data with the ultrasonic data decrease by 14 %. This is, however, error values lower than 25 % were used. Only the remarkably lower than the influence of scattering tomographic resistivity images obtained on the dry and obtained from data quality checks and could therefore the wet specimen using the four-sided Wenner array are be mostly affected by the choice of threshold velocity discussed. Here (Fig. 10; Table 3) show that CR value. The latter is also dependent on the size of the magnitudes range between 400 and 700 Xm for the area detected. Therefore, a quantitative evaluation of dry specimen, whereas for the wet specimen values are the level of scattering outside the void has to be between 12 and 24 Xm. From the tomographic image of examined carefully in regard to the size of the detected the dry specimen (Fig. 10a) relatively low resistivity area. However, as expected, both radar and ultrasonic magnitude values were obtained in comparison with the data exhibit higher levels of scattering in wet masonry. values which had been obtained earlier on the same It should be mentioned that the propagation of the specimen [14]. Even lower resistivities have been electromagnetic signal in wet materials is affected by detected at the edges. This becomes apparent at the top an increase in depth resolution due to decreased signal edge of the dry specimen where resistivity values reach velocity, as well as by increased attenuation. In the values as high as the void itself. This clearly shows addition, for the ultrasonic signal the attenuation also the problem of poor electrode coupling which greatly increases due to stronger scattering. According to our affects the reconstructed resistivities at the upper part of results, we can conclude that the increased attenuation the specimen. However, it has only small effect on the governs the propagation of both electromagnetic and reconstructed resistivities of the void area and thus on acoustic waves in wet masonry. the estimation of the efficiency value of void detection. On observing both images in Fig. 10, the lower edge of 4.2 Complex resistivity results the detected void seems to be slightly shifted down- wards. This means that CR has a lower depth resolution For the inversion routine performed on CR data, a 2-D than the other two methods, which could be explained inversion commercial software program called Wall- by the predominantly surface current flow in the BERT [11], based on FEM (finite element methods), specimen. Another noticeable point is the very low was used. Extensive data analysis using various elec- resistivity value of the void in the wet specimen (around trode arrays and varying numbers of electrodes and 50 Xm). As already stated for the radar and ultrasonic model elements for the inversion routine revealed that in results, the main reason for this is probably the the majority of cases the Wenner array produced the best restriction of the inversion process on the reconstruc- tomographic images. This could be explained by the tion of highly contrasting material properties. Despite greater depth of penetration and the better signal to noise all the drawbacks of CR, a relatively high efficiency ratio of the Wenner array in comparison to the dipole– value (20 %) of void detection can be observed even for dipole array [15]. Comparing Geotest and SIP256c the dry specimen, while for the wet specimen a measurement devices, SIP256c turned out to be more tremendous increase of more than 100 % can be noticed sensitive and consequently produced results with very from Table 3. Contrary to expectations, the higher level high errors. As already stated in Kruschwitz et al. [14], of scattering refers to the wet specimen. The reason for all phase data were relatively noisy and could not provide any satisfactory reconstructions. Therefore, Table 3 Tomographic results for complex resistivity data of only CR magnitudes were investigated. the dry and the wet specimen Moreover, besides the quality of the measured Surrounding complex Void detection Level of data, the parameter setting of the inversion routine resistivity magnitude efficiency (%) scattering could remarkably affect the quality of the resulting (Xm) (%) tomographic images. Usually, the best tomographic Dry 400–700 20 5 reconstructions were obtained by using irregular Wet 12–24 43 25 finite elements and a blocky model [15]. Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. C13 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy 1734 Materials and Structures (2013) 46:1723–1735 humidity levels. Radar-, ultrasonic- and CR-tomogra- phies were applied to a dry and a wet specimen. Comparing the radar and ultrasonic results, the latter proved to be more scattered and the variations also more sensitive to moisture. This phenomenon could be explained by both the greater effect of ambient vibrations on the ultrasonic data and the stronger difference in the acoustic properties of masonry materials (brick, mortar) compared to the change in the electromagnetic properties. To achieve a reliable evaluation of the influence of moisture on void detection with the three techniques, two estimators based on a threshold value were defined. The first determines the void detection efficiency directly and the second determines the level of scattering in a defined border area outside the void. Comparing results from the wet versus dry specimen, a decrease in void detection efficiency was observed for radar and ultrasonic data, whereas in CR the efficiency improved remarkably due to the increased conductiv- ity and sensitivity of CR with increased moisture content. The level of scattering was found to be highly sensitive to the threshold value chosen and the depth resolution of the method. However, both proposed estimators proved to be adequately applicable for determining the void efficiency. All analysed methods proved capable of detecting the large air void though differing significantly in respect to their efficiency. It was shown that radar tomography is by far the most reliable technique for assessing voids in both dry and wet masonry. On the other hand, ultrasonic tomography requires longer Fig. 10 Complex resistivity tomographic images of the dry (a) and the wet specimen (b), where the rectangle marks the measuring times, and a reliable tomographic image exact position of the void can be achieved solely by carefully performed auto- matic picking. CR tomography turned out to be a this is the shift in the detected area which becomes even reliable technique even at low humidity levels, larger due to increased detectability in wet masonry. although it is limited by lower depth resolution than Moreover, an inhomogeneous moisture distribution in the other two methods and suffers from poor electrode the wet specimen could also give rise to scattering. Due coupling. The latter might also greatly affect the to the high resistivity variations between the dry and interpretation of the reconstructed resistivities. wet state time-lapse resistivity inversion could not Even though the study took into account only one, contribute to better interpretation of the effect of relatively large sized void and only two moisture changing moisture distribution. levels, these first results already show that each method can be regarded as effective in detection of larger voids under particular masonry conditions, and their com- 5 Conclusion plementary use can contribute to better interpretation of more complex inner structures. However, to explore Three NDT methods were tested for their void the potential of the analysed methods more precisely, detection capability in masonry exposed to different more humidity conditions should be taken into account C14 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Author's personal copy Materials and Structures (2013) 46:1723–1735 1735 and also different tomography configurations that 13. Kong F, By TL (1995) Radar tomography for non-destruc- would not restrict the application only to pillars or tive testing. In: Proceedings of the international symposium on NDT in civil engineering (NDT-CE), Berlin, pp 681–688 walls accessible from all four sides as in our study. 14. Kruschwitz S, Niederleithinger E, Trela C, Wo¨stmann J (2010) Complex resistivity and radar investigation of Acknowledgments The first author gratefully acknowledges building material: first results of field scale measurements. the financial support of the Slovenian Research Agency through Acta Geophys 58(1):96–113 grant 1000-10-310156 and the Slovene Human Resources and 15. Loke M (2011) Tutorial: 2-D and 3-D electrical imaging Scholarship Fund through grant 11012-13/2011. The authors surveys. p 148. http://www.geotomosoft.com. Accessed 7 wish to thank Jens Wo¨stmann and Maria Grohmann for their April 2011 help during the measurements. 16. Maeda N (1985) A method for reading and checking phase times in auto-processing system of seismic wave data. J Seismol Soc Jpn 38:365–379 17. Maierhofer C, Colla C (2000) Influence of moisture content References on the detection of structural features in building con- struction using impulse radar. In: Alampali S (ed) Structural 1. Binda L, Saisi A (2009) Application of NDTS to the diagnosis materials technology IV—an NDT conference. Technomic of historic structures. In: Proceedings of the 7th international Publishing Co., Lancaster, pp 343–349 symposium on NDT in civil engineering (NDTCE 2009), p 28 18. Maierhofer C, Leipold S (2001) Radar investigation of 2. Binda L, Lenzi G, Saisi A (1998) NDE of masonry struc- masonry structures. NDT&E Int 34(2):139–147 tures: use of radar tests for the characterisation of stone 19. Maierhofer C, Roellig M (2009) Active thermography for masonries. NDT&E Int 31(6):411–419 the characterization of surfaces and interfaces of historic 3. Binda L, Zanzi L, Lualdi M, Condoleo P (2003) Comple- masonry structures. In: Proceedings of the 7th international mentarity of non destructive techniques in the diagnosis of symposium on NDT in civil engineering (NDTCE 2009), p 6 damaged historic structures. In: Proceedings of the inter- 20. Sadri A (2003) Application of impact-echo technique in national conference on structural faults and repair, p 9 diagnoses and repair of stone masonry structures. NDT&E 4. Binda L, Saisi A, Zanzi L (2003) Sonic tomography and flat- Int 36(4):195–202 jack tests as complementary investigation procedures for the 21. Sandmeier K (2004) Reflexw, Version 3.5, instruction stone pillars of the temple of S. Nicolo l’Arena (Italy). manual for processing of seismic, acoustic or electromag- NDT&E Int 36(4):215–227 netic reflection, refraction and transmission data 5. Binda L, Bosiljkov V, Saisi A, Zanzi L (2006) Guidelines for 22. Schuller M, Berra M, Atkinson R, Binda L (1997) Acoustic the diagnostic investigation of historic buildings. In: Pro- tomography for evaluation of unreinforced masonry. Constr ceedings of the 7th international masonry conference: London Build Mater 11(3):199–204 2006 (Proceedings of the British Masonry Society, No. 10) 23. Schuster G (1996) Resolution limits for crosswell migration 6. Binda L, Lualdi M, Saisi A, Zanzi L (2011) Radar investi- and traveltime tomography. Geophys J Int 127(2):427–440 gation as a complementary tool for the diagnosis of historic 24. Tronicke J (2007) The influence of high frequency uncorre- masonry buildings. Int J Mater Struct Integr 5(1):1–25 lated noise on first-break arrival times and crosshole travel- 7. Camplani M, Cannas B, Carcangiu S, Concu G, Fanni A, time tomography. J Environ Eng Geophys 12(2):173–184 Montisci A, Mulas M (2008) Acoustic tomography for non 25. Tronicke J, Dietrich P, Appel E (2002) Quality improvement destructive testing of stone masonry. In: Oea G (ed) ICCSA of crosshole georadar tomography: pre- and post-inversion 2008, part II, LNCS 5073. Springer, Berlin, pp 596–605 data analysis strategies. Eur J Environ Eng Geophys 7:59–73 8. Colla C, Das P, McCann D, Forde M (1997) Sonic, elec- 26. Tweeton D (2001) GEOTOMCG, instruction manual for the tromagnetic and impulse radar investigation of stone three dimensional tomography program masonry bridges. NDT&E Int 30(4):249–254 27. Valle S, Zanzi L, Rocca F (1999) Radar tomography for 9. Dyer B, Worthington M (1988) Some sources of distortion in NDT: comparison of techniques. J Appl Geophys 41(2–3): tomographic velocity images. Geophys Prospect 36(3):209–222 259–269 10. Flint R, Jackson P, McCann D (1999) Geophysical imaging 28. Wendrich A, Trela C, Krause M, Maierhofer C, Effner U, inside masonry structures. NDT&E Int 32(8):469–479 Wo¨stmann J (2006) Location of voids in masonry structures 11. Gu¨nther T, Ru¨cker C (2008) WallBERT, Version 0.9. by using radar and ultrasonic traveltime tomography. In: Benutzerhandbuch Proceedings of the 9th European NDT conference (ECNDT 12. Jol H (ed) (2009) Ground penetrating radar: theory and 2006), Berlin, vol 11, p 11 applications. Elsevier Science, Amsterdam Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D1 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. APPENDIX D: ARTICLE 4 Unsupervised clustering of laser-induced breakdown spectroscopy data from concrete P. Cotič, E. Niederleithinger & G. Wilsch JAAS (submitted for publication) D2 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Unsupervised clustering of laser-induced breakdown spectroscopy data from concrete Patricia Cotič,∗a,b Ernst Niederleithinger,∗a and Gerd Wilscha Received Xth XXXXXXXXXX 20XX, Accepted Xth XXXXXXXXX 20XX First published on the web Xth XXXXXXXXXX 200X DOI: 10.1039/b000000x Detailed knowledge on the content and distribution of contaminants, such as chlorine, sulphur or sodium is of utmost importance for damage assessment and lifetime prediction of concrete structures. In most cases, only the content of these elements in the cement matrix is relevant. Laser-Induced Breakdown Spectroscopy (LIBS) is able to provide this information as it features a millimetre spatial resolution. However, to associate spectra with the cement matrix only, an automatic method to discriminate the results would be beneficial. To solve issues associated with most conventional separation criteria of LIBS data, which rely on a subjectively estimated threshold value, we evaluated the use of statistical clustering methods. Several algorithms proved to resolve the concrete structure accurately even for large heterogeneity. On the basis of clustering results, smooth depth profiles of sodium content in the cement were obtained. With a parameter optimization procedure, an unsupervised evaluation of LIBS data is proposed. 1 Introduction speed and high resolution method (up to 100 Hz measurement frequency and 0.5 mm dot pitch for the application presented Concrete pavements became popular again in the 1990ies, here). It has been shown to be able to detect all the important when great stretches of motorway had to be rebuilt and ex- elements in concrete, e.g. chlorine, sodium and sulphur, which tended after the German reunion. Over several years damage are responsible for particular degradation mechanisms. The such as cracking or spalling occurred at many points, mainly method is renowned by its multi-element ability, i.e. deliver- related to the alkali-silica reaction (ASR) and reinforcement ing information from several elements during a single mea- corrosion, induced by de-icing salts in the pore system of the surement. With a more complex detector (echelle spectrome- cement matrix. To prevent deterioration in the condition of ter), up to 40 spectral lines can be detected simultaneously.4 the concrete, high resolution depth profiles of externally pro- Moreover, the method requires only minor sample preparation vided sodium and chlorine in the cement part are required to and can be performed on-site. The current state of the art of assess the effects of the ASR and corrosion, respectively. An LIBS for concrete is described e.g. in Molkenthin3 and Wilsch overview of the ASR together with other degradation phenom- and Molkenthin.4 ena is given e.g. in Kropp and Hilsberg,1 Glasser et al.2 and Molkenthin.3 One of the main aims of spatially acquired LIBS data on Current methods for evaluation of the sodium and chlorine concrete is the derivation of accurate depth profiles represent- content often lack speed, resolution or relevance. For conven- ing the content of alkali elements in the cement part of con- tional chemical analysis, cores are taken from structures and crete at a resolution and efficiency which cannot be met by larger portions are ground, which needs intensive laboratory other chemical methods.5 For that, proper identification of work. High resolution alternatives such as X-ray diffractome- the measurement points representing cement should be made. try do not detect all the important elements or minerals, cover However, in most cases its capability is restricted by the het- only a small sample volume or require costly sample prepa- erogeneity of concrete.6 The amount, type and size of aggre- ration. Consequently, in practice only few concrete cores are gate (mostly locally available gravel or crushed rock, with a investigated and assumed to be representative for the whole size from the sub-millimetre to the centimetre range) influence structure. the acquired spectra. In most cases it is impossible to predict Laser-induced breakdown spectroscopy (LIBS) is a high whether a single laser shot hits cement, aggregate or some kind of mixture even when the concrete surface is scanned a BAM Federal Institute for Materials Research and Testing, Div. 8.2: Non- with a resolution of at least 1 mm. Therefore, criteria based on Destructive Damage Analysis, Unter den Eichen 87, 12205 Berlin, Germany. the content of one or several elements (e.g. hydrogen, oxygen, Tel.: (+386) 1 4267178; E-mail: Ernst.Niederleithinger@bam.de calcium) have to be used on hundreds of spectra to identify b Institute of Mathematics, Physics and Mechanics, Jadranska 19, 1000 Ljubl- those which contain (almost) only information from the ce- jana, Slovenia. Tel.: (+386) 1 4267178; E-mail: patricia.cotic@imfm.si 1–12 | 1 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D3 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. ment matrix. Three separation criteria are currently used: the f = 10 Hz, pulse width = 7 ns, energy = 800 mJ, irradiance oxygen/calcium ratio, the position in ternary diagrams (silica, = 8 × 1011 W/cm2) was focused on the sample surface pro- iron+aluminium, calcium+magnesia) and the hydrogen con- ducing plasma, which vaporized a few µg of the material from tent (hydrogen being absent in most aggregate minerals). All the surface. To obtain a small spot size, a beam expander in these criteria contain a subjectively estimated threshold value combination with a lens ( f = 50 mm) was used. The spot size and cannot be used in automatic evaluation schemes.6 on the sample surface was in the order of 0.3 mm2. The vol- Objective statistical clustering or grouping criteria, which ume around the laser plasma was flooded by helium to remove can be used without detailed initial knowledge of the sample dust and in particular to enhance the chlorine signals in the and human supervision, have been applied to LIBS data be- spectra. Upon cooling of the plasma, the light of wavelengths fore. Methods from multivariate statistics such as principal characteristic of the chemical elements in the plasma (and component analysis (PCA), artificial neural networks or hi- thus in the specimen) was emitted. This light was guided by erarchical and other clustering algorithms have been used to an optical fibre to the spectrometer (Shamrock 303i, Czerny- identify groups or outliers in large numbers of samples.7–12 Turner, f = 303 mm, aperture = F/4, resolution of 0.1 nm with To our knowledge they have never been applied to the LIBS 1200 l/mm@800nm grating). For light detection, a charge- spatial data of a particular sample, i.e. to spectra coming from coupled device (CCD) camera (ANDOR IDUS 420A BR-DD, different positions on the same sample. 1024 × 256 pixels, 16 bit) with a quantum efficiency of 90 % In our study, clustering methods were applied for pattern up to 900 nm was used. The exposure time of the CCD was set recognition (here: cement identification) of spatial LIBS data to 50 ms to grab the whole plasma radiation of a single pulse. on concrete samples subjected to ASR. Clustering methods A time resolved measurement was not possible with that type were chosen since they can work fully unsupervised, which of CCD. For each point, the sum of three successive plasma means they do not need any prior knowledge about the sta- events was recorded. To map a particular area of the sample, a tistical distribution of data as conventional statistical methods translation stage was used to move the sample in the plane per- do. Moreover, they do not require a good training data set as pendicular to the laser beam. The system was fully automated most other objective statistical grouping criteria. This is es- and controlled by LabView-based software. pecially promising for LIBS data, where knowledge about the sample composition is usually limited. 2.2 Specimen 2 Material and methods We investigated one laboratory concrete test specimen, sub- jected to an ASR performance test, where one side of the spec- 2.1 LIBS acquisition system imen was immersed in a 3.6 % NaCl solution under cyclic cli- mate storage at 40 ◦C. The sample was split in the cross sec- The data presented here was acquired by the LIBS system used tion by dry sawing in order to avoid effects of drilling and to at BAM for the investigation of building materials (Fig. 1). obtain an even surface. A photograph of the concrete core in- A Nd:YAG-Laser (INNOLAS Spitlight 600, λ = 1064 nm, vestigated is shown in Fig. 2 (the top edge of the specimen Fig. 2 Concrete specimen with coordinate system. Black lines mark the investigated part, grey lines on aggregate are a result of the LIBS measurement. The top edge of the specimen was subjected to alkalis Fig. 1 Experimental configuration of the LIBS system with penetration in the y-direction 2 | 1–12 D4 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. was subjected to alkalis with penetration in the y-direction). spectra) may contain information from more than one compo- The sample is apparently undamaged and contains greywacke nent of the concrete. aggregates of different colour and size (ranging from the sub- millimetre size to 5 cm). The cement type CEM I was utilized 2.3.1 Fuzzy probabilistic clustering. The best known fuzzy probabilistic algorithm is the fuzzy c-means (FCM) al- with a water/cement ratio of 0.45. An area of 94.5 × 95 mm2 gorithm of Bezdek.15 It can be seen as the fuzzy extension of the sample was investigated using the LIBS setup described of the k-means algorithm, where u above with a spatial resolution of 0.5 mm by 1 mm, producing i j describes the member- ship of observation 190 x ×96 spectra. The resolution was high enough to map most j in the i-th cluster in terms of probability (thus probabilistic clustering). The fuzzing exponent or fuzzi- of the aggregates. The data acquisition time for a complete fier m, introduced in (2), is a positive real number and is rec- scan of the sample was 90 min. ommended to range from 1.2 to 2 (in our calculation m was fixed to 2, being the value most recommended by other au- 2.3 Clustering methods thors). To optimize J, uij and vi should be updated in each Clustering or cluster analysis refers to the task of grouping iteration by a set of data (observations) into subsets (clusters, groups or 1 classes) in such a way that the members of a cluster are as uij = c , (3a) similar as possible, whereas members belonging to different ∑(dij/dkj)2/(m−1) clusters are as dissimilar as possible.13 The term similarity k=1 refers to differences in observational properties. A common N measure of similarity between two observations x ∑ um 1, x2 is the i j · xj Euclidian distance d, which can be for M variables written as v j=1 i = , (3b) N M ∑ umij d2(x1,x2) = ∑ (x1i −x2i)2 = kx1 −x2k2. (1) j=1 i=1 with dij = kxj − vik, 1 ≤ i ≤ c, 1 ≤ j ≤ N. Starting with a random grouping of an unlabelled data set Due to the fact that the Euclidian distance is used, the FCM with N observations X = {x1,x2,...,xN} into c clusters, the method is based on the assumption of spherical clusters and is observations are moved in each step to the cluster whose cen- sensitive to outliers and disturbances. To account for the first, tre is the nearest to them until a convergence limit is reached. the Gustafson-Kessel (GK) algorithm16 can be used, which To partition the data set by iteratively assigning labels uij to uses individually adapted distance functions determined by vectors in X by partition matrix U = [uij]c×N, many clustering the covariance matrices of each cluster to account for their dif- models have been proposed. The target of all of them is to ferent shape (e.g. ellipsoidal) and size. However, the GK algo- minimize an objective function J, which is the sum of the dis- rithm cannot be used in a fully automatic mode as some pre- tances of N observations from their respective cluster centre liminary information about the cluster size needs to be given. v: Kruse et al.17 recommended using the final FCM partition for c N J the initialization of GK partitioning. This was also used in our m(U, V; X) = ∑ ∑ umijkxj − vik2. (2) i=1 j=1 version of the algorithm. As the result may be dependent on the initial random partition, The sensitivity to outliers can be overcome by using noise clustering should be repeated several times. The minimum clustering,18 where an additional noise cluster is introduced. number of clusters is 2 and all have to be non-empty. For the noise centre vc, the distance of observation xj to vc is With respect to the partition matrix, clustering algorithms dcj = δ for each observation. Therefore, the distances in (2) can be classified as hard (or crisp), fuzzy probabilistic or fuzzy and (3a) should be replaced according to possibilistic. For hard and fuzzy probabilistic algorithms, the d memberships u i j = kxj − vik for 1 ≤ j ≤ N, 1 ≤ i ≤ c − 1, (4a) i j of each observation have to sum to 1 over all clusters, whereas possibilistic clustering drops this con- dij = δ for 1 ≤ j ≤ N, i = c, straint. Moreover, for hard clustering (in this study we refer ∑c−1 i=1 ∑Nj=1 d2ij to the k-means algorithm14), u δ2 = λ · . (4b) i j possess values of only 0 or 1 N(c − 1) (uij ∈ {0,1}). On the other hand, fuzzy clustering is defined as soft partitioning, allowing observations to belong to more For the calculation of δ , parameter λ was estimated by using than one cluster with a calculated probability, i.e. uij may pos- the standard deviation of the inter-point distances.18 However, sess any number between 0 and 1 (uij ∈ [0,1]). This concept to achieve the best partition, we further multiplied this param- is promising for our application, since the observations (LIBS eter by factor L, using 1 as the default value (the values taken 1–12 | 3 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D5 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. into account for this parameter will be discussed within the re- to a = 1, b = 1, m = 2 and η = 2. While optimizing these pa- sults). Similarly as for FCM, noise clustering is applicable to rameters, m remained fixed to its default value since this was the GK or the k-means algorithms. the most suitable choice made by the authors of the FCM. Besides using the mixed c-means approach, we employed 2.3.2 Fuzzy possibilistic clustering. Apart from noise a PCM variant using a repulsion term proposed by Timm et clustering described above, fuzzy possibilistic clustering al- al.22 The objective function of PCM with repulsion is given gorithms are able to deal with noisy data. Since they drop by the following equation the constraint that the memberships of each observation have to sum to 1 over all clusters, memberships should rather be c N c N interpreted as the typicality of a data point to a specific clus- Jm(T,V;X,γ,ω) = ∑ ∑ tmij ·d2ij + ∑ γi ∑ (1−tij)m + ter. The original possibilistic c-means (PCM) algorithm by i=1 j=1 i=1 j=1 (8) c c Krishnapuram and Keller19 helps to identify outliers, but at ω · ∑ ∑ 1/d2(vi,vk) , the same time is also sensitive to initialization and very often i=1 k=1,k6=i generates coincident clusters. Therefore, we referred to two modifications of the PCM. where ω is a user-defined parameter. Here, the necessary con- Following Pal et al.20,21 we used a mixed c-means approach ditions are given by called the possibilistic-fuzzy c-means (PFCM) algorithm. The 1 model is based on generating both membership and typicality tij = , (9a) 1 1 /(m−1) values, where the objective function should be minimized ac- 1 + γ ·d2 i i j cording to N c ∑ tij · xj − ω · ∑ vk/d2(vi,vk) c N j=1 k=1,k6=i J v m . (9b) ,η (U, T, V; X, γ ) = ∑ ∑ a um i j + btη i j · d2ij+ i = N c i=1 j=1 (5) ∑ tij −ω · ∑ 1/d2(vi,vk) c N j k ∑ =1 =1,k6=i γi ∑ (1−tij)η , i As for the PFCM, the final FCM partition was used for initial- =1 j=1 ization. where a > 0, b > 0, m > 1 and η > 1 are user-defined param- 2.3.3 Selection of the number of clusters. All clustering eters and γi should be calculated by19 algorithms described above require the number of clusters c to N be determined in advance. The selection of the optimal cluster ∑ umij ·d2ij number, which yields the best partition is a user-defined pa- γ j=1 rameter. However, to perform partitioning without subjective i = K · , K > 0. (6) N ∑ um judgement, one can use validity measures to assess the suit- i j j=1 ability of a partition for a different c, taking into account both compactness and separation between clusters.15 Compactness To minimize Jm,η, the necessary conditions for memberships should be thought of as a measure of data scattering within uij, typicalities tij and cluster centres are a cluster and therefore the optimum c should minimize the compactness, but maximize the separation between clusters.23 1 uij = Several quality validity indices have been introduced (see e.g. c , (7a) ∑ d 2/(m−1) Abonyi and Feil24). In our case, we used three of them that i j/dk j k=1 account for both compactness and separation of clusters, as 1 well as fuzziness: tij = , (7b) 1/(η−1) 1 + b 1. The partition index SC(c), describing the ratio of the γ · d2 i i j sum of compactness and separation of the clusters, where N ∑ aum normalization is done by the fuzzy cardinality of each i j + btη i j · xj cluster Ni:23 v j=1 i = N . (7c) ∑ N aumij + btηij ∑ umijkxj −vik2 j=1 c SC j=1 (c) = ∑ c . (10) In our algorithm we used the final FCM partition for initial- i=1 Ni ∑ kvi −vkk2 ization as for the GK. The default parameter values were set k=1 4 | 1–12 D6 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 2. The separation index S(c), which uses the same nomi- nator as SC, but the minimal separation between clusters as the denominator:23 c N ∑ ∑ umijkxj −vik2 S i=1 j=1 (c) = N ·mini,k,i6=kkvi−vkk2. (11) 3. Xie and Beni’s index XB(c), which uses the same nom- inator as the SC and S indices, but the minimal distance of data points to cluster centres as the denominator:25 c N ∑ ∑ umijkxj −vik2 XB i=1 j=1 (c) = N ·mini,jkxj −vik2. (12) Fig. 3 A comparison between two typical spectra measured on As fuzzy cardinality normalization of the SC index makes coarse aggregate and the part with high fraction of cement for the it insensitive to cluster size, the S and XB indices were taken range from 808 nm to 862 nm with the identification of the spectral into account when searching for the optimal number of clus- lines ters, whereas the SC index was used to estimate the quality of different partitions for a specific number of clusters. A good partition should minimize all three validity indices. an enlarged part of the investigated sample, it can be seen that the cement part cannot be properly identified using Ca or O spectral intensities alone. Fig. 4 and 5 further show that even 3 Results and discussion larger aggregates which appear to be relatively homogeneous in Fig. 2, differ in their element content (see e.g. the Ca spec- 3.1 LIBS data tral line in Fig. 4a). In addition, some aggregates contain a Selected spectral lines of Ca (849.80 nm), O (844.64 nm), Na very significant content of Na (Fig. 4c), which would influ- (818.32 nm), Cl (837.59 nm), C (833.52 nm) and Ti (846.85 ence conventional chemical analysis. Therefore, for the eval- nm) were used for the investigation of the sample. A com- uation of clustering results, the reference for cement identifi- parison between two typical spectra measured on a coarse ag- cation was determined by a careful evaluation of the surface gregate (with an area large enough that any effect of mate- using the criterion based on the O/Ca ratio explained in the rial other than the aggregate itself can be excluded) and the Introduction, as well as by image analysis. The O/Ca criterion part with a high fraction of cement is shown in Fig. 3. The was assumed to be significant enough for the resolution used. spectrum assigned mainly to the cement part of the specimen The reference displayed in Fig. 6 refers to the subjectively es- shows a very high intensity of the Ca spectral line, whereas timated optimal threshold value of the O/Ca ratio. the same line is much lower in the case of the coarse aggre- gate. In addition, the spectrum measured on the cement part 3.2 Clustering results has a higher intensity of the Cl and Na spectral lines due to the ingress of the NaCl solution (note that the Cl spectral line The spatial intensities of all spectral lines shown in Fig. 4 were is very weak). Although the two spectra prove that a clear used as input data for classification with clustering methods. distinction between the aggregate and cement part is possible, As a prerequisite for any clustering technique, the optimal most often the ablated material is a mixture of both aggregate number of clusters was determined using the XB and S in- and cementious material, and thus more sophisticated separa- dices. The k-means, FCM and GK algorithms were applied tion criteria have to be applied. with cluster numbers between 2 and 10 and fifteen runs were The intensities of all spectral lines taken into account are used to avoid the influence of initialization (the parameter- presented for the whole investigated area of the sample in dependent algorithms were not applied to limit the computa- Fig. 4. The intensities were normalized according to the global tional effort). From the XB index, which proved more reliable minimum of the background. The figures show that hetero- than the S index, the optimal number of clusters was chosen geneity of the concrete sample is large. There is a large pro- to be five, as all methods taken into account exhibited a local portion of millimetre- or submilimetre-sized aggregates dif- minimum at this position. fering in their element content. From Fig. 5, which represents The resulting partitions of all the clustering methods dis- 1–12 | 5 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D7 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Fig. 4 LIBS normalized spatial intensities for (a) Ca 849.80 nm, (b) O 844.64 nm, (c) Na 818.32 nm, (d) Cl 837.59 nm, (e) C 833.52 nm and (f) Ti 846.85 nm spectral lines Fig. 5 (a) The sample photo of an area of 20 × 20 mm2; LIBS normalized spatial intensities for (b) Ca 849.80 nm and (c) O 844.64 nm spectral lines; classification result using (d) O/Ca conventional criterion and (e) the fuzzy c-means (FCM) algorithm for 5 clusters 6 | 1–12 D8 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Fig. 7 Clustering partitions for (a) k-means, (b) fuzzy c-means (FCM), (c) Gustafson-Kessel GK, (d) noise clustering FCM (NCFCM), (e) probabilistic c-means (PCM) with repulsion and (f) possibilistic FCM (PFCM) for 5 clusters. For the values of parameters used, refer to Table 1 cussed are visualized in Fig. 7 for five clusters. For fuzzy partitions, the cluster colour is displayed according to the hard association of the most probable cluster (the cluster with the highest membership degree), whereas the factor of trans- parency takes into account the membership degree (e.g. uij = 1: opaque, uij = 0.5 : 50% transparency). Also the mixed PFCM partition is displayed with respect to the memberships. It can be seen that the algorithms performed differently and as- signed both aggregates and the cement part to different clus- ters. It should be noted that the cluster labels cannot be di- rectly compared between different algorithms, since they de- pend on random initialization. For the noise clustering parti- tion (here noise clustering FCM NCFCM) it can be seen that cluster 6 collects noisy data, which fits data points with very high spectral intensities for Na (compare Fig. 7d to Fig. 4c). Further, the coincident problem of the PCM with repulsion stands out, where only three clusters are present (Fig. 7e). It Fig. 6 Result of cement identification (white) by conventional O/Ca has to be stressed that for the NCFCM, PCM with repulsion criterion and image analysis (used as a reference for the clustering and PFCM algorithms, the partitions presented refer to the op- results) timal user-defined parameter combination. An explanation of the choice of optimal parameters is given below. 1–12 | 7 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D9 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. For all partitions, the cluster identified as cement is repre- Table 1 Results of false negatives (FN) and false positives (FP) for sented by the blue colour. In the case of the FCM algorithm, it different clustering methods with 5 clusters can be seen that the detected cement area resembles the white area from the reference image closely (Fig. 5). However, to Clustering method FN (%) FP (%) quantitatively evaluate the efficiency of all clustering methods k-means 0.05 27.90 for cement identification, we used an objective measure based fuzzy c-means (FCM) 0.49 12.90 on the number of false negatives (FN) and false positives (FP) Gustafson-Kessel (GK) 14.50 24.60 noise clustering FCM (NCFCM, L = 1) 2.59 7.81 in relation to the reference image in Fig. 6. Here, FN quanti- probabilistic c-means (PCM) with rep. 0.62 35.40 fies the number of cement cells identified as aggregate by clus- (ω = 5) tering, whereas FP quantifies the number of aggregate cells possibilistic FCM (PFCM, a = 1.4, 6.02 2.77 identified as cement by clustering. The values of FN and FP b = 1.8, m = 2, η = 1.7) are expressed as the percentage of the total number of cement cells (where a cell represents an LIBS measurement point on the concrete surface). For fuzzy partitions, both values were clusters as the original PCM algorithm and was therefore not evaluated according to the hard association of the most prob- included in further studies as well as the k-means due to its able cluster. For the mixed PFCM, where both memberships initialization problems. and typicalities are involved, only memberships were taken into account for the calculation of FN and FP values since they are directly related to memberships from other algorithms and 3.3 Application of clustering results to the evaluation of thus enable quantitative comparison. sodium content Table 1 summarizes the results for FN and FP (as the mean Reliable, detailed depth profiles of Na in the cement part of value of fifteen runs) for the clustering partitions displayed samples subjected to ASR or other degradation phenomena in Fig. 7. In the interpretation of results, more weight was is the main outcome from an LIBS investigation. The basis put on the FP value since it directly reflects the overestimated for efficient evaluation is an appropriate cement identification, cement content, being of utmost importance for further cal- which has been demonstrated in the previous section. For the culation of Na content in the cement matrix. The results for sample discussed here, a dens grid (0.5 mm by 1 mm) was FP show that the GK performs worse than the FCM. No im- used in order to map most of the aggregates in the concrete provement could be achieved even by varying the number of and thus to provide a credible reference based on the O/Ca clusters. The GK generally delivered good partitions for LIBS criterion for a quantitative evaluation of clustering partitions. data on other concrete samples. The reason for this result In practice, however, the concrete cores taken from structures with the specimen investigated could be the relatively hetero- would be measured with a resolution of 1 mm by 2 mm for a geneous cement matrix with respect to the aggregates. The time efficient investigation. In the case of very heterogeneous hard k-means algorithm proved to be very susceptible to ini- concrete, the reduced resolution may greatly affect the classi- tialization and therefore could not provide stable and reliable fication outcome from the O/Ca criterion. To demonstrate this, results. Among the noise cluster variants, we only present the we present depth profiles of the sodium content in the cement results for the NCFCM algorithm as the conventional FCM part. performed best among the k-means and GK algorithms. Here, The depth profiles of Na were obtained by averaging the the user-defined parameter L = 1 proved to be the optimal one spectral intensity along the horizontal axis (x-direction), tak- and reduced the FP value by 40 % in comparison to the FCM. ing into account only the cement data, identified by the classi- However, the lowest value for FP, i.e. 2.77 %, was achieved by fication method (different clustering algorithms and the O/Ca using the PFCM algorithm with the parameters set to a = 1.4, criterion) for a spatial resolution of 1 mm by 2 mm. However, b = 1.8, m = 2 and η = 1.7. It should be noted, however, as a reference profile, the original resolution of 0.5 mm by that low FP values for the PFCM (5.55 %) could already be 1 mm and the classification shown in Fig. 6 were used. From achieved with the default parameters a = 1, b = 1, m = 2, Fig. 8 it can be seen that the reduced resolution affects the η = 2. Therefore, a slightly higher weight on the typicality O/Ca criterion to a large extent, which consequently yields a values (with b > 1 and η < 2) only further improved our re- very poor estimate of the Na content. Clustering methods on sults, which is in accordance with the predictions made by the the other hand derive smoother profiles, closer to the refer- authors of the method.21 In contrast to noise clustering and ence. However, small variations between different methods the PFCM, which already delivered good results for the de- can be observed, which corresponds to the conclusions drawn fault parameters, the PCM with repulsion was found to be a from the sensitivity evaluation of the methods. In particular, highly parameter-dependent algorithm. Moreover, as already the GK algorithm exhibits some deviations, but in general has observed (Fig. 7e) it still suffered from producing coincident still a similar profile to the reference one, whereas the differ- 8 | 1–12 D10 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Fig. 8 Depth profiles of Na in the cement matrix of the sample Fig. 9 Depth profiles of Na in the cement matrix for noise clustering determined using results from various clustering techniques and the FCM (NCFCM) before and after limiting the membership degree to conventional O/Ca criterion, compared to the reference. For the higher than 0.6, compared to the reference. For the parameter L, values of parameters used, refer to Table 1. The top of the sample is L = 1 was used. The top of the sample is left (y = 0) left (y = 0) to account for the diversity of concrete specimens, the prob- ence between the other fuzzy algorithms is not as apparent as lem of parameter optimization should be solved. One pos- in the sensitivity evaluation. All methods show a Na content sibility could be to use a cluster validity function evaluation which is in general higher near the surface exposed to NaCl algorithm. An example of this has been given in Wachs et solution and gradually decreases with depth. This is probably al.26 where the authors performed parameter optimization of due to de-icing salts. However, very close to the surface ex- the PCM with repulsion by the use of the XB index. Bensaid posed to alkalis (0–10 mm) the content is lower due to washout et al.23 suggested using the SC index for comparing the ef- by precipitation. This effect is commonly recognised, but not fectiveness of various clustering methods at the same number detectable by standard chemical analysis due to the low spatial of clusters. We followed both methods. For the PFCM, the resolution of the technique. SC and XB indices were extended to take into account both The depth profiles displayed in Fig. 8 do not restrict the membership and typicality as proposed in Zhang et al.27 The incorporation of data points according to their degree of mem- outcome was validated by the sensitivity calculation. bership to the cement cluster. By assuming that this could The noise clustering parameter L was varied over the range have an effect on the results, we additionally took into account 0.1:0.1:8, whereas the PFCM parameters a, b and η were only cells with a membership degree higher than 0.6. Inter- varied over the ranges a = 1 : 0.2 : 2, b = 1 : 0.2 : 2 and estingly, the new results did not bring any improvement, but η = 1 : 0.2 : 2, with m fixed to 2. In comparison to the SC rather produced a strong deviation of the depth profiles from index, the XB resulted in clear local minima which were in the reference profile (see Fig. 9 for NCFCM). Small improve- agreement with the sensitivity values of FN and FP. We there- ments could therefore probably only be achieved by suitably fore refer our results only to the XB index. Fig. 10 shows the adapting the limit with respect to the cluster number used. values obtained for the XB index and FN and FP while vary- ing parameter L in the region up to 2.5. The first local minima 3.4 Automation of the clustering algorithm of XB appear at L = 0.3, L = 1.0,.... According to the low- est FP value, L = 0.3 should be taken as the optimal parameter. A careful sensitivity evaluation showed that both the NCFCM However, Fig. 11 reveals that for L = 0.3, the Na profile would and PFCM algorithms yielded remarkably better partition than deviate greatly from the reference profile. This appeared to be the other clustering algorithms analysed. Moreover, contrary due to the fact that for L = 0.3 too many data points were clas- to the PCM with repulsion algorithm, both exhibited relatively sified as noise, corrupting the behaviour of the algorithm. Due robust performance and could therefore be suggested for spa- to the high noise level many cement cells were recognized as tial fuzzy clustering of LIBS data from heterogeneous con- noise (Fig. 12), which consequently resulted in high FN val- crete samples. ues. Thus, we suggest that the cluster validity function evalu- However, to fully automate the clustering algorithm in order ation algorithm should be restricted to minima of XB close to 1–12 | 9 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D11 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Fig. 12 Clustering partition for noise clustering FCM (NCFM) with the value of parameter L = 0.3 Fig. 10 Results of XB index, false negatives (FN) and false the default value. positives (FP) for the optimization of parameter L of noise clustering For the PFCM, Table 2 summarizes the results for the FCM (NCFCM) XB index and FN and FP for three combinations of param- eters that yielded the lowest XB values. Similarly as for the NCFCM, the parameter combination a = 1.4, b = 1.8, m = 2 and η = 1.7 with the lowest XB value did not give the lowest FP. However, Fig. 13 reveals that all three parameter combi- Fig. 13 Depth profiles of Na in the cement matrix for possibilistic FCM (PFCM) with the combination of parameter values set to a = 1.4, b = 1.8, m = 2, η = 1.7; a = 1.4, b = 2.0, m = 2, η = 1.8 Fig. 11 Depth profiles of Na in the cement matrix for noise and a = 1.0, b = 1.8, m = 2, η = 1.5, compared to the reference. clustering FCM (NCFCM) with the value of parameter L = 0.3 and The top of the sample is left (y = 0). L = 1, compared to the reference. The top of the sample is left (y = 0) nations would produce very similar depth profiles of the Na content. This suggests that the XB index is a robust validity measure and could be used for the parameter optimization al- gorithm. 10 | 1–12 D12 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. Table 2 Results of the XB index, false negatives (FN) and false construction and will bring LIBS into a productive environ- positives (FP) for possibilistic-fuzzy c-means (PFCM) with different ment. combinations of parameters. Parameter combination XB FN (%) FP (%) Acknowledgements a = 1.4, b = 1.8, m = 2, η = 1.7 64.1 6.02 2.77 a = 1.4, b = 2.0, m = 2, η = 1.8 71.6 5.24 3.45 a = 1.0, b = 1.8, m = 2, η = 1.5 78.1 11.0 1.09 This study was based on preliminary work carried out in our group by Julien Grandemenge. The sample discussed in this text was provided by our colleagues in Division 7.1 of BAM. 4 Conclusions The first author gratefully acknowledges the financial support of the Slovenian Research Agency through grant 1000-10- The use of clustering methods for cement identification of spa- 310156 and the Slovene Human Resources and Scholarships tial LIBS data from heterogeneous concrete samples subjected Fund through grant 11012-13/2012. to ASR was studied. The use of hard vs. fuzzy clustering, as well as fuzzy probabilistic, noise and fuzzy possibilistic clus- tering was explored. The separation of LIBS spectra domi- References nated by cement or aggregate content by different clustering 1 Performance Criteria for Concrete Durability, RILEM Report No. 12, ed. algorithms worked well in our application example. The sen- J. Kropp and H. K. Hilsdorf, E & FN Spon, London, 1995. sitivity evaluation based on the calculation of false negatives 2 F. P. Glasser, J. Marchand and E. Samson, Cem. Concr. Res, 2008, 38, and false positives showed that the noise clustering and the 226–246. mixed possibilistic-fuzzy c-means algorithm in particular pro- 3 A. Molkenthin, Ph.D. Thesis, BAM, Berlin, 2009. 4 G. Wilsch and A. Molkenthin, Non-Destructive Evaluation of Reinforced vided the optimal partitions. This was justified by the sodium Concrete Structures, Woodhead Publishing Limited, Cambridge, 2007. depth profiles, which gave an almost ideal fit to the reference 5 F. Weritz, A. Taffe, D. Schaurich and G. Wilsch, Constr. Build. Mater., profile and outperformed the standard classification based on 2009, 23, 275–283. the O/Ca criterion to a large extent. 6 F. Weritz, D. Schaurich, A. Taffe and G. Wilsch, Anal. Bioanal. Chem., To account for the problem of parameter optimization of 2006, 385, 248–255. 7 C. Bohling, D. Scheel, K. Hohmann, W. Schade, M. Reuter and G. Holl, noise and PFCM algorithms, we proposed a scheme based Appl. Opt., 2006, 45, 3817–3825. on the XB index. For the PFCM, the global minimum of XB 8 N. Labbé, I. M. Swamidoss, N. André, M. Z. Martin, T. M. Young and was found to characterize the optimal parameter combination, T. G. Rials, Appl. Opt., 2008, 47, G158–G165. whereas for noise clustering, the search for the optimal value 9 S. M. Clegg, E. Sklute, M. D. Dyar, J. E. Barefield and R. C. Wiens, of parameter L should be restricted to local minima in the re- Spectrochim. Acta B, 2009, 64, 79–88. 10 A. Koujelev, M. Sabsabi, V. Motto-Ros, S. Laville and S. Lui, Planet. gion close to its default value. Although our results proved Space. Sci., 2010, 58, 682–690. the PFCM to be the most reliable clustering algorithm, noise 11 S.-L. Lui and A. Koujelev, J. Anal. At. Spectrom., 2011, 26, 2419–2427. clustering could be seen as the optimal one when also taking 12 G. V´ıtková, K. Novotn´y, L. Prokeš, A. Hrdlička, J. Kaiser, J. Novotn´y, into account the computational effort for parameter optimiza- R. Malina and D. Prochazka, Spectrochim. Acta B, 2012, 73, 1–6. tion. For the PFCM, the computational time for optimization 13 F. Hoeppner, F. Klawonn, R. Kruse and T. Runkler, Fuzzy Cluster Anal- ysis: Methods for Classification, Data Analysis and Image Recognition, of the three parameters greatly increases with respect to the John Wiley & Sons Ltd., Chichester, 1999. time needed for the optimization of one parameter for noise 14 J. B. MacQueen, Proc. of the 5th Berkeley Symp. on Math. Statist. and clustering. Prob., Berkeley, U.S.A., Jun 21 – Jul 18, 1965, Dec 27 –Jan 7, 1966, pp. It has to be stressed that although results were presented 281–297. 15 J. Z. Bezdek, Pattern Recognition with Fuzzy Objective Function Algo- from only one concrete test specimen subjected to ASR, rithms, Plenum Press, New York, 1981. we also analysed several other laboratory and in-situ con- 16 D. Gustafson and W. Kessel, 1978 IEEE Conf. on Decision and Control crete cores (also subject to other degradation mechanisms). including the 17th Symp. on Adaptive Processes, San Diego, CA, Jan Overall, noise and possibilistic clustering performed best and 10–12, 1979, pp. 761–766. can therefore be regarded as capable of dealing with multi- 17 R. Kruse, C. Doering and M. J. Lesot, Advances in Fuzzy Clustering and its Applications, John Wiley & Sons Ltd., Chichester, England, 2007, pp. aggregate concrete specimens and noisy LIBS spectra. Con- 3–30. trary to the classification based on the subjectively estimated 18 R. N. Dave, Pattern. Recogn. Lett., 1991, 12, 11, 657–664. threshold value of the O/Ca ratio, our proposed parameter op- 19 R. Krishnapuram and J. M. Keller, IEEE Trans. Fuzzy Syst., 1996, 4, 3, timization model enables the algorithms to work fully unsu- 385–393. pervised. Therefore, the improved use of LIBS, being itself 20 N. R. Pal, K. Pal and J. C. Bezdek, Proc. of the 6th IEEE Int. Conf. on Fuzzy Systems, Barcelona, Spain, Jul 1–5, 1997, pp. 11–21. reliable and fast, will play an important role in the implemen- 21 N. R. Pal, K. Pal, J. M. Keller and J. C. Bezdek, IEEE T. Fuzzy Syst., tation of the in situ mobile system, which is currently under 2005, 13, 4, 517–530. 1–12 | 11 Cotič, P. 2014. The synthesis of multisensor non-destructive testing. . . clustering methods. D13 Doc. Dis. Ljubljana, UL FGG, Doctoral study Built Environment. 22 H. Timm, C. Borgelt and R. Kruse, Proc. of the Eur. Symp. on Intelli- gent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems, Tenerife, Spain, Dec 13–14, 2001, pp. 1–7. 23 A. M. Bensaid, L. O. Hall, J. C. Bezdek, L. P. Clarke, M. L. Silbiger, J. A. Arrington and R. F. Murtagh, IEEE T. Fuzzy Syst., 1996, 4, 2, 112–123. 24 J. Abonyi and B. Feil, Advances in Fuzzy Clustering and its Applications, John Wiley & Sons Ltd., Chichester, England, 2007, pp. 95–121. 25 X. L. Xie and G. Beni, IEEE T. Pattern Anal., 1991, 13, 8, 841–847. 26 J. Wachs, O. Shapira and H. Stern, Applied Soft Computing Technologies: The Challenge of Complexity, Springer Verlag, Berlin, 2006, pp. 77–87. 27 C. Zhang, Y. Zhou and T. Martin, Proc. of IPMU08, Torremolinos, Málaga, Jun 22–27, 2008, pp. 877–882. 12 | 1–12 Document Outline Bibliographic-documentalistic information and abstract Bibliografsko-dokumentacijska stran in izvleček Acknowledgement Introduction Scientific background and motivation Aims and scope Basic principles of selected NDT methods Overview of NDT methods used in civil engineering Ground penetrating radar Electromagnetic properties of building materials and the energy storage-loss mechanisms Wave nature of electromagnetic fields Reflection and refraction of electromagnetic waves Horizontal and vertical resolution Complex resistivity Electrical polarization Apparent and true resistivities Electrode configurations Infrared thermography Interaction of incident radiation with matter Thermal emission Conduction heat transfer Quantitative data analysis in pulsed thermography Thermal contrast Pulsed phase thermography Ultrasonic Acoustic wave theory Reflection and refraction of acoustic waves Acoustic attenuation and dispersion Piezoelectric transducers NDT Data Fusion with Clustering Introduction to data fusion A data fusion framework Cluster analysis Distance and similarity Objective function-based clustering – a general note Objective function-based clustering algorithms Hard clustering Fuzzy clustering Validity measures Objective function-based clustering algorithm scheme Non-destructive testing of laboratory concrete specimens with inbuilt defects and inclusions Experimental Specimen description Methods Results Characterization of defects and inclusions using infrared thermography Thermal contrast Pulsed phase thermography Numerical simulation Sensitivity evaluation of ground penetrating radar and infrared thermography Conclusion and Outlook Summary Razširjeni povzetek v slovenskem jeziku (Extended abstract in Slovene) Uvod Osnove izbranih neporušnih metod Združevanje podatkov z gručenjem Neporušne preiskave laboratorijskih betonskih preizkušancev z vgrajenimi anomalijami Eksperimentalne metode in materiali Rezultati Neporušne preiskave laboratorijskih ometanih večslojnih kamnitih zidov Eksperimentalne metode in materiali Rezultati Neporušne preiskave laboratorijskega opečnatega zidu pod vplivom različne vlažnosti Eksperimentalne metode in materiali Rezultati Neporušne preiskave z večsenzorskim robotnim sistemom BetoScan Eksperimentalne metode in materiali Rezultati Uporaba metod gručenja za avtomatsko klasifikacijo spektroskopskih podatkov na betonskih preizkušancih Eksperimentalne metode in materiali Rezultati Zaključki in obeti References List of Publications