Radiol Oncol 1997; 31: 358-63. Influence of the radiation source on the quality of transmission bone phantom images Terez Sera,1 Janos Mester,2 Arne Skretting,3 Laszlo Csernay1 'Albert Szent-Gyorgyi Medical University, Department of Nuclear Medicine, Szeged, Hungary, 2 University Hospital Eppendorf Department of Nuclear Medicine, Hamburg, Germany, 3The Norwegian Radium Hospital, Department of Medical Physics and Technology, Oslo, Norway Different types of flood sources were applied to produce bone phantom images, and the quality of the resulting phantom images was studied. The measurements were made with 37-PMT circular detector gamma cameras and with rectangular detectors SPECT devices, with the Scanflex Transbone transmission bone phantom. Phantom images were produced with a 99m-Tc fillable flood source, a 57-Co flood source and a dynamic line phantom (Veenstra Instrumenten B. V..). Images were acquired with the number of counts collected under routine clinical conditions (600 000-800 000 counts) and also with a high number of counts (2 million counts) and were documented on X-ray films. The images were interpreted by three independent experienced observers. From the scores, ROC (receiver operating characteristic) curves were generated. The quality of the images was characterized by the area under the ROC curves. There were no significant differences between the ROC curve areas of the images produced with the same parameters. The differences between the routine and the high-count images were significant. It is concluded that all of the investigated flood sources are suitable for the production of bone phantom images for the quality control of nuclear medicine imaging procedures. Key words: phantoms; imaging; quality control; bone and bones-radionuclide imaging Introduction For more than ten years the World Health Organization (WHO) and the International Atomic Energy Agency (IAEA) have organized interlaboratory comparison studies to analyse the quality of medical investigations using nuclear medicine imaging devices. To reproduce the clinical conditions, organ phantoms have been applied. Measurements with these phantoms allow the determination of parameters which simultaneously characterize the imaging Correspondence to: Dr. T. Sera, Department of Nuclear Medicine, Albert Szent-Gyorgyi Medical University, Koranyi fasor 8, H-6720 Szeged, Hungary. Tel: (36)-62-455375; Fax: (36)-62-311170; E-mail: serat@com-ser.szote.u-szeged.hu UDC: 616.71-073.756-092.4:539.163 device, the clinical investigation method, and the physician performance. 1,2 The reports returned to the laboratories permit the participants to re-examine their own performance and to correct any errors. The quality control measurements series organized by the WHO to date have involved liver,2'3 thyroid4 and heart5 phantoms. Bone scintigraphy is one of the most widely used nuclear medicine investigation procedures, and the production of a bone phantom is therefore also necessary. The prototype of the "black box" transmission bone phantom was developed by Skretting and co-workers. With this phantom, the first interlaboratory study was carried out in Norway6,7 as part of an ongoing WHO/IAEA coordinated study. Interlaboratory measurements with a version of this phantom were also performed in Hungary at the beginning of 1994, in 22 nuclear Influence of the radiation .source on the quality of transmission bone phantom images 359 medicine laboratories on 28 gamma cameras and SPECT devices; the results were reported earlier. The aim of the present study was to investigate whether the type of flood phantom utilized for imaging the transmission bone phantom influences the quality of the phantom images. Materials and methods Our investigations were performed on three 4C/15 type 37-PMT gamma cameras Hungarian made under Picker licence, equipped with a circular detector, and on two modem 59-PMT SPECT devices with a rectangular-shaped detector (Siemens Diacam, Elscint Helix). For the measurements, we utilized the Scanflex Transbone transmission bone phantom, which was applied in the international interlaboratory quality control study organized by the WHO and IAEA. The schematic diagram of the phantom is presented in Figure 1. The "black box" phantom represents the thoraco-lumbar region from the posterior view. It is rectangular in shape, with constant thickness, and contains absorbing material with a rela- tively high absorption coefficient, and filling material mostly with a water-equivalent absorption coefficient. The variation in the intensity of the gamma rays is caused by the variation in thickness of the absorbing material. The phantom is divided into 45 regions; 23 of them contain built-in lesions (22 are hot lesions with relative intensities varying between 1.10 and 2.30, and one is a cold lesion with a relative intensity of 0.58). Seven Iesions are Iocated on the vertebrae, 3 in the medial and 3 in the distal regions of the left ribs, and 5 in the medial and 5 in the distal parts of the right ribs (Figure 2). LEFT RIBS COL^^ RIGHT RIBS ^.TEBRA Distal Medial Medial Distal 1.8 1.93 1.65 2.04 1.48 1.41 (disc) 1.32 1.52 1.18 1.25 1.76 2.18 1.32 2.30 1.60 1.86 1.72 1.7 1.10 VERTOBRA 0.58 1.9 Figure l. Design of the bone phantom. Figure 2. Localization of lesions built into the bone phantom. In our investigations, the phantom was imaged by three different types of flood source: a 99m-Tc flood source, a 57-Co flood source and a dynamic line phantom. The 99m-Tc flood source was obtained by filling a plastic disc with bubble-free 99m-Tc solution; a commercially available 57-Co flood source with 120 MBq activity (Amersham) was purchased; and the dynamic line phantom was the one produced by Veenstra Instrumenten B.V. The first of these is a movable capillary tube, controlled by a microprocessor, filled with 350 MBq 99m-Tc solution. The diameters of the 99m-Tc and 57-Co circular flood sources corresponded to the diameter of the 4C/15 type 37-PMT detector. With the dynamic line phantom, a 500x500 mm surface uniform flood source was produced. 360 Sera T et al. Static images To obtain static images, the phantom was placed on the collimator of the upward-facing detector, and the 99m-Tc or the 57-Co flood source was then positioned on the phantom. When the dynamic line phantom was used, the bone phantom was placed on the special holder above the capillary tube and the images were made with the downward-facing detector. The parts of the detector which were not covered by the phantom were shielded with lead foil. Images were produced with routine clinical parameters, and were printed on X-ray film, using the exposure parameters elaborated for the clinical routine for each device and a standardized method for film development. In order to maximize the available information, high-count images, not applicable in clinical practice, were also produced. Whole body images With the SPECT devices, images were also produced in whole body scanning mode, utilizing 99m-Tc or 57-Co flood sources: the flood source was placed on the patient investigation bed, and the bone phantom was placed on top of it. The whole body clinical program was activated. To produce the whole body images with the Siemens SPECT, the phantom placed on the patient bed was moved under the surface of the detector at constant speed; with the Elscin SPECT, the images were obtained in a step mode, and the software then generated the whole body image of the phantom. Images were produced with clinical parameters, and high-count images were also obtained. The images were printed on X-ray film. All the X-ray films were exposed and developed within a few days. Interpretation of the images Images were interpreted independently by three experienced observers, using a score method. The WHO study protocol was applied, and the image of the phantom was divided into 45 regions (Figure 2). The observers were asked to decide whether they could observe lesions, and with what certainty of detection, within each of the defined regions of the images. The certainty of the detection was characterized by score values on the scale 1-4, 1 indi- cating a definitely negative region, and 4 a definitely positive region. Before the interpretation, the images were mixed and given to the observers in a random sequence at long intervals, so that they did not remember the position of the lesions in the already interpreted images. Images were interpreted by ROC (receiver operating characteristic) analyses [8, 9, 10]. The results of the images produced on the same gamma cameras with identical parameters, using the three flood sources, were analysed in pairs. Results With the 99m-Tc flood source, the 57-Co flood source and the dynamic line phantom, totals of 19, 14 and 17 images, respectively, were produced. From the results of the three observers, a total of 50 ROC curves were generated. Figure 3 relates to these ROC curves. The observers' performances are characterized by the area under the ROC curve. Source ROC area 99m-Tc flood source 0.863 dynamic line phantom 0.861 57-Co flood source 0.869 No. of collected counts = 600 000. Fi^^e 3. ROC curves and areas of phantom images produced with routine clinical parameters. on a 59-PMT gamma camera with three different flood sources. Influence of the radiation .source on the quality of transmission bone phantom images 361 The ROC areas vary between 0.73 and 0.96. The best value for the ROC curve area was obtained from the data pertaining to the image produced with the 59-PMT SPECT device with the 57-Co flood source and a high number of counts. The lower value for the ROC area derived from image data obtained with the 59-PMT SPECT device, using the 99m-Tc flood source in the whole body acquisition mode. The differences in the ROC areas obtained from the interpretations by the individual observers varied between 0.03 and 0.08. The highest differences were obtained with the 59-PMT SPECT device with the 57-Co flood source in the whole body acquisition mode, for a high number of counts, and with the image produced with the 59-PMT SPECT device with the 99m-Tc flood source, for static images with clinical parameters. In the evaluation of the high-count images, the important differences between the observers were caused by the three false-positive findings of one observer, while in the case of the low-count images, the differences stemmed from the relatively high number of false-negative findings of another observer. However, significant differences were not found between the results of the different observers (Table 1). Table 1. Comparison of ROC curve areas of images produced with the same gamma cameras but with different radiation sources. 99m-Tc flood source line phantom ROC area (mean ± SD) Observer 1 0.87±0.04 0.87±0.03 n.s. Observer 2 0.86±0.03 0.86±0.03 n.s. Observer 3 0.86±0.04 0.87±0.03 n.s. All 0.86±0.04 0.86±0.03 n.s. 99m-Tc flood source 57-Co flood source ROC area (mean ± SD) Observer 1 0.87±0.04 0.85±0.03 n.s. Observer 2 0.87±0.04 0.89±0.04 n.s. Observer 3 0.86±0.04 0.89±0.04 n.s. All 0.87±0.04 0.88±0.04 n.s. line phantom 57-Co flood source ROC area (mean ± SD) Observer 1 0.87±0.03 0.85±0.04 n.s. Observer 2 0.87±0.03 0.88±0.04 n.s. Observer 3 0.89±0.02 0.89±0.03 n.s. All 0.88±0.03 0.87±0.04 n.s. The results obtained from the images produced with the same cameras, with the same acquisition parameters, but with different flood sources, are presented in Table 1. From the images obtained using the 99m-Tc flood source, 15 were matched with the dynamic line phantom images, 12 with 57-Co images, and 8 of the images obtained with the 57-Co source were compared with their counterparts by using the dynamic line phantom. The quantitative interpretation of the ROC curves does not reveal significant differences in quality of the phantom images produced with the different flood sources. From the analysis of the influence of the different numbers of counts acquired for the phantom images on the ROC curve area, significant differences in quality were observed between the routine and high-count images (Table 2). The highest difference (0.2) was obtained with the 37-PMT gamma camera, with the dynamic line phantom. The results obtained with the SPECT devices were usually better than those obtained with the planar cameras, but the low number of SPECT devices involved in the study does not permit a statistical interpretation of the differences. Table 2. Comparison of ROC curve areas of images produced with the same gamma cameras but with the standard clinical number or a high number of counts. 99m-Tc flood source Acquisition routine optimal ROC area (mean ±SD) Observer l Observer 2 Observer 3 All 0.85±0.04 0.83±0.04 0.82±0.04 0.83±0.04 0.88±0.04 0.89±0.03 0.88±0.03 0.88±0.03 n.s. p<0.05 p<0.05 p<0.05 dynamic line phantom Acquisition routine optimal ROC area (mean ±SD) Observer 1 Observer 2 Observer 3 All 0.83±0.04 0.83±0.03 0.85±0.03 0.84±0.03 0.90±0.03 0.90±0.02 0.89±0.02 0.90±0.02 p<0.05 p<0.05 p<0.05 p<0.05 57-Co flood source Acquisition routine optimal ROC area (mean ±SD) Observer 1 Observer 2 Observer 3 All 0.83±0.01 0.87±0.01 0.86±0.02 0.85±0.01 0.87±0.03 0.92±0.03 0.92±0.02 0.90±0.03 p<0.05 p<0.05 p<0.05 p<0.05 Discussion Regular quality assurance is one of the important conditions of correct diagnostic investigations." Routine quality control investigations in nuclear medicine laboratories provide mostly data on the technical parameters of the imaging devices,12 but 362 Sera Tet al. the information relating to the clinical performance in patient investigation results is not reliable. The detectability of different lesions can be evaluated by using organ phantoms. "Black box" phantom measurements provide valuable data which simultaneously reflect the performances of the imaging instrument and the decision-making physician. It is important, of course, that the basis of the study is the evaluation of the phantom images produced in the same mode as the clinical reports on the patients. Various national and international organizations and working groups have so far reported results of intercomparison studies with thyroid, liver, heart and brain phantoms.2'4'13'14 Their conclusions indicate that the differences in the performances of the imaging instruments, in the acquisition and processing parameters of the images, and in the performances of the observers lead to significant differences in the clinical evaluation. The procedures applicable for intercomparison measurements of the information lines between the imaging device and the patient report are becoming more effective. The images were initially evaluated by detecting the numbers of true-positive and false-negative findings. Later, different semiquantitative score methods were introduced which take into account the accuracy of lesion detection.8 So, due to the evaluation of the images, ROC curves can be generated.8^10 The area under such a curve is an objective parameter for the characterization of the performance of an imaging and evaluating system. The list of available organ phantoms was supplemented by the bone phantom developed by Skret-ting and co-workers.1"•'4 to meet the increasing clinical importance of bone scintigraphic investigations. We consider transmission phantoms to be a useful compromise between practicability and reality, but a bone phantom based on the emission would have been closer to the clinical conditions. To establish the fine limitations of the devices, phantoms with lesions on the border of detectability are needed. Since examinations of the skeleton comprise a considerable proportion of nuclear medicine investigations, we considered it important to investigate how certain technical measurement conditions influence the quality of bone phantom images. We have studied how the type of the isotope source applied to generate the uniform radiation flood influences the medical performance measurable by the ROC curves while other technical parameters remaining constant. Having emphasized the clinical use, tests were performed on different types of cameras routinely used for bone scintigra-phy. In order to achieve realistic results, acquisition parameters corresponding to the clinical routine on each particular device were chosen for the phantom investigation. This was extended with a second, high-count data set in order to maximize the available information. The results of our measurements with three different radiation flood sources (a 99m-Tc flood source, a 57-Co flood source and the dynamic line phantom) demonstrate that all of these sources are suitable for imaging the transmission bone phantom. We concluded by saying that a higher number of counts than that applied in routine clinical investigations greatly improves the area under the ROC curves. This is especially important in whole body acquisitions, where an apparently good image quality is needed in a relatively short time. The radionu-clide imaging equipment has nowadays nearly reached the limits of resolution achievable. The time during which a patient can remain motionless is limited. The theoretical alternative is to increase the activity injected.15 This proposal has limitations. The approved levels of injected activity vary from country to country. It would seem that the activities used rely on the tradition and/or the expense, and there is no a generally accepted scientific basis for determining the activity employed.15 In accordance with the results Qf Skretting and co-workers,6 our investigations based on the images produced with routine clinical parameters demonstrated that whole body images usually produced lower ROC curve areas than in the case of static image evaluations. Our results reveal that the quality control of bone scintigraphy performed in a whole body or static mode with the transmission bone phantom is a procedure which can be included and applied in the quality assurance system of nuclear medicine laboratories. References 1. Hermann GA, Herrera NE, Hauser W, Oxley DK. Rationale, Techniques and Results of a Quality Control Programme of Imaging Procedures. In: Medical radionuclide imaging. Vol l.Vienna; IAEA, 1977: 55-66. 2. Herrera NE, Hermann GA, Hauser W, Paras P. College of American Pathologists Program Series X Survey Program. In: Medical radionuclide imaging. Vol 2. Vienna, IAEA, 1980: 177-87. Influence of the radiation .source on the quality of transmission bone phantom images 363 3. Mould R. TheLondon liver phantom. JNuclMed 1983; 24: 974-5. 4. Souchkevitch GN, Asikainen M, Bauml A, Bergmann H, Busemann-Sokole E, Carlsson S. The World Health Organization and International Atomic Energy Agency second interlaboratory comparison study in 16 countries onquality performance of nuclear medicine imaging devices. Eur J Nucl Med 1988; 13: 495-501. 5. Busemann Sokole E. Quality assurance in nuclear medicine imaging [Thesis]. Amsterdam, 1990. 6. Skretting A, Strandmyr E, Lindegaard MW. A Norwegian nationwide quality assurance project in nuclear medicine: total performance in bone scintigraphy measured with a new transmission phantom. Eur J Nucl Med 1990; 17:10-4. 7. Skretting A, Strandmyr E, Lindegaard MW. Properties of a transmission phantom for simulation of radionuclide bone studies and its use for quality assurance. In: HoferR, Bergmann H, Sinzinger H, eds. Stuttgart-New York: Schattauer, 1991:180-5. 8. Metz CE. ROC methodology in radiologic imaging. Invest Radiology 1986. 21:720-33. 9. Hanley JA,McNeilBJ. Themeaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29-36. 10. Hermann GA, Herrera N, Sugiura H. Comparison of interlaboratory data in terms of receiver operating characteristic (ROC) indices. JNucl Med 1982; 23: 525-31. 11. WHO. Quality assurance in nuclear medicine. Geneva, World Health Organization, 1982. 12. National Electrical Manufacturers Association. Standards Publication Nu 1-80. Performance Measurements of Scintillation Camera;:, Washington DC: NEMA, 1980. 13. Volodin V, Souchkevitch G, Racoveanu N, Bergmann H, Busemann-Sokole E, Delaloye B, etal. World Health Organization interlaboratory comparison study in 12 countries on quality performance of nuclear medicine imaging devices. Eur J Nucl Med 1985; 10:193-7. 14. Heikkinen J, KuikkaJT, AhonenA, JurvelinJ, Hartikai-nen K, Kvist G. A Finnish multicentre quality assurance project in bone scintigraphy and brain SPET: a phantom study. Nucl Med Commun 1994; 15: 795805. 15. McCready R, A'Hern R. A more rational basis for determining the activities used for radionuclide imaging? Eur J Nucl Med 1997; 24: 109-10.