i sciendo Zdr Varst. 2019;58(1):1-10 10.2478/sjph-2019-0001 Smigelskas K, Lukoseviciute J, Vaiciunas T, Mozuraityte K, Ivanaviciute U, Mileviciute I, Zemaitaityte M. Measurement of health and social behaviors in schoolchildren: randomized study comparing paper versus electronic mode. Zdr Varst. 2019;58(1):1-10. doi: 10.2478/sjph-2019-0001. MEASUREMENT OF HEALTH AND SOCIAL BEHAVIORS IN SCHOOLCHILDREN: RANDOMIZED STUDY COMPARING PAPER VERSUS ELECTRONIC MODE MERITVE ZDRAVSTVENEGA IN SOCIALNEGA VEDENJA PRI ŠOLOOBVEZNIH OTROCIH: RANDOMIZIRANA ŠTUDIJA, KI PRIMERJA UPORABO TISKANIH IN ELEKTRONSKIH VPRAŠALNIKOV Kastytis ŠMIGELSKAS12*, Juste LUKOŠEVIČIUTE2, Tomas VAIČIUNAS12, Kristina MOZURAITYTE1, Urte IVANAVICIUTE1, leva MILEVIČIUTE1, Monika ŽEMAITAIT/TE1 'Department of Health Psychology, Faculty of Public Health, Medical Academy, Lithuanian University of Health Sciences, Tilžes g. 18, Kaunas LT-47181, Lithuania 2Health Research Institute, Faculty of Public Health, Lithuanian University of Health Sciences, Tilžes g. 18, Kaunas LT-47181, Lithuania Original scientific article Introduction: Electronic survey mode has become a more common tool of research than it used to be previously. This is strongly associated with the overall digitization of modern society. However, the evidence on the possible mode effect on study results has been scarce. Therefore, the aim of this study is to investigate the comparability of findings on health and behaviours using a paper-versus-electronic mode of survey with randomization design among schoolchildren. Methods: A randomized study was conducted using a mandatory questionnaire on international Health Behaviour in School-aged Children (HBSC) study in Lithuania, enrolling 531 schoolchildren aged 11-15 years. The questionnaire included health and social topics about physical activity, risk behaviours, self-reported health and symptoms, life satisfaction, bullying, fighting, family and school environment, peer relationships, electronic media communication, sociodemographic indicators, etc. The schoolchildren within classes were randomly selected for electronic or paper mode. Results: It was found that by study mode differences are inconsistent and in the majority of cases do not exceed 5%-point difference between the modes. The only significant difference was that in the paper survey the participants reported more exercise than in the electronic survey (OR=8.08, P<.001). Other trends were nonsignificant and did not show a consistent pattern - in certain behaviours the paper mode was related to healthier choices, while in others - the electronic. Conclusions: The use of electronic questionnaires in surveys of schoolchildren may provide findings that are comparable with concurrent or previously conducted paper surveys. Uvod: Uporaba elektronskih vprašalnikov postaja vse bolj pogosto raziskovalno orodje, ki ga omogoča vsesplošna digitalizacija sodobne družbe. Dokazi o morebitnih učinkih elektronskih vprašalnikov na rezultate študije pa so pomanjkljivi. Cilj te študije je raziskati primerljivost dognanj o zdravstvenih vedenjih med šoloobveznimi otroki z uporabo tiskanih vs. elektronskih vprašalnikov. Metode: Randomizirano študijo smo izvajali v Litvi in je vključevala 531 šoloobveznih otrok med 11. in 15. letom starosti. Uporabili smo vprašalnik mednarodne raziskave Z zdravjem povezano vedenje šoloobveznih otrok (Health Behaviour in School-aged Children (HBSC)). Vprašalnik je zajemal vprašanja s področja zdravja in družbe; povpraševal je o fizični aktivnosti otrok, tveganih vedenjih, samoporočanem zdravju in simptomih, življenjskem zadovoljstvu, ustrahovanju, pretepanju, družinskem in šolskem okolju, odnosih z vrstniki, sociodemografskih dejavnikih, komunikaciji po elektronskih medijih itd. Šoloobvezni otroci znotraj razredov so bili naključno izbrani za odgovarjanje na vprašalnike v tiskani in elektronski obliki. Rezultati: Ugotovitve kažejo, da so razlike med obema oblikama vprašalnikov nekonsistentne in v večini primerov ne presegajo 5 % razlike med oblikama. Edina pomembna razlika je, da so v skupini, ki je odgovarjala na tiskani vprašalnik, poročali o več gibanja kot v skupini, ki je uporabljala elektronski vprašalnik (OR = 8,08, P < ,001). Drugi trendi niso znatni in ne prikazujejo konsistentnega vzorca; pri določenih vedenjih so se rezultati tiskanega vprašalnika nagibali k bolj zdravim izbiram, medtem ko so se v nekaterih drugih vedenjih nagibali k bolj zdravim izbiram rezultati elektronskega vprašalnika. Zaključek: Uporaba elektronskega vprašalnika v raziskavah pri šoloobveznih otrocih lahko prinaša rezultate, ki so primerljivi s sočasnimi ali predhodno izvedenimi raziskavami, ki so uporabljale tiskane vprašalnike. 'Corresponding author: Tel. +370 37 242 911; E-mail: kastytis.smigelskas@lsmuni.lt Received: Mar 19, 2018 Accepted: Nov 13, 2018 ABSTRACT Keywords: school-aged children, health behavior, social support, prevalence, validity, reliability, questionnaire design, Lithuania IZVLEČEK Ključne besede: šoloobvezni otroci, zdravstveno vedenje, socialna podpora, razširjenost, veljavnost, zanesljivost, oblika vprašalnika, Litva Z National Institute © National Institute of Public Health, Slovenia. 1 of Public Health This work is "censed under the Creative il«itarycbj6iifcilj3«®3id^eruntergeladen 23.03.20 10:37 UTC 10.2478/sjph-2019-0002 Zdr Varst. 2019;58(1):11-20 1 INTRODUCTION Information and communication technology has become an ever more demanded working tool to enhance the management, efficiency, and quality of surveys on health and social phenomena. There are several kinds of electronic questionnaires - online access, mobile device administered by the researcher, or computer/ device handled by respondent. The responses can be collected by participant, researcher or a proxy (if a participant is minor). Overall digitization of social life and communication suggests ever-increasing pressure to conduct digital surveys and, therefore, it is essential to assess how reliable and valid the digital methods are and, if they replace paper-and-pencil method, are the findings comparable? The online mode reduces the study costs by saving on the costs of paper and printing as well as from transportation (1). Besides, it ensures quick data with virtually no errors and suggests fewer no-response answers (2). Another important point is that these devices permit automatic checking of responses and complex skip patterns. However, in the digital survey mode, it is essential to ensure who is filling in the questionnaire, which is not always feasible. The literature on the effects of digital-based and computer-adaptive testing suggests that digitization of standardized tests is a precise and appropriate research mode both from a scientific and logistic point of view (3, 4). Nonetheless, some researchers propose that the reliability of data obtained by the web-based approach should be determined (5). There is also a potential for selection bias, where a particular type of participant may be more prone to a particular survey mode (e.g. preference for digital mode among younger, more affluent or educated people). Moreover, in online mode, the participants can be unknown, not meet eligibility criteria or make double entries. Therefore, due to the potential for selection bias a randomized controlled design could be regarded as the main choice in studies on potential mode effects. Even though many studies analysing the issue of mode effect on study results use randomization, quite a lot of them address the issue of response rate foremost, while content-specific comparison receives less attention. Also, such studies rarely investigate younger groups and the majority of them do not use randomization. For example, in the international Health Behaviour in School-aged Children (HBSC) study some countries use mixed mode design for more than a decade, e.g. Belgium (6), but they usually do not randomize the schools or children, leaving the choice of mode up to the school's or child's preference - which may be a subject to bias. Thus, even though research on the validity and reliability of digital versus paper mode is quite extensive, such assessment in adolescents is rarely addressed. Moreover, the randomized approach in the research of mode effect is not always applicable, leaving the findings with a potential for self-selection or school-specific bias. In addition, the health perceptions and behaviours have also been under-investigated from this perspective. Therefore, the objective of our study is to compare the findings from paper and electronic mode using a randomized controlled design among schoolchildren. 2 METHODS 2.1 Study Process and Sample The randomized controlled study was conducted in May 2017 at five secondary schools in Lithuania. All study subjects were informed about the details of the study and that the return of the filled questionnaire will be treated as the informed consent. The anonymity of study participants and confidentiality of the data was ensured. The study was conducted as a pilot project for an oncoming 2018 Health Behaviour in School-aged Children study in Lithuania. The schools were randomly selected from the national schools' list, by choosing the first five schools who agreed to participate in the study. The schools were from the second-largest city, other cities, and one town. In every school, the questionnaire was administered to 5th, 7th, and 9th grades (predominant age of children 11, 13 and 15 years, respectively). Then, the randomization was applied for every class in the school, with one-half of students filling the questionnaire in paper mode and the other half in electronic mode. Every class was randomized to define which half of the students' list filled the online and which the paper version of the questionnaire. Questionnaires (both electronic and paper mode) were administered in school classrooms by trained researchers who complied with written instructions. The electronic version of the questionnaire was uploaded to Google Forms, which was available only to the researchers. During the survey, the researchers shared the web link to study participants. The online questionnaire was filled in on desktop or tablet computers. The places for survey were usually classrooms, computer rooms or libraries. In some cases, the survey of paper and online mode was conducted simultaneously in the same room. Every researcher wrote the notes about the procedure of survey. 2.2 Measurements The tool for the study was based on the then-current version of the standardized international HBSC research protocol (7). The HBSC questionnaire covers a wide range of health and social topics about schoolchildren's physical activity, risk behaviours, self-reported health and symptoms, life satisfaction, bullying, fighting, family, school environment, peer relationships, electronic media Bereitgestellt von National & University Library Ljubljana | Heruntergeladen 23.03.20 10: 37 UTC 10.2478/sjph-2019-0001 Zdr Varst. 2019;58(1):1-10 communication, sociodemographic indicators, etc. Only the mandatory items were included. The sequence, formulation, and overall visualization of items did not differ by mode. Some items of the questionnaire were used from particular scales or subscales: • HBSC symptom checklist, 8 items (7), • Family Affluence Scale, 6 items (8), • Multidimensional Scale of Perceived Social Support: Family, 4 items (9), • Multidimensional Scale of Perceived Social Support: Friends, 4 items (adapted from (9)), • Teacher and Classmate Support Scale: Classmates, 3 items (adapted from (10)), • Teacher and Classmate Support Scale: Teachers, 3 items (adapted from (10)), • Online contact with friends and others, 4 items (11), • Preference for online communication, 3 items (12), • Social media addiction, 9 items (13). 2.3 Data Analysis Data was processed using MS Excel 2010 and analysed using IBM SPSS Statistics, version 20. The descriptive analysis included the calculation of the prevalence of different health behaviours (%). The items were dichotomized based on the cut-offs used in the 2014 Health Behaviour in School-aged Children study report (14). The main purpose of the analysis was to estimate whether various health-related items are similarly distributed among study groups in schoolchildren that filled in the questionnaire in paper-versus-electronic mode. For this, the percentage point differences were calculated, and logistic regression was used with the calculation of certain behaviours' risk when comparing the modes. The differences between the modes were estimated using percentage point difference and odds ratios with the reference group being electronic mode (0R=1.00). Given that despite randomization there were some imbalances between the study groups by gender, grade, and school, these indicators were adjusted for in the multivariate logistic regression model. Due to multiple comparisons of different indicators, the Bonferroni correction was used: in total, 78 variables were compared, so the conventional significance level of P<0.05 was decreased to P<0.001 (0.05/78=0.00064). The P-values between 0.001 and 0.05 were reported as trends. 3 RESULTS The study sample comprised 531 schoolchildren - 261 filled the electronic questionnaire and 270 the paper version. The overall response rate was 83.0% with higher rates among girls and elder schoolchildren. A detailed comparison of study groups by gender, grade, and school are presented in Table 1. Regardless of randomization, there were some differences observed between study groups and since they were definitely random (by design of the study) their statistical significance was not calculated. Table 1. The main characteristics of study sample. Characteristic Electronic Paper n Response mode mode rate Gender Boys 51.4 48.6 255 77.0 Girls 47.3 52.7 275 89.0 Grade 5th 49.7 50.3 187 77.9 7th 48.8 51.2 201 84.1 9th 49.0 51.0 143 88.8 School #1 (large city) 47.3 52.7 74 89.2 #2 (large city) 48.7 51.3 224 94.5 #3 (city) 50.5 49.5 103 60.6 #4 (city) 50.0 50.0 48 80.0 #5 (town) 50.0 50.0 82 91.1 In this study, the internal consistency of scales and subscales was acceptable and the difference between the modes was not more than .07 points - with no consistent superiority of either mode (Table 2). Table 2. Internal consistency of study scales and subscales by survey mode. Scale Number Internal consistency (a) of items Electronic mode Paper mode HBSC symptom checklist 8 .78 .79 Family Affluence Scale 6 .52 .58 Multidimensional Scale of Perceived Social Support: Family 4 .76 .69 Multidimensional Scale of Perceived Social Support: Friends 4 .90 .85 Teacher and Classmate Support Scale: Classmates 3 .77 .70 Teacher and Classmate Support Scale: Teachers 3 .75 .74 Online contact with friends and others 4 .54 .54 Preference for online communication 3 .84 .81 Social media addiction 9 .75 .76 Bereitgestellt von National & University Library Ljubljana | Heruntergeladen 23.03.20 10: 37 UTC 10.2478/sjph-2019-0001 Zdr Varst. 2019;58(1):1-10 3.1 Health Behaviours In the field of health behaviours (Table 3), the largest difference depending on survey mode was observed in extensive physical activity - in paper mode, the schoolchildren more frequently reported daily exercise until getting out of breath or sweating (OR=8.08, P<.001). Table 3. Health behaviours of schoolchildren by survey mode. Other indicators had no differences except the trends that students in paper mode more frequently reported, such as having a regular breakfast on weekends (OR=1.93, P=.009). Almost all aspects of health behaviours differed between the survey modes by no more than 5% points. Characteristic Prevalence, % % OR P difference Electronic Paper Eating habits Having breakfast during the weekdays Every day 58.8 62.7 3.9 1.18 .366 Having breakfast during the weekends Every day 79.6 87.7 8.1 1.93 .009 Having breakfast with parents Every day 41.0 40.7 -.3 1.00 .982 Having dinner with parents Every day 47.1 45.6 -1.5 .96 .816 Eating fruits Every day 41.8 38.5 -3.3 .87 .446 Eating vegetables Every day 32.6 34.2 1.6 1.07 .707 Eating sweets Every day 16.1 13.8 -2.3 .83 .453 Drinking soft drinks Every day 5.4 6.3 .9 1.23 .593 Drinking energy drinks Every day 2.3 .4 -1.9 .16 .097 Health and well-being Subjective health assessment Good 88.5 91.8 3.3 1.59 .132 Life satisfaction 6-10 (10 pts scale) 87.7 85.8 -1.9 .84 .510 Headache Rarely 84.3 82.5 -1.8 .91 .707 Stomach ache Rarely 93.5 93.7 .2 1.04 .912 Backache Rarely 91.6 92.1 .5 1.09 .794 Feeling low Rarely 80.1 81.3 1.2 1.14 .577 Irritability or bad temper Rarely 72.0 76.5 4.5 1.39 .115 Feeling nervous Rarely 70.5 69.7 -.8 1.01 .946 Difficulties in getting to sleep Rarely 79.7 83.2 3.5 1.33 .214 Feeling dizzy Rarely 89.7 89.9 .2 1.08 .798 Brushing the teeth More than once a day 61.3 62.8 1.5 1.05 .809 Body image A bit too thin 11.9 15.0 3.1 .71 .200 A bit too fat 29.1 30.7 1.6 .91 .654 About the right size 59.0 54.3 -4.7 1.00 - Physical activity Physical activity at least 60 minutes 7 days 18.9 20.6 1.7 1.16 .518 per day (last week) Exercise in free time until getting Every day 3.1 19.7 16.6 8.08 <.001 out of breath or sweating Risk behaviour Cigarette smoking (lifetime) Never 73.2 77.8 4.6 1.37 .187 Cigarette smoking (last month) Never 88.1 92.3 4.2 1.75 .097 Alcohol drinking (lifetime) Never 62.8 68.4 5.6 1.37 .129 Alcohol drinking (last month) Never 88.9 88.0 -.9 .89 .706 Cannabis taking (lifetime) Never 94.3 97.0 2.7 2.22 .101 Cannabis taking (last month) Never 98.1 99.2 1.1 2.13 .376 Sexual intercourse No 95.0 93.3 -1.7 .81 .626 Bereitgestellt von National & University Library Ljubljana | Heruntergeladen 23.03.20 10: 37 UTC 10.2478/sjph-2019-0001 Zdr Varst. 2019;58(1):1-10 3.2 Social Behaviours and School The selected indicators of social behaviours under study showed slightly bigger differences than health behaviours, though they were inconsistent and nonsignificant (Table 4). Here the trend in paper mode was that the children were more likely to report having friends to share joys and sorrows, but also more cyber-bullying and more treatment-needed injuries (.001