SLOVENIAN JOURNAL OF PUBLIC HEALTH ZDR VARST 2024 • YEAR 63 • No 1 ZDR VARST 2024 • LETNIK 63 • ŠTEVILKA 1 CODEN ZDVAFY • UDK 613 / 614 + 628 • ISSN 0351 - 0026 Zalika KLEMENC-KETIŠ,Uroš ZAFOŠNIK 1 MEDPOKLICNO IZOBRAŽEVANJE S SIMULACIJAMI V PRIMARNEM ZDRAVSTVENEM VARSTVU (1-4) Matic MIHEVC, Črt ZAVRNIK, Majda MORI LUKANČIČ, Tina VIRTIČ POTOČNIK, Marija PETEK ŠTER, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ ANALIZA STROŠKOV TELEMONITORINGA OD SPODAJ NAVZGOR: ŠTUDIJA PRIMERA V SLOVENSKEM PRIMARNEM ZDRAVSTVU (5-13) Andrej PANGERC, Marija PETEK ŠTER, Leja DOLENC GROŠELJ VALIDACIJA SLOVENSKE RAZLIČICE VPRAŠALNIKA STOP-BANG ZA UPORABO V AMBULANTAH NA PRIMARNEM NIVOJU ZDRAVSTVENEGA VARSTVA (14-20) Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ ZAZNAVANJE KULTURE VARNOSTI PACIENTOV NA PRIMARNI RAVNI ZDRAVSTVENEGA VARSTVA: RAZISKAVA V ZDRAVSTVENEM DOMU LJUBLJANA (21-29) Matija AMBROŽ, Candan KENDIR, Wienke BOERMA, Zalika KLEMENC-KETIŠ OCENA IZKUŠENJ IN IZIDOV ZDRAVSTVENE OSKRBE PACIENTOV S KRONIČNIMI NENALEZLJIVIMI BOLEZNIMI: METODOLOŠKI VIDIK PILOTNE IZVEDBE RAZISKAVE V SLOVENIJI (30-37) Črt ZAVRNIK, Nataša STOJNIĆ, Majda MORI LUKANČIČ, Matic MIHEVC, Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ PRILOŽNOSTI IN OVIRE ZA IZBOLJŠANJE CELOSTNE OSKRBE ARTERIJSKE HIPERTENZIJE IN SLADKORNE BOLEZNI TIPA 2 V SLOVENIJI: KVALITATIVNA ŠTUDIJA (38-45) Tadeja HOČEVAR, Tim ANSTISS, Danica ROTAR PAVLIČ VSEBINSKA VELJAVNOST IN KOGNITIVNO TESTIRANJE PRI RAZVOJU SAMO-OCENJEVALNEGA VPRAŠALNIKA O MOTIVACIJSKEM INTERVJUJU (46-54) Špela MIROŠEVIČ, Judith PRINS, Nikola BEŠIĆ , Simona BORŠTNAR , Andreja Cirila ŠKUFCA SMRDEL, Vesna HOMAR, Marko POPOVIĆ, Zalika KLEMENC-KETIŠ SIMPTOMI ANKSIOZNOSTI IN DEPRESIJE PRI SLOVENSKIH BOLNICAH Z RAKOM DOJKE PO ZDRAVLJENJU V ČASU PANDEMIJE COVIDA-19: PRESEČNA ŠTUDIJA (55-62) UVODNIK IZVIRNI ZNANSTVENI ČLANKI Zalika KLEMENC-KETIŠ,Uroš ZAFOŠNIK 1 INTERPROFESSIONAL EDUCATION WITH SIMULATIONS IN PRIMARY CARE (1-4) Matic MIHEVC, Črt ZAVRNIK, Majda MORI LUKANČIČ, Tina VIRTIČ POTOČNIK, Marija PETEK ŠTER, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ BOTTOM-UP ANALYSIS OF TELEMONITORING COSTS: A CASE STUDY IN SLOVENIAN PRIMARY CARE (5-13) Andrej PANGERC, Marija PETEK ŠTER, Leja DOLENC GROŠELJ VALIDATION OF THE SLOVENE VERSION OF THE STOP-BANG QUESTIONNAIRE IN A PRIMARY PRACTICE SETTING (14-20) Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ PERCEPTION OF PATIENT SAFETY CULTURE AT THE PRIMARY CARE LEVEL: THE CASE OF THE COMMUNITY HEALTH CENTRE LJUBLJANA (21-29) Matija AMBROŽ, Candan KENDIR, Wienke BOERMA, Zalika KLEMENC-KETIŠ PATIENT REPORTED INDICATOR SURVEYS (PARIS): METHODOLOGICAL CONSIDERATIONS OF A FIELD TRIAL IN SLOVENIA (30-37) Črt ZAVRNIK, Nataša STOJNIĆ, Majda MORI LUKANČIČ, Matic MIHEVC, Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ FACILITATORS AND BARRIERS TO SCALING-UP INTEGRATED CARE FOR ARTERIAL HYPERTENSION AND TYPE-2-DIABETES IN SLOVENIA: QUALITATIVE STUDY(38-45) Tadeja HOČEVAR, Tim ANSTISS, Danica ROTAR PAVLIČ CONTENT VALIDITY AND COGNITIVE TESTING IN THE DEVELOPMENT OF A MOTIVATIONAL INTERVIEWING SELF-ASSESSMENT QUESTIONNAIRE (46-54) Špela MIROŠEVIČ, Judith PRINS, Nikola BEŠIĆ , Simona BORŠTNAR , Andreja Cirila ŠKUFCA SMRDEL, Vesna HOMAR, Marko POPOVIĆ, Zalika KLEMENC-KETIŠ SYMPTOMS OF ANXIETY AND DEPRESSION AMONG SLOVENIAN BREAST CANCER SURVIVORS POST-TREATMENT DURING THE COVID-19 PANDEMIC: A CROSS-SECTIONAL STUDY (55-62) EDITORIAL ORIGINAL SCIENTIFIC ARTICLES CODEN ZDVAFY • UDK 613 / 614 + 628 • ISSN 0351 - 0026 Zdravstveno varstvo ISSN 0351-0026 Izdajatelj/Publisher: Nacionalni inštitut za javno zdravje generalni direktor: Branko Gabrovec Odgovorni urednik/Editor in Chief: Igor Švab Izvršna urednica/Executive Editor: Saša Zupanič Uredniški odbor/Editorial Board: Lektoriranje slovenščine/Proofreading for Slovenian: Mihaela Törnar Lektoriranje angleščine/Proofreading for English: AMIDAS d.o.o., Ljubljana Prevajanje anglelščine/English translation AMIDAS d.o.o., Ljubljana Naslov uredništva/Adress of the Editorlal Offi ce: Zdravstveno varstvo - Slovenian Journal of Public Health, Trubarjeva 2, 1000 Ljubljana, p.p. 260 Elektronski naslov uredništva/E-mail Address: Zdrav.Var@nijz.si Domača stran na internetu/lnternet Home Page: https://nijz.si/objave/revije/revija-zdravstveno-varstvo/ in https://sciendo.com/journal/SJPH Transakcijski račun/Current Account: 011006000043188, UJP Zdravstveno varstvo izhaja praviloma štirikrat letno v nakladi 250 izvodov. Naročnino zaračunavamo z računom za predplačilo v začetku leta. Upoštevamo le pisne odpovedi do 1. decembra za naslednje leto. Vsako spremembo naslova sporočite naročniški službi pravočasno. Revija Zdravstveno varstvo je pri Ministrstvu za kulturo RS vpisana v razvid medijev pod zaporedno številko 608. Izid revije je fi nančno podprla ARRS iz sredstev državnega proračuna iz naslova razpisa za sofi nanciranje domačih znanstvenih periodičnih publikacij. Letna naročnina/Year subscription rate: 52,50 EUR (z vključenim 5 % DDV/including 5% VAT). Naročniška služba: zdrav.var@nijz.si Gradivo navaja predvsem poglede avtorjev, za katere ni nujno, da se ujemajo z načelnimi stališči stroke oziroma uredniškega odbora. Naklada: 250 Likovna oprema ovitka: Jurij Kocbek Grafi čno oblikovanje in prelom: Urška Stariha Tisk: Tisk Žnidarič d.o.o., Laze 7, 4000 Kranj Tit Albreht (Slovenija), Marjan Bilban (Slovenija), Ivan Eržen (Slovenija), Zalika Klemenc Ketiš (Slovenija), Mitja Kos (Slovenija), Alenka Kraigher (Slovenija), Sara Atanasova (Slovenija), Ada Hočevar Grom (Slovenija), Mojca Gabrijelčič Blenkuš (Slovenija), Rado Pišot (Slovenija), Darja Barlič Maganja (Slovenija), Tanja Kamin (Slovenija), Lijana Zaletel-Kragelj (Slovenija), Valentina Prevolnik Rupel (Slovenija), Brigita Skela Savič (Slovenija), Ksenija Rener Sitar (Slovenija), Genc Burazeri (Nizozemska, Albanija), Niek Klazinga (Nizozemska), Jan de Maeseneer (Belgija), Jadranka Božikov (Hrvaška), Birgit Babitsch (Nemčija), Reinhard Burger (Nemčija), Björn Hibell (Švedska), David M. Salisbury (Velika Britanija), Debbie Tolson (Velika Britanija), John E. Morley (ZDA), Josep Figueras (Belgija), Lynne Friedli (Velika Britanija), Eva Kralikova (Češka), Douglas Crews (ZDA) IF(2022)=1.5 INTERPROFESSIONAL EDUCATION WITH SIMULATIONS IN PRIMARY CARE Zalika KLEMENC-KETIŠ 1,2,3* , Uroš ZAFOŠNIK 1 1 Community Health Centre Ljubljana, Metelkova 9, 1000 Ljubljana, Slovenia 2 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia 3 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia Received: Oct 16, 2023 Accepted: Nov 16, 2023 Invited editorial *Correspondence: zalika.klemenc-ketis@zd-lj.si 10.2478/sjph-2024-0001 Zdr Varst. 2024;63(1):1-4 1 MEDPOKLICNO IZOBRAŽEVANJE S SIMULACIJAMI V PRIMARNEM ZDRAVSTVENEM VARSTVU Klemenc Ketiš Z, Zafošnik U. Interprofessional education with simulations in primary care. Zdr Varst. 2024;63(1):1-4. doi: 10.2478/Sjph-2024-0001. ABSTRACT Keywords: Interprofessional primary care Simulations Teamwork Patient safety Education IZVLEČEK Ključne besede: medpoklicno primarno zdravstveno varstvo simulacije timsko delo varnost pacientov izobraževanje The introduction of interprofessional primary care (IPC) as a model of collaborative patient care is increasingly vital in the context of complex healthcare systems and the growing needs of patients. Its benefits include improved patient outcomes, enhanced efficiency, and reduced costs. However, the successful implementation of IPC faces challenges due to the differences in training and backgrounds among healthcare professionals, emphasising the importance of effective teamwork and collaborative education. Educational approaches utilising simulations have gained prominence, particularly in addressing the challenges of interprofessional primary care. Notably, simulations facilitate team learning, enhancing team management and confidence, which ultimately leads to improved performance in real-life scenarios. They also contribute to patient safety by providing comprehensive training and creating a safe environment for professionals to practice and refine their skills without risking real patient harm. Moreover, simulations promote psychological safety, allowing healthcare workers to manage stress effectively and prepare for critical situations. Ethical considerations are met through simulation-based education, ensuring patient confidentiality, and creating a standardised and just learning environment for all students. Simulations contribute to promoting equity in medical education by providing equal access to high-quality training opportunities for all healthcare professionals. In conclusion, successful IPC implementation requires a comprehensive approach that includes interprofessional education and the integration of simulations as an essential component of the curriculum at all levels of healthcare education. This approach fosters effective communication, teamwork, and confidence among primary care teams, ultimately leading to improved patient care and outcomes. Uvajanje medpoklicnega primarnega zdravstvenega varstva (MPZV) kot modela sodelovalne oskrbe bolnikov je vse pomembnejše v kontekstu kompleksnih zdravstvenih sistemov in naraščajočih potreb bolnikov. Njegove prednosti vključujejo boljše izide zdravljenja bolnikov, večjo učinkovitost in manjše stroške. Vendar se uspešno izvajanje MPZV sooča z izzivi zaradi razlik v usposabljanju in izobrazbi zdravstvenih delavcev, kar poudarja pomen učinkovitega timskega dela in sodelovalnega izobraževanja. Izobraževalni pristopi z uporabo simulacij so postali pomembni, zlasti pri obravnavi izzivov medpoklicnega primarnega zdravstvenega varstva. Zlasti simulacije olajšujejo timsko učenje, krepijo timsko vodenje in samozavest, kar na koncu privede do boljšega delovanja v resničnih scenarijih. Prispevajo tudi k varnosti bolnikov, saj zagotavljajo celovito usposabljanje in ustvarjajo varno okolje, v katerem lahko strokovnjaki vadijo in izpopolnjujejo svoje spretnosti, ne da bi tvegali resnično škodo za bolnika. Poleg tega simulacije spodbujajo psihološko varnost, saj zdravstvenim delavcem omogočajo učinkovito obvladovanje stresa in pripravo na kritične situacije. Etični vidiki so izpolnjeni z izobraževanjem na podlagi simulacij, zagotavljanjem zaupnosti bolnikov in ustvarjanjem standardiziranega, pravičnega učnega okolja za vse študente. Simulacije prispevajo k spodbujanju enakosti v medicinskem izobraževanju, saj vsem zdravstvenim delavcem zagotavljajo enak dostop do visokokakovostnih možnosti usposabljanja. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Interprofessional primary care (IPC) is a model of care in which healthcare professionals from different disciplines work together to provide coordinated and comprehensive care to patients. IPC is becoming increasingly important as the healthcare system becomes more complex, and patients have increasingly complex needs (1, 2). For example, a systematic review and meta-analysis found that IPC was associated with improved patient outcomes in a range of areas, including chronic disease management, mental healthcare, and preventive care (3). IPC can also improve the efficiency and effectiveness of the healthcare system. Caroon et al. found that those systems which used an IPC model had lower costs per patient and higher rates of patient satisfaction (4). 2 INTERPROFESSIONAL EDUCATION IN PRIMARY CARE Despite the benefits of IPC, there are still several challenges to its implementation. One challenge is that healthcare professionals from different disciplines may have different training and backgrounds. This can make it difficult for them to communicate and collaborate effectively, and thus successful teamwork is an emerging educational topic (5). Each team member has a different set of skills and knowledge, and it is important for all team members to be working together effectively to provide the best possible care to patients (6). Team education can help to improve communication and collaboration among team members. It can also help to ensure that all team members are up to date on the latest best practices in primary care. Additionally, team education can help to create a more positive and supportive work environment for all team members. The specific benefits of team education in primary care include improved patient care, increased efficiency, reduced costs, and improved staff satisfaction (7). 3 EDUCATION WITH SIMULATIONS IN PRIMARY CARE The use of simulations in medical education has recently been added as a standard to the European Resuscitation Guidelines from 2021 (8). Similarly, the WHO (9) also said that health professionals’ education and training institutions should use simulations. 3.1 Team development Teaching with simulations enables team learning, which is essential for working harmoniously in real-life situations. Through learning with simulations, teams gain self-esteem and increased confidence in the team’s ability to handle a situation properly. Primary care teams that were involved in team education indicated that their management and 10.2478/sjph-2024-0001 Zdr Varst. 2024;63(1):1-4 2 confidence had improved, that they were better able to take a leadership role, give instructions and delegate tasks as required. They also valued the chance to train as a team (10). It appears that team training improves the performance of the resuscitation team in simulated emergency scenarios (11). 3.2 Safety 3.2.1 Safety of patient management The use of new training methods, such as classroom simulations with 3-dimensional highly realistic simulators (12), can provide comprehensive training in handling medical situations and identifying potentially dangerous medical situations that are usually not part of the daily work of primary care physicians and other healthcare workers (13). Medical professionals can refine their skills through repetitive practice on simulators, which can lead to better patient outcomes when they encounter real medical situations. Medical simulation prevents harm to real patients who might otherwise be subjected to unnecessary procedures, misdiagnoses, or inexperienced practitioners during training. Healthcare professionals can make mistakes in a safe environment during simulations, providing valuable learning opportunities without causing harm to patients. This fosters a culture of accountability and continuous improvement, aligning with ethical principles. 3.2.2 Psychological safety Stress is prevalent in the field of medicine, especially in critical and time-sensitive situations. Since stress can hinder the safe execution of tasks and the delivery of high-quality care, it is crucial to impart early education to health workers on stress management. With simulations, we can replicate actual patient scenarios that mirror real-life clinical settings (14), allowing for the assessment of different teams’ competencies in a controlled and standardised manner. This approach provides a secure and effective way to acquire the skills necessary to handle challenging, uncommon, or critical clinical situations. The scenarios are designed to be both standardised and adaptable, making it possible to tailor them to the trainees’ proficiency levels. This uniform and standardised training process enhances the quality of learning and eliminates the need for extensive on-site exposure over several years. By immersing participants in lifelike and demanding scenarios using highly realistic simulations, we foster experiential learning, enabling primary healthcare teams to engage with virtual patients. During these simulations, participants learn how to effectively manage stress and independently care for the patient. They can explore their emotions and fears in a safe environment, acquiring the tools to confront and conquer them (15). 10.2478/sjph-2024-0001 Zdr Varst. 2024;63(1):1-4 3 3.3 Ethical aspects Obtaining informed consent can be challenging when using real patients for training, especially for invasive or risky procedures. Simulations eliminate this ethical concern since they involve consenting individuals who are aware of the simulated nature of the scenario. Simulations also ensure the confidentiality of patients’ medical records and personal information, as no real patients are involved. This maintains patient privacy and adheres to ethical standards. Simulations can replicate rare or complex medical cases that medical students or professionals may not encounter frequently in their clinical practice. This exposure helps them build expertise in handling such cases ethically. Simulations provide the opportunity to create highly realistic, interactive environments in which students can learn experientially in ways that would not be ethical or appropriate in real life (16). 3.4 Equity Simulations can help ensure that all medical students and professionals have equal access to high-quality training experiences, regardless of their background or location. This promotes equity and fairness in medical education, aligning with ethical principles of justice. 4 CONCLUSION Successful implementation of IPC depends on several factors, one of which is also the need to educate primary care teams to work, despite the diversity of their members’ backgrounds and training (17). There is a need to implement interprofessional education, which fosters effective communication and collaboration among team members with varying skill sets and knowledge. One successful method of interprofessional education in primary care is education with simulations. Such education not only improves clinical skills but also enhances team dynamics and confidence, translating into better performance in real-life scenarios. Therefore, we suggest implementing interprofessional education with simulations in the form of an obligatory curriculum in the field of healthcare education, including that provided for undergraduates and postgraduates, as well as part of continuous professional development. CONFLICTS OF INTEREST The authors declare that no conflicts of interest exist. FUNDING This editorial was produced without external funding. ETHICAL APPROVAL Ethical approval is not required for this editorial. AVAILABILITY OF DATA AND MATERIALS Data sharing is not applicable to this article as no datasets were generated or analysed during the current study. ORCID Zalika Klemenc Ketiš: https://orcid.org/0000-0002-0270-1754 Uroš Zafošnik: https://orcid.org/0000-0001-9334-2947 REFERENCES 1. Rawlinson C, Carron T, Cohidon C, Arditi C, Hong QN, Pluye P, et al. An overview of reviews on interprofessional collaboration in primary care: Barriers and facilitators. Int J Integr Care. 2021;21(2):32. doi: 10.5334/ijic.5589. 2. Virtič T, Mihevc M, Zavrnik Č, Mori Lukančič M, Poplas Susič A, Klemenc-Ketiš Z. Peer support as part of scaling-up integrated care in patients with type 2 diabetes and arterial hypertension at the primary healthcare level: A study protocol. Zdr Varst. 2023;62(2):93-100. doi: 10.2478/sjph-2023-0013. 3. Lee JK, McCutcheon LRM, Fazel MT, Cooley JH, Slack MK. Assessment of interprofessional collaborative practices and outcomes in adults with diabetes and hypertension in primary care: A systematic review and meta-analysis. JAMA Netw Open. 2021;4(2):e2036725. doi: 10.1001/jamanetworkopen.2020.36725. 4. Carron T, Rawlinson C, Arditi C, Cohidon C, Hong QN, Pluye P, et al. An overview of reviews on interprofessional collaboration in primary care: Effectiveness. Int J Integr Care. 2021;21(2):31. doi: 10.5334/ ijic.5588. 5. Ramanayake RP, Ranasingha S, Lakmini S. Management of emergencies in general practice: Role of general practitioners. J Family Med Prim Care. 2014;3(4):305-308. doi: 10.4103/2249-4863.148089. 6. Eddy K, Jordan Z, Stephenson M. Health professionals’ experience of teamwork education in acute hospital settings: A systematic review of qualitative literature. JBI Database Syst Rev Implement Rep. 2016;14(4):96-137. doi: 10.11124/jbisrir-2016-1843. 7. Carney PA, Thayer EK, Palmer R, Galper AB, Zierler B, Eiff MP. The benefits of interprofessional learning and teamwork in primary care ambulatory training settings. J Interprofessional Educ Pract. 2019;15:119-126. doi: 10.1016/j.xjep.2019.03.011. 8. Perkins GD, Graesner JT, Semeraro F, Olasveengen T, Soar J, Lott C, et al. European Resuscitation Council guidelines 2021: Executive summary. Resuscitation. 2021;161:1-60. doi: 10.1016/j. resuscitation.2021.02.003. 9. Martins JCA, Baptista RCN, Coutinho VRD, Fernandes MID, Fernandes AM. Simulations in nursing and midwifery education. Copenhagen: WHO; 2018. 10. Strachan AN, Graham AC, Hormis AP, Hilton G. What were the perceptions of primary care teams on learning from a single multidisciplinary simulation-based training intervention? Educ Prim Care. 2011;22(4):229-234. doi: 10.1080/14739879.2011.11494005. 11. Murphy M, Curtis K, McCloughen A. What is the impact of multidisciplinary team simulation training on team performance and efficiency of patient care? An integrative review. Australas Emerg Nurs J. 2016;19(1):44-53. doi: 10.1016/j.aenj.2015.10.001. 12. Sharmin N, Chow AK, King S. Effect of teaching tools in spatial understanding in health science education: A systematic review. Can Med Educ J. 2023;14(4):70-88. doi: 10.36834/cmej.74978. 13. Klemenc-Ketis Z, Zafošnik U, Poplas Susič A. An innovative approach to educating primary health care teams about medical emergencies. Educ Prim Care. 2020;31(1):44-47. doi: 10.1080/14739879.2019.1691471. 14. Zhu E, Hadadgar A, Masiello I, Zary N. Augmented reality in healthcare education: An integrative review. Peer J. 2014;2:e469. doi: 10.7717/ peerj.469. 15. Klemenc-Ketis Z, Zafosnik U, Poplas Susic A. An innovative approach to educating primary health care teams about medical emergencies. Educ Prim Care. 2019:1-4. doi: 10.1080/14739879.2019.1691471. 16. Meredith C, Heslop P, Dodds C. Simulation: Social work education in a third place. Social Work Educ. 2023;42(6):917-934. doi: 10.1080/02615479.2021.1991908. 17. Smit LC, Dikken J, Moolenaar NM, Schuurmans MJ, de Wit NJ, Bleijenberg N. Implementation of an interprofessional collaboration in practice program: A feasibility study using social network analysis. Pilot Feasib Stud. 2021;7(1):7. doi: 10.1186/s40814-020-00746-3. 10.2478/sjph-2024-0001 Zdr Varst. 2024;63(1):1-4 4 BOTTOM-UP ANALYSIS OF TELEMONITORING COSTS: A CASE STUDY IN SLOVENIAN PRIMARY CARE Matic MIHEVC 1,2* , Črt ZAVRNIK 1,2 , Majda MORI LUKANČIČ 1 , Tina VIRTIČ POTOČNIK 1,3 , Marija PETEK ŠTER 2 , Zalika KLEMENC-KETIŠ 1,3 , Antonija POPLAS SUSIČ 1,2 1 Community Health Centre Ljubljana, Primary Healthcare Research and Development Institute, Metelkova ulica 9, 1000 Ljubljana, Slovenia 2 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia 3 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska ulica 8, 2000 Maribor, Slovenia Received: Sep 15, 2023 Accepted: Nov 23, 2023 Original scientific article *Correspondence: mihevc.matic@zd-tr.si 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 5 ANALIZA STROŠKOV TELEMONITORINGA OD SPODAJ NAVZGOR: ŠTUDIJA PRIMERA V SLOVENSKEM PRIMARNEM ZDRAVSTVU Mihevc M, Zavrnik Č, Mori Lukančič M, Virtič Potočnik T, Petek Šter M, Klemenc-Ketiš Z, Poplas Susič A. Bottom-up analysis of telemonitoring costs: A case study in Slovenian primary care. Zdr Varst. 2024;63(1):5-13. doi: 10.2478/Sjph-2024-0002. ABSTRACT Keywords: Costs Telemonitoring Primary care Elderly Diabetes Hypertension IZVLEČEK Ključne besede: stroški telemonitoring primarno zdravstveno varstvo starejši sladkorna bolezen hipertenzija Introduction: Telemonitoring improves clinical outcomes in patients with arterial hypertension (AH) and type 2 diabetes (T2D), however, cost structure analyses are lacking. This study seeks to explore the cost structure of telemonitoring for the elderly with AH and T2D in primary care and identify factors influencing costs for potential future expansions. Methods: Infrastructure, operational, patient participation, and out-of-pocket costs were determined using a bottom-up approach. Infrastructure costs were determined by dividing equipment and telemonitoring platform expenses by the number of participants. Operational and patient participation costs were determined by considering patient training time, data measurement/review time, and teleconsultation time. The change in out-of-pocket costs was assessed in both groups using a structured questionnaire and 12-month expenditure data. Statistical analysis employed an unpaired sample t-test, Mann-Whitney U test, and chi-square test. Results: A total of 117 patients aged 71.4±4.7 years were included in the study. The telemonitoring intervention incurred an annual infrastructure costs of €489.4 and operational costs of €97.3 (95% CI 85.7-109.0) per patient. Patient annual participation costs were €215.6 (95% CI 190.9-241.1). Average annual out-of-pocket costs for both groups were €345 (95% CI 221-469). After 12 months the telemonitoring group reported significantly lower out- of-pocket costs (€132 vs. €545, p<0.001), driven by reduced spending on food, dietary supplements, medical equipment, and specialist check-ups compared to the standard care group. Conclusion: To optimise the cost structure of telemonitoring, strategies like shortening the telemonitoring period, developing a national telemonitoring platform, using patient devices, integrating artificial intelligence into platforms, and involving nurse practitioners as telemedicine centre coordinators should be explored. Uvod: Telemonitoring predstavlja učinkovit pristop za izboljšanje urejenosti bolnikov z arterijsko hipertenzijo (AH) in sladkorno boleznijo (SB) tipa 2, vendar analize stroškovne strukture niso na voljo. Namen raziskave je raziskati stroškovno strukturo telemonitoringa pri starejših bolnikih z AH in SB tipa 2 v primarnem zdravstvenem varstvu in ugotoviti dejavnike, ki vplivajo na stroške za morebitne prihodnje širitve. Metode: S pomočjo pristopa od spodaj navzgor smo ocenili infrastrukturne in operativne stroške, stroške sodelovanja bolnikov in stroške iz žepa. Infrastrukturne stroške smo izračunali tako, da smo stroške nakupa telemedicinske opreme in spletne platforme delili s številom sodelujočih bolnikov. Operativne stroške in stroške sodelovanja bolnikov smo izračunali z upoštevanjem časa za usposabljanje bolnikov, časa za pregled/opravljanje meritev ter časa za telekonzultacije. Spremembe v stroških iz žepa smo ocenili s pomočjo strukturiranega vprašalnika, v katerem so bolniki v obeh skupinah poročali o stroških iz žepa v preteklem letu. Pri statistični analizi smo uporabili t-test za neparne vzorce, Mann-Whitneyev U test in hi-kvadrat test. Rezultati: V raziskavo je bilo vključenih 117 bolnikov, starih povprečno 71,4 ± 4,7 leta. Letni infrastrukturni stroški telemonitoringa so znašali 489,4 €, operativni stroški pa 97,3 € (95 % interval zaupanja [IZ] 85,7–109,0) na bolnika. Letni stroški sodelovanja bolnikov so znašali 215,6 € (95 % IZ 190,9–241,1). Povprečni letni stroški iz žepa za obe skupini so znašali 345 € (95 % IZ 221–469). Po 12 mesecih je skupina s telemonitoringom poročala o bistveno nižjih stroških iz žepa (132 € proti 545 €, p < 0,001), pri čemer so se pomembno zmanjšali stroški za hrano in prehranska dopolnila, medicinsko opremo in samoplačniške specialistične preglede. Zaključek: Za optimizacijo stroškovne strukture telemonitoringa je potrebno preučiti strategije, kot so skrajšanje obdobja telemonitoringa po stabilizaciji kliničnih parametrov, razvoj nacionalne platforme za spremljanje na daljavo z možnostjo prenosa mobilne aplikacije na osebne naprave bolnikov, vključevanje umetne inteligence v spletne platforme in povečanje vloge diplomirane medicinske sestre na mestu koordinatorja telemedicinskega centra. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION The escalating demands on global health systems resulting from the management of chronic diseases have underscored the need for innovative solutions. Arterial hypertension (AH) and type 2 diabetes (T2D) are among the most prevalent chronic conditions worldwide, with projections indicating a surge in their prevalence, particularly among the elderly (1-3). In response to the mounting burden of chronic diseases in primary care, telemonitoring has emerged as a practical solution. Telemonitoring involves using medical devices to collect real-time physiological data, such as blood pressure (BP) and blood glucose (BG), which is then directly transmitted to a telemonitoring centre, where it triggers a response from a healthcare provider, often complemented by a teleconsultation (3, 4). Previous studies reported that telemonitoring can effectively lower overall costs by reducing interaction time with healthcare professionals, preventing early health deterioration, reducing hospital admissions, cutting patient travel costs, and transferring specific elements of care from professionals to patients (4-6). To seamlessly integrate telemonitoring into healthcare systems, it is crucial to pinpoint and optimise factors that influence costs. A successful strategy for identifying these factors involves employing a bottom-up approach that considers both healthcare provider and societal (patient) perspectives. This method entails a thorough examination of individual cost components, starting from specific aspects within the healthcare provider’s realm and extending to the broader societal context. The provider perspective primarily focuses on medical costs, encapsulating telemonitoring technology infrastructure and operational expenses. Simultaneously, the societal perspective encompasses more extensive effects, including indirect and non-health-related costs such as patient and caregiver time, out-of-pocket costs, and productivity loss (7, 8). Slovenia, a high-income country in central Europe with the Bismarck healthcare model, has made significant efforts to implement an integrated care package for individuals with AH and T2D in primary care settings (9). However, the national-scale implementation of telemonitoring in Slovenia is still pending despite its prior evaluation in pilot studies (10-13). This delay could be attributed to the absence of comprehensive clinical impact and cost analyses, which would enable decision-makers to extend financial support towards telemonitoring initiatives. To address this gap, we have designed a pilot multicentre randomised controlled trial (3) aimed at assessing the feasibility, acceptability, and clinical effectiveness of telemonitoring for older people with AH and T2D in 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 6 primary care in Slovenia. The aim of this sub-study is to explore the cost structure of telemonitoring for the elderly with AH and T2D in primary care and identify factors influencing costs for potential future expansions. 2 METHODS 2.1 Study design Upon completion of the 12-month follow-up period, we conducted a cross-sectional survey of patients who participated in the multicentre randomised controlled trial (3) between March 2021-March 2022 and May 2022- May 2023 as part of the SCUBY international project. 2.2 Study setting The study took place in three primary health centres (PHCs) in Slovenia. PHC Ljubljana represented the urban population, while the peripheral PHCs of Trebnje and Slovenj Gradec represented the rural population. 2.3 Study population and sampling strategy The study included patients aged 65 years or older who had both AH and T2D. Participants were conveniently sampled, as they were invited to take part in the study by their general practitioners (GPs). Once they agreed, they were randomly assigned in a 1:1 ratio to either telemonitoring or standard care groups. The inclusion criteria were: (a) ≥65 years of age, (b) confirmed diagnosis of AH and T2D for at least one year, and (c) the ability to use telemonitoring equipment. The exclusion criteria were: (a) <65 years of age, (b) T2D requiring insulin treatment, (c) gestational diabetes or type 1 diabetes, (d) cognitive impairment, or (e) an inability to use telemonitoring equipment for any reason. 2.4 Telemonitoring intervention We supplemented standard care with telemonitoring intervention (3). Participants were provided with a telemonitoring package including a smartphone and monitors for BP and BG. Over a 12-month period, participants were instructed to measure their BP twice a week and their BG once a month, with a more intensive regimen in case of derailments. The results were transmitted to a telemedicine platform for review by a telemedicine centre coordinator (GP). Patient management followed established clinical protocols, and if necessary the coordinator communicated with patients or their GPs through a mobile app or phone, providing additional health analysis. 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 7 2.5 Cost structure assessment and data collection Considering prior research on the cost dynamics in telemonitoring in patients with AH or T2D (14-22), we gathered data encompassing both the healthcare provider’s and patient’s perspectives (Table 1). Infrastructure costs were determined by the third-party telemonitoring solution provider and were independent of this study. Operational costs and patient participation costs were extracted from telemedicine platform data and medical records. Out-of-pocket costs were explored through a structured questionnaire, capturing self- reported expenses from the past year. 2.6 Data analysis In conducting the cost analysis, we employed a bottom- up approach, considering perspectives from both the healthcare provider and patient/society (7, 8). Healthcare provider Healthcare provider Patient or society Patient or society Costs for obtaining necessary devices Ongoing costs for device functionality and long service life Costs of spreading equipment cost over its service life Costs covering technology infrastructure, data security, and user support for telemonitoring and video calls Initial costs for patient training to ensure proper device usage Labour costs for the analysis of patient data Labour costs for conducting teleconsultations Costs associated with training in device usage Costs for BP and BG measurement time Costs for teleconsultation sessions Costs related to travel Costs for dietary needs Costs for physical activity Costs for educational materials or programmes Costs for rehabilitation or physiotherapy Costs for non-covered or quicker checkups at private institutions Costs for monitors and BG strips Costs for specialised diabetes footwear Infrastructure costs Operational costs Patient participation costs Change in out-of- pocket costs Equipment acquisition Equipment maintenance Equipment depreciation Telemedicine platform subscription Patient training investment Cost of data review by coordinator Teleconsultation costs Training time costs Measurements time costs Teleconsultation time costs Transportation and parking Food and dietary supplement Exercise and fitness Education Rehabilitative services Out-of-pocket checkups Medical devices Customised footwear DefinitionCategory Cost subcategoryPerspective Table 1. Breakdown of telemonitoring costs from various perspectives. Legend: BP – blood pressure; BG – blood glucose The infrastructure costs per patient were determined by dividing the total expenses incurred for equipment acquisition, maintenance, depreciation, and subscription to the telemedicine platform by the total number of participants. The operational costs per patient were determined by considering training time, data review time, and teleconsultation time. Training time was calculated by dividing the total training time by the number of patients. Time spent on data review was calculated by multiplying the average number of measurements per patient per year by the interpretation time for each measurement. Teleconsultation time was calculated by multiplying the average number of teleconsultations per patient per year by the average teleconsultation duration. Each cost subcategory’s average time was then multiplied by the corresponding gross hourly values for the service provider. Hourly rates for the nurse practitioner and GP were calculated using the rates agreed in the General Agreement of the Slovenian Health Insurance Institute for the year 2022 (23). Patient participation costs were computed based on training time, measurement time, and teleconsultation time, using the previously mentioned principles. Hourly rates for patients were calculated using the average gross salary data for Slovenia in 2022 (24). The choice of using average gross salary data for our population was deliberate. Assessing the value of time for retired individuals is complex due to the diversity of their activities, and there was a small minority of people who were still working. Furthermore, previous studies have predominantly centred around the working population, making the adoption of average gross salary data a strategic decision to ensure comparability of our results with existing research (5). Statistical analysis was performed using IBM SPSS Statistics for Windows, version 25.0 (IBM Corp, Armonk, NY). The distributional characteristics of the samples were assessed using the Shapiro-Wilk test. Socio- demographic characteristics were found to follow a normal distribution, while out-of-pocket costs followed a non-normal distribution. Differences between groups were examined using t-tests for numerical variables with a normal distribution and Mann-Whitney U-test for variables with a non-normal distribution. Categorical variables were analysed with the chi-square test. Statistical significance was defined as a p-value <0.05. 3 RESULTS 3.1 Randomisation process The randomisation process is presented in Figure 1. A total of 128 patients were randomised to either telemonitoring or standard care groups, of whom 120 (93.8%) attended follow-up visit at 6 months, and 117 (91.4%) at 12 months. Figure 1. CONSORT flow diagram illustrating the randomisation process. 3.2 Socio-demographic and clinical data The study comprised 117 participants, with an average age of 71.4±4.7 years, of whom 60.7% were male. Most of the patients had received primary or vocational school education, were married, and had slightly elevated BP and HbA1c values. There were no significant differences between groups in terms of their socio-demographic and clinical characteristics (Table 2). Age, years, mean (SD) Sex Male (N) Female (N) Highest education achieved Primary school (N) Vocational school (N) High school (N) Bachelor’s degree (N) Master’s degree (N) Region Urban (N) Rural (N) 71.4±4.7 71 46 18 69 16 10 4 54 63 70.6±4.3 34 21 7 33 10 3 2 27 28 72.0±5.0 37 25 11 36 6 7 2 27 35 0.123 0.813 0.518 0.548 All patients (N=117) Telemonitoring (N=55) Standard care (N=62) PVariable Table 2. Baseline socio-demographic and clinical data. 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 8 Patient training Data review Teleconsultations Mean annual total gross costs (EUR, 95% CI) Marital status Married (N) Divorced (N) Widowed (N) Single (N) Clinical data Duration of type 2 diabetes, years, mean (SD) Duration of hypertension, years, mean (SD) Systolic blood pressure, mmHg, mean (SD) Diastolic blood pressure, mmHg, mean (SD) Glycated haemoglobin, %, mean (SD) 82 (75-89) 201 (178-225) 38 (31-44) / 86 8 5 18 9.5±7.6 14.5±10.6 136.7±14.1 76.5±8.2 7.2±1.0 10.3 20.9 20.9 / 41 2 2 10 9.8±6.3 13.7±10.6 135.8±14.9 75.6±7.0 7.2±1.2 14.1 (12.9-15.3) 70.0 (62.0-78.4) 13.2 (10.8-15.3) 97.3 (85.7-109.0) 45 6 3 8 9.2±8.6 15.2±10.6 137.7±13.1 77.2±9.0 7.1±0.8 0.527 0.672 0.667 0.458 0.280 0.411 Mean annual time per action (min, 95% CI) All patients (N=117) Gross value per hour (EUR) Telemonitoring (N=55) Mean annual gross costs per patient (EUR, 95% CI) Standard care (N=62) P Cost subcategory Variable Table 3. Breakdown of operational costs per patient. Legend: N – number; SD – standard deviation Legend: min – minutes; 95% CI – 95% confidence interval; EUR – euros 3.2 Costs of telemonitoring intervention 3.2.1 Infrastructure costs The infrastructure costs related to equipment acquisition, maintenance, depreciation, and telemedicine platform subscription totalled €29,361 for 60 users. This equates to an annual cost of €489.4 per patient. 3.2.2 Operational costs Operational costs, as delineated in Table 3, comprised patient training costs, costs of data review, and teleconsultations costs. The average time for initial patient training was 82 minutes (95% CI 75-89). The telemedicine centre coordinator, who spent an average of 1 minute on each measurement interpretation, spent 201 minutes (95% CI 178-225) per year reviewing data per patient. Teleconsultations, which lasted an average of 15 minutes and took place 2.5 times per patient per year (95% CI 2.0-2.9), contributed to an average of 38 minutes (95% CI 31-44) of teleconsultation time per patient per year. Considering the costs of nurse training, data review, and teleconsultations, the average annual operational costs per patient were €97.3 (95% CI 85.7-109.0). 3.2.3. Patient participation costs Patient participation costs consisted of training time costs, measurement time costs, and teleconsultation time costs (Table 4). 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 9 Patient training Data review Teleconsultations Mean annual total gross costs (EUR, 95% CI) Transportation and parking (EUR) Food and dietary supplements (EUR) Exercise and fitness (EUR) Education (EUR) Rehabilitative services (EUR) Out-of-pocket checkups (EUR) Medical devices (EUR) Customised footwear (EUR) Miscellaneous (EUR) Mean annual total costs (EUR, 95% CI) 82 (75-89) 1,005 (890-1,125) 38 (31-44) / 27 (0-1,200) 151 (0-2,400) 38 (0-1,440) 1 (0-60) 20 (0-840) 23 (0-900) 56 (0-1,200) 6 (0-180) 23 (0-1,200) 345 (221-469) 11.5 11.5 11.5 / 9 (0-180) 83 (0-2,400) 20 (0-480) 0 (0-0) 0 (0-0) 0 (0-0) 9 (0-72) 2 (0-50) 9 (0-480) 132 (32-231) 15.7 (14.4-17.1) 192.6 (170.6-215.6) 7.3 (5.9-8.4) 215.6 (190.9-241.1) 43 (0-1,200) 214 (0-2,400) 56 (0-1,440) 1 (0-60) 39 (0-840) 45 (0-900) 101 (0-1,200) 10 (0-180) 36 (0-1,200) 545 (332-757) 0.149 0.015 0.784 0.334 0.050 0.016 <0.001 0.198 0.972 <0.001 Mean annual time per action (min, 95% CI) All patients (N=117), mean (min, max) Gross value per hour (EUR) Telemonitoring (N=55), mean (min, max) Mean annual gross costs per patient (EUR, 95% CI) Standard care (N=62), mean (min, max) P Cost subcategory Cost subcategory Table 4. Table 5. Breakdown of participation costs per patient. Comparison of out-of-pocket costs across groups. Legend: N – number; 95% CI – 95% confidence interval; min – minimum; max – maximum; EUR – euros Legend: min – minutes; 95% CI – 95% confidence interval; EUR – euros Patients took an average of 5 minutes for a single BP or BG measurement. On average, patients performed 140 (95% 122-158) BP measurements per year and 61 (95% CI 49-72) BG measurements per year, for a total of 201 (95% CI 178- 225) measurements per patient per year. This resulted in an average of 1,005 (95% CI 890-1125) minutes spent on measurements per patient per year. When considering costs for training, measurements, and teleconsultations, the average annual patient participation costs per patient were €215.6 (95% CI 190.9-241.1). 3.2.4 Change in out-of-pocket costs The average annual self-reported out-of-pocket costs for patients in both groups at the end of the 12-month follow- up period were €345 (95% CI 221-469). The telemonitoring group exhibited significantly lower costs compared to the standard care group (€132 vs. €545, p <0.001). Specifically, the telemonitoring group reported reduced expenses for food and dietary supplements, personal payments for specialist checkups, and the acquisition of medical devices (Table 5). 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 10 4 DISCUSSION In Slovenian primary care, the use of telemonitoring interventions for individuals with AH and T2D yielded notable economic benefits. The annual infrastructure costs per patient were €489.4, coupled with operational costs of €97.3 (95% CI 85.7-109.0). Additionally, patient participation costs were €215.6 (95% CI 190.9-241.1) annually, demonstrating the multifaceted financial benefits of telemonitoring. Importantly, the telemonitoring group exhibited a significant 12-month reduction in out-of-pocket costs compared to the standard care group, showcasing the potential economic benefits of our intervention (€132 vs. €545). Previous studies have revealed varying telemonitoring costs, influenced by factors such as healthcare systems, hourly rates, and intervention intensity. For instance, a Canadian study in 2019 reported BP telemonitoring costs of €279 for the first three months and €300 annually for the next 20 years (16). In a European context the rates were higher, such as €2,104 in the United Kingdom (5), €4,859 in the Netherlands (21), and €1,962 in Italy (22). However, previous studies often required more frequent interactions between patients and healthcare workers, with operational costs being the primary driver and infrastructure costs accounting for only about one-third of the total costs (9, 16). In our study, infrastructure costs emerged as the principal cost driver. To optimise these, we propose establishing a national or institutional telemonitoring platform. This platform, complete with the user-friendly mHealth application, has the potential to reduce additional expenses related to mobile phones and service subscriptions (4, 12). However, the feasibility of this approach hinges on factors such as the existing infrastructure, technological readiness, data protection, financial resources, and national healthcare system priorities (4, 12, 25). The operational costs in our study were predominantly driven by data review and teleconsultation costs led by GPs. Patients exceeded the expected number of BP and BG measurements by 46.5% and 252.8%, respectively. This is an important finding, as patients voluntarily continued to take measurements even when not required, especially for BG, driving up operational costs. To address this issue, we suggest shortening the telemonitoring interval to six months, when clinical parameters stabilise (3, 12), and conducting nurse follow-up meetings every three months after the telemonitoring period to maintain the self-management behaviour learned through telemonitoring (4, 12). Additionally, the delegation of measurement interpretation to nurses (12, 26) or the introduction of artificial intelligence for automated responses (27) could significantly lower operational costs. As nurse practitioners gain expertise, they could gradually handle teleconsultations, allowing GPs to focus primarily on making any changes to therapy that are needed (4, 12, 26). Due to our focus on an elderly demographic, we specifically calculated patient participation costs, omitting the impractical comparison of productivity losses. In our study the patients devoted 18.8 hours (95% CI 16.6-21.0) annually to participate in telemonitoring. This is less than in previous studies where patients reported spending 10- 12 hours per year on self-monitoring BP and 13-46 hours on self-monitoring BG (28, 29), and more than the expected 12 hours based on our measurement protocol (3). Implementing a less intensive BP measurement protocol (i.e., once a week) (12) and shortening the telemonitoring interval could further optimise patient participation costs. One critical aspect from a societal perspective is the impact of telemonitoring on out-of-pocket costs. Previous studies found that these payments made up almost one quarter of all AH and T2D treatment costs in Slovenia (30, 31). In our study, both groups had an average annual out-of-pocket cost of €345, with the telemonitoring group reporting significantly lower expenses after a 12-month follow-up. The reduction in medical device costs was in line with expectations, as telemonitored patients received BP and BG monitors with BG test strips, while decreases in expenses for food, dietary supplements, and specialist check-ups could be associated with improved patient education received during training and teleconsultations (32). Notably, there was no significant reduction in self-reported transportation costs. Older individuals in the intervention group continued regular preventative activities, including visits to their GPs for other health consultations. This was deemed essential for ethical reasons, given the study population’s various associated health conditions. However, a more restrictive approach might prove feasible and efficient in younger populations with isolated AH or T2D (17-19). The strength of this study lies in its integration into a randomised controlled trial with elderly participants from diverse backgrounds, an underexplored demographic in telemedicine research. Nevertheless, limitations include a small sample size and the inclusion of motivated participants, potentially limiting generalisability. Moreover, we only examined the change in out-of-pocket costs at the end of a 12-month period, while a baseline assessment should be performed to compare groups and verify the results. Additionally, there were costs related to educating GPs and registered nurses on the proper use of the telemedicine platform and devices. Given that this was a one-time expense that fell significantly with increased patient volume, we have excluded it from our calculations for clarity. In future research, it is advisable to estimate costs associated with unpaid caregiver time and delve into further medical aspects of telemonitoring’s cost-saving potential, encompassing the prevention of secondary complications and hospital admissions. 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 11 5 CONCLUSION In conclusion, this study provides valuable insights into the cost structure of integrating telemonitoring into established clinical pathways for older people with AH and T2D in primary care. To optimise the cost structure of telemonitoring, strategies like shortening the telemonitoring period, developing a national telemonitoring platform, using patient devices, integrating artificial intelligence into platforms, and involving nurse practitioners as telemedicine centre coordinators should be explored. Future research should build on these findings, testing new models and estimating the savings resulting from telemonitoring to provide evidence-based insights into the economic impact of telemonitoring in primary care. CONFLICTS OF INTEREST The authors declare that no conflicts of interest exist. FUNDING The research is financially supported by the SCUBY project, an international research initiative co-financed by the European Union through the H2020 – Health Programme (H2020-SC1) and identified by contract number 825432 – SCUBY. ETHICAL APPROVAL AND REGISTRATION The study was approved by the National Medical Ethics Committee of Slovenia (approval number: 0120-219/2019/4) and is registered in the ISRCTN registry (https://doi. org/10.1186/ISRCTN31471852). AVAILABILITY OF DATA AND MATERIAL The datasets can be obtained from the corresponding author upon a reasonable request. ORCID Matic Mihevc: https://orcid.org/0000-0003-4041-8682 Črt Zavrnik: https://orcid.org/0000-0003-0654-452X Majda Mori Lukančič: https://orcid.org/0009-0004-8507-833X Tina Virtič Potočnik: https://orcid.org/0000-0003-4925-3706 Marija Petek Šter: https://orcid.org/0000-0003-1736-2377 Zalika Klemenc-Ketiš: https://orcid.org/0000-0002-0270-1754 Antonija Poplas Susič: https://orcid.org/0000-0002-4328-3333 REFERENCES 1. Zhou B, Carrillo-Larco RM, Danaei G, Riley LM, Paciorek CJ, Stevens GA, et al. Worldwide trends in hypertension prevalence and progress in treatment and control from 1990 to 2019: A pooled analysis of 1201 population-representative studies with 104 million participants. Lancet. 2021;398(10304):957-980. doi: 10.1016/S0140-6736(21)01330-1. 2. Ong KL, Stafford LK, Mclaughlin SA, Boyko EJ, Vollset SE, Smith AE, et al. Global, regional, and national burden of diabetes from 1990 to 2021, with projections of prevalence to 2050: A systematic analysis for the Global Burden of Disease Study 2021. Lancet. 2023;402(10397):203– 234. doi: 10.1016/S0140-6736(23)01301-6. 3. Mihevc M, Zavrnik Č, Mori Lukančič M, Virtič T, Prevolnik Rupel V, Petek Šter M, et al. Telemonitoring of elderly with hypertension and type 2 diabetes at the primary care level: Protocol for a multicentric randomized controlled pilot study. Zdr Varst. 2022;61(4):216-223. doi: 10.2478/sjph-2022-0029. 4. Mihevc M, Podgoršek D, Gajšek J, Mikuletič S, Homar V, Kolšek M, et al. The rise of telemedicine in primary care: Understanding patients’ and healthcare workers’ perspectives on acceptability of the COVID-19 remote care model. Fam Med Prim Care Rev. 2023;25(3):297-301. doi: 10.5114/fmpcr.2023.130091. 5. Snoswell CL, Taylor ML, Comans TA, Smith AC, Gray LC, Caffery LJ. Determining if telehealth can reduce health system costs: Scoping review. J Med Internet Res. 2020;22(10):e17298. doi: 10.2196/17298. 6. Henderson C, Knapp M, Fernandez J-L, Beecham J, Hirani SP, Cartwright M, et al. Cost effectiveness of telehealth for patients with long term conditions (whole systems demonstrator telehealth questionnaire study): Nested economic evaluation in a pragmatic, cluster randomised controlled trial. BMJ. 2013;346:f1035. doi: 10.1136/bmj.f1035. 7. Kim DD, Silver MC, Kunst N, Cohen JT, Ollendorf DA, Neumann PJ. Perspective and costing in cost-effectiveness analysis, 1974-2018. Pharmacoeconomics. 2020;38(10):1135-1145. doi: 10.1007/s40273-020- 00942-2. 8. Garrison LP, Pauly MV, Willke RJ, Neumann PJ. An Overview of value, perspective, and decision context - A health economics approach: An ISPOR special task force report [2]. Value Health. 2018;21(2):124-130. doi: 10.1016/j.jval.2017.12.006. 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 12 9. Klemenc-Ketiš Z, Stojnić N, Zavrnik Č, Ružić Gorenjec N, Danhieux K, Lukančič MM, et al. Implementation of integrated primary care for patients with diabetes and hypertension: A case from Slovenia. Int J Integr Care. 2021;21(3):15. doi: 10.5334/ijic.5637. 10. Rudel D, Slemenik-Pušnik C, Epšek-Lenart M, Pušnik S, Balorda Z, Lavre J. Telemedicine support to patients with chronic diseases for better long-term control at home. Zdrav Vestn. 2017;85:676-685. doi: 10.6016/ZdravVestn.1553. 11. Iljaž R, Brodnik A, Zrimec T, Cukjati I. E-healthcare for diabetes mellitus type 2 patients – a randomised controlled trial in Slovenia. Zdr Varst. 2017;56(3):150-157. doi: 10.1515/sjph-2017-0020. 12. Mihevc M, Puntar Š, Petek Šter M. Telemonitoring of patients with arterial hypertension and/or type 2 diabetes in a family medicine practice: Results of a mixed-methods pilot project. Med Razgl. 2023;62(1):3-16. 13. Munda A, Mlinaric Z, Jakin PA, Lunder M, Pongrac Barlovic D. Effectiveness of a comprehensive telemedicine intervention replacing standard care in gestational diabetes: A randomized controlled trial. Acta Diabetol. 2023;60(8):1037-1044. doi: 10.1007/s00592-023-02099-8. 14. Warren R, Carlisle K, Mihala G, Scuffham PA. Effects of telemonitoring on glycaemic control and healthcare costs in type 2 diabetes: A randomised controlled trial. J Telemed Telecare. 2018;24(9):586-595. doi: 10.1177/1357633X17723943. 15. Bohingamu Mudiyanselage S, Stevens J, Watts JJ, Toscano J, Kotowicz MA, Steinfort CL, et al. Personalised telehealth intervention for chronic disease management: A pilot randomised controlled trial. J Telemed Telecare. 2019;25(6):343-352. doi: 10.1177/1357633X18775850. 16. Padwal RS, So H, Wood PW, Mcalister FA, Siddiqui M, Norris CM, et al. Cost-effectiveness of home blood pressure telemonitoring and case management in the secondary prevention of cerebrovascular disease in Canada. J Clin Hypertens (Greenwich). 2019;21(2):159-168. doi: 10.1111/jch.13459. 17. Zhang Y, Peña MT, Fletcher LM, Lal L, Swint JM, Reneker JC. Economic evaluation and costs of remote patient monitoring for cardiovascular disease in the United States: A systematic review. Int J Technol Assess Health Care. 2023;39(1):e25. doi: 10.1017/S0266462323000156. 18. Monahan M, Jowett S, Nickless A, Franssen M, Grant S, Greenfield S, et al. Cost-effectiveness of telemonitoring and self-monitoring of blood pressure for antihypertensive titration in primary care (TASMINH4). Hypertension. 2019;73(6):1231-1239. doi: 10.1161/ HYPERTENSIONAHA.118.12415. 19. Kaambwa B, Bryan S, Jowett S, Mant J, Bray EP, Hobbs FD, et al. Telemonitoring and self-management in the control of hypertension (TASMINH2): A cost-effectiveness analysis. Eur J Prev Cardiol. 2014;21(12):1517-1530. doi: 10.1177/2047487313501886. 20. Odnoletkova I, Ramaekers D, Nobels F, Goderis G, Aertgeerts B, Annemans L. Delivering diabetes education through nurse- led telecoaching: Cost-effectiveness analysis. PLoS One. 2016;11(10):e0163997. doi: 10.1371/journal.pone.0163997. 21. Greving JP, Kaasjager HAH, Vernooij JWP, Hovens MMC, Wierdsma J, Grandjean HMH, et al. Cost-effectiveness of a nurse-led internet- based vascular risk factor management programme: Economic evaluation alongside a randomised controlled clinical trial. BMJ Open. 2015;5(5):e007128. doi: 10.1136/bmjopen-2014-007128. 22. Zanaboni P, Landolina M, Marzegalli M, Lunati M, Perego GB, Guenzati G, et al. Cost-utility analysis of the EVOLVO study on remote monitoring for heart failure patients with implantable defibrillators: Randomized controlled trial. J Med Internet Res. 2013;15(5):e106. doi: 10.2196/jmir.2587. 23. Health Insurance Institute of Slovenia. General agreement for contract year 2022 [Internet]. 2022 [cited 2023 Sept 3]. Available from: https:// www.zzzs.si/ ?id=126&detail=8B561838D2A02999C12587D600415B4B 24. Statistical Office of the Republic of Slovenia. Average gross salary data for Slovenia in 2022 [Internet]. 2022 [cited 2023 Sept 3]. Available from: https://www.stat.si/StatWeb/News/Index/10891 25. Osei E, Mashamba-Thompson TP. Mobile health applications for disease screening and treatment support in low-and middle-income countries: A narrative review. Heliyon. 2021;7(3):e06639. doi: 10.1016/j.heliyon.2021.e06639. 26. Powers MA, Bardsley JK, Cypress M, Funnell MM, Harms D, Hess-Fischl A, et al. Diabetes self-management education and support in adults with type 2 diabetes: A consensus report of the American Diabetes Association, the Association of Diabetes Care & Education Specialists, the Academy of Nutrition and Dietetics, the American Academy of Family Physicians, the American Academy of PAs, the American Association of Nurse Practitioners, and the American Pharmacists Association. Diabetes Care. 2020;43(7):1636-1649. doi: 10.2337/dci20- 0023. 27. Makroum MA, Adda M, Bouzouane A, Ibrahim H. Machine learning and smart devices for diabetes management: Systematic review. Sensors (Basel). 2022;22(5):1843. doi: 10.3390/s22051843. 28. Chernyak N, Jülich F, Kasperidus J, Stephan A, Begun A, Kaltheuner M, et al. Time cost of diabetes: Development of a questionnaire to assess time spent on diabetes self-care. J Diabetes Complications. 2017;31(1):260-266. doi: 10.1016/j.jdiacomp.2016.06.016. 29. Icks A, Haastert B, Arend W, Konein J, Thorand B, Holle R, et al. Time spent on self-management by people with diabetes: Results from the population-based KORA survey in Germany. Diabet Med. 2019;36(8):970-981. doi: 10.1111/dme. 30. Prevolnik Rupel V, Mori Lukančič M, Ogorevc M. Costs and quality of life in patients with systemic arterial hypertension in Slovenia. Value Health Reg Issues. 2023;33:49-55. doi: 10.1016/j.vhri.2022.09.001. 31. Prevolnik Rupel V, Ogorevc M, Mori Lukančič M, Poplas Susič A. Costs and quality of life in patients with type 2 diabetes. In: Proceedings of the 1st International Scientific Conference of Primary Care: Interprofessional teamwork and quality in healthcare. Ljubljana: Community Health Centre Ljubljana; 2021:1-6. 32. Tur-Sinai A. Out-of-Pocket expenditure on medical services among older adults: A longitudinal analysis. Front Public Health. 2022;10:836675. doi: 10.3389/fpubh.2022.836675. 10.2478/sjph-2024-0002 Zdr Varst. 2024;63(1):5-13 13 VALIDATION OF THE SLOVENE VERSION OF THE STOP-BANG QUESTIONNAIRE IN A PRIMARY PRACTICE SETTING Andrej PANGERC 1* , Marija PETEK ŠTER 1 , Leja DOLENC GROŠELJ 2,3 1 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia 2 University Medical Centre Ljubljana, Institute of Clinical Neurophysiology, Division of Neurology, Zaloška cesta 7, 1000 Ljubljana, Slovenia 3 University of Ljubljana, Faculty of Medicine, Department of Neurology, Zaloška cesta 2, 1000 Ljubljana, Slovenia Received: Oct 06, 2023 Accepted: Nov 27, 2023 Original scientific article *Correspondence: andrej.pangerc@gmail.com 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 14 VALIDACIJA SLOVENSKE RAZLIČICE VPRAŠALNIKA STOP-BANG ZA UPORABO V AMBULANTAH NA PRIMARNEM NIVOJU ZDRAVSTVENEGA VARSTVA Pangerc A, Petek Šter M, Dolenc Grošelj.L Validation of the Slovene version of the STOP-BANG questionnaire in a primary practice setting. Zdr Varst. 2024;63(1):14-20. doi: 10.2478/Sjph-2024-0003. ABSTRACT Keywords: Family medicine Obstructive sleep apnea Primary practice setting STOP-BANG questionnaire Validation IZVLEČEK Ključne besede: družinska medicina obstruktivna apneja v spanju primarno zdravstveno varstvo vprašalnik STOP-BANG validacija Aim: The aim of our study was to validate the Slovene translation of the STOP-BANG (SBQ) questionnaire for use in the primary practice setting. Methods: We recruited 158 randomly selected visitors at four primary practice clinics who came to the practice for any reason. Participants completed the Slovene SBQ and underwent type 3 respiratory polygraphy, which was analysed by an experienced somnologist. The SBQ was previously translated in to Slovene and validated for the sleep clinic. Results: Of 158 participants, 153 had valid recordings. The mean age of the participants was 49.5 years (±13.0 years), and 47.7% were male. OSA was identified in 49.0% of the participants. The questionnaire, with a cutoff of ≥3, demonstrated an area under the curve of 0.823 for any OSA (REI≥5), 0.819 for moderate and severe OSA (REI≥15) and 0.847 for severe OSA (REI≥30). Sensitivity was 65.3%, 81.8%, and 90.0%, and specificity was 87.2%, 73.3% and 65.0% for any, moderate to severe and severe OSA, respectively. Conclusions: The Slovene translation of the SBQ is a reliable instrument for OSA risk stratification in the primary practice setting. Namen: Namen naše študije je bil preveriti veljavnost slovenskega prevoda vprašalnika STOP-BANG (SBQ) za uporabo v ambulantah na primarni ravni zdravstvenega varstva. Metode: Naključno smo izbrali 158 obiskovalcev v štirih ambulantah družinske medicine, ki so tja prišli iz kateregakoli razloga. Udeleženci so izpolnili slovensko različico SBQ in doma opravili respiratorno poligrafijo tipa 3, katero je analiziral izkušen somnolog. SBQ je bil v Slovenščino že preveden in validiran za uporabo v laboratorijih za motnje spanja. Rezultati: Od 158 udeležencev jih je 153 imelo veljavne posnetke. Povprečna starost preiskovancev je bila 49,5 leta (± 13,0 leta); 47,7 % jih je bilo moških. OSA je bila identificirana pri 49,0 % preiskovancev. Vprašalnik z mejno vrednostjo ≥ 3 je pokazal površino pod krivuljo 0,823 za katerokoli OSA (REI ≥ 5), 0,819 za zmerno in hudo OSA (REI ≥ 15) in 0,847 za hudo OSA (REI ≥ 30). Občutljivost je bila 65,3 %, 81,8 % in 90,0 %, specifičnost pa 87,2 %, 73,3 % in 65,0 % za katerokoli, zmerno do hudo in hudo OSA odnosno. Zaključek: Slovenski prevod vprašalnika STOP-BANG je zanesljivo orodje za stratifikacijo tveganja za OSA na primarnem nivoju zdravstvenega varstva. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Obstructive sleep apnea (OSA) is the most common sleep related respiratory disorder (1) and a standalone risk factor for various clinical conditions, such as hypertension, stroke, depression and diabetes (2). Furthermore, OSA constitutes a notable contributor to motor vehicle accidents (1). It is linked to a rise in overall mortality rates, especially attributed to coronary artery disease (1, 3). In order to establish a diagnosis, a sleep study is required. There are several types of sleep study available, and the gold standard is a traditional laboratory type 1 polysomnography (PSG). Increasingly, however, home- based type 3 polygraphy (PG) is being used as it is easier to perform, cheaper, and more mobile (4, 5). The various types of sleep studies are listed in Table 1. The result of such a study is expressed by the apnea- hypopnea index (AHI) for PSG and respiratory event index (REI) for PG, which indicate the average number of apneas and hypopneas per hour of sleep or bedrest, respectively (6). Based on the number of these events, OSA is categorised as mild (5≤AHI/REI<15), moderate (15≤AHI/REI<30) and severe (AHI/REI≥30) (7). Estimates suggest that nearly 1 billion adults aged 30– 69 years worldwide could have OSA (8). Roughly 80% of individuals experiencing moderate-to-severe OSA are believed to remain undiagnosed (7). 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 15 Sleep studies are time consuming, labour intensive, and can be costly (9, 10). For this purpose several risk stratification questionnaires have been developed in order to assess the pretest probability of OSA (5). Such screening methods have become important, especially in primary care (11). The STOP-BANG questionnaire (SBQ) was developed as a preoperative screening tool for OSA (12). Due to its practicality and high sensitivity, the SBQ has been validated in surgical and sleep clinic settings worldwide. However, its validity has been explored to a much lesser extent in the general population and in primary care settings (13-15). We have previously published the details pertaining to the translation, adaptation, test-retest reliability, and internal consistency as well as the validation of the Slovene SBQ in a sleep laboratory setting (16). The aim of the current study was to validate the Slovene version of the SBQ in a primary practice setting. 2 METHODS 2.1 Study design and setting We conducted a cross-sectional study which took place in four family medicine practices in Slovenia. Primary practice physicians (specialists in family medicine) recruited patients by using a randomisation protocol. This protocol was based on randomly generated numbers and would select one out of the first ten patients to visit the practice on a given day. If the doctor wished, he or she could invite additional consecutive patients on the same day. Location At home Sleep disorder laboratory (sleep lab) Under real-time technician supervision Channels Chest movement Snoring Airflow Arterial blood oxygen saturation Heart rate Electroencephalography Electromyography Electrooculography Electrocardiography - ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ - - ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ - - ✓ ✓ ✓ ✓ ✓ - - - - ✓ - - - - - ✓ ✓ - - - - Type 1 PSG Type 2 PSG Type 3 PG PG Type 4 PG Table 1. Types of sleep studies. Legend: PSG = polysomnography, PG = polygraphy Adapted from: Patil SP. What every clinician should know about polysomnography. Respir Care. 2010;55(9):1179–1195. 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 16 Primary practice physicians briefly introduced OSA and the study to their randomly selected patients, taking care to inform them of the potential impact of an OSA diagnosis, especially its more severe forms, on their ability to drive safely. They also emphasised that OSA treatment could affect this. After the initial verbal presentation, patients were given a comprehensive written explanation of the purpose of the study, the protocol and the risks involved. Candidates were encouraged to ask additional questions before giving their informed consent and were reassured that they could choose to discontinue their participation in the study at any time without the need for further procedures or giving any reasons for discontinuation. 2.2 Participants The participating physicians invited randomly selected adult patients who had visited their practice for any reason. Participants had to be between 18 and 70 years of age at recruitment. Exclusion criteria were pre-existing sleep-disordered breathing, regular use of sedatives, tranquilizers or opioids (including tramadol), heart failure, neuromuscular disease, psychiatric disorders, severe COPD (stage D), use of psychoactive substances or excessive alcohol consumption. 2.3 Data collection The inclusion of patients in each primary practice took place between August 1, 2018, and August 1, 2022. A registered nurse employed in each primary practice setting facilitated communication with the participating candidates and scheduled their appointments. Upon arrival, patients completed a simple questionnaire asking them their age and gender and rechecking the inclusion and exclusion criteria, and then completed the Slovenian version of the SBQ. Each affirmative answer to the eight SBQ questions yielded a score of one, giving the SBQ score a range from zero to eight. The nurse then gave detailed instructions on how to use the device correctly, including a test dawn of the PG device, and a simple diagram of the process was also given to the participant for future reference. All candidates underwent at home ambulatory type 3 polygraphy, putting the equipment on themselves later that evening. PG recordings were made using the Alice NightOne (Phillips Respironics, Murrysville, Pennsylvania, USA), a type 3 PG, which has an effort belt, cannula, oximeter, and a built-in body position sensor with microphone providing seven channels of data (body position, pressure flow, snoring, respiratory effort, blood oxygen saturation, plethysmography and pulse rate). Manual scoring of all PG recordings was performed by a European accredited somnologist and neurologist at the University Hospital of Ljubljana, who was blinded to the SBQ scores. Based on the REI, OSA was categorised as mild (5≤REI<15), moderate (15≤REI<30) or severe (REI≥30). Recordings of less than 3 hours were considered insufficient. The evaluation was conducted in accordance with the current standards of the American Academy of Sleep Medicine (AASM) (17, 18). A total of 158 patients were included. Fifteen had technically unsuitable PG recordings due to issues such as dislocation of nasal cannula, pulse oximeter malfunction and missing or short recordings. In nine cases, the recording was repeated. Six patients declined to repeat the recordings. The final analysis included 153 patients with a mean age of 49.5 years (±13.0 years), of whom 73 (47.7%) were male. 2.4 Statistical analysis The patients’ characteristics were presented with the mean (standard deviation) in the case of normally distributed numerical variables, median (interquartile range) in the case of non-normally distributed numerical variables, and frequencies (%) in the case of categorical variables. The correlation between SBQ and REI was assessed using Pearson’s correlation coefficient (r). To assess the predictive validity of the SBQ, the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated for different REI cutoff values. We conducted an analysis of the receiver operating characteristics (ROC) and utilised Youden’s index, which provided the optimal threshold value based on the longest vertical distance from the diagonal line to the ROC curve (or the point on the curve closest to the upper-left corner). Statistical analysis was conducted using SPSS version 15.0 (SPSS Inc., Chicago, Illinois, USA), and JASP version 0.16.4 (Jasp Team, University of Amsterdam, Netherlands). 2.5 Ethical approval Ethical approval to conduct the study was obtained from the National Medical Ethics Committee of the Republic of Slovenia (NMEC), No. 0120-80/2018/7. 3 RESULTS A total of 153 participants were included, of whom 75 (49.0%) were diagnosed with OSA based on a manual REI≥5. The detailed classification into OSA severity levels and descriptive statistics of the sample are provided in Table 2. N age (years) sex (m) BMI (kg/m2) 1 2 3+ 4 5 6 7 8 1 2 3+ 4 5 153 49.7±13.1 73 (47.7%) 28.0±4.9 74 (48.4) 67 (43.8) 49 (32) 26 (17) 12 (7.8) 5 (3.3) 0 (0) 0 (0) 33 (21.6) 31 (20.3) 27 (17.6) 15 (9.8) 11 (7.2) 78 (51%) 43.6 (±12.9) 30 (38.5%) 26.1 (±4.1) 98.7 (94.7–100) 89.3 (81.3–96) 65.3 (54.7–76) 34.7 (24–45.4) 16 (8–25.3) 6.7 (1.3–13.3) 0 (0–0) 0 (0–0) 100 (100–100) 93.9 (84.8–100) 81.8 (66.7–93.9) 45.5 (27.3–63.6) 33.3 (18.2–48.5) 42 (27.5%) 55 (±11.7) 21 (50%) 29.1 (±4.9) 11.5 (5.1–19.2) 56.4 (44.9–66.7) 87.2 (79.5–93.6) 96.2 (91–100) 97.4 (93.6–100) 97.4 (93.6–100) 98.7 (96.2–100) 100 (100–100) 8.3 (4.2–13.3) 41.7 (33.3–50) 73.3 (65–80.8) 88.3 (82.5–94.2) 97.5 (94.2–100) 23 (15%) 58.5 (±7) 13 (56.5%) 30.6 (±4.4) 51.7 (49.7–54) 66.3 (60.6–72.2) 83.1 (74.6–91.1) 90 (77.8–100) 86.7 (65–100) 72.7 (33.3–100) 0 (0–0) 0 (0–0) 23.1 (22.–24.1) 30.8 (27.2–34.7) 45.8 (37.7–55.4) 51.7 (36.7– 68.8) 78.9 (57.1–100) 10 (6.5%) 54.1 (±7.6) 9 (90%) 33 (±5.4) 90.9 (66.7– 100) 84.8 (75.5–93.8) 72.3 (66.3–79.3) 60.5 (56.6– 65) 54.7 (52.1–57.6) 52.1 (50.3–53.9) 50.7 (50–51) 51.0 (51–51) 100 (100–100) 96.3 (90.6–100) 93.7 (89–97.9) 85.5 (81.7–89.6) 84.2 (81.1–87.4) ALL n (%) Non-OSA (REI 0–4.9) Sensitivity (95% CI) REI≥15 REI≥5 Mild (REI 5–14.9) Specificity (95% CI) Moderate (REI 15–29.9) PPV (95% CI) Severe REI≥30 NPV (95% CI) SBQ cutoff Table 2. Table 3. Figure 1. Descriptive statistics of the primary screening sample for OSA in a primary practice setting. Diagnostic characteristics of the Slovene SBQ in a primary practice setting at different SBQ cutoff values for different severities of OSA. Scatterplot of manual REI against the Slovene SBQ in a primary practice setting. Legend: REI = respiratory event index; SBQ = STOP-BANG questionnaire Legend: BMI = body mass index; OSA = obstructive sleep apnea; REI = respiratory event index; N = number of participants 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 17 The correlation between the SBQ and manual REI, as assessed by the Pearson correlation coefficient, was significant (p<0.00), and the details are presented in Figure 1. The sensitivity, specificity, positive (PPV) and negative predictive values (NPV) at different SBQ cutoff values and for different severities of OSA are given in Table 3. 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 18 6 7 8 1 2 3+ 4 5 6 7 8 5 (3.3) 0 (0) 0 (0) 10 (6.5) 9 (5.9) 9 (5.9) 7 (4.6) 6 (3.9) 4 (2.6) 0 (0) 0 (0) 15.2 (3–27.3) 0 (0–0) 0 (0–0) 100 (100–100) 90.0 (70–100) 90.0 (70–100) 70.0 (40–100) 60.0 (30–90) 40.0 (10–70) 0 (0– 0) 0 (0–0) 98.3 (95.8–100) 99.2 (97.5–100) 100 (100–100) 7.0 (3.5–11.9) 35.7 (28 –44.1) 65.0 (57.3–72.7) 84.6 (78.3–90.2) 94.4 (90.2–97.9) 97.9 (95.1–100) 99.3 (97.9–100) 100 (100–100) 72.7 (33.3–100) 0 (0–0) 0 (0–0) 7.0 (6.8–7.4) 9.0 (6.8–10.6) 15.4 (11.4–19.6) 24.2 (14.3–36) 42.9 (23.5–66.7) 58.3 (25–100) 0 (0–0) 0 (0–0) 80.8 (78.8–83.2) 78.3 (78–78.4) 78.4 (78.4–78.4) 100 (100–100) 98.1 (94–100) 99.0 (96.7–100) 97.6 (95.2–100) 97.1 (95–99.3) 95.9 (94–97.9) 93.4 (93.3–93.5) 93.5 (93.5–93.5) n (%) Sensitivity (95% CI) REI≥30 Specificity (95% CI) PPV (95% CI) NPV (95% CI) SBQ cutoff Legend: CI = confidence interval; NPV = negative predictive value; OSA = obstructive sleep apnea; PPV = positive predictive value; REI = respiratory event index; SBQ = STOP-BANG questionnaire. +The shaded areas indicate optimal values according to Youden’s index, which was 3 in for all severities of OSA. ROC analysis revealed AUC values of 0.823 (95% CI: 0.758– 0.888) for any OSA (REI≥5), 0.819 (95% CI: 0.742–0.869) for moderate to severe OSA (REI≥15), and 0.847 (95% CI: 0.695–0.999) for severe OSA (REI≥30). The receiver operating characteristic curves are shown in Figure 2. Figure 2. ROC for the Slovene SBQ in the primary practice setting at the threshold values of manual REI≥5, ≥15, and ≥30. Legend: REI = respiratory event index; SBQ = STOP-BANG questionnaire 4 DISCUSSIONS The Slovene translation of the SBQ showed good correlation with and diagnostic accuracy for OSA in the primary practice setting. At the standard cutoff of ≥3, the SBQ demonstrated an area under the curve of 0.82 for any OSA (REI≥5), 0.82 for moderate and severe OSA (REI≥15) and 0.85 for severe OSA (REI≥30). The prevalence of OSA (REI≥5) in our primary practice setting was notably high, at 49.0%, with 21.5% exhibiting moderate to severe OSA (REI≥15). Two key factors that may account for this high rate of OSA in primary practice are the age of the patients (with an average age of 49.7 years ±13.1) and the presence of comorbidities. Notably, OSA is more frequently observed in older individuals (19), it often coexists with other chronic diseases (20) and it is older individuals with chronic illness who are more frequent visitors to primary practice clinics (21). Muñoz-Gómez, who validated the Spanish version of the SBQ, using type 3 PG, for primary practice setting, found that 61.5% of the participants had OSA (REI≥5) with 38.8% having moderate and severe OSA (REI≥15). Even higher rates of OSA among patients were reported by Bailes and Fichten and their colleagues, who recruited patients older than 45 years in primary practice. The average age of patients in this study was 57.5 (±11.5) years, with 75% of patients having an AHI≥10, as assessed by PSG (22). There is no data for the prevalence of OSA in the Slovene general population. A previous study showed an OSA prevalence of 69.6% (REI≥5) in a Slovene sleep clinic, with 47.2% having moderate and severe OSA (REI≥15) (16). The differences in prevalence and severity of OSA in the two environments are to be expected, as there is a gatekeeper system in place in Slovenia and thus all the patients referred to a sleep clinic are in a sense pre-screened. 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 19 We found a good correlation between the Slovene version of the SBQ in the primary practice setting and REI. The sensitivity, specificity, NPV and PPV were also impressive. Sensitivity was, however, somewhat lower than in the sleep lab. These differences could be attributed to the small sample sizes, especially in the primary practice setting where the disease prevalence was lower and cases of OSA were on average milder. The diagnostic ability of the Slovene SBQ, at a cutoff of ≥3, for any OSA (REI≥5) assessed by ROC analysis yielded a high AUC of 0.82, this being even higher than the 0.76 in the Slovene sleep clinic (16). Muñoz-Gómez, who validated the Spanish version of the SBQ for primary practice setting, found an AUC of 0.69 for any OSA (23) and a meta- analysis of sleep clinics showed an AUC of 0.74 (24). The AUC improved for the detection of moderate to severe OSA with the Spanish version in the primary practice setting with an AUC of 0.77 (23), whereas it stayed more or less the same with our version. Youden’s index, which provides the optimal threshold value based on the longest vertical distance from the diagonal line to the ROC curve, was 3 for all OSA severity levels. This threshold value aligns with the classic threshold recommended by Chung (12). Many studies have been published validating the SBQ in sleep laboratory setting (25), including a Slovene version (16). There have also been studies validating the SBQ in the general population (25), however, publications validating the SBQ in primary practice setting, the most ubiquitous medical setting, are relatively rare. Compounding the problem, there have also been articles claiming to validate the SBQ in a primary practice setting even though only patients suspected of having a sleep disorder were included and then referred to a sleep clinic for a sleep study (11), something we would consider a pre-screened sleep clinic population. In our study, we made a methodological restriction by setting the maximum age of the participants at 70 years. While this probably had some impact on the representativeness of our sample compared to the broader family medicine clinic demographic, it was strategically employed to optimise participant engagement. We anticipated that this age limit would allow for a more seamless integration of participants into the study, particularly in terms of understanding instructions, correctly completing questionnaires, and proficiently utilising the designated equipment. Furthermore, compliance with continuous positive airway pressure (CPAP) therapy decreases with age, with the adherence of patients decreasing significantly from the age of 65-69 years, and decreasing further with increasing age (26). The study was further hindered by the COVID-19 pandemic, extending the patient recruitment period from the initially planned one year to four years. This included a two-year hiatus, after which the pace of patient recruitment was slower than anticipated. In addition, the lack of funding for patient recruitment was a significant constraint. Another limitation was the decision to use type 3 PG instead of the gold standard, that is type 1 PSG. Type 3 PG is now routinely used in clinical practice for the diagnosis of OSA, as it has been shown to be a reliable, cost-effective and simpler alternative to PSG (26, 27). However, we were conscious of the limitations of type 3 PG and therefore excluded patients who were regularly taking sedatives, opioids and tranquilizers, as well as patients with heart failure, neuromuscular disease, COPD stage D, and so on, in whom central or mixed types of apnea and other sleep disorders are more common and for whom a type 1 PSG would be preferable according to the AASM guidelines. Our study was not the first to utilize PG for the validation of the SBQ questionnaire, as Reis et al. (27) and Muñoz- Gómez et al. (23) also did so. We are also of the opinion PSG is too complex and impractical for use in a primary practice setting. 5 CONCLUSIONS With this study, we have confirmed the validity of the Slovene translation of the SBQ as a reliable instrument for OSA risk stratification in the primary practice setting. ACKNOWLEDGEMENTS We thank the SBQ, Dr. Frances Chung and the University Health Network for allowing us to use the STOP-BANG questionnaire. CONFLICTS OF INTEREST The authors have no conflicts of interest to report. FUNDING This study was funded by the Slovenian Research Agency (Grant No.: P3-0338). AVAILABILITY OF DATA AND MATERIALS Data and material are available upon request. LLM STATEMENT During the preparation of this work the authors used Chat GPT-4 to shorten the abstract. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. ORCID Andrej Pangerc: https://orcid.org/0000-0003-2428-6501 Marija Petek Šter: https://orcid.org/0000-0003-1736-2377 Leja Dolenc Grošelj: https://orcid.org/0000-0002-8350-951X REFERENCES 1. Kendzerska T, Mollayeva T, Gershon AS, Leung RS, Hawker G, Tomlinson G. Untreated obstructive sleep apnea and the risk for serious long-term adverse outcomes: A systematic review. Sleep Med Rev. 2014;18(1):49-59. doi: 10.1016/j.smrv.2013.01.003. 2. Eastwood PR, Malhotra A, Palmer LJ, Kezirian EJ, Horner RL, Ip MS, et al. Obstructive sleep apnoea: From pathogenesis to treatment: Current controversies and future directions. Respirology. 2010;15(4):587-595. doi: 10.1111/j.1440-1843.2009.01699.x. 3. Punjabi NM, Caffo BS, Goodwin JL, Gottlieb DJ, Newman AB, O’Connor GT, et al. Sleep-disordered breathing and mortality: A prospective cohort study. PLoS Med. 2009;6(8):e1000132. doi: 10.1371/journal. pmed.1000132. 4. Gottlieb DJ, Punjabi NM. Diagnosis and management of obstructive sleep apnea: A review. JAMA. 2020;323(14):1389-1400. doi: 10.1001/ jama.2020.3514. 5. Patel SR. Obstructive sleep apnea. Ann Intern Med. 2019;171(11):Itc81- itc96. doi: 10.7326/aitc201912030. 6. Patil SP, Schneider H, Schwartz AR, Smith PL. Adult obstructive sleep apnea: Pathophysiology and diagnosis. Chest. 2007;132(1):325-337. doi: 10.1378/chest.07-0040. 7. Young T, Evans L, Finn L, Palta M. Estimation of the clinically diagnosed proportion of sleep apnea syndrome in middle-aged men and women. Sleep. 1997;20(9):705-706. doi: 10.1093/sleep/20.9.705. 8. Benjafield AV, Ayas NT, Eastwood PR, Heinzer R, Ip MSM, Morrell MJ, et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: A literature-based analysis. Lancet Resp Med. 2019;7(8):687- 698. doi: 10.1016/s2213-2600(19)30198-5. 9. Punjabi NM. The epidemiology of adult obstructive sleep apnea. Proc Am Thorac Soc. 2008;5(2):136-143. doi: 10.1513/pats.200709-155MG. 10. Chai-Coetzer CL, Antic NA, Rowland LS, Catcheside PG, Esterman A, Reed RL, et al. A simplified model of screening questionnaire and home monitoring for obstructive sleep apnoea in primary care. Thorax. 2011;66(3):213-219. doi: 10.1136/thx.2010.152801. 11. Rebelo-Marques A, Vicente C, Valentim B, Agostinho M, Pereira R, Teixeira MF, et al. STOP-Bang questionnaire: The validation of a Portuguese version as a screening tool for obstructive sleep apnea (OSA) in primary care. Sleep Breath. 2018;22(3):757-765. doi: 10.1007/ s11325-017-1608-0. 12. Chung F, Yegneswaran B, Liao P, Chung SA, Vairavanathan S, Islam S, et al. STOP questionnaire: A tool to screen patients for obstructive sleep apnea. Anesthesiology. 2008;108(5):812-821. doi: 10.1097/ ALN.0b013e31816d83e4. 13. Chen L, Pivetta B, Nagappa M, Saripella A, Islam S, Englesakis M, et al. Validation of the STOP-Bang questionnaire for screening of obstructive sleep apnea in the general population and commercial drivers: A systematic review and meta-analysis. Sleep Breath. 2021;25(4):1741- 1751. doi: 10.1007/s11325-021-02299-y. 14. Silva GE, Vana KD, Goodwin JL, Sherrill DL, Quan SF. Identification of patients with sleep disordered breathing: Comparing the four-variable screening tool, STOP, STOP-Bang, and Epworth Sleepiness Scales. J Clin Sleep Med. 2011;7(5):467-472. doi: 10.5664/jcsm.1308. 15. Tan A, Yin JD, Tan LW, van Dam RM, Cheung YY, Lee CH. Predicting obstructive sleep apnea using the STOP-Bang questionnaire in the general population. Sleep Med. 2016;27-28:66-71. doi: 10.1016/j. sleep.2016.06.034. 16. Pangerc A, Petek Šter M, Dolenc Grošelj L. Translation and validation of the STOP-Bang questionnaire into Slovene. Eur J Med Res. 2021;26(1):32. doi: 10.1186/s40001-021-00503-z. 17. Iber C, American Academy of Sleep Medicine. The AASM manual for the scoring of sleep and associated events : Rules, terminology and technical specifications. Westchester: American Academy of Sleep Medicine; 2007. 59 p. 18. Duce B, Milosavljevic J, Hukins C. The 2012 AASM respiratory event criteria increase the incidence of hypopneas in an adult sleep center population. J Clin Sleep Med. 2015;11(12):1425-1431. doi: 10.5664/ jcsm.5280. 19. Glasser M, Bailey N, McMillan A, Goff E, Morrell MJ. Sleep apnoea in older people. Breathe. 2011;7(3):248-256. doi: 10.1183/20734735.021910. 20. McNicholas WT, Pevernagie D. Obstructive sleep apnea: Transition from pathophysiology to an integrative disease model. J Sleep Res. 2022;31(4):e13616. doi: 10.1111/jsr.13616. 21. Elliott J, Stolee P, Boscart V, Giangregorio L, Heckman G. Coordinating care for older adults in primary care settings: Understanding the current context. BMC Fam Pract. 2018;19(1):137. doi: 10.1186/s12875- 018-0821-7. 22. Bailes S, Fichten CS, Rizzo D, Baltzan M, Grad R, Pavilanis A, et al. The challenge of identifying family medicine patients with obstructive sleep apnea: Addressing the question of gender inequality. Fam Pract. 2017;34(4):467-672. doi: 10.1093/fampra/cmx008. 23. Muñoz-Gómez R, Navarrete-Martínez E, Serrano-Merino J, Silva-Gil F, Roldán-Villalobos A, Martín-Rioboó E, et al. The usefulness of the Spanish version of the STOP-Bang questionnaire for screening for moderate or severe sleep apnea syndrome in primary care. Front Public Health. 2022;10:975114. doi: 10.3389/fpubh.2022.975114. 24. Nagappa M, Liao P, Wong J, Auckley D, Ramachandran SK, Memtsoudis S, et al. Validation of the STOP-BANG questionnaire as a screening tool for obstructive sleep apnea among different populations: A systematic review and meta-analysis. PLoS One. 2015;10(12):e0143697. doi: 10.1371/journal.pone.0143697. 25. Chung F, Abdullah HR, Liao P. STOP-BANG questionnaire: A practical approach to screen for obstructive sleep apnea. Chest. 2016;149(3):631-638. doi: 10.1378/chest.15-0903. 26. Posadas T, Oscullo G, Zaldívar E, Garcia-Ortega A, Gómez-Olivas JD, Monteagudo M, et al. Treatment with CPAP in elderly patients with obstructive sleep apnoea. J Clin Med. 2020;9(2). doi: 10.3390/ jcm9020546. 27. Reis R, Teixeira F, Martins V, Sousa L, Batata L, Santos C, et al. Validation of a Portuguese version of the STOP-Bang questionnaire as a screening tool for obstructive sleep apnea: Analysis in a sleep clinic. Rev Port Pneumol 2015;21(2):61-68. doi: 10.1016/j.rppnen.2014.04.009. 10.2478/sjph-2024-0003 Zdr Varst. 2024;63(1):14-20 20 PERCEPTION OF PATIENT SAFETY CULTURE AT THE PRIMARY CARE LEVEL: THE CASE OF THE COMMUNITY HEALTH CENTRE LJUBLJANA Tina VIRTIČ POTOČNIK 1, 2* , Zalika KLEMENC-KETIŠ 1,2, 3 1 Community Health Centre Ljubljana, Primary Healthcare Research and Development Institute, Metelkova 9, 1000 Ljubljana, Slovenia 2 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia 3 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia Received: Oct 16, 2023 Accepted: Nov 27, 2023 Original scientific article *Correspondence: tina.virtic@gmail.com 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 21 ZAZNAVANJE KULTURE VARNOSTI PACIENTOV NA PRIMARNI RAVNI ZDRAVSTVENEGA VARSTVA: RAZISKAVA V ZDRAVSTVENEM DOMU LJUBLJANA Virtič Potočnik T, Klemenc-Ketiš Z. Perception of patient safety culture at the primary care level: The case of the Community Health Centre Ljubljana. Zdr Varst. 2024;63(1):21-29. doi: 10.2478/Sjph-2024-0004. ABSTRACT Keywords: Safety culture Patient safety Quality healthcare Primary healthcare Health promotion IZVLEČEK Ključne besede: kultura varnosti varnost bolnikov kakovostno zdravstveno varstvo primarno zdravstveno varstvo promocija zdravja Background: Patient safety is a crucial element of quality healthcare, and endeavours to enhance it are vital for attaining universal health coverage and improving patient outcomes. This study aimed to evaluate the perception of patient safety culture among staff at the Community Health Centre Ljubljana (CHCL). Methods: A cross-sectional study was conducted in December 2022. All CHCL staff (N=1,564) from different professional groups were invited to participate in an anonymous electronic survey using the validated Slovenian version of the “Medical Office Survey on Patient Safety Culture” (MOSPSC). Mean percent positive scores for all items in each composite were calculated. Results: The final sample included 377 participants (response rate, 24.1%), most of whom were women (91.5%, N=345) with different professional profiles. The mean age of the participants was 44.5 years (SD 11.1) with a mean work experience of 20.1 years (SD 12.1). The percentage of positive overall MOSPSC composite scores was 59.6%. A strong patient safety culture perception was identified in the following dimensions: Information exchange with other settings (93.5%), Organisational learning (90.2%), List of patient safety and quality issues (88.1%), Patient care tracking/follow-up (76.2 %) and Teamwork (75.0%). Weak patient safety culture was identified in the dimensions of Work pressure and pace (10.7%), Leadership support for patient safety (27.1%), Communication openness (40.9%), Office processes and standardisation (48.2%) and Overall ratings on quality and patient safety (49.4%). Conclusions: CHCL leadership should address weaknesses, redesign processes, and implement strategies to reduce patient safety incidents. Establishing a just culture that encourages employees to report errors fosters transparency and facilitates learning from errors. Izhodišča: Za kakovostno zdravstveno varstvo je ključna varnost pacientov in prizadevanja za njeno izboljšanje so bistvenega pomena za doseganje splošnega zdravja in izboljšanje izidov za paciente. V Sloveniji kultura varnosti pacientov še ni sistemsko urejena. Redno ocenjevanje kulture varnosti je pomembno za opredelitev področij, kjer so potrebne izboljšave in za zagotovitev najboljše možne oskrbe pacientom. Namen raziskave je bil oceniti kulturo varnosti pacientov med zaposlenimi v Zdravstvenem domu Ljubljana (ZDL). Metode: V decembru 2022 je bila izvedena presečna raziskava, v katero so bili povabljeni vsi zaposleni v ZDL (n = 1564) iz različnih poklicnih skupin. V anonimni elektronski anketi so izpolnili slovensko različico “Medical Office Survey on Patient Safety Culture” (MOSPSC). Za vsako dimenzijo varnosti je bil v skladu z navodili izračunan povprečni odstotek pozitivnih ocen. Rezultati: Končni vzorec je vključeval 377 udeležencev (24,1 % odzivnost) različnih poklicnih profilov, med katerimi je bilo 345 (91,5 %) žensk. Njihova povprečna starost je bila 44,5 leta (SD 11,1), s povprečno delovno dobo 20,1 leta (SD 12,1). Delež pozitivnih ocen vseh MOSPSC dimenzij je znašal 59,6 %. Kultura varnosti je bila identificirana kot močna v 5 dimenzijah: Izmenjava informacij z drugimi okolji (93,5 %), Organizacijsko učenje (90,2 %), Seznam težav glede varnosti in kakovosti bolnikov (88,1 %), Sledenje/nadaljnja oskrba bolnikov (76,2 %) in Timsko delo (75,0 %). Šibka kultura varnosti je bila prepoznana v naslednjih dimenzijah: Delovni pritisk in tempo (10,7 %), Podpora vodstva za varnost bolnikov (27,1 %), Odprtost komunikacije (40,9 %), Pisarniški procesi in standardizacija (48,2 %) ter Skupna ocena kakovosti in varnosti bolnikov (49,4 %). Ugotovljene so bile razlike med dimenzijami MOSPSC in posameznimi enotami ZDL, starostjo zaposlenih, različnimi poklicnimi profili in trajanjem zaposlitve. Zaključek: Zaznava kulture varnosti pacientov v ZDL je bila ocenjena kot pozitivna v petih dimenzijah MOSPSC lestvice, vendar so bila prepoznana tudi šibka področja, ki potrebujejo izboljšave. Pomembno je, da vodstvo ZDL obravnava ta vprašanja, preoblikuje procese in izvaja strategije za zmanjšanje števila incidentov, povezanih z varnostjo pacientov. Potrebno je spodbujati pravično kulturo in ustvariti okolje, kjer bodo zaposleni brez zadržkov poročali o potrebnih izboljšavah, storjenih napakah in se iz njih učili. Nenehno prizadevanje, spremljanje in izboljševanje prispevajo k zagotavljanju varne, učinkovite in kakovostne oskrbe pacientov. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Patient safety is a critical component of quality healthcare, and efforts to improve it are essential for achieving universal health coverage and improving patient outcomes. The World Health Organization defines patient safety as a “framework of organised activities that creates cultures, processes, procedures, behaviours, technologies and environments in healthcare that consistently and sustainably lower risks, reduce the occurrence of avoidable harm, make errors less likely and reduce impact of harm when it does occur” (1). Prioritising patient safety in healthcare settings has gained greater recognition in recent years (2). Achieving optimal patient safety requires a safety culture that promotes open communication, transparency, and continuous learning (1). It entails a shared understanding that safety is a top priority and everyone’s responsibility (3), necessitating systems that encourage error reporting, root cause analysis, and effective preventive strategies (1, 3). Patient safety in primary healthcare in Slovenia varies significantly from secondary to tertiary levels. Acting as a gatekeeper to hospital care, primary care is accessible and involves frequent patient contact (4, 5) with diverse health issues. Health providers are often required to make complex decisions with limited information (6). Hence, it is crucial to be comfortable with uncertainty, explore probabilities, and minimise risks (7). Patient safety culture remains unregulated in Slovenia, and thus there is a need for more quality assessment with regard to enhancing the quality of work in this sector. Unfortunately, punitive measures and fault-based liability persist (8). As Slovenia increasingly focuses on “value-based healthcare” to optimise patient treatment outcomes, a stakeholder expert panel has been formed to develop a roadmap for its implementation (9). The safety culture in Slovenian out-of-hours primary healthcare settings was recently assessed, with the results showing the need to standardise working conditions (10, 11). While leaders in Slovenian primary healthcare organisations generally express positive perceptions of safety culture, there remains room for improvement in communication and stress recognition (12). The only study evaluating the perception of patient safety culture among employees at the Community Health Centre Ljubljana (CHCL) was conducted in 2017, and this revealed an overall good patient safety culture. However, variations between professions were noted, and certain areas require further evaluation (4). Regularly assessing patient safety culture is crucial for identifying areas of improvement and ensuring optimal patient care (13, 14). This study thus aimed to evaluate employee perceptions of patient safety culture at the 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 22 CHCL and explore the relationships among different dimensions of patient safety and the sociodemographic characteristics of the study participants. 2 MATERIALS AND METHODS 2.1 Research design and setting We conducted a cross-sectional study at CHCL, the largest community healthcare centre in Slovenia, which provides health services for the Municipality of Ljubljana with over 450,000 registered patients. The CHCL consists of eight units, located in various parts of Ljubljana. In 2022, there were 1,564 employees from different areas of healthcare. The study was approved by the Slovenian National Medical Ethics Committee (No. 107/07/16). 2.2 Participants We invited all employees (N=1,564) of the CHCL to participate in the study (physicians, dentists, healthcare assistants, registered nurses, management, administrative or clerical staff, clinical support staff and employees working at other positions). 2.3 Tool We utilised the validated Slovenian version of MOSPSC (15), developed by the Agency for Healthcare Research and Quality (16). This tool facilitates the assessment of patient safety culture at the primary healthcare level, the detection of possible differences, better understanding with regard to the safety of a particular organisation, and evaluating the impact of specific interventions for improving patient safety culture (13, 16, 17). The structure of MOSPSC is outlined in Table 1. Domains A and B were answered according to a 6-point frequency scale from “daily” to “not in the past 12 months”. Domain C consists of four dimensions and domain D of three dimensions, both answered according to a 5-point Likert scale. Domain E included four items on leadership support, answered only by individuals without a leadership function, using a 5-point Likert scale. Those with a leadership function were required to respond to Domain F. Domain G describes overall ratings in five areas of healthcare quality (patient-centred, effective, timely, efficient, and equitable) and an overall rating on patient safety, using a 5-point Likert scale. All 52 items also included the response option “Does not apply” or “I don’t know” (16, 18). Finally, data on sociodemographic characteristics (gender, age, function, work experience, working hours, and location of work) were collected. 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 23 A B C D E F G Age (years), mean (SD) Age group (years), n (%) <25 26-40 41-55 >56 Male Female Profile, n (%) Physician, dentist Registered nurse Management Administrative or clerical staff Healthcare assistants Other clinical staff Other position Health centre unit, n (%) Center Moste-Polje List of patient safety and quality issues Information exchange with other settings Teamwork Work pressure and pace Staff training Office processes and standardisation Communication openness Patient care tracking/follow-up Communication about error Leadership support for patient safety Organisational learning Overall perceptions of patient safety and quality Overall ratings on quality Overall rating on patient safety 44.5 (11.1) 14 (1.8) 135 (17.8) 142 (18.7) 86 (11.3) 32 (8.5) 345 (91.5) 84 (21.8) 102 (26.5) 12 (3.1) 12 (3.1) 97 (25.2) 47 (12.2) 31 (8.1) 79 (21.0) 85 (22.5) DIMENSION DESCRIPTIVES DOMAIN CHARACTERISTIC Table 1. Table 2. Structure of the MOSPSC tool (16). Sociodemographic characteristics of the participants. 2.4 Data collection The survey was completed electronically, and the link was sent to the participants’ email addresses in December 2022. A reminder was sent after two weeks. Participation was anonymous, as possible identifiers such as email and IP address were removed by the administrative coordinator of the project. It was not possible for the researchers to link the participants to their responses. 2.5 Data analysis We performed a univariate analysis with frequency distributions and descriptive statistics. The MOSPSC analysis was performed according to the AHRQ recommendations. Responses with the highest scores on the Likert scale indicate a more positive patient safety culture evaluation at each item level. Negatively worded items (C3, C6, C8, C10, C12, C14, D4, D7, D10, E1, E2, E4, F3, F4, and F6) were reversed so that higher scores always indicated a more positive evaluation of the safety culture. “Does not apply” or “I don’t know” responses were treated as missing data. A positive response was considered when the response score was equal to or above 4 on any 5-point or 6-point Likert scale (16). We calculated a percent positive score for each of the composites and the overall MOSPSC composite percent positive score using the mean percent score of positive responses of all dimensions. A positive patient safety culture was considered when the composite percent positive score was above 60%. The patient safety culture was considered strong when composite percent positive score was equal to or greater than 75% and identified as weak when composite percent positive score was less than 50%, suggesting the need for improvements (16). For the statistical analysis, the composite percent positive scores are expressed as the mean ± standard deviation (SD) and median and interquartile range (25–75%). According to the Kolmogorov-Smirnov test with Lilliefors correction, the overall MOSPSC composite percent positive score had a normal distribution, and all composite percent positive scores of MOSPSC dimensions had a non-normal distribution. The association between the composite percent positive scores of the MOSPSC dimensions and the participants’ gender or professional profile was evaluated using the Mann-Whitney U test, resulting in the calculation of the U value. Additionally, the Kruskal-Wallis test was employed to compare the medians of the composite percent positive scores of the MOSPSC dimensions across various sociodemographic characteristics of the employees, yielding the H value. A p-value of <0.05 was considered statistically significant. 3 RESULTS 3.1 Demographic characteristics The final sample comprised 377 participants who returned eligible and complete surveys, resulting in a response rate of 24.1%. The majority were women (91.5%, N=345). Participants had a mean age of 44.5 years (SD 11.1) and a mean work experience of 20.1 years (SD 12.1), with 78.5% having more than 16 years of experience. On average, the participants worked 37.1 hours per week (SD 10.1) and the average length of work in the current medical office location was 12.4 years (SD 10.9). List of patient safety and quality issues, % Information exchange with other settings, % Teamwork, % Work pressure and pace, % Staff training, % Office processes and standardisation, % Communication openness, % Patient care tracking/follow-up, % Communication about error, % Leadership support for patient safety, % Organisational learning, % Overall perceptions of patient safety and quality, % Overall ratings on quality and patient safety, % Overall MOSPSC composite percent positive score, % G1: Overall ratings on quality G2: Overall rating on patient safety Overall ratings on quality and patient safety, % 88.1 (18.9) 93.5 (18.8) 75.0 (29.7) 10.7 (20.3) 51.6 (36.7) 48.2 (28.2) 40.9 (33.7) 76.2 (28.6) 58.3 (31.6) 27.1 (31.2) 90.2 (22.9) 70.1 (24.2) 49.4 (38.3) 59.6 (15.7) 48.4 (50.0) 51.7 (50.0) 47.1 (50.0) 47.1 (50.0) 69.8 (46.0) 52.8 (40.3) 32.6 (46.9) 49.4 (38.3) G1A: Patient-centred, % G1B: Effective, % G1C: Timely, % G1D: Efficient, % G1E: Equitable, % G1: Average overall rating on quality, % G2, % 100.0 (77.8-100.0) 100.0 (100.0-100.0) 75.0 (50.0-100.0) 0.0 (0.0-25.0) 66.7 (33.3-66.7) 50.0 (25.0-72.0) 25.0 (0.0-75.0) 75.0 (50.0-100.0) 75.0 (25.0-75.0) 25.0 (0.0-50.0) 100.0 (100.0-100.0) 75.0 (50.0-100.) 50.0 (16.7-83.3) 75.0 (50.0-100.0) 0.0 (0.0-100.) 100.0 (0.0-100.0) 0.0 (0.0-100.0) 0.0 (0.0-100.0) 100.0 (0.0-100.0) 60.0 (0.0-100.0) 0.0 (0.0-100.0) 50.0 (16.7-83.3) Mean (SD) Mean (SD)ITEMS Median (IQ 25-75) Median (IQ 25-75) MOSPSC DIMENSIONS DOMAIN G Table 3. Table 4. Composite percent positive scores of the Medical Office Survey on Patient Safety Control (MOSPSC) and its dimensions. Percentage of positive responses on Overall ratings on quality and patient safety. Legend: SD – standard deviation; IQ 25–75 – 25%–75% interquartile range Legend: SD – standard deviation; IQ 25–75 – 25%–75% interquartile range Legend: SD – standard deviation 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 24 Management Šentvid Vič-Rudnik Bežigrad Emergency Care Unit Šiška Work experience (years), n (%) <4 4-7 8-15 >16 5 (1.3) 12 (3.2) 82 (21.8) 56 (14.9) 6 (1.6) 52 (13.8) 13 (3.4) 25 (6.6) 43 (11.4) 296 (78.5) DESCRIPTIVESCHARACTERISTIC 3.2 Attitudes to patient safety culture Table 3 shows the composite percent positive scores of the MOSPSC and its dimensions. Strong patient safety culture was detected in five dimensions: Information exchange with other settings (93.5%), Organisational learning (90.2%), List of patient safety and quality issues (88.1%), Patient care tracking/follow-up (76.2 %) and Teamwork (75.0%). Weak patient safety culture was identified in the following five dimensions: Work pressure and pace (10.7%), Leadership support for patient safety (27.1%), Communication openness (40.9%), Office processes and standardisation (48.2%) and Overall ratings on quality and patient safety (49.4%). The overall MOSPSC composite percent positive score was 59.6%. Table 4 shows the percentage of positive scores for items of Domain G. The percentage of positive responses on Overall ratings on quality was 52.8% and that for Overall rating on patient safety was 32.6%, with the composite percent positive score of 49.4%. Regarding the items of the G1 sub-domain, only Equitable showed a positive safety culture, with 69.8%. 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 25 List of patient safety and quality issues, % Mean (SD) Median (IQ 25-75) Information exchange with other settings, % Mean (SD) Median (IQ 25-75) Teamwork, % Mean (SD) Median (IQ 25-75) Work pressure and pace, % Mean (SD) Median (IQ 25-75) Staff training, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 85.6 (24.7) 80.0 (70.0-100.0) 94.2 (17.4) 95.0 (90.0-100.0) 76.8 (29.3) 70.0 (60.0-87.5) 8.1 (18.8) 0.0 (0.0-33.3) 43.8 (40.0) 50.0 (0.0–75.0), (U=9362.0; p=0.019) 88.9 (17.0) 85.0 (80.0.-100.0) 95.0 (16.8) 95.0 (90.0-100.0) 75.8 (29.2) 70.0 (60.082.5) 11.2 (19.0) 15.0 (0.0-25.0) 50.5 (36.7) 50.0 (25.0-75.0) 96.3 (6.4) 100.0 (90.0-100.0) 100 (0.0) 100.0 (100.0-100.0) 85 (24.1) 100.0 (75.0-100.0 12.5 (13.4) 0.0 (0.0- 33.3) 77.8 (29.6) 75.0 (60–100.0), (U=2870.5; p=0.014) / / 100 (0.0) 100.0 (100.0-100.0) 87.5 (25.0) 100.0 (75.0-100.0 37.5 (53.0) 30.0 (25.0-47.5) 38.1 (35.6) 25.0 (25.0-75.0) 90.8 (13.8) 90.0 (90.0-100.0) 92.6 (20.4) 100. (80.0-100.0) 70.7 (32.7) 70.0 (50.0-87.5) 8.3 (16.8) 0.0 (0.0-33.3) 50.2 (36.0) 50.0 (25.0-87.5) 100 (0.0) 100.0 (100.0-100.0) 91.1 (20.3) 100.0 (75.0- 100.0) 68.6 (31.2) 75.0 (50.0-100.0 18.1 (29.0) 25.0 (25.0-25.0) 48.8 (30.0) 50.0 (25.0-80.0) Physician, dentist (n=84) Registered nurse (n=102) Management (n=12) Administrative or clerical staff (n=12) Healthcare assistants (n=97) Other clinical staff (n=47) MOSPSC dimensions Table 5. Analysis of composite percent positive score of the MOSPSC dimensions based on different professional profiles. 3.3 Associations between the MOSPSC dimensions and the employees’ characteristics Bivariate analyses revealed relationships among the various sociodemographic characteristics of participants and MOSPSC dimensions. No differences were found between gender and the assessment of MOSPSC dimensions. However, significant variations were observed among different age groups in their perception of patient safety culture dimensions. Employees aged 55 and older rated Staff training (H=13.801; p=0.003), Office processes and standardisation (H=12.287; p=0.006) and Patient care tracking/follow- up (H=10.451; p=0.015) higher than their younger co- workers. Management workers aged over 41 evaluated Organisational learning (H=7.944; p=0.019) more positively than their younger colleagues. Participants with less than three years of employment rated Staff training (H=13.455; p=0.004) and Patient care tracking/follow-up (H=19.333; p<0.001) significantly lower than others. Those employed more than 16 years of employment assessed Office processes and standardisation better than their co- workers (H=9.963; p=0.019). Leadership support received a higher assessment from those with more than eight years of work experience (H=8.440; p=0.038). No significant differences were found between the number of weekly working hours and the employees’ evaluation of MOSPSC dimensions. Teamwork was the only dimension assessed differently across CHCL units, with the Emergency care unit and management personnel providing the highest scores (H=15.441, p=0.031). Table 5 illustrates the relationship between the composite percent positive score of the MOSPSC dimensions and professional profiles. Physicians rated Office processes and standardisation lower than employees in other positions (U=8942.0; p=0.037), whereas management staff rated this dimension higher than their co-workers (U=2963.0; p=0.004). Physicians also evaluated Patient care tracking/ follow-up with lower score compared to others (U=9362.0; p=0.049). Management personnel evaluated Staff training (U=2870.5; p=0.014), Office processes and standardisation (U=2963.0; p=0.003), Communication openness (U=1715.5; p=0.043) and Communication about error (U=1592.0; p=0.043) higher than other employees. Other clinical staff (e.g. physiotherapist, laboratory assistant, etc.) rated Communication openness (U=2269.5; p=0.011) and Leadership support for patient safety (U=2074.5; p=0.002) lower than employees in other positions. 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 26 Physician, dentist (n=84) Registered nurse (n=102) Management (n=12) Administrative or clerical staff (n=12) Healthcare assistants (n=97) Other clinical staff (n=47) MOSPSC dimensions Office processes and standardisation, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 Communication openness, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 Patient care tracking/ follow-up, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 Communication about error, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 Leadership support for patient safety, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 Organisational learning, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 /Overall perceptions of patient safety and quality, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 /Overall ratings on quality and patient safety, % Mean (SD) Median (IQ 25-75), U value if p-value <0.05 43.4 (26.1) 37.5 (20.0–80.0), (U=8942.0; p=0.037) 52.9 (31.0) 62.5 (25.0-75.0), (U=9542.5; p<0.01) 70.8 (31.1) 75.0 (75.0-100.0), (U=3937.5; p=0.049) 56.2 (32.2) 50.0 (50.0-75.0) 31.5 (32.6) 25.0 (0.0-50.0) 66.7 (38.4) 75.0 (50.0-100.0) 62.5 (32.2) 66.7 (33.3-100.0) 53.4 (36.3) 66.7 (33.3-100.0) 49.7 (28.9) 50.0 (33.3-80.0) 34.1 (33.3) 25.0 (0.0-50.0) 77.7 (27.7) 75.0 (75.0-100.0) 56.9 (32.0) 50.0 (50.0-75.0) 28.7 (28.9) 33.3 (0.0-33.3) / / / / 46.4 (39.8) 50.0 (33.3-66.7) 72.9 (24.9) 75.0 (80.0–100.0), (U=2963.0; p=0.003) 63.9 (35.6) 75.0 (50.0– 100.0), (U=1715.5; p=0.043) 68.8 (23.9) 75.0 (50.0-100.0) 87.5 (18.9) 100.0 (75.0-100.0), (U=1592.0; p=0.043) 50.0 (25.0) 50.0 (25.0-75.0) 100 (0.0) 100.0 (100.0- 100.0) 75.0 (25.0) 66.7 (33.3-100.0) 52.8 (33.2) 50.0 (33.3- 100.0) 55.0 (41.1) 75.0 (50.0–75.0) 33.3 (28.9) 25.0 (0.0-50.0) / / 75.0 (0.0) 80.0 (75.0-100.0) 58.3 (52.0) 50.0 (50.0-100.0) / / / / 20.8 (34.9) 20.0 (0.0-50.0), (U=1260.5; p=0.009) Legend: SD – standard deviation; IQ 25–75 – 25%–75% interquartile range; U – value in the Mann-Whitney U test if p-value <0.05 47.8 (25.9) 50.0 (25.0-75.0) 37.3 (31.6) 25.0 (0.0-50.0) 80.3 (26.9) 75.0 (75.0–100.0) 58.6 (28.0) 50.0 (25.0-75.0) 27.9 (32.8) 25.0 (0.0–50.0) 87.2 (25.6) 100.0 (75.0- 100.0 58.3 (14.4) 50.0 (50.0-75.0) 47.6 (40.4) 50.0 (33.3- 66.7) 50.0 (27.1) 50.0 (33.3-66.7) 25.0 (32.3) 25.0(10.0-50.0), (U=2269.5; p=0.011) 70.5 (36.8) 70.0 (60.0-82.5) 58.8 (30.6) 50.0 (25.0-75.0) 12.5 (24.7) 0.0 (0.0–33.3), (U=2074.5; p=0.002) / / / / 56.0 (34.3) 66.7 (33.3-66.7) 4 DISCUSSION This study provides valuable insights into the perception of safety culture at CHCL, highlighting areas that require attention to enhance patient safety. While five dimensions of MOSPSC are identified as strong (Information exchange with other settings, Organisational learning, List of patient safety and quality issues, Patient care tracking/ follow-up, and Teamwork), there are still areas in need of improvement, namely Work pressure and pace, Leadership support for patient safety, Communication openness, Office processes and standardisation, and Overall ratings on quality and patient safety. Additionally, the research indicates variations in the MOSPSC domains based on CHCL’s individual units, employee age, different professional profiles, and length of employment. The present study found no significant relationship between any of the safety culture composites and gender, similar to a study in Greece (19). In contrast, Polish (20), Spanish (21) and Slovenian (4) studies revealed that women rated certain domains higher than men. Our study showed a positive relation between seniority and responses referring to Staff training, Office processes and standardisation and Patient care tracking/follow-up. A study in Spain also showed the best perception of safety culture was among respondents 56 to 69 years old (21). We found differences in the patient safety culture among the professional profiles and so did other studies (4, 21–23). Information exchange with other settings (93.5%) received the highest score, indicating effective communication and collaboration. This is an essential component of a strong patient culture in healthcare, as it enables healthcare providers to share important patient information and coordinate care across different settings (16). Teamwork (75.0%) and Organisational learning (90.2%) were also among the highest rated MOSPSC dimensions which is consistent with studies from Spain (21), Brazil (24), Yemen (25), Iran (26), and Poland (20). These findings highlight the importance of comprehensive primary care in promoting and strengthening teamwork, as multi-professional healthcare stakeholders are involved in managing patients (10). Moreover, this dimension is essential because it influences the health professionals’ satisfaction and participation, and promotes personal and professional well-being (24). In addition, the emphasis on organisational learning is also an important aspect of promoting quality and safety. Through ongoing evaluation and improvement of healthcare practices, healthcare professionals can continuously learn and adapt to changing patients’ needs and healthcare environments (21). The dimension List of patient safety and quality issues considers daily tasks such as access to care, patient identification, accessibility of medical records, medical equipment, medication and diagnostics (16), and these were assessed as having a strong safety culture (88.1%). This dimension was the highest scoring for all professional categories in the Spanish study (21). A strong safety culture was also evident in the Patient care tracking/ follow-up dimension (76.2%), especially when compared to two Brazilian studies (56.1%) (23) and (60.1%) (24) and a Yemeni study (52%) (26). This dimension ensures that patients receive timely and appropriate care (17). In our study, doctors assessed it worse (p=0.049) than registered nurses and healthcare assistants, and it is essential to identify the reasons for this discrepancy, which may involve a lack of communication and coordination among healthcare providers. We identified a weak safety culture in Work pressure and pace (10.7%) and Leadership support for patients’ safety (27.1%), with the lowest scores. Notably, the dimension of Work pressure and pace is a significant concern for patient safety in primary care, consistently associated with low scores across various studies (4, 20, 21, 24, 25, 27). Research highlights challenges in primary care settings, emphasising issues such as inadequate staffing levels to manage patient loads (4, 14, 26, 27). Healthcare professionals in CHCL gave the lowest scores for this dimension, aligning with the shortage of personnel and high workload among family medicine doctors (8) and nurses in Slovenia, with negative impacts on healthcare quality and safety (28). Understanding and addressing factors influencing workload is crucial for developing interventions to improve efficiency, reduce work pace, and prevent burnout (17, 27). For the sustainability of the Slovenian healthcare system it is essential to transfer health competences and responsibility from healthcare professionals to empowered patients (8, 29). To this end, CHCL recently introduced a model of peer support with trained lay individuals who are CHCL’s volunteers and ambassadors, with the aim of empowering patients, family members and informal caregivers in the local community (29). In the Overall ratings on quality and patient safety dimension (49.4%), administrative staff scored significantly lower than employees in other profiles, aligning with Hickner’s et al. study (17). Continuous education and training are instrumental in improving the skills and knowledge of all employees (14). Regarding the overall quality assessment, Equitability (69.8%) obtained the highest score. This can be attributed to the CHCL’s pivotal role as Slovenia’s largest primary health centre, in promoting health equity and reducing disparities. By addressing social determinants, it can enhance the health and well-being of its diverse population. While the overall MOSPSC composite percent positive score of 59.6% falls below the threshold for a positive assessment, it surpasses the observed scores in other primary healthcare settings (23, 24). In contrast, studies in Poland (20), Yemen (25), Iran (26), and Greece (19) 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 27 reported positive overall MOSPSC composite percent positive scores. Nonetheless, CHCL has demonstrated ongoing efforts over the years to enhance patient safety culture and care quality. This commitment is further underscored by its recent participation in the OECD initiative, focusing on the outcomes and experiences of patients receiving care in family medicine practices (30). A notable strength of this article could be in its chronological consistency, employing the same methodology to assess the environment at CHCL over time. In contrast with the one prior study at CHCL in 2017, where 37.3% of the respondents reported a positive score for the Overall ratings on quality and patient safety dimension (4), the current result of 49.4% is promising. However, it must be interpreted carefully, given our study’s low response rate of only 24.1%, compared to 67.8% in the 2017 study (4). However, several limitations of this study should be acknowledged. The CHCL represents just one of numerous health centres in Slovenia, and the sample may not accurately reflect the diversity of Slovenian primary healthcare. Additionally, the findings may not be generalisable to the entire CHCL, given the study’s low response rate of 24.1%, possibly indicating organisational or leadership culture issues. The participants might not have felt entirely comfortable expressing safety concerns, despite confidentiality assurances. Those who chose to participate may also have had a heightened interest or knowledge about the topic, introducing potential bias. Encouraging healthcare professionals to identify and report errors without fear or blame is crucial, fostering a shift from an accountability to a just culture, focused on learning to prevent errors (21, 31). As most respondents were registered nurses, healthcare assistants and physicians, the results did not adequately reflect the perceptions of management and administrative staff. This aligns with the common misconception that only healthcare staff bear responsibility for patient safety and incidents. As such, delving into the reasons for the limited participation of other professional groups is crucial. A thorough understanding of the barriers these groups face in this regard can guide the development of more effective strategies to engage them in research. Additionally, it is important to note a limitation related to the MOSPSC’s validation, which originally focused on participants in leadership positions (15). Further psychometric evaluation is necessary, considering the diverse roles within our participant population. Moreover, not using open-ended questions may limit understanding healthcare professionals’ perceptions of safety culture (24), emphasising the need for a combined quantitative and qualitative approach. Given these limitations, it is crucial to interpret the study results carefully. 5 CONCLUSIONS This study offers insights into the perception of the safety culture at CHCL, pinpointing areas for attention to further enhance patient safety. Although five dimensions of the MOSPSC were identified as strong, there are still areas needing improvement. It is crucial for CHCL leadership to address these issues, redesign processes, and implement improvement strategies that reduce patient safety incidents. Fostering a just culture within the organisation is imperative with regard to enhancing future participation rates, where employees are comfortable reporting errors, learning from them, and promoting transparency. Continuous effort, monitoring and improvement can ultimately contribute to the delivery of safe, effective, and high-quality patient care. ACKNOWLEDGMENT We would like to acknowledge and thank all the subjects who participated in the survey. CONFLICTS OF INTEREST The authors declare that no conflicts of interest exist. FUNDING The study received no funding. ETHICAL APPROVAL The study was approved by the National Ethics Committee of the Republic of Slovenia (No. 107/07/16). AVAILABILITY OF DATA AND MATERIALS All data and materials used in this study are available upon reasonable request. ORCID Tina Virtič Potočnik: https://orcid.org/0000-0003-4925-3706 Zalika Klemenc-Ketiš: https://orcid.org/0000-0002-0270-1754 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 28 REFERENCES 1. World Health Organization. Global patient safety action plan 2021– 2030: Towards eliminating avoidable harm in health care [Internet]. 2021 [cited 2023 Apr 14]. Available from: https://www.who.int/ publications/i/item/9789240032705 2. Bates DW, Singh H. Two decades since to err is human: An assessment of progress and emerging priorities in patient safety. Health Affairs. 2018;37(11):1736–1743. doi: 10.1377/hlthaff.2018.0738. 3. Vincent C, Burnett S, Carthey J. Safety measurement and monitoring in healthcare: A framework to guide clinical teams and healthcare organisations in maintaining safety. BMJ Qual Saf. 2014;23(8):670–677. doi: 10.1136/bmjqs-2013-002757. 4. Tevžič Š, Poplas-Susič A, Klemenc-Ketiš Z. The safety culture of The Ljubljana Community Health Centre’s employees. Zdr Varst. 2021;60(3):145–151. doi: 10.2478/sjph-2021-0021. 5. Tušek-Bunc K, Petek Šter M, Petek D. Correlation of coronary heart disease patient assessments of chronic illness care and quality of care procedures. Acta Medico-Biotechnica. 2018;11(1):45–53. 6. Ivetic V, Poplas-Susic T, Pašić K, Selič P. Beliefs and viewpoints of family medicine physicians on approaches to identify and treat medically unexplained symptoms. Acta Medico-Biotechnica. 2016;9(2):47-57. 7. Parker D, Wensing M, Esmail A, Valderas JM. Measurement tools and process indicators of patient safety culture in primary care: A mixed methods study by the LINNEAUS collaboration on patient safety in primary care. Eur J General Practice. 2015;21:26–30. doi: 10.3109/13814788.2015.1043732. 8. Švab I, Klemenc-Ketiš Z, Poplas Susič T, Rotar Pavlič D, Petek Šter M, Kolšek M, et al. Family medicine in Slovenia: Overview of the state of affairs and suggestions for improvement. Zdrav Vestn. 2021;90(1– 2):112–125. doi: 10.6016/ZdravVestn.3010. 9. Prevolnik Rupel V, Došenović Bonča P. Is value-based health care just the latest fad or can it transform the Slovenian health care system? Zdr Varst. 2023;62(2):55–58. doi: 10.2478/sjph-2023-0008. 10. Klemenc-Ketiš Z, Deilkås ET, Hofoss D, Bondevik GT. Patient safety culture in Slovenian out-of-hours primary care clinics. Zdr Varst. 2017;56(4):203–210. doi: 10.1515/sjph-2017-0028. 11. Klemenc-Ketis Z, Maletic M, Stropnik V, Deilkås ET, Hofoss D, Bondevik GT. The safety attitudes questionnaire – ambulatory version: Psychometric properties of the Slovenian version for the out-of- hours primary care setting. BMC Health Serv Res. 2017;17(1):36. doi: 10.1186/s12913-016-1972-7. 12. Klemenc-Ketis Z, Makivić I, Poplas-Susič A. Safety culture in the primary health care settings based on workers with a leadership role: The psychometric properties of the Slovenian-language version of the safety attitudes questionnaire – short form. BMC Health Serv Res. 2018;18(1):767. doi: 10.1186/s12913-018-3594-8. 13. Vasconcelos PF, Arruda LP, Sousa Freire VEC, Carvalho REFL. Instruments for evaluation of safety culture in primary health care: Integrative review of the literature. Public Health. 2018;156:147–151. doi: 10.1016/j.puhe.2017.12.024. 14. Lawati MHA, Dennis S, Short SD, Abdulhadi NN. Patient safety and safety culture in primary health care: A systematic review. BMC Fam Pract. 2018;19(1):104. doi: 10.1186/s12875-018-0793-7. 15. Klemenc-Ketiš Z, Makivić I, Poplas-Susič T. The psychometric properties of the Slovenian version of the medical office survey on patient safety culture. Acta medico-biotechnica. 2021;13(2):38–45. 16. Rockville W, Sorra J, Gray L, Famolaro T, Yount N, Behm J. AHRQ medical office survey on patient safety culture: User’s guide. AHRQ Publication 18-0037-EF [Internet]. 2018 [cited 2023 Apr 9]. Available from: https://www.ahrq.gov/sites/default/files/wysiwyg/ professionals/quality-patient-safety/patientsafetyculture/medical- office/userguide/medicalofficeusersguide.pdf 17. Hickner J, Smith SA, Yount N, Sorra J. Differing perceptions of safety culture across job roles in the ambulatory setting: Analysis of the AHRQ medical office survey on patient safety culture. BMJ Qual Saf. 2016;25(8):588–594. doi: 10.1136/bmjqs-2014-003914. 18. El Zoghbi M, Farooq S, Abulaban A, Taha H, Ajanaz S, Aljasmi J, et al. Improvement of the patient safety culture in the primary health care corporation – Qatar. J Patient Saf. 2021;17(8):e1376–e1382. doi: 10.1097/PTS.0000000000000489. 19. Antonakos I, Souliotis K, Psaltopoulou T, Tountas Y, Kantzanou M. Patient safety culture assessment in primary care settings in Greece. Healthcare. 2021;9(7):880. doi: 10.3390/healthcare9070880. 20. Raczkiewicz D, Owoc J, Krakowiak J, Rzemek C, Owoc A, Bojar I. Patient safety culture in Polish primary healthcare centers. Int J Qual Health Care. 2019;31(8):G60-G66. doi: 10.1093/intqhc/mzz004. 21. Astier-Peña MP, Torijano-Casalengua ML, Olivera-Cañadas G, Silvestre- Busto C, Agra-Varela Y, Maderuelo-Fernández JÁ. Are Spanish primary care professionals aware of patient safety? Eur J Public Health. 2015;25(5):781–787. doi: 10.1093/eurpub/ckv066. 22. Desmedt M, Bergs J, Willaert B, Schrooten W, Vlayen A, Hellings J, et al. Exploring and evaluating patient safety culture in a community- based primary care setting. J Patient Saf. 2021;17(8):e1216–e1222. doi: 10.1097/PTS.0000000000000458. 23. Raimondi DC, Bernal SCZ, Matsuda LM. Patient safety culture from the perspective of workers and primary health care teams. Rev Saude Publica. 2019;53:42. doi: 10.11606/S1518-8787.2019053000788. 24. Araújo GL, Amorim FF, de Miranda RCPS, Amorim FFP, Santana LA, Göttems LBD. Patient safety culture in primary health care: Medical office survey on patient safety culture in a Brazilian family health strategy setting. PLoS ONE. 2022;17(7):e0271158. doi: 10.1371/journal. pone.0271158. 25. Webair HH, Al-assani SS, Al-haddad RH, Al-Shaeeb WH, Bin Selm MA, Alyamani AS. Assessment of patient safety culture in primary care setting, Al-Mukala, Yemen. BMC Fam Pract. 2015;16(1):136. doi: 10.1186/s12875-015-0355-1. 26. Tabrizchi N, Sedaghat M. The first study of patient safety culture in Iranian primary health centers. Acta Med Iran. 2012;50(7):505–510. 27. Inácio ALR, Rodrigues MCS. Aplicação do Medical office survey on patient safety culture: revisão integrativa. Acta Paulista de Enfermagem. 2022;35:eAPE001222. doi: 10.37689/acta-ape/2022AR01223. 28. Skela-Savič B, Albreht T, Sermeus W, Lobe B, Bahun M, Dello S. Patient outcomes and hospital nurses’ workload: A cross-sectional observational study in Slovenian hospitals using the RN4CAST survey. Zdr Varst. 2023;62(2):59–66. doi: 10.2478/sjph-2023-0009. 29. Virtič T, Mihevc M, Zavrnik Č, Mori Lukančič M, Poplas Susič A, Klemenc-Ketiš Z. Peer support as part of scaling-up integrated care in patients with type 2 diabetes and arterial hypertension at the primary healthcare level: A study protocol. Zdr Varst. 2023;62(2):93–100. doi: 10.2478/sjph-2023-0013. 30. De Boer D, Van Den Berg M, Ballester M, Bloemeke J, Boerma W, De Bienassis K, et al. Assessing the outcomes and experiences of care from the perspective of people living with chronic conditions, to support countries in developing people-centred policies and practices: Study protocol of the International Survey of People Living with Chronic Conditions (PaRIS survey). BMJ Open. 2022;12(9):e061424. doi: 10.1136/bmjopen-2022-061424. 31. Van Baarle E, Hartman L, Rooijakkers S, Wallenburg I, Weenink JW, Bal R, et al. Fostering a just culture in healthcare organizations: Experiences in practice. BMC Health Serv Res. 2022;22(1):1035. doi: 10.1186/s12913-022-08418-z. 10.2478/sjph-2024-0004 Zdr Varst. 2024;63(1):21-29 29 PATIENT REPORTED INDICATOR SURVEYS (PaRIS): METHODOLOGICAL CONSIDERATIONS OF A FIELD TRIAL IN SLOVENIA Matija AMBROŽ 1* , Candan KENDIR 2,3 , Wienke BOERMA 4 , Zalika KLEMENC-KETIŠ 1,5,6 1 Community Health Centre Ljubljana, Metelkova 9, 1000 Ljubljana, Slovenia 2 Organisation for Economic Co-operation and Development (OECD), 2, rue André Pascal 75016 Paris, France 3 Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands 4 NIVEL Netherlands Institute for Health Services Research, Otterstraat 118, 3513 CR Utrecht, The Netherlands 5 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia 6 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia Received: Oct 11, 2023 Accepted: Nov 27, 2023 Original scientific article *Correspondence: matija.ambroz@zd-lj.si 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 30 OCENA IZKUŠENJ IN IZIDOV ZDRAVSTVENE OSKRBE PACIENTOV S KRONIČNIMI NENALEZLJIVIMI BOLEZNIMI: METODOLOŠKI VIDIK PILOTNE IZVEDBE RAZISKAVE V SLOVENIJI Ambrož M, Kendir C, Boerma W, Klemenc-Ketiš Z. Patient reported indicator surveys (PaRIS): Methodological considerations of a field trial in Slovenia. Zdr Varst. 2024;63(1):30-37. doi: 10.2478/Sjph-2024-0005. ABSTRACT Keywords: Patient Reported Indicator Surveys (PaRIS) Primary care Quality of healthcare Patient experience Data collection methodology IZVLEČEK Ključne besede: ocena izkušenj in izidov zdravstvene oskrbe primarno zdravstvo kakovost zdravstvene oskrbe pacientove izkušnje metodologije zbiranja podatkov Introduction: Healthcare systems collect little information about the experiences and outcomes of care from the perspectives of patients. Patient Reported Indicator Surveys (PaRIS) is an OECD initiative to measure the outcomes and experiences of people living with chronic conditions, who are managed in primary care. Objectives: To evaluate the feasibility of the methodology employed in the Field Trial of the PaRIS survey in Slovenia and propose adjustments to enhance sampling in the Main Survey. Methods: In 2022, we conducted a cross-sectional observational study in 50 family medicine practices in Slovenia with a target of recruiting 70 patients per practice. We used the Slovenian version of the PaRIS questionnaires, and evaluated sampling and data collection. Results: The sample contained 21 providers (42.0% response rate) and 454 patients (50.7% response rate). The provider sample did not differ from the population characteristics, while the patient sample differed significantly from the patient population. All providers completed the survey online, in 20.9±11.1 minutes and had 1.5±1.5 restarts. Most patients (74.9%) completed the survey online and needed 36.0±22.6 minutes, and the mean number of restarts was 1.4±2.2. Conclusion: Based on the results, we recommend conducting a methodology test for quality assessment studies before initiating the main survey. Legal issues should be addressed and considered early when developing the methodology. It is also necessary to be aware of the feasibility of the study in practice, to avoid a low participation rate. Uvod: Zdravstveni sistemi zbirajo malo podatkov o izkušnjah in izidih oskrbe z zornega kota pacientov. Ocena izkušenj in izidov zdravstvene oskrbe pacientov s kroničnimi nenalezljivimi boleznimi (projekt PaRIS) je pobuda OECD za merjenje izidov in izkušenj ljudi s kroničnimi boleznimi, ki so obravnavani na primarnem zdravstvenem nivoju. Cilj je oceniti izvedljivost in ustreznost uporabljene metodologije pilotne izvedbe raziskave PaRIS v Sloveniji in predlagati izboljšave vzorčenja in zbiranja podatkov v glavni raziskavi. Metode: Leta 2022 smo v Sloveniji izvedli presečno opazovalno študijo v 50 ambulantah družinske medicine, s ciljem povabiti k sodelovanju po 70 pacientov iz vsake ambulante. Uporabili smo slovensko različico vprašalnikov PaRIS. Ocenili smo vzorčenje in zbiranje podatkov. Rezultati: Vzorec je zajel 21 izvajalcev zdravstvenih storitev (42,0-% odzivnost) in 454 pacientov (50,7-% odzivnost). Vzorec izvajalcev zdravstvenih storitev se ni razlikoval od značilnosti populacije, medtem ko se je vzorec pacientov bistveno razlikoval od populacije pacientov. Vsi izvajalci zdravstvenih storitev so anketo izpolnili prek spleta v 20,9 ± 11,1 minute pri čemer so imeli povprečno 1,5 ± 1,5 ponovnih zagonov. Večina pacientov (74,9 %) je anketo izpolnila prek spleta. Čas izpolnjevanja spletne ankete je bil 36,0 ± 22,6 minute, povprečno število ponovnih zagonov pa 1,4 ± 2,2. Zaključki: Na podlagi rezultatov lahko predlagamo, da se pri študijah ocenjevanja kakovosti, test metodologije izvede pred izvedbo glavne raziskave. Pravne prepreke in značilnosti je treba nasloviti in upoštevati zgodaj v razvoju metodologije. Prav tako je treba biti pozoren na izvedljivost študije v praksi, da se prepreči nizka stopnja sodelovanja. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Healthcare systems that place greater emphasis on the quality of primary care have better clinical outcomes and are more economically efficient and sustainable [1–5]. The patient’s experience with the healthcare system is being recognised as an important dimension of the quality of healthcare. It is thus an increasingly common strategy to monitor the quality of healthcare at all levels [6–9]. Tools that assess patient experience enable patients to provide feedback and potentially strengthen a patient- centred health system [10]. A standardised data collection procedure is preferred in international surveys. Due to a wide variety of legal, ethical and cultural differences in primary care, a data collection strategy is adapted for each country in agreement with the consortium [11]. To date the largest international research on the quality of primary care (Quality and Costs of Primary Care in Europe (QUALICOPC)) was carried out in 31 countries between 2010 and 2013, and it included assessment of the patients’ experiences [12]. This study was crucial in developing the methodology for future large-scale international studies such as the Patient Reported Indicator Surveys (PaRIS). In 2017, the OECD launched the PaRIS initiative to systematically measure the outcomes and experiences of healthcare that matter most to people [13]. The flagship project within the PaRIS initiative is a survey of people living with chronic conditions who are managed in primary care, namely the PaRIS survey [11]. The PaRIS survey has seen three different phases: 1) design and development phase (2017-2020), including the study design and the development of PaRIS questionnaires; 2) field trial (2021- 2022), including the testing of the PaRIS survey design and PaRIS questionnaires in participating countries; and 3) main survey (2022-2023), including the full implementation of the PaRIS survey in participating countries and data collection [14]. In this paper, we aimed to evaluate the feasibility of the methodology employed in the field trial of the PaRIS survey in Slovenia and propose adjustments to enhance sampling in the main survey. While the manuscript does not present content-related field trial results, it provides a comprehensive assessment of the survey’s methodology. 2 METHODS 2.1 Design of the study The PaRIS survey is a cross-sectional observational study, and the study protocol is described in detail elsewhere [11]. The study presented here took place in Slovenian family medicine practices. Family medicine physicians work either as private contractors or in primary care 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 31 centres, and both are part of national public primary care system. One family medicine team consists of 1 full time equivalent (FTE) family physician, 1 FTE practice nurse and 0.5 FTE registered nurse. Practice nurses are mainly involved in tasks including administration, appointments, and clinical work, while registered nurses work as nurse practitioners and carry out preventive activities and care for stable chronic patients [15, 16]. 2.2 Study population 2.2.1 Providers We defined providers as primary care practices in line with the study design of the PaRIS survey. In Slovenia one family physician works in one family medicine practice, and the registry of family medicine practices is available online, with the system led by the Health Insurance Institute of Slovenia (ZZZS). We used this registry for random sampling, by assigning consecutive numbers to every physician in the registry and then using a random number generator to select them. The registry contains all family physicians working primarily with registered patients. Physicians that provide healthcare only to institutionalised patients or children, or who practice occupational healthcare services as their primary area of work, were excluded. There were 978 eligible family medicine physicians in Slovenia in February 2022. As this was a field trial, we aimed for 25 family medicine practices in a final sample. Based on previous studies, we estimated at least a 50% positive response rate from the contacted family physicians, and thus to achieve the target we invited 50 practices to participate. 2.2.2 Patients All patients aged 45 years old or more, living in the community and having had any contact (including via phone or email) with the practice in the six months preceding the study, were eligible to participate. People living in a long-term care facility, healthcare or other residential institution were excluded, as were those in the terminal phase of illness or with an advanced cognitive impairment. Sampling of patients was random, and we aimed at 70 patients per practice. Practices autonomously generated a list of eligible patients, utilising either a random number generator or systematically selecting every Nth patient. Each eligible patient was first contacted by the practice and given the necessary information about the study, and could then provide informed consent and contact data if they wanted to participate. After that, the information gathered by the practice was sent to the researchers. 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 32 2.3 Data collection To collect data, we used the PaRIS provider and patient questionnaires. 2.3.1 Provider questionnaire The provider questionnaire includes 40 questions and defines the structure and characteristics of the team providing primary care and of the provider responsible – the family physician. The questionnaire was developed by the OECD and an international consortium (PaRIS-SUR consortium) in collaboration with patients and provider organisations, including the World Organization of Family Doctors (WONCA), and other stakeholders [11]. The questionnaire covers domains related to care delivery such as urbanisation, practice type, care model, information and administration systems, remote consultations, skill- mix, and remuneration. The estimated time for completion is 10-15 minutes. 2.3.2 Patient questionnaire The patient questionnaire includes 121 questions addressing individual and sociodemographic factors, health and healthcare capabilities, health behaviours and lifestyle, Patient Reported Outcome Measures (PROMs) such as symptoms and functioning, and Patient Reported Experience Measures (PREMs) such as coordination of care and safety. The PaRIS patient questionnaire is based on the PaRIS conceptual framework and consists of validated and established tools, already described elsewhere [11]. The estimated time to complete the patient questionnaire is 25 minutes. 2.3.3 Translation and cognitive testing The PaRIS questionnaires has two source questionnaires in English and French. The base for developing the Slovenian questionnaires was the original English versions developed and cognitively tested by the PaRIS consortium. We first translated the original English questionnaires into Slovene using the TRAP-D approach, as recommended by the guidelines [17]. The resulting questionnaires were then cognitively tested using the four-stage model of cognitive testing developed by Tourangeau [18]. Cognitive testing was only performed on questions that were identified as necessary for testing by the consortium. Changes were incorporated in the final version of the field trial questionnaires. 2.3.4 Data collection process and contact strategy After acquiring the sample of providers those who were selected were contacted via post in which they received information about the study and an invitation to participate. Within a week after sending the invitation letter, all providers were contacted again via phone by the research team. Those who agreed to participate received a link to the survey for providers and a spreadsheet to sample their patients. The practitioners had a month to sample 70 eligible patients. They were reminded weekly by the research team via email to complete the survey and provide the completed spreadsheet (Figure 1). Symbolic incentives were used to motivate physicians to participate, such as one-time free registration to national congresses of primary care and feedback information for their team. The patients in the sample were contacted via their preferred method. All patients who provided an email address received the survey link via email. Those who did not provide an email address received the survey link via SMS. Those who did not provide either an email or telephone number, or who opted for a paper survey, received this via the regular mail system. Non-respondents who provided an email address received a reminder via email for two consecutive weeks after receiving the link to the survey, and those who did not give an email address received two reminders via regular mail. 2.4 Evaluation of results For the quality check of the samples, we identified the characteristics of the provider and patient population and performed chi-square and independent samples t-test. For the providers, we used the data from the aforementioned list of practices. We performed the sample check according to geographic distribution (10 health regions) and status of the providers (public or private with a concession). For the patients, we used the data from the participating practices with regard to the age and gender of their registered patients who were age 45 years or older. For the evaluation of data collection, we used the following variables: Mode of completion, participation time for the e-survey, number of restarts with the e-survey, method of access to the e-survey, return code (partial/complete), and type of device used to complete the e-survey. The appropriateness and feasibility of the methodology were assessed using predefined criteria, including achieving at least 80% of the target sample size and a response rate exceeding 30%. The research team systematically documented reasons for non-participation by directly querying both providers and patients. 3 RESULTS 3.1 Response rate On the providers’ side, 21 family medicine practices of the 50 sampled practices participated in the study (42.0%). 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 33 The family medicine practices managed to recruit 896 patients, out of which 454 responded to the patient questionnaire (50.7%) (Figure 1). 3.1.1 Characteristics of the participants Most providers that completed the questionnaire were family physicians, working in the public sector, located in a city and in a solo practice (Table 1). Most patients were female, 60 years of age and older, with secondary education, retired and living in a rural area (Table 2). Figure 1. Schematic presentation of the process of recruiting of family physicians and patients. Characteristics of the providers. Characteristics of the patients. Professional background Family physician Family medicine trainee Type of practice Public (in a primary care centre) Private with a concession No answer Practice location City Town or suburb Rural area Work type Solo practice Group practice with own patients Group practice with shared patients Sex Male Female I prefer not to say Not answered Age group 44 years old or younger 45-49 years old 50-54 years old 55-59 years old 60-64 years old 65-69 years old 70-74 years old 75-79 years old 80-84 years old 85 years or older Not answered Education Primary school not completed Primary school Vocational school Secondary school University education Master’s degree or doctorate Not answered Employment status Self-employed In paid employment Looking for work Looking after the home Unable to work due to sickness or ill-health Retired Other Not answered Living area City Town or suburb Rural area Don’t know Not answered 18 3 10 3 8 9 5 7 16 4 1 170 249 2 33 2 32 45 69 74 95 61 27 13 4 32 9 52 123 230 19 54 35 14 150 4 7 16 227 2 34 105 69 245 1 34 85.7 14.3 47.6 14.3 38.1 42.9 23.8 33.3 76.2 19.0 4.8 37.4 54.8 0.4 7.3 0.4 7.0 9.9 15.2 16.3 20.9 13.4 5.9 2.9 0.9 7.0 2.0 11.5 27.1 50.7 4.2 11.9 7.7 3.1 33.0 0.9 1.5 3.5 50.0 0.4 7.5 23.1 15.2 54.0 0.2 7.5 N N % % Characteristic Characteristic Table 1. Table 2. 3.2 Sampling evaluation The size of the provider population was 978 and the size of the sample was 50. There was no significant difference between the sample and the population regarding geographical distribution and status of the practice (Table 3). 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 34 Provider quality sample check. Patient quality sample check. Geographical distribution Region 1 Region 2 Region 3 Region 4 Region 5 Region 6 Region 7 Region 8 Region 9 Region 10 Status Public Private with a concession Gender Male Female Age group 45-49 years old 50-54 years old 55-59 years old 60-64 years old 65-69 years old 70-74 years old 75-79 years old 80-84 years old 85 years or older 5 vs. 100 4 vs.74 2 vs. 33 5 vs. 100 15 vs. 303 7 vs. 142 3 vs. 55 3 vs. 53 3 vs. 56 3 vs. 62 36 vs. 710 14 vs. 268 170 vs. 84944 249 vs. 95749 32 vs. 19672 45 vs. 11343 69 vs. 15712 74 vs. 14526 95 vs. 14544 61 vs. 12801 27 vs. 16070 13 vs. 8791 4 vs. 7378 9 1 1 8 0.151 0.009 6.954 98.120 1.000 0.926 0.008 <0.001 Number (sample vs. population) Number (sample vs. population) Degrees of freedom Degrees of freedom Pearson chi-square Pearson chi-square p value p value Characteristic Characteristic Table 3. Table 4. The sample and population of patients differed significantly regarding gender and age (Table 4). 3.3 Data collection evaluation All the providers completed the survey online. They mostly used a desktop computer and accessed the survey via a link sent by email (Table 5). The providers completed the e-survey in 20.9±11.1 minutes and had 1.5±1.5 restarts. Most patients completed the survey online (e-survey), others used a paper-based questionnaire. They mostly accessed the e-survey via a link sent by email and used a smartphone to do so. Only a small percentage of patients left the e-survey incomplete (Table 5). The completion time was 36.0±22.6 minutes and the mean number of restarts was 1.4±2.2. Data collection evaluation – providers and patients. Mode of completion Online Paper Method of access to e-survey Link via email Link via SMS Device used for e-survey Desktop Smartphone Tablet Unknown Return code Complete Partial (incomplete e-survey) 21 (100) 0 21 (100) 0 18 (85.7) 2 (9.5) 1 (4.8) 0 21 (100) 0 340 (74.9) 114 (25.1) 199 (58.5) 141 (41.5) 127 (28.0) 195 (43.0) 17 (3.7) 1 (0.2) 420 (92.5) 34 (7.5) Providers N (%) Patients N (%) Characteristic Table 5. 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 35 3.4 Reasons for non-participation Some of the providers refused to participate in the study due to the workload expected from them regarding the sampling and data collection. They stated that they were overburden with their usual work and did not have time to engage in the survey. Some of the patients refused to participate due to low interest in the survey, the questionnaire being seen as too long for them, and not having enough time to complete it. 4 DISCUSSON 4.1 Main findings The evaluation of the PaRIS survey field trial in Slovenia showed satisfactory results regarding the sampling of providers and data collection among them. However, the sampling of the patients and data collection among them did not yield the desired evaluation outcomes, the main problem being the small sample size. The response rate of the practices was satisfactory. The literature shows that the response rate can vary significantly (from 20-80%) [19–21], although there are several methods that can be used to improve it [14, 20]. We used personal contact and reminders, which seemed to be effective. The survey was mostly completed by family physicians. This was expected, as the physician is a team leader in primary care practices in Slovenia, and access to the patient list through the physician was required for further sampling. Regarding the work type of the practice, most providers were in a solo practice. Here, it should be noted that in Slovenia around 70% of the family medicine practices are in community health centres, and 30% work as private practices with a concession [22]. Family physicians in health centres are practicing in their own practice with their own registered patients, working with a team of their own nurses. However, a health centre composed of a family physician, practice nurse and registered nurse is not considered as a group practice in Slovenia. A group practice in Slovenia is described as a practice where two or more physicians are working in the same practice with other healthcare professionals [23]. Hence, what is considered to be a group practice in the international PaRIS survey was recorded as solo practice in Slovenia. The sample size of the patients was much smaller than anticipated, which is probably a consequence of how the GDPR is interpreted in Slovenia. With this, practices are not allowed to send the contact information of their patients to researchers without the patients’ consent. This forced the researchers to adapt the international methodology, and to ask the practices to both sample the patients and acquire their informed consent. As expected, this placed a significant additional burden on the practices, which then failed to provide a sufficient patient sample size. All the providers accessed the questionnaire online, and in the literature no significant differences were found regarding the mode of survey [19, 21]. The survey was completed by the providers in 20 minutes, whereas we anticipated 15 minutes would be needed. However, we still consider this feasible. A quarter of the patients completed a paper questionnaire. Others completed an online survey via link sent by email or SMS. A meta-analysis showed that the response rate to online surveys is approximately 40% [23], and this is consistent with our results. On the other hand, the literature shows that using a paper questionnaire does not yield a lower response rate [24]. This might indicate that paper questionnaires can now be dropped, but this would be a mistake as some respondents still prefer paper versions [24]. Patients on average needed 36 minutes to complete the online survey, while there is no data on the time needed for the paper questionnaire. We expected that the online survey should be completed in 25 minutes, which turned out to be rather optimistic. The literature shows that questionnaires should include 25 to 30 questions, and be possible to complete in 30 minutes [24, 25]. In the case of longer questionnaires, we can expect a larger percentage of non-responders, a larger amount of missing data, and an increase in providing answers automatically with the last items [24, 25]. In our study, only a small number of patients left the questionnaire incomplete, which indicates that although the questionnaire was long, it was still manageable. 4.2 Strengths and limitations One limitation of the study was that we could not evaluate the same indicators on the paper-based survey. It is thus possible that we missed some lessons for the main survey. Another limitation is the missing characteristics of the population and sample, which would have helped with regard to detecting additional significant differences. Moreover, the patient sample did not reflect the characteristics of the population, and there was also a problem with sampling according to inclusion criteria, as two patients were younger than 45 years old, although this could be due to an error when completing the questionnaire. Therefore, we cannot assume that the patient sample was representative, and one solution might be to omit this age category within the questionnaire. 4.3 Implications We evaluated the sampling method of the practices as satisfactory, so no adjustments will be made for the main survey. The same is true for data collection of the practices. For patient sampling, the evaluation indicated the need for revision of the methodology to achieve a better response rate. We are thus planning to provide the practices with printed survey invitations for patients containing a QR code with the link to the online questionnaire. This seems to be a good method of data collection [26]. We also plan to provide the practices with paper questionnaires. The practices will draw up a patient sample as in the field trial, but will not need to obtain the patients’ informed consent for sending the contact information. Instead, the practices will mail either the QR code or a paper questionnaire to the patients in the sample. With such methodology, we expect a lower burden on the practices and higher response rate. On the other hand, such an approach will make it impossible to only send the reminders to the non- respondents. Shortening the patient questionnaire might also contribute to better response rates. 5 CONCLUSION This study provides insights into the methodological challenges when conducting a cross-sectional observational study on patient outcomes and experiences in Slovenian primary care. The results suggest a need for pre-testing the methodology of quality assessment studies before the main survey, enabling researchers to refine their approaches for optimal results. Early consideration of legal issues in methodology development is essential. Additionally, awareness of study feasibility is crucial to mitigate low participation rates. ACKNOWLEDGEMENTS The PaRIS survey is an OECD initiative. The PaRIS-SUR Consortium, consisting of Nivel, Exeter University, Ipsos, Avis Donabedian Research Institute, and Optimedis assisted the OECD in the development and implementation. The authors thank the Working Party for PaRIS, the international PaRIS Patient Panel and the Technical Advisory Community and various experts for their roles throughout the study. The authors also thank the national project management teams, participating primary care professionals and patients. PaRIS is funded by the participating countries and receives a contribution from the European Commission. The views expressed and arguments employed herein are solely those of the author(s) and do not necessarily reflect the views of the OECD or its member countries. The OECD cannot be held responsible for possible violations of copyright resulting from the posting of any written material on this website. FUNDING The authors acknowledge that the project (Assessment of experiences and outcomes of healthcare for patients with chronic non-communicable diseases in family medicine in Slovenia, ID V3-2102) was financially supported by the Slovenian Research Agency. PaRIS is funded by the participating countries and receives a contribution from the European Commission. ETHICAL APPROVAL The study in Slovenia was approved by the National Ethics Committee (No. 0120-260/2021/3). CONFLICTS OF INTEREST The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. AVAILABILITY OF DATA AND MATERIALS The data that support the findings of this study are available from OECD. Restrictions apply to the availability of these data, which were used under license for this study. Data are available from the authors with the permission of OECD. ORCID Matija Ambrož: https://orcid.org/0009-0008-3406-5693 Candan Kendir: https://orcid.org/0000-0001-6877-4836 Wienke Boerma: https://orcid.org/0000-0001-7698-0015 Zalika Klemenc Ketiš: https://orcid.org/0000-0002-0270-1754 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 36 REFERENCES 1. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83:457–502. doi: 10.1111/j.1468- 0009.2005.00409.x. 2. Macinko J, Starfield B, Shi L. The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970-1998. Health Serv Res. 2003;38:831–865. doi: 10.1111/1475-6773.00149. 3. Kringos D, Boerma W, Bourgueil Y, Cartier T, Dedeu T, Hasvold T, et al. The strength of primary care in Europe: An international comparative study. Br J Gen Pract. 2013;63:e742-e750. doi: 10.3399/bjgp13X674422. 4. Klemenc Ketiš Z, Rochfort A. Sustainability for planetary health: A seventh domain of quality in primary care. Zdr Varst. 2022;61:198–200. doi: 10.2478/sjph-2022-0026. 5. Virtič T, Mihevc M, Zavrnik Č, Mori Lukančič M, Poplas Susič A, Klemenc-Ketiš Z. Peer support as part of scaling-up integrated care in patients with type 2 diabetes and arterial hypertension at the primary healthcare level: A study protocol. Zdr Varst. 2023;62:93–100. doi: 10.2478/sjph-2023-0013 6. 6Gleeson H, Calderon A, Swami V, Deighton J, Wolpert M, Edbrooke- Childs J. Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open. 2016;6:e011907. doi: 10.1136/bmjopen-2016-011907. 7. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3:e001570. doi: 10.1136/bmjopen-2012-001570. 8. de Boer D, Delnoij D, Rademakers J. Do patient experiences on priority aspects of health care predict their global rating of quality of care? A study in five patient groups. Health Expect. 2010;13:285–297. doi: 10.1111/j.1369-7625.2010.00591.x. 9. Prevolnik Rupel V, Došenović Bonča P. Is value-based health care just the latest fad or can it transform the Slovenian health care system? Zdr Varst. 2023;62:55–58. doi: 10.2478/sjph-2023-0008 10. Holt JM. Patient experience in primary care: A systematic review of CG-CAHPS surveys. J Patient Exp. 2019;6:93–102. doi: 10.1177/2374373518793143. 11. de Boer D, van den Berg M, Ballester M, Bloemeke J, Boerma W, de Bienassis K, et al. Assessing the outcomes and experiences of care from the perspective of people living with chronic conditions, to support countries in developing people-centred policies and practices: Study protocol of the International Survey of People Living with Chronic Conditions (PaRIS survey). BMJ Open. 2022;12:e061424. doi: 10.1136/ bmjopen-2022-061424. 12. Schäfer WLA, Boerma WGW, Kringos DS, De Maeseneer J, Gress S, Heinemann S, et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract. 2011;12:115. doi: 10.1186/1471-2296-12-115. 13. Recommendations to OECD ministers of health from the high level reflection group on the future of health statistics: Strengthening the international comparison of health system performance through patient-reported indicators [Internet]. Paris: OECD; 2017 [cited 2023 Aug 25]. Available from: https://www.oecd.org/health/ Recommendations-from-high-level-reflection-group-on-the-future-of- health-statistics.pdf 14. 14. Kendir C, Naik R, Bloemeke J, Bienassis K de, Larrain N, Klazinga N, et al. All hands on deck: Co-developing the first international survey of people living with chronic conditions [Internet]. 2023 [cited 2023 Aug 25]. Available from: https://www.oecd-ilibrary.org/content/ paper/8b31022e-en 15. Poplas Susic A, Svab I, Klemenc Ketis Z. Upgrading the model of care in family medicine: A Slovenian example. Public Health Panorama. 2018:550–555. 16. Susič AP, Klemenc-Ketiš Z. Successful implementation of integrated care in Slovenian primary care. Zdr Varst. 2021;60:1–3. doi: 10.2478/ sjph-2021-0001. 17. Mohler P, Dorer B, De Jong J, Hu M. Guidelines for best practice in cross-cultural surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan; 2016. 18. Collins D. Cognitive interviewing practice. London; 2023. doi: 10.4135/9781473910102 19. Taylor T, Scott A. Do physicians prefer to complete online or mail surveys? Findings from a national longitudinal survey. Eval Health Prof. 2019;42:41–70. doi: 10.1177/0163278718807744. 20. Brtnikova M, Crane LA, Allison MA, Hurley LP, Beaty BL, Kempe A. A method for achieving high response rates in national surveys of U.S. primary care physicians. PLoS One. 2018;13:e0202755. doi: 10.1371/ journal.pone.0202755. 21. Weaver L, Beebe TJ, Rockwood T. The impact of survey mode on the response rate in a survey of the factors that influence Minnesota physicians’ disclosure practices. BMC Med Res Methodol. 2019;19:73. doi: 10.1186/s12874-019-0719-7. 22. Klančar D, Švab I. Primary care principles and community health centers in the countries of former Yugoslavia. Health Policy. 2014;118:166–172. doi: 10.1016/j.healthpol.2014.08.014. 23. Muhlestein DB, Smith NJ. Physician consolidation: Rapid movement from small to large group practices, 2013–15. Health Aff (Millwood). 2016;35:1638–1642. doi: 10.1377/hlthaff.2016.0130. 24. Horevoorts NJ, Vissers PA, Mols F, Thong MS, van de Poll-Franse LV. Response rates for patient-reported outcomes using web-based versus paper questionnaires: Comparison of two invitational methods in older colorectal cancer patients. J Med Internet Res. 2015;17:e111. doi: 10.2196/jmir.3741. 25. Sahlqvist S, Song Y, Bull F, Adams E, Preston J, Ogilvie D, et al. Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: Randomised controlled trial. BMC Med Res Methodol. 2011;11:62. doi: 10.1186/1471-2288-11-62. 26. Azad A, Laidlaw DAH, Orlans HO. Using QR smartphone technology to improve patient communication and information distribution. Eye. 2022;36:1321–1322. doi: 10.1038/s41433-021-01757-x. 10.2478/sjph-2024-0005 Zdr Varst. 2024;63(1):30-37 37 FACILITATORS AND BARRIERS TO SCALING-UP INTEGRATED CARE FOR ARTERIAL HYPERTENSION AND TYPE 2 DIABETES IN SLOVENIA: QUALITATIVE STUDY Črt ZAVRNIK 1,2* , Nataša STOJNIĆ 1 , Majda MORI LUKANČIČ 1 , Matic MIHEVC 1,2,3 , Tina VIRTIČ POTOČNIK 1,4,5 , Zalika KLEMENC-KETIŠ 1,2,5 , Antonija POPLAS SUSIČ 1,2 1 Community Health Centre Ljubljana, Primary Healthcare Research and Development Institute, Metelkova ulica 9, 1000 Ljubljana, Slovenia 2 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia 3 Primary Healthcare Centre Trebnje, Goliev trg 3, 8210 Trebnje, Slovenia 4 Primary Healthcare Centre Slovenj Gradec, Partizanska pot 16, 2380 Slovenj Gradec, Slovenia 5 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska ulica 8, 2000 Maribor, Slovenia Received: Oct 31, 2023 Accepted: Nov 27, 2023 Original scientific article *Correspondence: crt.zavrnik@zd-lj.si 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 38 PRILOŽNOSTI IN OVIRE ZA IZBOLJŠANJE CELOSTNE OSKRBE ARTERIJSKE HIPERTENZIJE IN SLADKORNE BOLEZNI TIPA 2 V SLOVENIJI: KVALITATIVNA ŠTUDIJA Zavrnik Č, Stojnić N, Mori Lukančič M, Mihevc M, Virtič Potočnik T, Klemenc-Ketiš Z, Poplas Susič A. Facilitators and barriers to scaling-up integrated care for arterial hypertension and type 2 diabetes in Slovenia: Qualitative study. Zdr Varst. 2024;63(1):38-45. doi: 10.2478/Sjph-2024-0006. ABSTRACT Keywords: Integrated care Hypertension Type 2 diabetes Barriers Facilitators IZVLEČEK Ključne besede: integrirana oskrba arterijska hipertenzija sladkorna bolezen tipa 2 ovire priložnosti Introduction: Arterial hypertension and type 2 diabetes are significant contributors to global non-communicable disease-related mortality. Integrated care, centred on person-centred principles, aims to enhance healthcare quality and access, especially for vulnerable populations. This study investigates integrated care for these diseases in Slovenia, providing a comprehensive analysis of facilitators and barriers influencing scalability. Methods: Qualitative methods, including focus group discussions and semi-structured interviews, were employed in line with the grounded theory approach. Participants represented various levels (micro, meso and macro), ensuring diverse perspectives. Data were collected from May 2019 to April 2020, until reaching saturation. Transcripts were analysed thematically using NVivo software. Results: Nine categories emerged: Governance, Health financing, Organisation of healthcare, Health workforce, Patients, Community links, Collaboration/Communication, Pharmaceuticals, and Health information systems. Some of identified barriers were political inertia and underutilisation of research findings in practice; outdated health financing system; accessibility challenges, especially for vulnerable populations; healthcare workforce knowledge and burnout; patients’ complex role in accepting and managing their conditions; collaboration within healthcare teams; and fragmentation of health information systems. Peer support and telemedicine were the only two potential solutions identified. Conclusions: This study offers a comprehensive evaluation of integrated care for hypertension and type 2 diabetes in Slovenia, featuring insights into facilitators and barriers. These findings have implications for policy and practice. Monitoring integrated care progress, refining strategies, and enhancing care quality for patients with these two diseases should be priorities in Slovenia. Uvod: Arterijska hipertenzija in sladkorna bolezen tipa 2 sta kronični bolezni, ki pomembno prispevata k smrtnosti zaradi nenalezljivih bolezni. Integrirana oskrba, osredotočena na posameznika, si prizadeva izboljšati kakovost in dostopnost zdravstvenega varstva, zlasti za ranljive skupine prebivalstva. Namen te raziskave je prepoznati ovire in spodbujevalce za izboljšanje integrirane oskrbe teh dveh bolezni v Sloveniji. Metode: Izvedena je bila kvalitativna raziskava po principih utemeljitvene analize z uporabo fokusnih skupin in polstrukturiranih intervjujev. Udeleženci so predstavljali različne ravni (mikro, mezo in makro), kar je zagotavljalo raznolikost stališč. Podatki so bili zbrani od maja 2019 do aprila 2020, dosežena je bila nasičenost. Transkripte smo tematsko analizirali z uporabo programske opreme NVivo. Rezultati: Prepoznanih je bilo devet kategorij: politika, financiranje zdravstva, organizacija zdravstvenega varstva, zdravstveni delavci, pacienti, povezava s skupnostjo, sodelovanje/komunikacija, farmacija in zdravstveni informacijski sistemi. Identificirane ovire so bile: politična inertnost in nezadostna implementacija raziskovalnih ugotovitev v prakso, zastarel sistem financiranja, izzivi glede dostopnosti do zdravstvenih storitev (zlasti za ranljive skupine prebivalstva), pomanjkanje znanja in izgorelost zdravstvenih delavcev, kompleksna vloga pacientov pri sprejemanju in obvladovanju njihove bolezni, sodelovanje znotraj zdravstvenih timov in razdrobljenost zdravstvenih informacijskih sistemov. Med predlaganimi priložnostmi sta izstopala implementacija laičnega svetovalca in telemedicine. Zaključki: Ta študija prinaša celovito oceno integrirane oskrbe hipertenzije in sladkorne bolezni tipa 2 v Sloveniji ter vpogled v raznolika stališča deležnikov. Prednostno nalogo v Sloveniji predstavljajo sledenje napredku integrirane oskrbe, izboljševanje strategij in povečanje kakovosti oskrbe pacientov s tema dvema boleznima. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Arterial hypertension (HTN) and type 2 diabetes (T2D) stand as the leading chronic non-communicable diseases, accounting for nearly 70% of all deaths. Consequently, they have emerged as a growing global public health concern, driven by rapid urbanisation, an aging population, and the worldwide spread of unhealthy lifestyles (1, 2). Integrated care has been recognised as a crucial approach to confront this challenge, aligning with the concept of person- centred care. It aims to enhance access to healthcare, as well as its quality and continuity (3). The integrated care model for chronic non-communicable diseases, endorsed by the World Health Organization (WHO), encompasses the following elements: (a) identification, (b) primary healthcare (PHC) treatment, (c) health education, (d) self-management support, and (e) cooperation between care providers (4). Although many countries have partially implemented this model within their health systems, various facilitators and barriers persist across different levels, including patients, healthcare providers, and decision-makers (5, 6). In addition, different health systems worldwide exhibit distinct facilitators and barriers when it comes to scaling up the integrated care for these two diseases (7). At the PHC level in Slovenia, a capitation system is established, and family physicians play a vital role as gatekeepers. They act as the central coordinators of care, both horizontally – collaborating with other healthcare professionals at the PHC, such as clinical pharmacists, physiotherapists, social workers, clinical psychologists, etc. – and vertically, liaising with specialists at the secondary and tertiary levels, as well as with the community (8). Advanced nurses assume responsibility for screening and managing patients with specific chronic non-communicable diseases, including HTN and T2D, using established protocols (9). Several studies have been conducted to appraise various dimensions of integrated care, encompassing care models (10, 11), quality indicators (12), the level of knowledge and adherence to clinical guidelines (13, 14), financial considerations (15), the impact on the quality of life (15, 16), extent of patient knowledge (17), and health professionals’ perspectives (18). Despite the aforementioned studies, a comprehensive evaluation of integrated care for HTN and T2D, involving participants from various levels, remains unexplored in Slovenia. To address this gap, this paper utilises qualitative data to (1) identify the perspectives of participants at different levels regarding the implementation of integrated care for HTN and T2D, and (2) identify the facilitators and barriers that affect the scalability of integrated care for HTN and T2D in Slovenia. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 39 2 METHODS 2.1 Study design and settings This study employed a qualitative study design, utilising focus group discussions (FGDs) and semi-structured in- depth interviews, while applying the grounded theory approach. It was a part of the SCUBY project (SCaleUp diaBetes and hYpertension care) (19). 2.2 Sample A multi-level WHO-based qualitative framework was employed, which categorises participants into three levels: micro (patients and health professionals), meso (healthcare providers), and macro (regulatory, financial, professional, and scientific stakeholders) (20). Participants were purposively put on the list and then randomly selected. Inclusion criteria were their relevance to the research themes and roles in the health system, ability to communicate verbally, representation of different geographic locations and coverage of all PHC team members (general practitioners, nurses, advanced nurses, prevention nurses, and community nurses). As data collection progressed, additional key informants relevant to the study were identified using the snowballing technique and subsequently added to the participant list. 2.3 Data collection A common thematic guide was initially developed by a research team based on potential facilitators and barriers, relying on both literature research and contextual knowledge (data is available on request). The main themes were identification, treatment in PHC, health education, self-management, and cooperation between healthcare providers. All interviews and FGDs were conducted in person by a researcher, accompanied by an observer who documented non-verbal communication. A team consisted of three researchers who had received advanced training from the experienced researcher APS prior to the commencement of the study, and were actively engaged in their roles as researchers during the study period. The research objective was introduced again prior to conducting interviews and FGDs, and participants were asked to provide written informed consent. Each participant was introduced using their first name. The meetings took place at a designated location with audio recording equipment. Each session lasted between 30 to 90 minutes and was audio recorded. Data saturation was achieved. The interviews and FGDs were transcribed verbatim. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 40 2.4 Analysis Analysis was conducted using the QSR NVivo software. Thematic mapping facilitated the systematic identification of recurring themes and their interrelationships. The final codebook was developed through a stepwise process consistent with the grounded theory approach, incorporating both inductive (bottom-up) and deductive principles. Two independent researchers carried out the analysis for each interview or FGD. They identified, compared, and categorised elements and concepts, employing open- coding principles to generate a list of emerging themes. The independence and clarity of each theme, as well as the criteria for its application and potential areas of overlap, were thoroughly evaluated. After achieving a consensus on the utility of each proposed theme, two researchers independently coded a transcript using a “chunking” approach, where subsets of text were assigned one or more themes and used to represent the specific context (21, 22). Themes were then gathered in categories and split in subthemes. The iterative analysis employed participant, method, researcher and supervisor triangulation, enabling a comprehensive exploration of facilitators and barriers to integrated care for HTN and T2D. 2.5 Ethical consideration The protocol of the overarching SCUBY project has received approval from the Institutional Review Board of the Institute of Tropical Medicine (ref: 1323/19) and the National Ethics Committee of Slovenia (ref: 0120- 219/2019/4). 3 RESULTS 3.1 Participants In total, 15 FGDs were conducted with participants at the micro level (seven with patients and eight with health professionals), along with 23 interviews (11 at the meso level and 12 at the macro level), spanning the period from May 2019 to April 2020. A detailed description of the FGDs with patients, FGDs with health professionals, and interviews is provided in Tables 1, 2, and 3, respectively. 3.2 Identified facilitators and barriers to scale-up 3.2.1 Governance In the theme of “Political interest, commitment, and power dynamics”, the barrier was the inactive and unresponsive political structure in Slovenia. Additional barriers emerged in the “Stakeholder collaboration” theme, with participants highlighting that innovations and adjustments are introduced without consulting relevant Participants in the focus group discussions with patients. Participants in the focus group discussions with patients. Legend: HTN – arterial hypertension; T2D – type 2 diabetes; N – number Legend: HTN – arterial hypertension; T2D – type 2 diabetes; N – number Gender male female Place of residence urban rural n/a Age group ≤49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 ≥85 Disease HTN T2D HTN and T2D Employment status employed unemployed retired n/a Gender male female Workplace location urban rural Profession general practitioner practice nurse advanced nurses prevention nurse community nurse 14 28 32 9 1 1 1 2 2 11 10 10 3 2 14 18 10 5 1 35 1 4 44 26 22 11 1 20 6 10 33.3 66.7 76.2 21.4 2.4 2.4 2.4 4.8 4.8 26.2 23.8 23.8 7.1 4.8 33.3 42.9 23.8 11.9 2.4 83.3 2.4 8.3 91.7 54.2 45.8 22.9 2.1 41.7 12.5 20.8 N N % % Characteristic Characteristic Table 1. Table 2. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 41 Participants in the interviews. Legend: HTN – arterial hypertension; T2D – type 2 diabetes Meso Macro Community Health Centre Ljubljana (3 interviews) and Postojna Institution for informal home care Municipality of Ljubljana (2 interviews) Associations of patients with chronic diseases (2 interviews) Associations of patients with chronic diseases Nursing home The National Institute of Public Health (2 interviews) The Health Insurance Institute of Slovenia Ministry of Health (2 interviews) Chamber of Pharmacies of Slovenia Health Council at Ministry of Health Nurses and Midwives Association of Slovenia Medical Faculty of Ljubljana, Department of Family medicine (3 interviews) Medical Chamber of Slovenia ParticipantParticipants level Table 3. experts. Moreover, already published research findings are not acknowledged by policymakers and remain unimplemented. “Innovations are implemented without prior piloting, but the pilot should be first and then implementation.” (Meso level, female, code ZDL-113) Within the “Policy, regulation, strategy” theme, potential facilitators encompass reduced taxes on healthy food choices and health-promoting recreational and educational activities. Additionally, possible improvements in food labelling, including larger and more comprehensive nutrient information, were identified. Ensuring that accurate content related to HTN and T2D is included in the media (internet, radio, television, etc.) was also deemed beneficial. Moreover, creating a national-level unified website for T2D and HTN, offering verified medical information in one location, implementing prescription training, and introducing a nominal pharmacy fee for medication collection to reduce excess medications at home were other facilitators that were identified. 3.2.2 Health financing In the “Health insurance and social protection” theme, the outdated healthcare financing system, which prioritises service quantity over quality, was acknowledged as a barrier. A consensus emerged that reform is needed to support an integrated care model. In the “User financial payment” theme, the increasing demand for patient co- payments for services and medications was recognised, with an agreement that patients should not bear the financial burden of healthcare financing. Within the “Budget/Sources of funding” theme, participants linked the aforementioned issue to budget constraints, inefficient spending, and inadequate resource allocation. “Financing models should be modernised to better suit integrated care, e.g., by focusing on health outcomes.” (Macro level, female, code MB-36) 3.2.3 Organisation of healthcare In the “Primary healthcare level” theme, the barriers related to healthcare facility accessibility based on location and the related financial burdens were identified. Urban areas struggle with a shortage of parking spaces near healthcare facilities, while rural regions face difficulties due to long distances to healthcare facilities. The introduction of advanced nurses in family practices was seen as a facilitator, enhancing monitoring and patient education. Patients less comfortable with technology faced a barrier when transitioning to new electronic communication methods, such as email or web portals. In the “Secondary and tertiary healthcare level” theme, prolonged waiting times, which lead patients to opt for self-funded healthcare services, were acknowledged as a significant barrier. Within the “Integration throughout the healthcare continuum” theme, participants noted the barrier of the duplication of services across various healthcare providers (e.g., patients with T2D often receive redundant treatments from family medicine doctors, advanced nurses, and diabetologists). They emphasised that expanding the range of services available at the PHC level and enhancing home care could be potential facilitators. Tailoring care to individual patient needs could also be encouraged by implementation of comprehensive and specific treatment plans, which include well-defined tasks for patients. Additionally, the adoption of telemedicine, especially for vulnerable populations such as geographically distant, elderly, or immobile patients, was highlighted as an effective strategy. “Our experience with telemedicine pilot projects taught us that some patients with HTN have been over-treated, while for patients with T2D it was empowering to receive feedback on their measurements.” (Macro level, female, code ZDT-5) In the “Teamwork” theme, the barrier that was identified was the lack of clearly defined responsibilities among healthcare providers. Conversely, the introduction of dietitians, mental health consultants, and social advisors into PHC teams, along with the well-defined delegation of tasks, were all identified as facilitators. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 42 Within the “Quality of care” theme, the implementation of regular supervision meetings aimed at improving patient care, education, risk management, and overall well-being was a recognised facilitator. 3.2.4 Health workforce Within the “Time burden” theme, the barrier of a shortage of medical personnel leading to excessive workloads and an elevated risk of burnout was identified. The burden of administrative tasks was also acknowledged as hindering curative and, particularly, preventive activities. “Physicians are burdened by computer tasks, reducing their ability to fully engage during check-ups.” (Patient, male, code FSFB-304) Within the “Education and training” theme, the barrier of insufficient knowledge and a lack of regular refreshment, including understanding patient motivation, were discerned. It was also acknowledged that healthcare workers at times fell short in offering adequate support, displayed impatience, were resistant to alternative treatment methods, and set a poor example by consuming unhealthy food and smoking in the presence of patients. 3.2.5 Patients Barriers identified in the “Patient empowerment” theme were that some patients find it challenging to accept a new illness and may not fully accept responsibility for their health and self-care, often shifting it onto the healthcare system. Although they have access to reliable and verified information, they frequently struggle to follow medical instructions and recommendations, such as regular medication use or attendance at workshops. In the “Lifestyle” theme, participants recognised the barriers associated with the motivation to make lifestyle changes. Incentivising patients who adopt a healthy lifestyle was thus identified as a facilitator. “If you’ve lived unhealthily for 60 years, your body initially resists change. It is hard, but progress can be made with gradual steps.” (Patient, male, code FSBA-180) 3.2.6 Community link Within the “Patients associations” theme, patient associations were identified as a facilitator by playing a crucial role in bridging the gap between healthcare professionals and the general public. They are seen as valuable sources of reliable information about diseases, empowerment, and emotional support. Patients put a high level of trust in these associations. In the “Individuals” theme, participants recognised the positive impact of educating patients’ family members. This education can improve patient care and reduce the burden on the healthcare system. Within the “Informal caregivers” theme, the introduction of peer supporters was highlighted as a facilitator to address the shortage of healthcare professionals and empower patients and their families. These would be individuals living with HTN and T2D who receive specialised training to offer reliable information to other patients. “Patients are more inclined to trust peer supporters who can relate through shared experiences – these cannot be received from health professionals.” (Meso level, male, code DDO-162) Within the “Local community” theme, multiple facilitators were recognised. Participants underscored the authority of municipalities (local authorities) and the need for them to play a more active role in promoting a healthy lifestyle. Additionally, there was a call for systematic education on maintaining a healthy lifestyle to be included in primary schools. Moreover, participants emphasised the importance of employers encouraging their employees to embrace healthy habits. 3.2.7 Collaboration/communication Within the “Horizontal and vertical collaboration” theme, participants noted barriers in both horizontal (within healthcare teams) and vertical collaboration (between different levels in the healthcare system). The latter is limited to formalised systems of referrals and reports, which are considered inadequate. Consequently, the exchange of information is challenging. “Collaboration between healthcare professionals and social workers is severely lacking, despite the pivotal role of social issues in elder care.” (Meso level, female, code ZDL1-215) 3.2.8 Pharmaceutical Within the “Pharmaceutical” theme, the participants noted that one facilitator was the role of the pharmaceutical industry and pharmacies in educating and advising patients on the safe and effective use of medications. Nonetheless, the participants also pointed out the barrier of an excessive focus on product sales by these entities, which, in their opinion, hinders the provision of trustworthy information regarding the effectiveness of specific drugs and dietary supplements. “I have reservations about pharmacies due to the overwhelming advertising in pharmacies, making me question the reliability of their information.” (Patient, female, code FSBA-315) 3.2.9 Health information systems In the “Fragmentation” theme, a barrier to the flow of information was found in the excessive number of health information systems, which lacked interconnectedness. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 43 Within the “Data Management System” theme, data protection regulations were identified as an additional complicating factor. Conversely, in the “E-health” theme, the introduction of improved and more user-friendly interfaces for information systems was recognised as a facilitator. 4 DISCUSSION This study represents a comprehensive exploration of integrated care for HTN and T2D in Slovenia, providing insights into the perspectives of participants at various levels within the healthcare system. It contributes to the existing body of knowledge from recent studies on this subject (18, 23). The study identified a range of barriers and facilitators related to the scaling-up of integrated care for these chronic conditions in various categories, including Governance, Health financing, Organisation of healthcare, Health workforce, Patients, Community links, Collaboration/Communication, Pharmaceutical, and Health information systems. In our study, patients emphasised the challenges associated with accepting a new illness and taking responsibility for their health. These findings align with a previous study that reported a significant impact of T2D on dietary choices and reliance on others, including family life (16). Our study underscores the crucial role of patient empowerment through education and self-management to improve disease management outcomes, consistent with previously published research (10, 18, 24, 25). However, another study suggests that, despite the implementation of the National Diabetes Prevention and Care Development Programme in Slovenia from 2011 to 2020, knowledge levels among elderly patients with T2D either remained stagnant or worsened (17). Therefore, our study advocates for the exploration of innovative approaches to structured patient education, such as national websites providing verified and reliable information on chronic diseases. In the literature several concepts have been identified as promising approaches, such as T2D-care groups in the Netherlands and group appointments in Canada (24, 25). The findings of our study highlight that patients have acknowledged the growing demand for co-payments for healthcare services and medications, as well as the increasing costs of transportation and healthy food. These factors present a significant challenge to effective care (18). Out-of-pocket expenses account for the second- largest portion (28.2%) of all HTN-related treatment costs in Slovenia, only behind expenditures on medicines (15). Potential solutions to address these barriers, as identified in our study, could include reducing taxes on activities and products that encourage a healthy lifestyle and implementing reforms in healthcare financing. In addition to the shortage of healthcare personnel, our study has revealed that certain participants observed that some healthcare professionals do not consistently update and improve their knowledge about HTN and T2D. Another published study also indicated that the management strategies of Slovenian family physicians for HTN may not always align with accepted HTN- guidelines (14). Similar barriers have also been identified in international contexts (26). In our study, healthcare workers emphasised their need for enhanced knowledge in motivating patients, recognising its potential to reduce their daily responsibilities. The results of our study reveal a predominant trend in healthcare delivery at the PHC level. The introduction of advanced nurses within family practices, actively screening patients for HTN and T2D, and providing regular check-ups, have all been recognised as making a significant contribution to holistic care (11). Similar findings were also reported in Switzerland (27). In Belgium, although the contribution of advanced nurses is acknowledged and they are increasingly employed in PHC practices, the lack of well-defined task descriptions still hinders their broader implementation (23, 28, 29). Another study that assessed six aspects of integrated care for HTN and T2D in Slovenia revealed that while patient identification was nearly fully implemented, self-management support lagged behind (10). These findings align with the results of our own study, and two possible solutions were identified. The introduction of peer supporters, who share their own experiences in managing HTN or T2D, is likely to be accepted by all participants (30). In addition, the implementation of telemonitoring can improve care for vulnerable patients, such as the elderly or those living in geographically distant areas (31). 4.1 Strengths and limitations of the study The strengths of this study lie in its comprehensive examination of integrated care for HTN and T2D in Slovenia. To establish credibility, we employed a triangulation approach by using multiple data collection methods, engaging various researchers and supervisors, and involving participants from different healthcare levels and diverse data collection settings. To ensure the transferability of our findings, we provided not only a detailed account of the participants’ experiences, but also the contextual factors. To maintain dependability and confirmability, we applied precise descriptions and maintained consistency in both data collection and analysis methods. Nonetheless, this study has certain limitations. Firstly, the study’s participants from the healthcare sector were solely recruited from PHC settings. Professionals working in secondary and tertiary healthcare may possess differing viewpoints that were not included in this study. Secondly, transcripts were not shared with the participants for their feedback or correction, which could have limited their ability to provide input and validate the data. 5 CONCLUSION This qualitative study has illuminated the perspectives of participants at different levels in Slovenia concerning integrated care for HTN and T2D. It underscores the imperative for multifaceted strategies addressing governance, health financing, the workforce, patient education, and healthcare system organisation. The insights garnered from this study hold significant value for guiding future healthcare policies and practices. To build upon this foundational knowledge, further research is imperative to monitor the progress of integrated care initiatives, gauge their impact on patient outcomes, and refine strategies for mitigating the identified barriers. The findings should act as a driving force for sustained efforts aimed at enhancing the quality of care for patients with HTN and T2D in Slovenia. CONFLICTS OF INTEREST The authors declare that no conflicts of interest exist. FUNDING The overarching SCUBY project was funded by the Horizon 2020 Framework Programme of the European Union (grant no: 825432). The study sponsor and funders were not involved in the study design; collection, management, analysis, and interpretation of data; writing of the report; or the decision to submit the report for publication. ETHICAL APPROVAL The protocol of the overarching SCUBY project has been approved by the National Ethics Committee of Slovenia (ref: 0120-219/2019/4). AVAILABILITY OF DATA AND MATERIALS The data presented in this study can be obtained upon request from the corresponding author. ORCID Črt Zavrnik: https://orcid.org/0000-0003-0654-452X Nataša Stojnić: https://orcid.org/0000-0002-1338-4774 Majda Mori Lukančič: https://orcid.org/0009-0004-8507-833X Matic Mihevc: https://orcid.org/0000-0003-4041-8682 Tina Virtič Potočnik: https://orcid.org/0000-0003-4925-3706 Zalika Klemenc Ketiš: https://orcid.org/0000-0002-0270-1754 Antonija Poplas Susič: https://orcid.org/0000-0002-4328-3333 REFERENCES 1. Zhou B, Carrillo-Larco RM, Danaei G, Riley LM, Paciorek CJ, Stevens GA, et al. Worldwide trends in hypertension prevalence and progress in treatment and control from 1990 to 2019: A pooled analysis of 1201 population-representative studies with 104 million participants. Lancet. 2021;398(10304):957–980. doi: 10.1016/S0140-6736(21)01330-1. 2. Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: Results from the International Diabetes Federation Diabetes Atlas, 9th edition. Diabetes Res Clin Pract. 2019;157:107843. doi: 10.1016/j.diabres.2019.107843. 3. Ferrer L, Goodwin N. What are the principles that underpin integrated care? Int J Integr Care. 2014;14:e037. doi: 10.5334/ijic.1884. 4. Demaio AR, Nielsen KK, Tersbøl BP, Kallestrup P, Meyrowitsch DW. Primary health care: A strategic framework for the prevention and control of chronic non-communicable disease. Glob Health Action. 2014;7:24504. doi: 10.3402/gha.v7.24504. 5. Threapleton DE, Chung RY, Wong SYS, Wong E, Chau P, Woo J, et al. Integrated care for older populations and its implementation facilitators and barriers: A rapid scoping review. Int J Qual Heal Care. 2017;29(3):327–334. doi: 10.1093/intqhc/mzx041. 6. Monaco A, Palmer K, Marengoni A, Maggi S, Hassan TA, Donde S. Integrated care for the management of ageing-related non- communicable diseases: Current gaps and future directions. Aging Clin Exp Res. 2020;32(7):1353–1358. doi: 10.1007/s40520-020-01533-z. 7. Kringos DS, Boerma WG, Bourgueil Y, Cartier T, Hasvold T, Hutchinson A, et al. The European primary care monitor: Structure, process and outcome indicators. BMC Fam Pract. 2010;11(1):81. doi: 10.1186/1471- 2296-11-81. 8. Albreht T, Pribakovic Brinovec R, Josar D, Poldrugovac M, Kostnapfel T, Zaletel M, et al. Slovenia: Health system review. Health Syst Transit. 2016;18(3):1–207. 9. Klemenc-Ketiš Z, Švab I, Stepanović A, Susič AP. Transition from a traditional to a comprehensive quality assurance system in Slovenian family medicine practices. Int J Qual Heal Care. 2019;31(4):319–322. doi: 10.1093/intqhc/mzy157. 10. Klemenc-Ketiš Z, Stojnić N, Zavrnik Č, Gorenjec NR, Danhieux K, Lukančič MM, et al. Implementation of integrated primary care for patients with diabetes and hypertension: A case from Slovenia. Int J Integr Care. 2021;21(3):15. doi: 10.5334/ijic.5637. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 44 11. Petek D, Mlakar M. Quality of care for patients with diabetes mellitus type 2 in “model practices” in Slovenia - first results. Zdr Varst. 2016;55(3):179–184. doi: 10.1515/sjph-2016-0023. 12. Klemenc-Ketiš Z, Švab I, Poplas Susič A. Implementing quality indicators for diabetes and hypertension in family medicine in Slovenia. Zdr Varst. 2017;56(4):211–219. doi: 10.1515/sjph-2017-0029. 13. Petek-Ster M, Kersnik J, Svab I. Compliance with hypertension guidelines in general practice in Slovenia. Srp Arh Celok Lek. 2007;135(3–4):191–196. 14. Šter MP, Žagar BB. Obravnava bolnikov z arterijsko hipertenzijo v ambulantah družinske medicine v Sloveniji – Ocena stanja po uvedbi referenčnih ambulant. Zdr Vestn. 2021;90(9–10):478–487. 15. Prevolnik Rupel V, Mori Lukančič M, Ogorevc M. Costs and quality of life in patients with systemic arterial hypertension in Slovenia. Value Heal Reg Issues. 2023;33:49–55. doi: 10.1016/j.vhri.2022.09.001. 16. Turk E, Prevolnik Rupel V, Tapajner A, Leyshon S, Isola A. An Audit of Diabetes-Dependent Quality of Life (ADDQOL) in older patients with diabetes mellitus type 2 in Slovenia. Value Heal Reg Issues. 2013;2(2):248–253. doi: 10.1016/j.vhri.2013.05.001. 17. Prevolnik Rupel V, Divjak M, Turk E. Changes in the level of knowledge of diabetes among elderly with diabetes in Slovenia in the period 2011-2020. Prim Care Diabetes. 2021;15(5):879–883. doi: 10.1016/j. pcd.2021.07.001. 18. Stojnić N, Klemenc-Ketiš Z, Mori Lukančič M, Zavrnik Č, Poplas Susič A. Perceptions of the primary health care team about the implementation of integrated care of patients with type 2 diabetes and hypertension in Slovenia: Qualitative study. BMC Health Serv Res. 2023;23(1):362. doi: 10.1186/s12913-023-09353-3. 19. van Olmen J, Menon S, Poplas Susič A, Ir P, Klipstein-Grobusch K, Wouters E, et al. Scale-up integrated care for diabetes and hypertension in Cambodia, Slovenia and Belgium (SCUBY): A study design for a quasi-experimental multiple case study. Glob Health Action. 2020;13(1):1824382. doi: 10.1080/16549716.2020.1824382. 20. Gilson L. Health policy and systems research: A methodology reader. Geneva: WHO; 2013. 21. Leech NL, Onwuegbuzie AJ. Qualitative data analysis: A compendium of techniques and a framework for selection for school psychology research and beyond. Sch Psychol Q. 2008;23(4):587–604. 22. Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–339. doi: 10.1370/afm.818. 23. Zavrnik Č, Danhieux K, Monarres MH, Stojnić N, Lukančič MM, Martens M, et al. Scaling-up an integrated care for patients with non-communicable diseases: An analysis of healthcare barriers and facilitators in Slovenia and Belgium. Zdr Varst. 2021;60(3):158–166. doi: 10.2478/sjph-2021-0023. 24. Lemmens LC, De Bruin SR, Struijs JN, Rijken M, Nijpels G, Baan CA. Patient involvement in diabetes care: experiences in nine diabetes care groups. Int J Integr Care. 2015;15:e044. doi: 10.5334/ijic.2207. 25. Sims Gould J, Tong C, Ly J, Vazirian S, Windt A, Khan K. Process evaluation of team-based care in people aged >65 years with type 2 diabetes mellitus. BMJ Open. 2019;9(8):e029965. doi: 10.1136/ bmjopen-2019-029965. 26. Joudeh AI, Elderawy NM, Alhitmi MA, Alnuaimi AS, Aabdien M, Alotaibi M, et al. Assessment of knowledge, attitude, and practice of primary healthcare physicians in Qatar towards hypertension management: An online cross-sectional study. Medicine (Baltimore). 2023;102(38):e34751. doi: 10.1097/MD.0000000000034751. 27. Gysin S, Sottas B, Odermatt M, Essig S. Advanced practice nurses’ and general practitioners’ first experiences with introducing the advanced practice nurse role to Swiss primary care: A qualitative study. BMC Fam Pract. 2019;20(1):163. doi: 10.1186/s12875-019-1055-z. 28. Aerts N, Van Bogaert P, Bastiaens H, Peremans L. Integration of nurses in general practice: A thematic synthesis of the perspectives of general practitioners, practice nurses and patients living with chronic illness. J Clin Nurs. 2020;29(1–2):251–264. doi: 10.1111/jocn.15092. 29. Matthys E, Remmen R, Van Bogaert P. Practice nurse support and task suitability in a general practice: a cross-sectional survey in Belgium. J Interprof Care. 2019;33(6):661–669. doi: 10.1080/13561820.2019.1569602. 30. Virtic T, Mihevc M, Zavrnik C, Mori Lukancic M, Poplas Susic A, Klemenc-Ketiš Z. Peer support as part of scaling-up integrated care in patients with type 2 diabetes and arterial hypertension at the primary healthcare level: A study protocol. Zdr Varst. 2023;62(2):93–100. doi: 10.2478/sjph-2023-0013. 31. Mihevc M, Zavrnik C, Mori Lukančič M, Virtič T, Prevolnik Rupel V, Petek Šter M, et al. Telemonitoring of elderly with hypertension and type 2 diabetes at the primary care level: Protocol for a multicentric randomized controlled pilot study. Zdr Varst. 2022;61(4):216–223. doi: 10.2478/sjph-2022-0029. 10.2478/sjph-2024-0006 Zdr Varst. 2024;63(1):38-45 45 CONTENT VALIDITY AND COGNITIVE TESTING IN THE DEVELOPMENT OF A MOTIVATIONAL INTERVIEWING SELF-ASSESSMENT QUESTIONNAIRE Tadeja HOČEVAR 1* , Tim ANSTISS 2 , Danica ROTAR PAVLIČ 3 1 National Institute of Public Health, Trubarjeva 2, 1000 Ljubljana, Slovenia 2 University of Reading, Henley Business School, Whiteknights Rd, Reading RG6 6UD, United Kingdom 3 University of Ljubljana, Medical Faculty, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia Received: Oct 05, 2023 Accepted: Nov 28, 2023 Original scientific article *Correspondence: Tadeja.Hocevar@nijz.si 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 46 VSEBINSKA VELJAVNOST IN KOGNITIVNO TESTIRANJE PRI RAZVOJU SAMO-OCENJEVALNEGA VPRAŠALNIKA O MOTIVACIJSKEM INTERVJUJU Hočevar T, Anstiss T, Rotar Pavlič D. Content validity and cognitive testing in the development of a motivational interviewing self-assessment questionnaire. Zdr Varst. 2024;63(1):46-54. doi: 10.2478/Sjph-2024-0007. ABSTRACT Keywords: Alcohol screening and brief intervention Cognitive testing Content validity Motivational interviewing Self-assessment questionnaire IZVLEČEK Ključne besede: kratko svetovanje za opuščanje tveganega in škodljivega pitja alkohola kognitivno testiranje vsebinska veljavnost motivacijski intervju samo-ocenjevalni vprašalnik Aim: To develop and content validate a self-assessment questionnaire on motivational interviewing (MI) practice as the first stages in forming the questionnaire to be used in cross-sectional studies involving practitioners conducting the MI-based alcohol screening and brief intervention (ASBI). Methods: A comprehensive mixed methods approach included a literature review, 3 rounds of expert panel (EP) opinions (n=10), cognitive testing (CT) with 10 MI-based ASBI practitioners, and questionnaire piloting with 31 MI-based ASBI practitioners. Based on the EP opinions in the second round, content validity indices (CVIs) and the modified kappa coefficient (k*) were calculated, focusing on the relevance and understandability of questions and comprehensiveness and meaningfulness of the response options. This analysis was performed in 2020, at the conclusion of the national “Together for a Responsible Attitude Towards Alcohol Consumption” (“Skupaj za odgovoren odnos do pitja alkohola”, SOPA) project’s pilot implementation. Results: On a scale level, CVI values based on universal agreement for the entire questionnaire were high for 3/4 categories (S-CVI-UA>0.80), and CVI values based on average agreement were high across all categories (S-CVI-Ave>0.90). At the item level, CVI values (I-CVI) were never <0.50 (automatic item rejection), and the modified kappa value (k*) indicated poor validity for two items in the understandability category (k*=0.33). All problematic parts of the questionnaire were further tested and successfully modified based on the results of CT, and accepted in the third round of testing. Conclusions: The final version of the questionnaire demonstrated appropriate content validity for use in studies among Slovenian MI-based ASBI practitioners and is now ready for further psychometric testing. Namen: Razviti samo-ocenjevalni vprašalnik o izvajanju motivacijskega intervjuja (MI) in preveriti njegovo vsebinsko veljavnost oz. izvesti prvi dve fazi oblikovanja vprašalnika za uporabo v presečnih raziskavah med izvajalci na MI temelječega kratkega svetovanja za opuščanje tveganega in škodljivega pitja alkohola (KSTŠPA). Metode: Izvedli smo celovit pristop mešanih metod, ki je vključeval pregled literature, tri kroge mnenj skupine strokovnjakov s področja (SS) (n = 10), kognitivno testiranje (KT) vprašalnika z 10 izvajalci in pilotno testiranje vprašalnika s 31 izvajalci na MI temelječega KSTŠPA. Na podlagi mnenj SS v drugem krogu testiranja smo izračunali indekse vsebinske veljavnosti (IVV) in modificirani koeficient kappa (k*). Osredotočili smo se na 4 vsebinske kategorije: bistvenost in razumljivost vprašanj ter smiselnost in izčrpnost možnosti odgovorov. Razumljivost vprašalnika smo dodatno preverjali s kognitivnim testiranjem. Analizo smo izvedli v letu 2020 ob zaključku izvajanja pilota nacionalnega projekta Skupaj za odgovoren odnos do pitja alkohola (SOPA). Rezultati: Vrednosti IVV na podlagi univerzalnega strinjanja strokovnjakov za celotni vprašalnik so bile ustrezno visoke (> 0,80) v 3 od 4 kategorij, IVV na podlagi povprečnega strinjanja strokovnjakov pa je bila visoka (> 0,90) v vseh kategorijah. Vrednosti IVV na ravni postavk niso bile manjše od 0,50, kar bi pomenilo avtomatično zavrnitev postavke, vrednosti k* pa so pokazale slabo veljavnost pri dveh postavkah v kategoriji razumljivost (k* = 0,33). Vse problematizirane dele vprašalnika smo dalje kognitivno testirali in na podlagi rezultatov uspešno spremenili ter so bili nato sprejeti kot ustrezni v tretjem krogu testiranja. Zaključki: Končna različica vprašalnika ima ustrezno vsebinsko veljavnost za uporabo med slovenskimi izvajalci na MI temelječega KSTŠPA in je pripravljena na preverjanje psihometričnih lastnosti. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Motivational interviewing (MI) is a widely used and effective conversational approach for helping people change their behaviour (1). It seeks to strengthen a person’s self-determined motivation by evoking their inner resources and strengths (1). Several studies indicate its effectiveness in counselling for excessive alcohol consumption. (2). An increasing number of primary healthcare and other professionals in the helping professions use this approach, and assessing its quality or use is essential for programme and outcome optimisation. Tools for evaluating the integrity of MI practice include the MISC (Manual for the Motivational Interviewing Skill Code) (3-5) and the MITI (Motivational Interviewing Treatment Integrity Code) (6, 7). These tools involve an expert assessing (part of) the counselling session and the related demands with regard to time, financial sources and knowledge (6, 7). Another tool is supervisory, MIA:STEP (Motivational interviewing assessment: Supervisory tools for enhancing proficiency) (8), which can be self-administered by the practitioner and used for subsequent supervision and discussion. This also addresses a single session or part of a session. To the best of our knowledge, there are no comprehensive, easy to administer and validated self-assessment questionnaires regarding MI practice, which might be used to help self- assess longer time periods of MI usage to help shape practice and inform research on the effectiveness and outcomes in a practical way. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 47 This study aims to address this gap through the initial development and content validation of a comprehensive self-assessment questionnaire to be used as an instrument in cross-sectional studies among Slovenian experts that use MI in their work (MI practitioners), focusing specifically on those conducting alcohol screening and brief intervention (ASBI) in primary healthcare settings and social work centres. 2 METHODS A mixed methods approach was applied. We adhered to the content validity protocol as described by Lynn (9), upgraded with cognitive testing procedures. This iterative process involved 10 steps spanning two distinct stages (Figure 1). 2.1 Stage one – questionnaire development The authors of this paper conducted a comprehensive literature review, including the foundational work of the MI authors Miller and Rollnick (1), literature on the main MI practice coding systems (3-8), and a literature review focusing on self-assessment of MI practice. This helped generate the initial pool of items for the first version of the questionnaire. Stages and steps in testing the content validity of the MI practice self-assessment questionnaire. Legend: [] – numbers in brackets define steps in testing content validity;MIQ 1.0/2.=/3.0 – version of MI questionnaire Figure 1. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 48 The literature review was conducted using the PubMed bibliographic database in 2018, 2020, and during the summer of 2023. Keywords used in the title/abstract search included “self-evaluation questionnaire”, “self-assessment scale”, “self-evaluation”, and “self- assessment”. An article was considered relevant if it was an original research paper or a scientific review article that discussed self-assessment questionnaires related to the practice of MI. We excluded articles that focused on self-assessment of health outcomes in patients/clients or that were self-assessment questionnaires not specific to the practice of MI (e.g., attitudes toward practicing MI, its effects, satisfaction with MI training, etc.). We assessed the following data in the articles: the purpose of the tool, including who it was intended for and the time period it assessed, the MI elements selected, number of items, response categories defined, and number of response options on the response scales. 2.2 Stage two – judgment and quantification 2.2.1 Participants, materials, procedures, data collection, and analysis regarding expert panels We established two expert panels for our study. The first panel consisted of five MI experts, four of whom were foreign members of the Motivational Interviewing Network of Trainers, one being the second author of this article. The fifth member was a national expert who had collaborated in the national project “Together for a Responsible Attitude Towards Drinking Alcohol” (TRATAC, and in Slovenian “Skupaj za odgovoren odnos do pitja alkohola”, SOPA) and helped to deliver MI-based ASBI training for primary healthcare and social workers. We sought the experts’ opinions on each item and the questionnaire as a whole, considering four perspectives: the relevance and understandability of the questions and comprehensiveness and meaningfulness of the response options. We used 4-point response scales (1 = not 2 = somewhat 3 = quite 4 = highly relevant/understandable/ comprehensive/meaningful). The text for foreign MI experts was translated into English by a Slovenian-English translator and by the first author of this article, then proofread by the second author. Email was used to both interact with the experts and administer the questionnaire. The second round of the expert panel involved six national experts, all of whom were SOPA project MI trainers, with one having previously participated in the first round. Testing with both rounds of expert panels took place in the fall of 2020, with a three-week gap between rounds. In the first round, not all the experts completed the feedback form in its entirety; two of them provided more general opinions. Consequently, during data analysis, we considered general comments and removed items if at least one expert deemed them irrelevant. In the second round, all the experts completed the entire form and provided more general opinions. Data analysis included calculation of three content validity indices, following the guidelines of Lynn (9), Polit (10), and Halek (11): the item- level content validity index (I-CVI) and both versions of the scale-level content validity indexes (S-CVI) – the universal agreement (S-CVI-UA) and its more liberal variant, the average agreement (S-CVI-Ave). S-CVI-UA was defined as the proportion of the items the experts scored as valid (ratings 3 or 4), with the cut-off point: S-CVI-UA≥0.80 (10). S-CVI-Ave was defined as the average proportion of the items rated 3 or 4, with a cut-off score: S-CVI-Ave≥0.90 (10). I-CVI was defined as the number of experts providing a rating of 3 or 4/number of experts, with a cut-off score: I-CVI≥0.78 (9) and automatic item rejection value: I-CVI<0.50 (11). Additionally, we calculated the modified kappa coefficient (k*) as per Polit (12) to assess chance agreement. The formula for k* was (I-CVI-pc) (1-pc), with pc as the probability of chance occurrence calculated with formula: [N!/A!(N-A)!] x 0.5 N, where N is the number of experts and A is the number of experts agreeing on a rating of 3 or 4 (11). The third round of the expert panel involved the same experts from the second round. This time, the questionnaire was administered using the survey app (1KA), and the experts were asked to comment on specific parts and confirm their broad agreement with version MIQ 3.0. 2.2.2 Participants, materials, procedures, data collection, and analysis regarding cognitive testing and pilot study We conducted cognitive testing with the SOPA MI-based ASBI practitioners as potential respondents to help check the understandability of the items and the questionnaire as a whole. We conducted this testing in two rounds, each proceeding the expert panels’ assessments. In total, we included 10 practitioners, consisting of a family medicine specialist, a specialist in sports medicine, two registered nurses in family medicine practice, two nurses in home care, and four social workers in social work centres. We employed a cognitive interviewing method based on Willis (13), and combined two techniques: think-aloud and verbal probing. Following the reading aloud of the accompanying instructions and questions, respondents were asked to answer questions item by item. They shared in their own words what each question was about, their level of confidence in their understanding, how they interpreted specific terms, their reasoning behind their answers, the difficulty they encountered in responding, and their perception of the comprehensiveness of the response scale. At the end of the interview, we posed additional meta-questions exploring which patients/clients or users they had in mind while responding, whether they provided principle-based answers to any questions, and whether they anticipated answering any questions differently when completing the questionnaire in a conventional 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 49 manner. Throughout the process we encouraged the participants to express their thoughts and suggestions, especially when they detected areas for improvement. These interviews were conducted during national COVID restrictions, primarily via telephone or Zoom, and were audio-recorded. The duration of the interviews ranged from 47 to 123 minutes, and due to their length two of them were conducted in two parts. We conducted a preliminary analysis during the interviews, followed by a more in-depth analysis upon reviewing the recordings. After we aligned the feedback from MI experts and practitioners for version 3.0, we additionally sought comments from respondents in the succeeding pilot study regarding the questionnaire. As with the cognitive testing, the respondents were SOPA MI practitioners. Due to the small sample size (n=31) and potential data identifiability, we did not collect further details on the sampled individuals. The pilot version of the questionnaire was administered via a survey app (1KA) in the autumn of 2020. Participation in all steps of the questionnaire validation process was entirely voluntary and without any financial incentives. 3 RESULTS 3.1 Stage one – questionnaire development We initially identified 19 articles and subsequently excluded four articles either because they focused on self- assessment in patients/clients (14-16) or did not address the practice of MI (17). Analysis of the remaining articles revealed the following: All of the current self-assessment tools were developed or published after 2003 (18-32), with almost half from 2020 onwards (10, 27-32). Most of the self-assessment tools focused on evaluating the implementation of MI in a single conducted session (20, 21, 23-27, 29-31). In some cases, these tools were derived from instruments used to assess MI integrity, such as MISC (27) or MITI (20, 31), or from a supervisory tool according to MIA:STEP (21). In some of the other instances, they took the form of checklist-style inventories (27, 28). Various tools addressed the use of different elements of MI. Almost all of them encompassed selected aspects of the spirit of MI, with many focusing on skills (21-24, 27, 29) and emphasizing strategies for assessing readiness for change (18, 21, 26, 29, 32). The range of relevant items in these tools varied from one (19) to 20 (23). Some items were ‘double/triple etc.-barreled’ (actually contained two/three etc. different questions within one) (21, 8). Response scales were often 5-point (18-20, 23, 27, 31) or 3-point (20, 29, 31), but some were 4-point (24, 25), binary (26), 6-point (29), or 7-point (21). The scales measured frequency (18), agreement (23), the extent of behaviour (19-21, 31), or the number of occurrences of behaviour (20, 31), expertise (24, 25), optimality (27), or capability (29). In three cases, the possible response scales were not described (28, 30, 32). From our literature review, we generated a pool of 58 items addressing five important aspects of MI: partnership, acceptance, evoking, resisting the righting reflex, and strengthening self-efficacy. Some aspects of MI we did not assess include: focussing, planning, compassion, and developing discrepancy. For practical reasons, we reduced the number of items to 30. We introduced a 7-point frequency scale and included instructions for completing the questionnaire. This marked the creation of version one of the MI questionnaire (MIQ 1.0). 3.2 Stage two – judgment and quantification The first expert panel round revealed concerns about the clarity of instructions and the understandability of items. Specifically, there were questions about what the period or frequency referred to, whether it was about the total number of times the element was practiced, the number of times in one session, with one or all patients/ clients, or the duration when it occurred. Some experts raised concerns about the questionnaire’s length and abundant response options, and some questioned the often indifferent neutral middle option. Certain sections were questioned regarding their understandability, and these concerns were given special consideration during the subsequent cognitive testing. In the first round of cognitive testing all five respondents quickly adapted to the instructions and almost instantly discussed all the required aspects in one flow. For example: KT1_1_36-39/1/ (in item P3): “Yes - (reads the question:) How often have you checked if you and the patient (skips the words ‘slash client’) are working together towards the same goal? (short pause, thinking) How often? Well... this actually refers to, it refers to one patient, if I understand correctly, I would interpret it this way: it refers to one patient over several sessions or encounters (note: it means meetings), and I would understand it as, do I check with the patient at each encounter if we are on the same path (short pause)... yes, I would answer (short pause) ‘almost always’. Almost every time the patient came for an encounter, I somehow checked, actually, even between the lines, if we were heading toward the same goal. I would answer ‘almost always’.” Moderator: (waits for a moment) “I see, okay, now you’ve also told me how you came to your thoughts. What do you think of this question - is it difficult/easy, understandable?” KT1_1_36-39(1): (short pause) “I find this question quite okay. It’s fine with me. Good.” 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 50 Moderator: “And what about the appropriateness of the answers, are they fine? The options, are they okay?” KT1_1_36-39(1): “Yes, ‘never’ is out, well, ‘almost every time’, yes, you kind of refresh or check at almost every encounter if we are both working toward the same goal. I could choose ‘frequently’, well, either ‘frequently’ or ‘almost always’ I would choose.” Moderator: “I see, what would you choose?” KT1_1_36-39(1): (pause) “Now, if there were only ‘never’, ‘sometimes’, ‘always’, I would choose ‘sometimes’, well, now, because I have two more sub-questions, ‘frequently’ and ‘almost always’. yes, I chose ‘almost always.’” Some answers were based less on actual experience (or memory of it) and were more principle-based or considered less thoughtfully. This was primarily the case for some items related to the spirit of MI, particularly partnership and acceptance. Items containing the verbs “ask” and “tell” were affected to a lesser extent. At times different respondents or even the same respondent had particular patients/clients in mind. Respondents showed a good general understanding of the optimal practice of MI elements. Some testers liked the multiple response options, some found them unnecessary. Typically, respondents tended to select the middle answer with less consideration, and they did not encounter difficulty in choosing an adjacent option when prompted. Using the respondents’ answers, we adapted the instructions to be more precise and direct in completing the questionnaire based on actual experience over principle-based answers. We added adverbial or adjectival emphasis to certain words and underlined them (e.g., actively strive). We also removed the middle option for answers. This resulted in the creation of version two of the MI questionnaire (MIQ 2.0). In the second expert panel round the indices and the modified kappa coefficient indicated that some experts found understandability problematic with regard to the elements of partnership, acceptance and resisting the righting reflex, and relevance in element evoking according to the S-CVI-UA value. However, no item had any of the four categories indices with values lower than 0.50, at which point an item would automatically be removed, as indicated by Halek (11). As suggested in the literature (9, 10), they were instead taken into special consideration for further adaptation and/or testing. Detailed values of the indices and k*s in all four categories are presented in Tables 1, 2 and 3. The content validity of the measurement instrument as a whole and by specific MI elements, with the universal agreement of experts (S-CVI-UA). The content validity of the measurement instrument as a whole and by specific MI elements, with the universal agreement of experts (S-CVI-UA). Legend: 1 S-CVI-UA = the proportion of the items the experts scored as valid (ratings 3 or 4); cut-off point: S-CVI- UA≥0.80 (10) Legend: 1 S-CVI- Ave = the average proportion of the items rated 3 or 4; cut-off score: S-CVI-Ave≥0.90 (10) Scale as a whole Partnership Acceptance Evoking Resisting the righting reflex Strengthening self-efficacy Scale as a whole Partnership Acceptance Evoking Resisting the righting reflex Strengthening self-efficacy 0.93 1.00 0.80 0.75 1.00 1.00 0.99 1.00 0.97 0.96 1.00 1.00 0.93 1.00 0.80 1.00 0.83 1.00 0.98 0.94 0.97 1.00 0.95 1.00 0.67 0.33 0.40 1.00 0.67 1.00 0.93 0.86 0.73 1.00 0.94 0.95 0.85 0.67 0.80 1.00 0.83 1.00 0.98 0.94 0.97 1.00 0.97 1.00 RELEVANCE of the question RELEVANCE of the question COMPLETENESS of response options COMPLETENESS of response options UNDERSTANDABILITY of the question UNDERSTANDABILITY of the question S-CVI-UA1 S-CVI-Ave1 MEANINGFULNESS of response options MEANINGFULNESS of response options MI scale/element MI scale/element Table 1. Table 2. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 51 Values of the validity index for individual items (I-CVI) and the modified kappa coefficient (k*) for 27 items. Legend: * reverse scaling 1 N(exp3-4) = number of experts providing a rating of 3 or 4 2 I-CVI (content validity index) = number of experts providing a rating of 3 or 4/number of experts; cut-off score: I-CVI≥0.78 (9); automatic item rejection: I-CVI<0.50 (11) 3 pc (probability of chance occurence) = [N!/A!(N-A)!] x 0,5N N = number of experts; A = number of experts agreeing on a rating of 3 or 4 (11) 4 k* (modified kappa) = (I-CVI-pc)(1-pc) 5 P/C = patient/client PARTNERSHIP ACCEPTANCE EVOKING STRENGTHENING SELF-EFFICACY RESISTING THE RIGHTING REFLEX P1 make P/C5 feel comfortable P2 being supportive P3 working together P4 P/C’s input P5 incorporate P/C’s ideas P6 be there in case P/C changes their mind A1 P/C’s view is relevant A2 strive to understand A3 P/C’s choice to change A4 respect P/C’s decision A5 P/C’s personal growth E2 P/C’s own reasons E3 P/C’s own strategies E4 encourage P/C’s thinking E5 P/C’s inner strenghts and sources S1 ask about confidence S2 ask about needed S3 P/C’s past experiences S5 affirmations S6 change talk S8 other resources R1* explaining without first exploring R2* talking about own knowing R3* reasons without permission and inquire R4* ideas without permission and inquire R5* talk P/C into R6 suggestions after permission and inquire 6 6 6 6 6 6 6 6 6 6 5 6 6 5 6 6 6 6 6 6 6 6 6 6 6 6 6 5 5 6 6 6 6 6 5 6 6 6 6 6 6 6 6 6 6 6 6 6 6 5 6 6 6 6 5 5 6 4 6 5 6 5 5 6 4 6 6 6 6 6 6 6 6 5 6 6 5 6 6 5 6 5 5 6 6 6 6 6 5 6 6 6 6 6 6 6 6 6 6 6 6 6 6 5 6 6 6 6 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.s000 0.000 0.094 0.000 0.000 0.094 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.094 0.094 0.000 0.000 0.000 0.000 0.000 0.094 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.094 0.000 0.000 0.000 0.000 0.094 0.094 0.000 0.234 0.000 0.094 0.000 0.094 0.094 0.000 0.234 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.094 0.000 0.000 0.094 0.000 0.000 0.094 0.000 0.094 0.094 0.000 0.000 0.000 0.000 0.000 0.094 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.094 0.000 0.000 0.000 0.000 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 0.83 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83 0.83 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 1.00 1.00 0.83 0.83 1.00 0.67 1.00 0.83 1.00 0.83 0.83 1.00 0.67 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 0.83 1.00 1.00 0.83 1.00 0.83 0.83 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.83 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 0.67 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.67 0.67 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 1.00 1.00 0.67 0.67 1.00 0.33 1.00 0.67 1.00 0.67 0.67 1.00 0.33 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 0.67 1.00 1.00 0.67 1.00 0.67 0.67 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.67 1.00 1.00 1.00 1.00 MI element Item code* and content Table 3. RELEVANCE of the question N1(exp3-4) N 1 (exp3-4) N 1 (exp3-4) N 1 (exp3-4)pc 3 pc 3 pc 3 pc 3k*4 k*4 k*4 k*4I-CVI2 I-CVI2 I-CVI2 I-CVI2 COMPLETENESS of response options UNDERSTANDABILITY of the question MEANINGFULNESS of response options In the second round of cognitive testing, the respondents properly understood both items that were considered potentially problematic in terms of understandability by the expert panel. For example, the expression “personal growth”, considered too broad and not understandable by two panel experts in round two, was consistently viewed by respondents in the second round of cognitive testing as one’s general ability to change one’s way of thinking and behaving, to undergo the necessary behaviour change, or to stop drinking (excessively). In this round the respondents also demonstrated appropriate knowledge regarding the optimal practice of different MI elements and remembered different patients/clients and situations. However, they provided fewer principle-based answers (although some instances still occurred, again in the partnership and acceptance subscales) and relied more on their memory of actual situations. Based on insights and suggestions from the second round of the expert panel and the cognitive testing we made changes to some expressions, and divided some items into two separate questions, made further improvements to the instructions, and created the third version of the MI questionnaire (MIQ 3.0). This final version of the questionnaire was then approved by the expert panel in the third round, and no further comments were received from respondents during the questionnaire piloting. 4 DISCUSSION The main aim of this study was to develop a comprehensive self-assessment questionnaire about practicing MI in conducting ASBI and to test its content validity. We used an iterative process involving a literature review, expert panel method and cognitive testing. This resulted in a content-valid 30-item long self-assessment questionnaire with a 6-point response scale exploring five elements of MI practice when conducting ASBI. Based on our review, previous studies have neither generated nor used a comprehensive and content-valid self-assessment questionnaire for MI practitioners that can be used for assessing MI practice over extended time periods – e.g. weeks or months. One study, however (18), did ask practitioners two MI questions (out of 39) about past practices in smoking cessation counselling. These two items focused on the self-assessment of the importance of change and confidence in making the change. This earlier questionnaire showed good content validity and internal consistency (18), and we included these aspects of those items in our questionnaire. The expert panel’s opinion can be analysed in different ways (e.g. 34, 35). In our case this involved calculating different content indexes in four content categories, allowing us to analyse the experts’ opinions very systematically and to pinpoint exactly where the potential problem was and what we needed to do about it. The otherwise acceptable to high or even optimal values of the CVIs (I-CVI, S-CVI-UA/Ave) and k* were most negatively affected by two items due to the expert panel’s concerns about understandability. Due to the fact that none of the items had their index value lower than 0.5, they were not automatically rejected. Similar to Halek et al. (11) and Carli et al. (34) in such cases, these items were further tested with potential respondents. In the iterative process of cognitive interviewing, the respondents understood both previously problematised items by the expert correctly and so the questions remained. In some other questions, at first some expressions were less understandable, and some items were answered in a more principle-based manner. These items were adjusted and in the subsequent testing the questions were understood accurately and were answered more based on the memory of the respondents’ actual experiences. Similarly, Robinson et al. (36) succeeded in enhancing the understandability of the questionnaire substantially by conducting this iterative process. In this way, our results confirmed the value of cognitive interviewing as a powerful tool for gaining insight into the thought process of the respondents and for improving the understandability of the questionnaire (as per Willis) (13). Our study has potential limitations that need to be addressed. Firstly, we focused on five MI elements, a mixture of selected aspects of the MI spirit, processes and principles, whilst leaving some of the aspects of these as well as skills, strategies and techniques out. This is not unique to our questionnaire, but is rather a common feature of other questionnaires and MI assessment tools which also cover different selected aspects of the MI spirit and/or different selected behaviours as stated earlier in this article. Which MI practice variables are selected and how they are captured varies at least to a certain degree. As per Moyers et al. (6), it is acceptable not to include some aspects to reduce the complexity of the tool whilst also being clear about those elements or aspects that are included. Next, according to our cognitive testing results, respondents might answer some questions in a more principle-based manner and/or less thoughtfully, usually more with those items seeking to capture the spirit of MI. We tried to reduce this tendency by adding instructions about the importance of answering according to actual personal experience rather than the professional ideal, and emphasising the practical value of completing the questionnaire in a manner which encourages reflection on one’s personal MI practice. Whilst principle-based answering may reflect a respondent’s difficulty in assessing their personal performance, Beckman et al. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 52 (31) comment on the effect of metacognition, as (self-) estimates may become more accurate during repeated testing and subjective ratings become more aligned with the objective ones. Regular use of check-lists and supervision may further add to this alignment of subjective and objective performance rating (31), especially when divergence encourages reflection and deliberate practice. Nonetheless, combining self-assessment instruments with objective ratings of performance may be optimal (31). The questionnaire we developed is not short. The MI questionnaires mentioned earlier have up to 20 items, while ours has 30, and some of the experts felt that the length might lower the response rates. However, the respondents in the cognitive testing part of this study did not comment on the questionnaire being too long. As per Robinson (33), to fully capture the richness of multidimensional variables, a larger number of items is required. In our case, the five MI elements we chose to incorporate could potentially mean five different dimensions of the questionnaire. Having approximately six items per element before testing the psychometric properties and potentially needing to narrow down the number of items per element/dimension/subscale to three, as the generally recommended minimum (33), makes this a rational decision. Finally, some of the items are alcohol-risk-factor-specific, and the language of the questionnaire is Slovenian. These specifics call for additional content validity testing when planning to use the instrument in a broader context and/ or different languages. There have been very few published studies on self- assessment of practicing MI. This is a rather young research field, as the majority of identified studies were published after 2015, half of them after 2020. Our study focused on the content validity of the questionnaire, leaving it open for further validation processes, including testing its psychometric properties, as in, for example, Sočan et al. (37). 5 CONCLUSIONS To the best of our knowledge, this is the only study in the MI research field that has deployed such a rigorous and comprehensive procedure for establishing the content validity of a self-assessment questionnaire. The questionnaire’s final version demonstrates appropriate content validity and is ready for testing its psychometric properties. With regard to reducing its length, we suggest the first items to be removed are those with a potentially higher likelihood of principle-based responses. ACKNOWLEDGMENTS We would like to thank all ten MI experts for their opinions on the content validity of the questionnaire as well as all the ten MI-based ASBI practitioners in the TRATAC (Slovenian: SOPA) project who assessed the questionnaire’s understandability in the cognitive testing, and 31 who collaborated it in the pilot study. CONFLICTS OF INTEREST The authors declare that there are no conflicts of interest. FUNDING This research was co-financed by the European Commission (European Social Fund) and Republic of Slovenia within the Operational Programme for the Implementation of the EU Cohesion Policy 2014-2020. ETHICAL APPROVAL The study was approved by the National Medical Ethics Committee of the Republic of Slovenia (No. 0120- 246/2018/21). All participants gave informed consent prior to study participation. AVAILABILITY OF DATA AND MATERIALS The data and materials are securely stored at the facilities of Slovenia’s National Institute of Public Health. Further information on the questionnaire’s development stages can be made available by the first author upon request. ORCID Tadeja Hočevar: https://orcid.org/0009-0000-2713-9801 Tim Anstiss: https://orcid.org/0000-0002-7762-532X Danica Rotar Pavlič: https://orcid.org/0000-0001-7575-3195 REFERENCES 1. Miller WR, Rollnick S. Motivational interviewing: Helping people change 3rd ed. New York: Guilford Press; 2013. 2. DiClemente CC, Corno CM, Graydon MM, Wiprovnick AE, Knoblach DJ. Motivational interviewing, enhancement, and brief interventions over the last decade: A review of reviews of efficacy and effectiveness. Psychol Addict Behav. 2017; 31(8):862-887. doi: 10.1037/adb0000318. 3. Miller WR. Motivational interviewing skill code. Unpublished document; 2000. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 53 4. Moyers T, Martin T, Catley D, Harris KJ, Ahluwalia JS. Assessing the integrity of motivational interviewing interventions: Reliability of the motivational interviewing skills code. Behav Cogn Psychother. 2003;31(2),177–184. doi: 10.1017/S1352465803002054. 5. Amrhein P, Miller WR, Moyers T, Ernst D. Manual for the motivational interviewing skill code (MISC). Department of Psychology Faculty Scholarship and Creative Works; 27 [Internet]. 2008 [cited 2023 Aug 8]. Available from: https://digitalcommons.montclair.edu/psychology- facpubs/27 6. Moyers TB, Martin T, Manuel JK, Miller, WR, Ernst D. Revised global scales: Motivational Interviewing Treatment Integrity 3.1.1 (MITI 3.1.1). New Mexico: Center on Alcoholism, Substance Abuse, and Addictions, The University of New Mexico [Internet]. 2010 [cited 2023 Aug 8]. Available from: https://www.motivationalinterviewing.org/sites/ default/files/MITI%203.1.pdf 7. Moyers TB, Rowell, LN, Manuel JK. The Motivational Interviewing Treatment Integrity code (MITI 4): Rationale, preliminary reliability and validity. J Subst Abuse Treat. 2016;65:36-42. doi: 10.1016/j. jsat.2016.01.001. 8. Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University. Motivational interviewing assessment: Supervisory tools for enhancing proficiency (MIA: STEP). Salem: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University. [Internet]. 2006 [cited 2023 Aug 8]. Available from: https://motivationalinterviewing.org/sites/default/ files/mia-step.pdf 9. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986; 35:382–385. doi: 10.1097/00006199-198611000-00017. 10. Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29:489–497. doi: 10.1002/nur.20147. 11. Halek M, Holle D, Bartholomeyczik S. Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia. BMC Health Serv Res. 2017;17(1):554. doi: 10.1186/s12913-017-2469-8. 12. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations Res Nurs Health. 2007;30:459– 467. doi: 10.1002/nur.20199. 13. Willis GB. Cognitive interviewing. Thousand Oaks: SAGE Publications; 2005. 14. Krampe H, Salz AL, Kerper LF, Krannich A, Schnell T, Wernecke KD, et al. Readiness to change and therapy outcomes of an innovative psychotherapy program for surgical patients: Results from a randomized controlled trial. BMC Psychiatry. 2017;17(1):417. doi: 10.1186/s12888- 017-1579-5. 15. Freyer-Adam J, Baumann S, Bischof G, Staudt A, Goeze C, Gaertner B, et al. Social equity in the efficacy of computer-based and in-person brief alcohol interventions among general hospital patients with at- risk alcohol use: A randomized controlled trial. JMIR Ment Health. 2022;9(1):e31712. doi: 10.2196/31712. 16. Bredie SJ, Fouwels AJ, Wollersheim H, Schippers GM. Effectiveness of nurse based motivational interviewing for smoking cessation in high risk cardiovascular outpatients: A randomized trial. Eur J Cardiovasc Nurs. 2011;10(3):174-179. doi: 10.1016/j.ejcnurse.2010.06.003. 17. Ramseier CA, Hübschi C, Crnić T, Woelber JP. Implementation of a communication curriculum in undergraduate dental education- students’ opinions during a 5-year development phase. Eur J Dent Educ. 2023 Aug 9. doi: 10.1111/eje.12940. 18. Scal P, Hennrikus D, Ehrlich L, Ireland M, Borowsky I. Preparing residents to counsel about smoking. Clin Pediatr (Phila). 2004;43(8):703-708. doi: 10.1177/000992280404300803. 19. Coyle K, Carcone AI, Butame S, Pooler-Burgess M, Chang J, Naar S. Adapting the self-assessment of contextual fit scale for implementation of evidence-based practices in adolescent HIV settings. Implement Sci Commun. 2022;3(1):115. doi: 10.1186/s43058-022-00349-4. 20. Mullin DJ, Saver B, Savageau JA, Forsberg L, Forsberg L. Evaluation of online and in-person motivational interviewing training for healthcare providers. Fam Syst Health. 2016; 34(4):357-366. doi: 10.1037/ fsh0000214. 21. Widder R. Learning to use motivational interviewing effectively: Modules. J Contin Educ Nurs. 2017;48(7):312-319. doi: 10.3928/00220124- 20170616-08. 22. Stenov V, Wind G, Skinner T, Reventlow S, Hempler NF. The potential of a self-assessment tool to identify healthcare professionals’ strengths and areas in need of professional development to aid effective facilitation of group-based, person-centered diabetes education. BMC Med Educ. 2017;17(1):166. doi: 10.1186/s12909-017-1003-3. 23. Vallabhan MK, Kong AS, Jimenez EY, Summers LC, DeBlieck CJ, Feldstein Ewing SW. Training primary care providers in the use of motivational interviewing for youth behavior change. Res Theory Nurs Pract. 2017;31(3):219-232. doi: 10.1891/1541-6577.31.3.219. 24. Muzyk AJ, Tew C, Thomas-Fannin A, Dayal S, Maeda R, Schramm-Sapyta N, et al. Utilizing Bloom’s taxonomy to design a substance use disorders course for health professions students. Subst Abus. 2018; 39(3):348-353. doi: 10.1080/08897077.2018.1436634. 25. Galal S, Vyas D, Mayberry J, Rogan EL, Patel S, Ruda S. Use of standardized patient simulations to assess impact of motivational interviewing training on social⁻emotional development. Pharmacy (Basel). 2018;6(3):65. doi: 10.3390/pharmacy6030065. 26. Boykan R, Blair R, Baldelli P, Owens S. Using motivational interviewing to address tobacco cessation: Two standardized patient cases for pediatric residents. MedEdPORTAL. 2019;15:10807. doi: 10.15766/ mep_2374-8265.10807. 27. Spangler JG, Shull CN, Hildebrandt CA, Jones KB, Brewer AL, Knudson MP, et al. Opioid use disorder and assessment of patient interactions among family medicine residents, medical students, and physician assistant students. MedEdPORTAL. 2020;16:11012. doi: 10.15766/ mep_2374-8265.11012. 28. Sadasivam RS, Kamberi A, DeLaughter K, Phillips B, Williams JH, Cutrona SL, et al. Secure asynchronous communication between smokers and tobacco treatment specialists: Secondary analysis of a web-assisted tobacco intervention in the QUIT-PRIMO and national dental PBRN networks. QUITPRIMO; National Dental PBRN Collaborative Group. J Med Internet Res. 2020;22(5):e13289. doi: 10.2196/13289. 29. Arnett MC, Evans MD, Stull C. Dental hygiene students’ perceptions regarding the importance of and confidence with using brief motivational interviewing during HPV patient counseling. J Dent Hyg. 2022;96(2):50-58. 30. Arnett MC, Evans MD, Stull CL. Brief motivational interviewing: Evaluation of a skills-based education program. J Dent Hyg. 2022;96(4):46-56. 31. Beckman M, Lindqvist H, Öhman L, Forsberg L, Lundgren T, Ghaderi A. Correspondence between practitioners’ self-assessment and independent motivational interviewing treatment integrity ratings. Front Psychol. 2022;13:890579. doi: 10.3389/fpsyg.2022.890579. 32. Fezza GC, Sansone S, Nolan RP. Therapeutic components of digital counseling for chronic heart failure. Front Psychiatry. 2022;13:888524. doi: 10.3389/fpsyt.2022.888524. 33. Robinson MA. Using multi-item psychometric scales for research and practice in human resource management. Hum Resour Manage. 2018;57:739-750. doi: 10.1002/hrm.21852. 34. Carli T, Košnik M, Zaletel-Kragelj L, Burazeri G, Kukec A. The APISS questionnaire: A new tool to assess the epidemiology of systemic allergic reactions to bee venom in beekeepers. Zdr Varst. 2023;62(3):137-144. doi: 10.2478/sjph-2023-0019. 35. Šlosar L, Puš K, Marušič U. Validation of the Slovenian version of the Movement Imagery Questionnaire for Children (MIQ-C): A measurement tool to assess the imagery ability of motortasks in children. Zdr Varst. 2023;62(3):113-120. doi: 10.2478/Sjph-2023-0016. 36. Robinson KM, Scherer AM, Nishimura TE, Laroche HH. Value of cognitive interviewing in the development of the weight stigma in healthcare inventory. Patient Educ Couns. 2023;113:107767. doi: 10.1016/j. pec.2023.107767. 37. Sočan G, De Boer D, Murko E, Kralj M, Ropret N, Zaletel M. Psychometric validation of an instrument for measuring patient experiences with outpatient healthcare.Zdr Varst. 2023;62(3):153-161. doi: 10.2478/sjph- 2023-0021. 10.2478/sjph-2024-0007 Zdr Varst. 2024;63(1):46-54 54 SYMPTOMS OF ANXIETY AND DEPRESSION AMONG SLOVENIAN BREAST CANCER SURVIVORS POST-TREATMENT DURING THE COVID-19 PANDEMIC: A CROSS-SECTIONAL STUDY Špela MIROŠEVIČ1,2* , Judith PRINS3, Nikola BEŠIĆ4 , Simona BORŠTNAR5 , Andreja Cirila ŠKUFCA SMRDEL6, Vesna HOMAR1 , Marko POPOVIĆ1, Zalika KLEMENC-KETIŠ1,2,7 1 University of Ljubljana, Faculty of Medicine, Department of Family Medicine, Poljanski nasip 58, 1000 Ljubljana, Slovenia 2 Community Health Centre Ljubljana, Metelkova 9, 1000 Ljubljana, Slovenia 3 Radboud University Medical Centre, Department of Medical Psychology, Nijmegen, Netherlands 4 Institute of Oncology, Department of Surgical Oncology Ljubljana, Zaloška 2, 1000 Ljubljana, Slovenia 5 Institute of Oncology, Department of Medical Oncology Ljubljana, Zaloška 2, 1000 Ljubljana, Slovenia 6 Institute of Oncology Ljubljana, Department of Psycho-Oncology, Zaloška 2, 1000 Ljubljana, Slovenia 7 University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia Received: Oct 09, 2023 Accepted: Nov 30, 2023 Original scientific article *Correspondence: spela.mirosevic@mf.uni-lj.si 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 55 SIMPTOMI ANKSIOZNOSTI IN DEPRESIJE PRI SLOVENSKIH BOLNICAH Z RAKOM DOJKE PO ZDRAVLJENJU V ČASU PANDEMIJE COVIDA-19: PRESEČNA ŠTUDIJA Miroševič Š, Prins J, Bešić N, Borštnar S, Škufca Smrdel AC, Homar V, Popović M, Klemenc-Ketiš Z. Symptoms of anxiety and depression among Slovenian breast cancer survivors post-treatment during the COVID-19 pandemic: A cross-sectional study. Zdr Varst. 2024;63(1):55-62. doi: 10.2478/Sjph-2024-0008. ABSTRACT Keywords: Anxiety Depression Psychological distress Breast neoplasms COVID-19 IZVLEČEK Ključne besede: anksioznost depresija psihološki distres rak dojke covid -19 Background: Although anxiety and depression are important determinants of mental health, the literature in this area is sparse as most studies focus on the period during treatment. Mental health problems can affect cancer recovery as well as quality of life and survival. In this cross-sectional study, we investigated the prevalence of anxiety and depression in Slovenian cancer survivors after treatment and assessed the associated correlates during the COVID-19 pandemic. Methods: From September 2021 to January 2022, we collected data from 430 breast cancer survivors one to five years after receiving post-local treatment and (neo)adjuvant chemotherapy. We used the Hospital Anxiety and Depression Scale (HADS) to measure anxiety and depression levels. Multivariate linear regression was used to identify factors associated with higher levels of anxiety and depression. Results: Key findings from this study are increased levels of psychological distress and identification of relevant factors associated with those elevated levels. Approximately one-third of breast cancer survivors exhibited symptoms of elevated anxiety and depression, with one in eight meeting clinical thresholds. Multivariate linear regression revealed that age, lower quality of life, heightened fear of cancer recurrence (FCR), reduced resilience, limited social support, and unmet psychosocial and emotional needs correlated with increased anxiety symptoms. Additionally, lower quality of life, higher FCR, diminished resilience, and limited social support were associated with higher depression symptomatology. Conclusions: Our study of Slovenian breast cancer survivors one to five years post-treatment observed a significant increase in anxiety and depression symptoms, possibly exacerbated by the COVID-19 pandemic. The demographic and psychosocial factors identified in this study offer valuable insights for future research. The study emphasises the importance of recognising and addressing the psychological needs of breast cancer survivors and the need to follow them throughout their cancer journey. Uvod: Čeprav sta anksioznost in depresija pomembna dejavnika duševnega zdravja, ni veliko študij, ki bi se osredotočale na obdobje po koncu zdravljenja. Težave v duševnem zdravju lahko vplivajo na okrevanje po raku ter na kakovost življenja in preživetje. V tej presečni študiji smo preučevali prevalenco anksioznosti in depresije v času pandemije covida-19 pri slovenskih preživelih bolnicah z rakom dojke po koncu zdravljenja in ocenili z njima povezane korelacije. Metode: Med septembrom 2021 in januarjem 2022 smo zbrali podatke 430 preživelih bolnic z rakom dojk, ki so bile 1–5 let po koncu zdravljenja in ki so prejele post-lokalno zdravljenje in (neo)adjuvantno kemoterapijo. Anksioznost in depresija sta bili merjeni z bolnišnično lestvico anksioznosti in depresije (HADS). Za ugotavljanje dejavnikov, povezanih z višjimi stopnjami anksioznosti in depresije, je bila uporabljena multivariatna linearna regresija. Rezultati: Ključne ugotovitve te študije so visoka pojavnost simptomov psihološkega distresa in identifikacija spremenljivk povezanih z več simptomov psihološkega distresa. Približno tretjina preživelih bolnic z rakom dojke ima višje od normalnih ravni simptomov anksioznosti in depresije. Pri eni od osmih oseb ugotavljamo klinično pomembno anksioznost in depresijo. Z multivariatno linearno regresijo je bilo ugotovljeno, da so starost, slabša kakovost življenja, višja raven strahu pred ponovitvijo raka, manjša psihološka odpornost, manjša socialna podpora ter nezadovoljene potrebe po psihosocialni in čustveni podpori pomembni korelati simptomov anksioznosti. Poleg tega je bilo ugotovljeno, da so nižja kakovost življenja, višje ravni strahu pred ponovitvijo bolezni, nižja odpornost in nižja socialna podpora povezani z večjo simptomatiko depresije. Zaključek: V našem vzorcu slovenskih preživelih bolnic z rakom dojk 1–5 let po zdravljenju je bila prevalenca simptomov anksioznosti in depresije visoka, kar bi lahko bilo povezano s pandemijo covida -19. Demografski in psihosocialni dejavniki, ugotovljeni v tej študiji, ponujajo obetavne usmeritve za prihodnje študije. Ključne ugotovitve študije so pomembnost prepoznavanja in obravnave psiholoških potreb preživelih bolnic z rakom dojke po končanem zdravljenju ter potreba po dolgoročnem spremljanju. This article was presented at the 2nd ISCPC conference, which took place in Cankarjev dom, Ljubljana, Slovenia, on 23 and 24 November, 2023. The conference was organised by the Community Health Centre Ljubljana and Medical Faculty, University of Ljubljana, Slovenia. © Nacionalni inštitut za javno zdravje, Slovenija. 1 INTRODUCTION Breast cancer (BC) is a common and significant cause of cancer-related deaths in women (1). The five-year survival rate has improved, exceeding the 90% threshold in high- income countries (2), resulting in a growing number of BC survivors. According to data extracted from GLOBOCAN, in the year 2020 there were approximately 2.3 million newly reported cases of breast cancer and 685.000 deaths. In Slovenia alone, around 1,500 women are diagnosed annually, and there are currently 20,000 women who have received a BC diagnosis at some point in their lives (3). Breast cancer stands as the most frequently identified cancer, representing a quarter of all cancer cases and contributing to one-sixth of cancer-related fatalities. In most countries, it is one of the main causes of cancer- related deaths in women (3). Over the years there has been a significant enhancement in the five-year survival rate for breast cancer survivors, surpassing 90% in high- income nations. This has translated into a multitude of breast cancer survivors leading fulfilling lives well beyond their initial diagnosis. A cancer diagnosis profoundly affects patients’ physical and mental well-being (4). Depression and anxiety can complicate the recovery process (5), reduce quality of life (6), and impact cancer recurrence and survival (7). Unfortunately, mental health often receives inadequate attention during and after cancer treatment, as the main focus is on physical symptoms, treatments, and potential side effects. This challenge was compounded by the additional challenges faced by BC patients during the COVID-19 crisis, such as increased cancer-related deaths, disruptions in treatment services, and delayed diagnoses (8). Studies show that cancer patients are more likely to suffer from stress, anxiety and depression, especially during prolonged treatment (9). Therefore, it is important to understand the impact of the pandemic on patients’ mental health. Previous research has identified several factors linked to increased anxiety and depression in BC survivors. These include socio-demographic factors like younger age (10, 11), no partner (12), unemployment (12, 13), lower education (13) and living at home while being treated (14). Clinical characteristics such as undergoing chemotherapy (11) and experiencing physical symptoms (12, 14) are also significant contributors. Additionally, psychosocial factors, including quality of life (10, 11, 13, 15), social support (16), fear of cancer recurrence (FCR) (17) and resilience (18) have been identified as impacting symptoms of anxiety and depression. Anxiety and depression rates are notably higher in cancer survivors than in the general population (19). However, these rates vary due to multiple factors such as time since treatment, the type of treatment, cancer stage, choice 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 56 of measurement tools for anxiety/depression and their thresholds, study location, and overall design (20, 21). Most importantly, there is a lack of rigorous, methodologically sound research on psychological distress, especially in post-treatment cancer survivors. In this study, we aimed to examine the prevalence of anxiety and depression symptoms among BC survivors in Slovenia and compare these rates with those in similar studies. Additionally, we sought to identify socio-demographic, psychological, and clinical factors associated with higher levels of depression and anxiety in this group. Through these objectives, we aim to gain a better understanding of the extent of this problem and uncover potential factors associated with higher levels of psychological distress. This information can then be used to develop more effective strategies for cancer care. 2 PATIENTS AND METHODS 2.1 Patients and procedure Participants were recruited at the Institute of Oncology Ljubljana between September 2021 and January 2022. All eligible female BC survivors attending hospital follow- up appointments during this period were invited to participate. The detailed inclusion and exclusion criteria for this cross-sectional study have already been described in another publication (22). Prior to participation, all individuals were informed about the purpose of the study and gave informed consent. 2.2 Methods In this study, we collected socio-demographic, clinical and patient-reported data. Comprehensive descriptions of these measures with validation reports can be found in the original study (22). Symptoms of anxiety and depression were assessed using the Hospital Anxiety and Depression Scale (HADS), a validated tool with strong psychometric properties in its Slovenian version (23). HADS consists of two subscales, HADS-A for anxiety symptoms and HADS-D for depression symptoms, with scores ranging from 0 to 21, where higher scores indicate more severe symptoms (24). We categorized participants based on symptom severity, with scores of 11 or higher as clinically significant (probable cases), 8 to 10 as “possible cases”, and below 8 as normal (25). To determine the overall prevalence of anxiety and depressive symptoms, we used a cutoff point of 8 and above, consistent with other similar studies (20, 21, 26). Demographic and clinical characteristics were collected in self-reported questionnaires. This information included socio-demographic data, such as age, marital status, employment status, education, place of residence, and behavioural and clinical characteristics, such as smoking 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 57 status type of primary treatment, cancer stage, time since the diagnosis and presence of hormonal therapy. The presence of comorbidities was measured using Comorbidity Questionnaire (SCQ-19) (27). For this study, the comorbidity status (SCQ-19 total score) and the number of comorbidities were analysed. Quality of life was measured with the EuroQol Five-Dimension Questionnaire (EQ-5D), which includes five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) and five different severity levels (no, mild, moderate, severe, extreme or unable to perform the activity) (28). The utility scores for this study were determined based on the norm set for the Slovenian population, using the EQ5DL crosswalk method SL (29). Fear of cancer recurrence was assessed with the Fear of Cancer Recurrence Inventory (FCRI), a 42-item scale that measures the extent of patients’ fear in this context. In this study, only the severity subscale was used, as this is the most commonly used report of the FCRI scale according to previous studies and best reflects FCR scores (30). Social support was measured using the Multidimensional Scale of Perceived Social Support (MSPSS), a 12-item, self- administered scale that assesses perceived social support using three subscales: Significant Others, Family and Friends (31). For the purposes of this study, only the MSPSS total score was analysed. Resilience was assessed with the 14-item Resilience Scale (RS-14), a widely used scale measuring adjustment and recovery from the effects of stressors (32). The RS-14 total score serves as a meaningful representation of the concept of resilience. Unmet needs were measured with the Cancer Survivors’ Unmet Needs (CaSUN) measure (33). The CaSUN is a 34- item instrument with five domains, including existential survivorship, comprehensive cancer care, psychological and emotional support, relationship and information. For the purpose of this study, only unmet needs (moderate or strong need) in all five domains were analysed. 2.3 Statistical analysis The methodology for sample size calculation, responder/ non-responder analysis, and handling missing data was detailed elsewhere (22). Here, we performed two stepwise multivariate linear regression analyses using IBM® SPSS® Statistics Version 27, focusing on anxiety and depression as primary outcomes. All variables were first described with the mean and standard deviation (SD) or frequency and percentage. We then assessed the distribution of HADS-A and HADS-D scores and examined the normality of the residuals. Multicollinearity between potential factors associated with depression/ anxiety was rigorously assessed using tolerance statistics, variance inflation factors and correlation values for each candidate variable. After confirming that the HADS-A and HADS-D scores were normally distributed and that there was no multicollinearity, we proceeded with univariate analysis to identify potential factors associated with symptoms of increased anxiety and depression symptoms. A univariate analysis (t-test for independent samples or ANOVA) was used to compare the scores of anxiety and depression with theoretically or clinically relevant factors. Variables that had a significance level of less than 0.2 in the univariate analysis were included in a stepwise linear regression model divided into three different groups: sociodemographic, clinical and psychological factors. A significance level below 0.1 was required for variables to be included in the final model. For the final regression model, variables were considered statistically significant if their P-values were below 0.05. 3 RESULTS 3.1 Patients’ characteristics Sociodemographic and clinical variables are summarised in Table 1. On average, patients were 56 years old, with the majority being married or in a partnership (77.2%). Approximately half of the participants had a university degree or doctorate and 39.8% were employed full-time. In terms of clinical status, 58.6% were diagnosed at stage II and 37.9% had received chemotherapy and radiotherapy, with a mean duration since treatment of 30 months. Sociodemographic, disease and treatment-related variables (N=430). Age (mean±SD), range: 18-90 Marital status, n (%) Married/partnered Single, divorced Widowed Education (n, %) Primary Education Secondary Education University, PhD Employment (n, %) Full-time employed Half-time employed Unemployed Retired Disabled retired Place of residence (n, %) Urban Suburban Rural 55.7±12.4 332 (77.2) 60 (14) 38 (8.8) 28 (6.5) 194 (45.1) 208 (48.4) 171 (39.8) 86 (20.0) 13 (3.0) 146 (34.5) 14 (3.3) 161 (37.4) 155 (36.0) 114 (26.5) Sample of BC survivors Characteristic Table 1. Distribution of depression and anxiety based on the HADS score (n=430). Comparisons between the mean values of symptoms of anxiety and depression across age groups. The abbreviation ‘d’ represents Cohen’s d effect size, while ‘**’ denotes statistical significance with a p-value below 0.01. Legend: SD – standard deviation; n – number of participants; SCQ-19 – The Self Administered Comorbidity Questionnaire Figure 1. Figure 2. 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 58 3.2 Prevalence of anxiety and depression symptoms and its severity The prevalence of anxiety and depression symptoms (HADS≥8) among BC survivors was 34.2% and 30.2%, respectively. Specifically, this included 22.8% possible cases and 11.4% probable cases of anxiety, while for depression the figures were 17.9% for possible cases and 12.3% for probable cases (Figure 1). The mean scores for HADS-A and HADS-D were 6.08 and 5.85, respectively. Smoking status (n, %) Yes No No, but smoked in the past Cancer stage (n, %) 0-I II III Primary treatment, besides surgery (n, %) Chemotherapy only (C) Radiotherapy only (R) C and R None Time since treatment (mean±SD), range: 8-66 months Hormonal therapy (n, %) Yes No 41 (9.5) 313 (72.8) 76 (17.7) 90 (20.9) 252 (58.6) 88 (20.5) 48 (11.6) 169 (39.3) 163 (37.9) 50 (11.6) 29.9±18.2 274 (63.7) 156 (36.3) Sample of BC survivors Characteristic We also conducted an analysis of the mean scores on the HADS-A and HADS-D in different age groups. While there were statistically significant differences in mean scores between all groups (p<0.05), the largest effect in the differences (Cohen’s d=0.5) was observed in the youngest patient group (18-35 years), indicating a remarkable discrepancy in anxiety and depression scores in that age group (Figure 2). 3.3 Factors associated with higher levels of anxiety and depression The results of the final model are presented below. Higher levels of anxiety were significantly associated with younger age (β=0.13, P<0.05), lower quality of life (β=-0.27, P<0.001), higher levels of FCR (β=0.29, P<0.001), lower resilience (β=-0.15, P<0.01), and more unmet PES (β=0.16, P<0.01). The model demonstrated was found to be statistically significant (F[22, 144]=23.82, P<0.01) and explained 57% of the variance. Higher levels of depression were significantly associated with lower quality of life (β=-0.27, P<0.01), higher levels of FCR (β=0.20, P<0.001), lower resilience (β=0.36, P<0.01) and lower perceived social support (β=-0.14, P<0.01). The model was statistically significant (F[19, 202]=30.23, P<0.01) and explained 59% of the variance. 4 DISCUSSION The aim of this study was to determine the prevalence of anxiety and depression symptoms while identifying potential factors that may be associated with higher levels of these. This study shows that a significant proportion of BC survivors, about one third, have symptoms that indicate anxiety and depression above normal levels. One in eight people in this group suffers from clinically relevant anxiety or depression. Importantly, several factors have been identified that correlate with and contribute to increased levels of anxiety and depression. 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 59 Final linear regression model predicting symptoms of anxiety and depression. Age Marital status (ref. married) Single, divorceda Widoweda Education (ref. Primary) Secondarya University, PhDa Employment status (ref. full-time) Half-time Unemployeda Disabled retireda Place of residence (ref. urban) Rurala Time since treatment Treatment type (ref. None) Chemotherapya Chemotherapy and radiotherapya No. of comorbidities Psychological support EQ-5D - index, quality of life FCRI, fear of cancer recurrence RS-14, resilience MPSS-total, social support CaSUN-PES, unmet needs -0.13* 0.04 -0.2 0.05 -0.05 0.04 0.06 -0.01 0.07 0.00 -0.01 0.03 -0.27** 0.29** -0.15** -0.10* 0.16** 0.06 0.03 -0.05 0.03 0.04 0.01 0.02 0.05 0.06 -0.27** 0.20** -0.36** -0.14** HADS-A Standardized beta HADS-DVariable Table 2. In our sample, we observed a significant prevalence of anxiety and depression (HADS≥8) with rates of 34.2% and 30.2%, respectively. Of these cases, 11.4% exhibited clinically relevant levels of anxiety (HADS≥11), while 12.3% met the same criteria for depression. Previous studies (11, 13, 34) conducted with BC survivors’ post-treatment and before COVID-19 using the same measurement tools and cut-off point reported that prevalence for anxiety symptoms (HADS≥8 ) ranged from 12% to 29.1% and between 9% to 11.9% for clinically relevant cases (HADS≥11), placing our prevalence on the higher end of that range. For depression, on the other hand, the prevalence of depressive symptoms (HADS≥8) in previous studies ranged from 7% to 12.5% and for clinically relevant cases from 2.1% to 6.1% (11, 13, 34), which is notably lower compared to the prevalence found in our study. This discrepancy highlights the increased burden of depressive symptoms among our study’s BC survivor population. A recent systematic review reported that anxiety and depression prevalence (HADS≥8) among cancer patients post-COVID-19 was 36% for anxiety and 28% for depression (26), which closely matches the results of our study. However, it is essential to note that these rates were predominantly drawn from patients currently undergoing treatment, where higher prevalence rates are anticipated. The high prevalence observed in our study may be attributed to COVID-19-related factors, including reduced clinical consultations, increased home confinement, and limited interactions (8). These factors could potentially worsen anxiety and depression symptoms. In Slovenia, studies have provided different insights into the prevalence of symptoms of depression. A study conducted by Klemenc-Ketiš et al. between 2010 and 2012 using Zung’s self-rating depression scale found a prevalence of depression of 3.4% (35). The National Institute of Public Health reported a prevalence of depression of 5.5% in 2014, and 5.4% in the 25-34 age group. By 2019, the overall prevalence of depression had risen to 7.5%, while it remained relatively stable in the 25-34 age group at 5.7% (36). In the following year, a study by Jerala et al. used a two-question scale to assess depression and found a prevalence of 4.2% in the middle-aged people (37). In the context of the COVID-19 pandemic, Mihevc et al. (38) conducted a study in which they used the PHQ-9 and GAD-7 questionnaires to assess symptoms of depression and anxiety in the general population. The prevalence of depression and anxiety was 24.4% and 12.9%, respectively (38). In 2021, Rus-Prelog et al. found that during the pandemic 25.5% of the population suffered from moderate- severe depressive symptoms and 21.6% from moderate- severe anxiety symptoms (39). The prevalence of depressive symptoms observed in our study reflects the rates observed in the general population during the COVID-19 pandemic, as described in the two articles above. While these studies provide valuable insights, it is important to remember that they use different assessment tools and methods. On the other hand, the prevalence of anxiety appears to be more pronounced in BC survivors. The pandemic brought with it unique stressors, including social isolation and disrupted access to healthcare, which may have contributed to these changes. Understanding these dynamics is critical to developing effective interventions and support systems for BC survivors. Further research, including longitudinal studies, is needed to examine these trends in depth and develop strategies to improve the mental well-being of this population in times of crisis. The results of our study also show a remarkable discrepancy in the prevalence rates of anxiety and depression (see Figure 2), especially in younger patient groups (18-35 years), consistent with previous research (40, 41). In the youngest group, 21.4% had clinically relevant cases, compared to 12% in the 36-50 and 51-65 age groups, and 5.8% in the 65+ group. This disparity may be due to the greater impact of a cancer diagnosis and treatment on younger patients, who often have a lot of professional and family responsibilities (6). Additionally, the financial strain of taking time off work can heighten their concerns. Facing a life-threatening illness can be especially stressful for younger people, as older patients may have already developed coping strategies to deal with these challenges (42). Our analysis identified age, quality of life, social support, FCR, resilience, and unmet psychosocial and emotional support needs as significant factors associated with increased anxiety levels, consistent with previous studies (11, 43, 44). Notably, increased anxiety correlated with a greater number of unmet psychological and emotional support needs, emphasising the importance of addressing these needs in psychosocial interventions. Our results show that, aside from age, no socio-demographic factors were associated with increased anxiety. Furthermore, marital status showed no significant relationship with the level of anxiety. However, a stronger perceived social support network was associated with lower anxiety scores, highlighting the central role of the quality of support in this context. It is important to recognise that having a partner is no guarantee that a patient’s needs will be met. Our results show that sociodemographic factors are not linked to higher depression levels in post-treatment BC survivors. In our study, depression was significantly associated only with quality of life, resilience, FCR and social support. While this aligns with prior research linking depressive symptoms to these factors (43), our study yielded a surprising result: no statistical significance between the number of comorbidities, treatment type, and depression levels. This suggests that other factors like resilience, social support, or quality of life may be stronger factors associated with depressive symptoms, possibly masking the impact of comorbidities. Notably, both clinical variables showed significant associations with depressive symptoms in the univariate analysis. There could also be interaction effects involving some unaccounted for variables, such as personality traits like pessimism (45) or coping self-efficacy, which have been shown to explain a significant portion of depression variance in post-treatment cancer survivors (46). The limitations and strengths of the design of our study are described in detail in the original work (22), and we mention them only briefly here. Due to the General Data Protection Regulation (GDPR), it was not possible to assess the differences between those who responded to the survey and those who did not. A significant drawback is that our study was not originally designed to identify correlates of anxiety and depression, which meant that some relevant variables (e.g., coping strategies, personality type) were not taken into account. This limited our ability to account for all relevant factors that could contribute to the model we propose here. As this study had a cross- sectional design, no information is available as to whether the symptoms of anxiety and depression remain stable or have a natural decrease. However, with a substantial sample of 430 BC survivors, this study does provide robust insights into anxiety and depression prevalence during the COVID-19 pandemic. Importantly, we have clearly distinguished between possible and probable cases of anxiety and depression, bringing attention to this issue for future research. 5 CONCLUSION In conclusion, this study shows a high prevalence of anxiety and depression in BC survivors, highlighting the need for further research in Slovenia. Comparative data are crucial for a more accurate assessment of the severity of the problem. Importantly, this study highlights the mental health challenges faced by BC survivors following treatment during the COVID-19 pandemic. The study emphasises the importance of recognising and addressing the psychological needs of BC survivors. It also underlines the need for a multidisciplinary approach to cancer care, including mental health, and emphasises the importance of early intervention and of following-up BC survivors over a period of time to monitor and address ongoing mental health problems. ACKNOWLEDGEMENTS The authors would like to thank the study participants for their contribution to the research. CONFLICTS OF INTEREST The authors have no conflicts of interest to declare. FUNDING The project was funded by the Slovenian Research Agency ARRS (Programs MR-39262 and P3-0339). ETHICAL APPROVAL The research was conducted in accordance with the Declaration of Helsinki and its subsequent amendments and with the permission of the National Medical Ethics Committee (number: 0120-25/2019/6) and the Ethics Committee of the Institute of Oncology Ljubljana (number: EK-OI -16092021). AVAILABILITY OF DATA AND MATERIALS The data and materials used in this study are available upon request. 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 60 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 61 ORCID Špela Miroševič: https://orcid.org/0000-0001-5004-0607 Judith Prins: no ORCID number Nikola Bešić: https://orcid.org/0000-0002-9874-8767 Simona Borštnar: https://orcid.org/0000-0003-1980-4992 Andreja Cirila Škufca Smrdel: no ORCID number Vesna Homar: https://orcid.org/0000-0003-1670-6625 Popović Marko: no ORCID number Zalika Klemenc-Ketiš: https://orcid.org/0000-0002-0270-1754 REFERENCES 1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209-249. doi: 10.3322/caac.21660. 2. Allemani C, Matsuda T, Di Carlo V, Harewood R, Matz M, Niksic M, et al. Global surveillance of trends in cancer survival 2000-14 (CONCORD-3): Analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries. Lancet. 2018;391(10125):1023-1075. doi: 10.1016/ S0140-6736(17)33326-3. 3. The organization for economic cooperation and development. EU Country Cancer Profile: Slovenia 2023, EU Country Cancer Profiles. [Internet]. 2023 [cited 2023 Sept 22]. Available from: https://www. gov.si/assets/ministrstva/MZ/DOKUMENTI/DJZ-Preventiva-in-skrb- za-zdravje/nenalezljive-bolezni/DPOR-2022-2026/Country-Cancer- Profile-Slovenija-OECD-in-EK-2023.pdf 4. Pitman A, Suleman S, Hyde N, Hodgkiss A. Depression and anxiety in patients with cancer. BMJ. 2018;361:k1415. doi: 10.1136/bmj.k1415. 5. Wu HS, Harden JK. Symptom burden and quality of life in survivorship: A review of the literature. Cancer Nurs. 2015;38(1):E29-E54. doi: 10.1097/NCC.0000000000000135. 6. Niedzwiedz CL, Knifton L, Robb KA, Katikireddi SV, Smith DJ. Depression and anxiety among people living with and beyond cancer: A growing clinical and research priority. BMC Cancer. 2019;19(1):943. doi: 10.1186/s12885-019-6181-4. 7. Wang X, Wang N, Zhong L, Wang S, Zheng Y, Yang B, et al. Prognostic value of depression and anxiety on breast cancer recurrence and mortality: A systematic review and meta-analysis of 282,203 patients. Mol Psychiatry. 2020;25(12):3186-3197. doi: 10.1038/s41380-020- 00865-6. 8. Swainston J, Chapman B, Grunfeld EA, Derakshan N. COVID-19 lockdown and its adverse impact on psychological health in breast cancer. Front Psychol. 2020;11:2033. doi: 10.3389/fpsyg.2020.02033. 9. Momenimovahed Z, Salehiniya H, Hadavandsiri F, Allahqoli L, Gunther V, Alkatout I. Psychological distress among cancer patients during COVID-19 pandemic in the world: A systematic review. Front Psychol. 2021;12:682154. doi: 10.3389/fpsyg.2021.682154. 10. Gotze H, Friedrich M, Taubenheim S, Dietz A, Lordick F, Mehnert A. Depression and anxiety in long-term survivors 5 and 10 years after cancer diagnosis. Support Care Cancer. 2020;28(1):211-220. doi: 10.1007/s00520-019-04805-1. 11. Breidenbach C, Heidkamp P, Hiltrop K, Pfaff H, Enders A, Ernstmann N, et al. Prevalence and determinants of anxiety and depression in long-term breast cancer survivors. BMC Psychiatry. 2022;22(1):101. doi: 10.1186/s12888-022-03735-3. 12. Smith AB, Rutherford C, Butow P, Olver I, Luckett T, Grimison P, et al. A systematic review of quantitative observational studies investigating psychological distress in testicular cancer survivors. Psychooncology. 2018;27(4):1129-1137. doi: 10.1002/pon.4596. 13. Ho SS, So WK, Leung DY, Lai ET, Chan CW. Anxiety, depression and quality of life in Chinese women with breast cancer during and after treatment: a comparative evaluation. Eur J Oncol Nurs. 2013;17(6):877- 882. doi: 10.1016/j.ejon.2013.04.005. PubMed PMID: 23727448. 14. Kim K, Park H. Factors affecting anxiety and depression in young breast cancer survivors undergoing radiotherapy. Eur J Oncol Nurs. 2021;50:101898. doi: 10.1016/j.ejon.2021. 15. Park EM, Gelber S, Rosenberg SM, Seah DSE, Schapira L, Come SE, et al. Anxiety and depression in young women with metastatic breast cancer: A cross-sectional study. Psychosomatics. 2018;59(3):251-258. doi: 10.1016/j.psym.2018.01.007. 16. Puigpinos-Riera R, Graells-Sans A, Serral G, Continente X, Bargallo X, Domenech M, et al. Anxiety and depression in women with breast cancer: Social and clinical determinants and influence of the social network and social support (DAMA cohort). Cancer Epidemiol. 2018;55:123-129. doi: 10.1016/j.canep.2018.06.002. 17. Schapira L, Zheng Y, Gelber SI, Poorvu P, Ruddy KJ, Tamimi RM, et al. Trajectories of fear of cancer recurrence in young breast cancer survivors. Cancer. 2022;128(2):335-343. doi: 10.1002/cncr.33921. 18. Aizpurua-Perez I, Perez-Tejada J. Resilience in women with breast cancer: A systematic review. Eur J Oncol Nurs. 2020;49:101854. doi: 10.1016/j.ejon.2020.101854. 19. Hartung TJ, Brahler E, Faller H, Harter M, Hinz A, Johansen C, et al. The risk of being depressed is significantly higher in cancer patients than in the general population: Prevalence and severity of depressive symptoms across major cancer types. Eur J Cancer. 2017;72:46-53. doi: 10.1016/j.ejca.2016.11.017. 20. Hashemi SM, Rafiemanesh H, Aghamohammadi T, Badakhsh M, Amirshahi M, Sari M, et al. Prevalence of anxiety among breast cancer patients: a systematic review and meta-analysis. Breast Cancer. 2020;27(2):166-178. doi: 10.1007/s12282-019-01031-9. 21. Pilevarzadeh M, Amirshahi M, Afsargharehbagh R, Rafiemanesh H, Hashemi SM, Balouchi A. Global prevalence of depression among breast cancer patients: A systematic review and meta-analysis. Breast Cancer Res Treat. 2019;176(3):519-533. doi: 10.1007/s10549-019-05271- 3. 22. Miroševič Š, Prins, J, Borštnar S, Bešić N, Homar V, Selič-Zupančič P, et al. Factors associated with a high level of unmet needs and their prevalence in the breast cancer survivors 1-5 years after post local treatment and (neo) adjuvant chemotherapy during the COVID-19: A cross-sectional study. Front Psychol. 2022:1-12. doi: 10.3389/ fpsyg.2022.969918. 23. Miklavčič IV, Snoj Z, Mlakar J, Pregelj P. Validation of the Slovenian version of Hospital Anxiety and Depression Scale in female cancer patients. Psychiatria Danubina. 2008;20(2):148-152. doi: 10.1016/j. eurpsy.2008.01.1149. 24. Annunziata MA, Muzzatti B, Bidoli E, Flaiban C, Bomben F, Piccinin M, et al. Hospital anxiety and depression scale (HADS) accuracy in cancer patients. Support Care Cancer. 2020;28(8):3921-3926. doi: 10.1007/ s00520-019-05244-8. 25. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361-370. doi: 10.1037/t03589-000. 26. Ayubi E, Bashirian S, Khazaei S. Depression and anxiety among patients with cancer during COVID-19 pandemic: A systematic review and meta-analysis. J Gastrointest Cancer. 2021;52(2):499-507. doi: 10.1007/s12029-021-00643-9. 27. Sangha O, Stucki G, Liang MH, Fossel AH, Katz JN. The self-administered comorbidity questionnaire: A new method to assess comorbidity for clinical and health services research. Arthritis Rheum. 2003;49(2):156- 163. doi: 10.1002/art.10993. 28. EuroQol G. EuroQol--a new facility for the measurement of health- related quality of life. Health Policy. 1990;16(3):199-208. doi: 10.1016/0168-8510(90)90421-9. 29. Prevolnik Rupel V, & Ogorevc, M. EQ-5D-5L Value Set for Slovenia. Pharmacoeconomics. 2023:1-10. doi: 10.1007/s40273-023-01280-9. 30. Costa DSJ, Smith AB, Fardell JE. The sum of all fears: Conceptual challenges with measuring fear of cancer recurrence. Support Care Cancer. 2016;24(1):1-3. doi: 10.1007/s00520-015-2943-y. 31. Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. Journal of personality assessment. 1988;52(1):30-41. doi: 10.1207/s15327752jpa5201_2. 32. Miroševič Š, Selič-Zupančič P, Prins J, Klemenc-Ketiš Z. Cross-sectional study examining psychometric properties of the Slovenian version of the 14-item Resilience Scale (RS-14-SL). Qual Life Res. 2023;32(6):1567- 1580. doi: 10.1007/s11136-022-03316-7. 33. Miroševič Š, Selič-Zupančič P, Prins J, Homar V, Klemenc-Ketiš Z. Psychometric properties of the Slovenian version of the Cancer Survivors’ Unmet Needs (CaSUN-SL) measure in post-treatment cancer survivors. BMC Psychol. 2022;10(1):1-10. doi: 10.1186/s40359-022- 00878-6. 34. Brunault P, Suzanne I, Trzepidur-Edom M, Garaud P, Calais G, Toledano A. Depression is associated with some patient-perceived cosmetic changes, but not with radiotherapy-induced late toxicity, in long-term breast cancer survivors. Psychooncology. 2013;22(3):590-597. doi: 10.1002/pon.3038. 35. Klemenc-Ketis Z, Peterlin B. Correlates of depression in the Slovenian working population. Arh Hig Rada Toksikol. 2013;64(4):489-495. doi: 10.2478/10004-1254-64-2013-2360. 36. NIJZ Podatkovni Portal [Internet]. 2023 [cited 2023 Nov 29]. Available from: https://podatki.nijz.si/pxweb/sl/NIJZ%20podatkovni%20 p o r t a l/ ? px _ l anguage = s l&px _ db = N I JZ %20 p o dat kov n i%20 portal&rxid=a4228b6f-0fc4-4e63-8492-a01631aac882 37. Jerala N and Selič-Zupačič P. The onset of depression in middle- aged presumed healthy slovenian family practice attendees and its associations with genetic risk assessment, quality of life and health status: a contribution for family medicine practitioners’ early detection. Int J Environ Res Public Health. 2021;18(15). doi: 10.3390/ ijerph18158197. 38. Mihevc M and Petek Šter M. Stigma and low sense of coherence as long- term predictors of depressive and anxiety symptoms in the population amid the Covid-19 pandemic. Psychiatr Danub. 2023;35(2):250-259. doi: 10.24869/psyd.2023.250. 39. Rus Prelog P, Matic T, Pregelj P, Sadikov A. Risk of depression, anxiety, and stress during the decond wave of Covid-19 in Slovenia. Front Psychiatry. 2021;12:788898. doi: 10.3389/fpsyt.2021.788898. 40. Hartl K, Schennach R, Muller M, Engel J, Reinecker H, Sommer H, et al. Quality of life, anxiety, and oncological factors: a follow-up study of breast cancer patients. Psychosomatics. 2010;51(2):112-123. doi: 10.1176/appi.psy.51.2.112. 41. Hinz A, Herzberg PY, Lordick F, Weis J, Faller H, Brahler E, et al. Age and gender differences in anxiety and depression in cancer patients compared with the general population. Eur J Cancer Care (Engl). 2019;28(5):e13129. doi: 10.1111/ecc.13129. 42. Linden W, Vodermaier A, Mackenzie R, Greig D. Anxiety and depression after cancer diagnosis: prevalence rates by cancer type, gender, and age. J Affect Disord. 2012;141(2-3):343-351. doi: 10.1016/j. jad.2012.03.025. 43. Wang F, Liu J, Liu L, Wang F, Ma Z, Gao D, et al. The status and correlates of depression and anxiety among breast-cancer survivors in Eastern China: a population-based, cross-sectional case-control study. BMC Public Health. 2014;14:326. doi: 10.1186/1471-2458-14-326. 44. Ng CG, Mohamed S, See MH, Harun F, Dahlui M, Sulaiman AH, et al. Anxiety, depression, perceived social support and quality of life in Malaysian breast cancer patients: A 1-year prospective study. Health Qual Life Outcomes. 2015;13:205. doi: 10.1186/s12955-015-0401-7. 45. Faye-Schjoll HH, Schou-Bredal I. Pessimism predicts anxiety and depression in breast cancer survivors: A 5-year follow-up study. Psychooncology. 2019;28(6):1314-1320. doi: 10.1002/pon.5084. 46. Philip EJ, Merluzzi TV, Zhang Z, Heitzmann CA. Depression and cancer survivorship: importance of coping self-efficacy in post-treatment survivors. Psychooncology. 2013;22(5):987-994. doi: 10.1002/ pon.3088. 10.2478/sjph-2024-0008 Zdr Varst. 2024;63(1):55-62 62 These instructions are in accordance with the ICMJE recommendations for the conduct, reporting, editing and publication of scholarly work in medical journals. Complete instructions can be found in the following PDF: http://www.icmje.org/icmje-recommendations.pdf. SHORT DESCRIPTION OF THE SLOVENIAN JOURNAL OF PUBLIC HEALTH The Slovenian Journal of Public Health has been published since 1962 by the National Institute of Public Health in Slovenia. Since 2003, the journal has been a peer-reviewed scientific journal with English abstracts, and since 2014, an international scientific public health journal in English only. The journal’s mission is to promote new achievements in the broad field of public health in Slovenia and Central and South-East Europe. The Slovenian Journal of Public Health publishes internationally oriented articles and encourages an interdisciplinary approach to public health. The journal is a source for exchanging new public health concepts and solutions among researchers. The journal mainly publishes original scientific articles, and on occasion also systematic reviews, methodological articles, and invited editorials. It is published four times a year, with up to 35 articles each year, and has an average annual rejection rate of around 80%. The journal is indexed in major international databases, such as PubMed, Web of Science, and Scopus, and has had an impact factor since 2011, ranging from 0.16 to 1.6. As an open- AUTHOR LOGIN Log in to the EM application as an author (Author Login). The first time you log in, you will need to register by entering your author data. For all subsequent logins, you will only need the login information sent to your email address after the initial login to the system. SLOVENIAN JOURNAL OF PUBLIC HEALTH INSTRUCTIONS FOR AUTHORS October 2023 access journal it is available online on De Gruyter, Sciendo https://sciendo.com/journal/SJPH. The manuscripts are peer-reviewed by three international reviewers, and the process is double-blinded, fair and constructive. MANUSCRIPT SUBMISSION We recommend the use of the video instructions for authors. The journal welcomes submissions in electronic form to the web-based peer-review system, Editorial Manager at http://www.editorialmanager.com/sjph/, and no longer accepts submissions by email or post. SUBMIT NEW MANUSCRIPT After successfully logging into the system, this menu will appear. To submit a new manuscript, click on ‘Submit New Manuscript’ DATA INPUT Submitting the manuscript requires the sequential input of several pieces of data: Article Type Selection, Attach Files, Review Preferences, Additional Information, Comments, and Manuscript Data. MANUSCRIPT CATEGORY (Article Type Selection) Choose the Article Type for your submission from the drop- down menu, then click the ‘Proceed’ button. ATTACHMENTS (Attach Files) You must then upload three Word attachments: 1) the Manuscript, 2) the Title Page, and 3) the Acknowledgement and Declaration Page. Combine the signed Authorship and Copyright Transfer Agreements for all the authors into one PDF document. It is desirable, but not mandatory, to submit this document at this stage. Signed Agreements are mandatory for article publication, but they can also be sent to the editorial office later by email zdrav.var@nijz.si. Please enter reviewers with their names, titles, email addresses, and employment affiliations, and provide specific reasons for your suggestions. Please note that the editorial office may not use your suggestions, but your help is appreciated and may speed up the selection of appropriate reviewers. Click the ‘Browse’ button, find the Manuscript on your computer, and upload the file. Do the same for the Title Page, Declaration Page, and Author Agreements. Then click on the ‘Proceed’ button. REVIEWERS (Review Preferences) In the Review Preferences field, please enter your suggestion of three internationally recognized potential reviewers for this submission. You can also add up to two opposed reviewers. SUPPLEMENTARY INFORMATION (Additional Information) In the Additional Information field please confirm the following statements: • the article has not yet been published or sent for publication to some other journal, • the manuscript has been read and approved by all the authors, • any experiments on humans were carried out following the ethical standards of the Helsinki-Tokyo Declaration, and • any experiments on animals were performed by following the related ethical principles. In the second window, please describe the novelties that your manuscript brings to the field of public health. The text here is limited to 20,000 characters, including spaces. NOTES (Comments) Please enter any additional comments you would like to send to the editorial office. These comments will not appear directly in your submission. METADATA (Manuscript Data) The metadata (Manuscript Data) are the title, abstract, keywords, and authorship. The title, abstract, and keywords are submitted in English and Slovene (the latter not applicable for foreign authors) in structured fields. There is a separate field for entering a different language only for the abstract, and the remaining data must be entered in English and Slovene (the latter not applicable for foreign authors) in the appropriate identical field in the second line. The first abstract is always in English (up to 250 words – the system counts the words for you), and the second in Slovene (extended abstract – up to 400 words). Please list 4 to 6 keywords in plural form and both languages (if applicable). If possible, align these keywords with the MeSH thesaurus. Enter the names of the authors as accurately as possible. You may reorder the authors by dragging and dropping an author’s summary line to the correct position in the Current Author List. Please select a corresponding author, who will be responsible for communication with the editorial office, and the co- authors (we need the address, telephone number, and email address of each author). MANUSCRIPT PDF The last step in submitting a manuscript is to build a PDF of your manuscript. Building the PDF takes some time, so please wait while the system builds it. Then open the PDF, review it, and confirm it by clicking on the ‘Approve Submission’ button. You have now submitted the manuscript to the editorial office, and the editorial process can now begin. Thank you for choosing the Slovenian Journal of Public Health to submit your manuscript. If you have any problems, please do not hesitate to contact the editorial office. DETAILED DESCRIPTION OF THE REQUIREMENTS FOR ALL ATTACHMENTS MANUSCRIPT The language of the manuscript is English and UK spelling is preferred. The name of the manuscript file must not include the author’s personal data or the names of the institutions involved in the preparation of the manuscript. Texts should be written using the Word for Windows word processor. The margins should be 25 mm wide, the font should be Ariel and the size 12. In Word, please use the Layout / Line numbers / Continuous format (this will add a line number in the margin of each line of the manuscript for reviewers to refer to when writing their reviews). The manuscript should have the following sections: introduction, methods, results, discussion, conclusion and references. Invited editorials may be structured differently, but the division into chapters and sub-chapters should be clearly indicated by the bolding of the letters in the headings. Chapters and subchapters should be numbered in decimal form according to SIST ISO 2145 and SIST ISO 690 (e. g. 1, 1.1, 1.1.1, etc.). Avoid abbreviations and acronyms, with the exception of internationally valid unit designations. If an abbreviation of a term will be used, then the first time the term appears in the text it should be written in full, followed by the abbreviation used thereafter in parentheses. Include graphic material (tables and figures) in the text where it belongs, with headings and legends explaining abbreviations. Use black and white images, and the background should always be white. Letters, numbers or symbols in figures should be clear, uniform and large enough to be readable even when reproduced in a smaller size. Units of measurement should be in accordance with the International System of Units (SI). Required length for invited editorial is 250 to 2000 words and for research article 2000 to 4500 words with tables and references. The revision may has 5000 words. References: The journal follows the Vancouver numerical referencing commonly used in biomedicine. Formatted citations should be numbered consecutively in a reference list as they are cited in the manuscript text. References cited in tables or figures legends should also be numbered to be in sequence with the references cited in the text. Please do not use the footnote or endnote feature to cite or create a reference list. Try to avoid using personal communications, unpublished data, and manuscripts in preparation as references. In-text citations: Each mention of the statements or findings by other authors should be cited. Reference numbers (Arabic numerals) in the text should appear in normal type and curved parentheses: Adam et al. state that the data is ‘unreliable’ (1, p. 122). This argument is increasingly relevant to the topic (2, 3) … Several studies (1, 4-8, 12) … Each entry starts with the author’s surname and initials. When a source has more than one author, their names are separated by commas. If a source has more than six authors, list the first six followed by ‘et al.’ Only the first word of the title and subtitle, along with any proper nouns, are capitalized. Titles in Vancouver referencing are consistently written in plain text, without italics or quotation marks. Journal titles should be abbreviated according to the National Library of Medicine’s List of Journals Indexed for Medline; for unlisted journals, please provide complete journal titles. If the article/book has a DOI number, please include it at the end of the reference. Below are some examples of the most commonly cited types of sources. Journal article Vodička S, Zelko E. Remote consultations in general practice: A systematic review. Zdr Varst. 2022 Sep 28;61(4):224-230. doi: 10.2478/ sjph-2022-0030. de Villiers TJ. The role of menopausal hormone therapy in the management of osteoporosis. Climacteric. 2015;18 Suppl 2:19-21. doi: 10.3109/13697137.2015.1099806. Book Wilkinson IB, Raine T, Wiles K, Goodhart A, Hall C, O’Neill H, et al. Oxford handbook of clinical medicine. 10th ed. Oxford: Oxford University Press; 2017. 123 p. Kaplan SJ. Post-hospital home health care: the elderly’s access and utilization [dissertation]. St. Louis (MO): Washington University; 1995. Book chapter Goldberg BW. Population-based health care. In: Taylor RB, Robin S, editors. Family medicine. 5th ed. Cambridge: Cambridge University Press; 1999. p. 32–36. Website Cancer Research UK. Current research into breast cancer [Internet]. 2020 [cited 2022 Dec 14]. Available from: https://www. cancerresearchuk.org/our-research/our-research-by-cancer-type/our- research-into-breast-cancer/current-breast-cancer-research McNeil DG. Vaccines against HIV, malaria and tuberculosis unlikely, study says. New York Times. 2018 Sep 7. [cited 2018 Nov 14]. Available from: https://www.nytimes.com/2018/09/07/health/vaccines-hiv- malaria-tuberculosis.html Before publishing an article in the journal, the editorial office reviews the literature list and, if necessary, corrects it. You can use a reference manager programme to cite your literature. Please choose the Vancouver citation style. When preparing the manuscript, the authors should check the Slovenian literature on the topic that was published in the last five years. TITLE PAGE The title page should include the following information: title, authors, affiliations, ORCID numbers of authors, email address of corresponding author, abstract, and keywords. The title should be informative and precise, descriptive and not assertive (full sentences are not allowed in titles). The title should not contain abbreviations. The names of the authors should be given in the order preferred, with the full addresses of the institutions where the authors are employed. Authors must meet the conditions for authorship. They must contribute to the conception and design or analysis and interpretation of the data, they must intellectually conceive of and critically review the manuscript, and they must agree with the final version of the manuscript. Simply collecting data is not sufficient for authorship. Shared first authorship is allowed for up to two authors. Please give the ORCID numbers of the authors and the email address of the corresponding author. The abstract should be structured in the IMRC structure and no longer than 250 words in English and 400 words in Slovenian. The abstract for invited editorials may be unstructured. The abstract should summarize the content of the manuscript, written in the third person, and avoid abbreviations and acronyms. Between 3 and 6 keywords should be listed. ACKNOWLEDGEMENT AND DECLARATION PAGE The acknowledgement should be placed first in the document. It should thank all contributors who did not receive authorship of the manuscript. The document should then include the following statements: CONFLICTS OF INTEREST (The authors declare that no conflicts of interest exist.) FUNDING (The study was financed by ...) ETHICAL APPROVAL (Received from the… or description of the ethical aspect of the research) AVAILABILITY OF DATA AND MATERIALS (All data and materials used in this study were collected from publicly available sources and are available upon reasonable request. or Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.) LLM STATEMENT (Authors should only use generative AI and AI-assisted technologies (like ChatGTP) to improve readability and language of the manuscript. Authors must disclose the use of these technologies in the writing process by adding the LLM statement. Example: During the preparation of this work the author(s) used [NAME TOOL /SERVICE] in order to [REASON]. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.) PREPRINT STATEMENT Deposition of preprints in recognized preprint repositories, such as bioRxiv, medRxiv, and others commonly used academic preprint repositories, is allowed before manuscript submission to the SJPH, but they must not have been published or submitted elsewhere. Authors are required to disclose the existence of a preprint in the Preprint statement. Please include the name of the preprint server and the DOI or URL of the preprint. Example: The preprint has been deposited in a preprint server Research Square, and is available from https://www.researchsquare.com/article/rs-2351315/v2. It is also recommended to cite the preprint in a reference list. For the SJPH public comments on preprints are not acceptable as reviews. After the submission is accepted for publication in the SJPH and before the publication, authors are required to link the preprint to the SJPH article with its DOI number. It is recommended that authors cite the final, published version of a work, not the preprint. Research involving human subjects (including human material or human data) must have been performed in accordance with the Declaration of Helsinki and must have been approved by an appropriate ethics committee. A statement detailing this, including the name of the ethics committee and the reference number where appropriate, must appear in all manuscripts reporting research on human subjects. If a study has been granted an exemption from requiring ethics approval, this should also be detailed. The authors must explain the rationale for their approach, and demonstrate that the institutional review body explicitly approved any questionable aspects of the study. Further information and documentation to support this should be made available to the editors on request. A manuscript may be rejected if the editors consider that the research has not been carried out within an ethical framework. In rare cases, the editors may contact the related ethics committee for further information. For all research involving human subjects, informed consent to participate in the study should be obtained from participants (or their parent or guardian in the case of minors) and a statement to this effect should be provided. For all manuscripts that include information or images relating to individual participants, written informed consent for the publication of these images must be obtained from the participant (or their parent or guardian in the case of minors) and a statement to this effect should be provided. These documents must be made available to editors if requested, and will be treated confidentially. For research carried out on animals, authors are encouraged to comply with the “Animal Research: Reporting in Vivo Experiments” - ARRIVE guidelines, and must comply with local or institutional ethics approval requirements on the care and use of animals for research. A statement detailing such ethics approval and/or guidelines must be provided. Declarations must disclose any financial or other interests of the pharmaceutical industry or equipment manufacturers and institutions associated with the manuscript. You can find examples of such attachments on the journal’s website. EDITORIAL WORK Submitted manuscripts with a public health theme of international relevance are forwarded to two editors for review after technical integrity and plagiarism checking with Crossref Ithenticate (if the editorial office finds a manuscript to be plagiarized, the manuscript is immediately excluded from the editorial process). If a manuscript is judged worthy of the peer review process, the editor sends it to three internationally renowned peer reviewers for review, at least one of whom must be from outside Slovenia. The review process is double-blind. Once the review process is complete, the manuscript is returned to the corresponding author for confirmation and consideration of corrections. The revision of the manuscript must be submitted to the Editorial Manager system within two months (first revision) or within one month (subsequent revisions). The journal allows a maximum of three revisions. If the third revision does not take into account all the comments of the reviewers, the manuscript will be rejected. After acceptance of the manuscript, a linguistic proofreading follows. During the editorial process, the confidentiality of the content of the manuscript is guaranteed. The author will receive the first, so-called brush proofs, but at this stage only corrections of typographical errors are taken into account. The brush proofs must be returned within three days, otherwise we consider that the author has no objections. The editorial team is committed to making the editorial process as fast as possible. Authors must respect the deadlines otherwise the manuscript may be removed from the process. Any complaints from authors are dealt with by the Editorial Board of the journal. To publish an article the authors transfer copyright to the publisher, the National Institute of Public Health, and sign Authorship and Copyright Transfer Agreements. Infringement of copyright and other related rights is a criminal offence. We do not pay for articles or reviews. We also do not charge any article publication fee. Slovenian Journal of Public Health is an open-access journal available online on Sciendo. Authors also receive author copies of the printed journal in which their article appears. If you encounter any difficulties in submitting your manuscript, please contact the editorial office at zdrav.var@nijz.si for assistance. NAVODILA AVTORJEM REVIJE ZDRAVSTVENO VARSTVO Navodila so v skladu s priporočili ICMJE recommendations for the conduct, reporting, editing and publication of scholarly work in medical journals. Popolna navodila so objavljena na spletni strani http://www.icmje.org/icmje- recommendations.pdf. KRATEK OPIS REVIJE ZDRAVSTVENO VARSTVO Revija Zdravstveno varstvo (SJPH) izhaja od leta 1962 in danes predstavlja temeljno znanstveno revijo s področja javnega zdravja na območju centralne in JV Evrope. Revija objavlja članke s širšo mednarodno tematiko s področja javnega zdravja in spodbuja objavo rezultatov interdisciplinarnih raziskav na tem področju. Objavlja izvirne znanstvene članke, v manjši meri tudi sistematične pregledne znanstvene članke in metodološke članke ter vabljene uvodnike. Letno objavi štiri številke, skupno do 35 člankov. Povprečni letni osip je okoli 80 %. Revija je vključena v številne mednarodne podatkovne zbirke, tudi v PubMed in v oba citatna indeksa WoS in Scopus ter ima faktor vpliva neprekinjeno že od leta 2011; giblje se med 0,16 in 1,6. Revija Zdravstveno varstvo se v e-obliki nahaja na straneh založbe De Gruyter, Sciendo https://sciendo. com/journal/SJPH. ELEKTRONSKA ODDAJA ROKOPISA Priporočamo ogled videoposnetka z navodili za avtorje. Rokopise oddajte v elektronski obliki v spletno uredniško aplikacijo Editorial Manager (EM), ki se nahaja na spletnem naslovu http://www.editorialmanager.com/sjph/. PRIJAVA V EM SISTEM V uredniško aplikacijo se prijavite kot avtor (Author Login). Prva prijava – registracija zahteva vnos podatkov o avtorju, vse nadaljnje prijave pa le še vnos podatkov za prijavo, ki jih na svoj elektronski naslov prejmete po prvi prijavi v sistem. Oktober 2023 VNOS PODATKOV Oddaja rokopisa zahteva zaporeden vnos več podatkov, ki si jih bomo ogledali v nadaljevanju: Article Type Selection, Attach Files, Review Preferences, Additional Information, Comments in Manuscript Data. ODDAJA NOVEGA ROKOPISA Po uspešni prijavi se pojavi ta meni. Za oddajo novega rokopisa kliknete na Submit New Manuscript. Potrdite izjavo, da vaš prispevek še ni bil objavljen ali poslan v objavo kakšni drugi reviji, da so prispevek prebrali in se z njim strinjajo vsi soavtorji, in da so raziskave na ljudeh oz. živalih opravljene v skladu z načeli Helsinško- Tokijske deklaracije oz. v skladu z etičnimi načeli. V drugo okno zapišite novosti, ki jih vaš rokopis prinaša področju javnega zdravja. KOMENTARJI (Comments) Polje Comments je namenjeno vnosu komentarjev, ki bi jih avtorji želeli dodatno posredovati uredništvu revije. KATEGORIJA ROKOPISA (Article Type Selection) Kategorijo rokopisa določite z izborom vrste rokopisa. Objavljamo izvirne znanstvene članke in metodološke članke (izberete Original Study), sistematične pregledne znanstvene članke (izberete Systematic Review) in vabljene uvodnike (izberete Invited Editorial). Po izboru kliknete na gumb Proceed. PRIPONKE (Attach Files) Sledi vnos priponk: rokopisa, naslovne strani, izjav z morebitno zahvalo in avtorskih pogodb. Obvezno oddate tri Wordove priponke: Manuscript, Title Page, Acknowledgement and Declaration Page. Podpisane pogodbe Authorship and Copyright Transfer Agreements za vse soavtorje združite v en PDF dokument. Oddaja v tej fazi je zaželena, a ni obvezna. Podpisane pogodbe so pogoj za objavo članka. V uredništvo jih lahko oddate tudi naknadno po sprejemu članka v objavo po e-pošti zdrav.var@nijz.si. Nujni podatki so ime, priimek, zaposlitev in e-naslov. Razlog za izbor je zaželen podatek. DODATNE INFORMACIJE (Additional Information) Ta del je namenjen potrditvi dodatnih izjav avtorjev in opisu novosti vašega rokopisa. S klikom na gumb Browse vstopite v svoj računalnik in naložite rokopis. Enako naredite za naslovno stran, izjave z morebitno zahvalo in avtorske pogodbe. Nato kliknite na gumb Proceed. RECENZENTI (Review Preferences) Vnesite prosim predlog treh mednarodno priznanih recenzentov. Pribeležite lahko tudi do dva neželena recenzenta. METAPODATKI (Manuscript Data) Metapodatki so naslov, izvleček, ključne besede in avtorstvo. Naslov, izvleček in ključne besede se oddajajo dvojezično v angleščini in v slovenščini (ne velja za tuje avtorje) v strukturirana polja. Posebno polje za zapis v drugem jeziku obstaja le za izvleček, preostale podatke vnesite v obeh jezikih v ustrezno isto polje v drugo vrstico. Prvi izvleček je vselej v angleškem jeziku (do 250 besed - sistem vam besede sproti šteje), drugi pa v slovenskem jeziku (razširjen izvleček - do 400 besed). Zapišite še 4 do 6 ključnih besed v množini in v obeh jezikih; če se le da, naj bodo usklajene s tezavrom MeSH. Podatke o avtorju in soavtorjih vnesite kar se da natančno in popolno. Naveden naj bo korespondenčni avtor (s polnim naslovom, telefonsko številko in elektronskim naslovom), ki bo skrbel za komunikacijo z uredništvom in ostalimi avtorji. Tako ste rokopis oddali v uredništvo, sledi uredniški postopek. Hvala, da ste za oddajo svojega rokopisa izbrali revijo Zdravstveno varstvo. V primeru težav se prosim obrnite na uredništvo. NATANČEN OPIS ZAHTEV ZA VSE PRIPONKE ROKOPIS Jezik rokopisa je angleščina. Ime datoteke z rokopisom ne sme zajemati avtorjevih osebnih podatkov, prav tako ne imen ustanov, vključenih v pripravo rokopisa. Besedila naj bodo napisana z urejevalnikom besedil Word for Windows. Robovi naj bodo široki 25 mm, izberite črke Ariel in velikost črk 12. V Wordu uporabite možnost Postavitev strani/Številke vrstic/Zaporedno (tako bo na robu vsake vrstice rokopisa dodana številka vrstice, na katero se lahko referirajo recenzenti pri pisanju recenzij). Rokopis naj ima naslednja poglavja: uvod, metode, rezultati, razprava, zaključek in reference. Vabljeni uvodniki so lahko zasnovani drugače, vendar naj bo razdelitev na poglavja in podpoglavja jasno razvidna iz odebelitve črk v naslovih. Poglavja in podpoglavja naj bodo številčena dekadno po standardu SIST ISO 2145 in SIST ISO 690 (npr. 1, 1.1, 1.1.1 itd.). Kraticam in okrajšavam se izogibajte, izjema so mednarodno veljavne oznake merskih enot. Na mestu, kjer se kratica prvič pojavi v besedilu, naj bo izraz, ki ga nadomešča, polno izpisan, v nadaljnjem besedilu uporabljano kratico navajajte v oklepaju. Grafično in slikovno gradivo vključite v besedilo na mesto, kamor le-to sodi in ga opremite z naslovi in legendami, v katerih pojasnite okrajšave. Uporabite črno-bele prikaze, ozadje naj bo vselej belo. Črke, številke ali simboli na slikah naj bodo jasni, enotni in dovolj veliki, da so berljivi tudi na pomanjšanem prikazu. Merske enote naj bodo v skladu z mednarodnim sistemom enot (SI). Zahtevana dolžina rokopisa je za vabljeni uvodnik od 250 do 2000 besed, za ostale vrste rokopisov pa od 2000 do 4500 besed s slikovnim gradivom in literaturo vred. Revizija sme obsegati 5000 besed. PDF ROKOPISA Zadnji korak pri oddaji rokopisa je izgradnja PDF rokopisa. Postopek malo traja, zato počakajte, da sistem PDF zgradi, PDF odprite, preglejte in potrdite s klikom na gumb Approve Submission. Reference: Zdravstveno varstvo uporablja Vancouverski numerični stil citiranja in navajanja literature, ki je v biomedicini splošno v uporabi. Urejeni citati si v seznamu literature na koncu rokopisa sledijo zaporedno, kot so zapisani v besedilu rokopisa. V to zaporedje vključite tudi citate, ki se pojavljajo v tabelah, njihovih legendah ali v slikovnem gradivu. Za citiranje in navajanje literature ne uporabljajte opomb pod črto. Izogibajte se citiranju osebnih pogovorov, neobjavljenih podatkov in rokopisov, ki so v uredniškem postopku. Citiranje v besedilu: Vsako navajanje trditev ali dognanj drugih avtorjev morate podpreti s citatom. Številka reference naj bo navedena v običajni velikosti na koncu citirane trditve v okroglih oklepajih. Uporbljajte arabske številke, navedete lahko tudi stran citata: Adam et al. state that the data is ‘unreliable’ (1, p. 122). This argument is increasingly relevant to the topic (2, 3) … Several studies (1, 4-8, 12) … Seznam literature: Numerično urejen seznam literature poimenujte z besedo “References” in ga postavite na konec rokopisa. Avtorje beležite s priimkom in kraticami imena, med posameznimi avtorji postavite vejico. Navedite imena vseh avtorjev; v primeru, da je avtorjev šest ali več, navedite prvih šest avtorjev in dodajte kratico et al. Naslov in podnaslov pišite z malimi začetnicami z izjemo prve besede in lastnih imen. Uporabljajte običajno pisavo in se izogibajte ležeči pisavi ali zapisu v navednicah. Naslove revij krajšajte tako kot baza Medline/PubMed. Popoln seznam kratic revij najdete na naslovu National Library of Medicine’s List of Journals Indexed for Medline. Naslovov revij, katerih kratic v seznamu ni, ne krajšajte. Če ima objava DOI številko, jo navedite na koncu reference. Primeri navajanja najbolj pogosto uporabljanih vrst objav: Članek v reviji Vodička S, Zelko E. Remote consultations in general practice: A systematic review. Zdr Varst. 2022 Sep 28;61(4):224-230. doi: 10.2478/ sjph-2022-0030. de Villiers TJ. The role of menopausal hormone therapy in the management of osteoporosis. Climacteric. 2015;18 Suppl 2:19-21. doi: 10.3109/13697137.2015.1099806. Knjiga Wilkinson IB, Raine T, Wiles K, Goodhart A, Hall C, O’Neill H, et al. Oxford handbook of clinical medicine. 10th ed. Oxford: Oxford University Press; 2017. 123 p. Kaplan SJ. Post-hospital home health care: the elderly’s access and utilization [dissertation]. St. Louis (MO): Washington University; 1995. Poglavje v knjigi Goldberg BW. Population-based health care. In: Taylor RB, Robin S, editors. Family medicine. 5th ed. Cambridge: Cambridge University Press; 1999. p. 32–36. Spletna stran Cancer Research UK. Current research into breast cancer [Internet]. 2020 [cited 2022 Dec 14]. Available from: https://www. cancerresearchuk.org/our-research/our-research-by-cancer-type/our- research-into-breast-cancer/current-breast-cancer-research McNeil DG. Vaccines against HIV, malaria and tuberculosis unlikely, study says. New York Times. 2018 Sep 7. [cited 2018 Nov 14]. Available from: https://www.nytimes.com/2018/09/07/health/vaccines-hiv- malaria-tuberculosis.html Primere navajanja redkeje uporabljanih vrst objav lahko najdete na spletni strani NLM knjižnice. Uredništvo pred objavo članka v reviji Zdravstveno varstvo seznam literature pregleda in ga po potrebi popravi v skladu z navodili. Za navajanje literature lahko uporabljate urejevalnike referenc, pri čemer izberete Vancouverski stil citiranja. Avtorjem priporočamo, da ob pripravi rokopisa pregledajo slovensko literaturo na temo svojega rokopisa, objavljeno v obdobju zadnjih petih let. NASLOVNA STRAN Naslovna stran naj zajema sledeče podatke: title / naslov, avtorji, zaposlitve, ORCID številke avtorjev, e-poštni naslov korespondenčnega avtorja, abstract / izvleček, keywords / ključne besede. Naslov v angleškem in slovenskem jeziku naj bo informativen in natančen, opisen in ne trdilen (povedi v naslovih niso dopustne). V naslovu naj ne bo kratic. Imena avtorjev naj bodo navedena v želenem zaporedju, dodani naj bodo popolni naslovi ustanov, kjer so avtorji zaposleni. Avtorji morajo izpolnjevati pogoje za avtorstvo. Prispevati morajo k zasnovi in oblikovanju oz. analizi in interpretaciji podatkov, rokopis morajo intelektualno zasnovati in ga kritično pregledati, strinjati se morajo s končno različico rokopisa. Zgolj zbiranje podatkov ne zadostuje za avtorstvo. Deljeno prvo avtorstvo je dovoljeno za največ dva avtorja. Dopišite ORCID številke avtorjev in e-poštni naslov korespondenčnega avtorja. Izvleček v angleškem in slovenskem jeziku naj bo strukturiran v IMRC strukturi in naj ne bo daljši od 250 besed v angleščini in 400 besed v slovenščini. Izvleček pri vabljenih uvodnikih je lahko nestrukturiran. Izvleček naj vsebinsko povzema in ne le našteva bistvene vsebine rokopisa. Napisan naj bo v 3. osebi. Izogibajte se kraticam in okrajšavam. Navedenih naj bo med 3 in 6 ključnih besed, ki bodo v pomoč pri indeksiranju. ZAHVALA IN IZJAVE Zahvala se naj nahaja na prvem mestu v dokumentu. Vsebuje naj zahvalo vsem sodelujočim pri rokopisu, ki niso prejeli avtorstva rokopisa. Dokument naj nato zajema še sledeče izjave: CONFLICTS OF INTEREST (The authors declare that no conflicts of interest exist.) FUNDING (The study was financed by ...) ETHICAL APPROVAL (Received from the… ali opis etičnega vidika raziskave) AVAILABILITY OF DATA AND MATERIALS (All data and materials used in this study were collected from publicly available sources and are available upon reasonable request. ali Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.) LLM STATEMENT (V LLM izjavi morajo avtorji obvezno navesti morebitno uporabo generativnih jezikovnih modelov (kot je ChatGTP) za izboljšanje jezika in berljivosti rokopisa. Primer: During the preparation of this work the author(s) used [NAME TOOL /SERVICE] in order to [REASON]. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.) PREPRINT STATEMENT Shranjevanje preprintov v priznanih repozitorijih preprintov, kot so bioRxiv, medRxiv in drugi akademski repozitoriji preprintov, je dovoljeno pred oddajo rokopisa v SJPH, vendar ne smejo biti objavljeni ali oddani v postopek še kje drugje. Avtorji morajo v Izjavi o preprintih navesti obstoj preprinta. Navedite ime strežnika preprintov in DOI ali URL preprinta. Primer: The preprint has been deposited in a preprint server Research Square, and is available from https://www.researchsquare.com/article/rs- 2351315/v2. Priporočljivo je tudi citirati preprint v seznamu referenc. Za SJPH javni komentarji o preprintih niso sprejemljivi kot recenzije. Po sprejemu rokopisa v objavo v naši reviji in pred samo objavo morajo avtorji povezati preprint s sprejetim člankom z DOI številko. Priporočljivo je, da avtorji citirajo končno objavljeno različico dela, ne preprinta. Raziskave na ljudeh (vključno s človeškimi materiali in osebnimi podatki) morajo biti izpeljane v skladu s Helsinško deklaracijo in potrjene s strani nacionalne etične komisije. V izjavi o etiki morajo avtorji podati izjavo o etiki raziskav na ljudeh, ki mora vsebovati ime etične komisije in referenčno števiko obravnave. Poročanje o raziskavah na ljudeh brez potrdila etične komisije zahteva dodatno razlago v poglavju o metodah dela. Na zahtevo Uredništva je avtor dolžan predložiti vso dokumentacijo o obravnavi raziskovalne etike njegovega rokopisa. Uredništvo si pridržuje pravico, da kontaktira etično komisijo. Prav tako morajo avtorji, ki poročajo o ljudeh ali posredujejo javnosti njihovo slikovno gradivo, pridobiti dovoljenja vseh sodelujočih, da se z vključitvijo v raziskavo strinjajo (v primeru otrok so to starši ali skrbniki). Izjavo o pridobitvi teh dovoljenj morajo avtorji podati v poglavju o metodah dela. Uredništvo si pridržuje pravico vpogleda v to dokumentacijo. Raziskave na živalih morajo biti izpeljane v skladu z navodili “Animal Research: Reporting in Vivo Experiments”- (ARRIVE) in potrjene s strani nacionalne etične komisije. V poglavju o metodah dela in med izjavami morajo avtorji podati izjavo o etiki raziskav na živalih z veljavno številko dovoljenja. V izjavah morajo biti zapisani morebitni finančni ali drugi interesi farmacevtske industrije ali proizvajalcev opreme ter inštitucij, povezanih z rokopisom. Primere priponk najdete na spletni strani revije. UREDNIŠKO DELO Prispele rokopise z javnozdravstveno tematiko mednarodnega pomena posreduje uredništvo po tehnični brezhibnosti in plagiatorskem pregledu s programom Crossref Ithenticate (če uredništvo ugotovi, da je rokopis plagiat, se rokopis takoj izloči iz uredniškega postopka) v pregled dvema urednikoma. Če je rokopis ocenjen kot vreden recenzentskega postopka, ga uredništvo pošlje v recenzijo trem mednarodno priznanim recenzentom, vsej eden mora biti iz tujine. Recenzijski postopek je dvojno slep. Po končanem recenzentskem postopku vrnemo rokopis korespondenčnemu avtorju, da popravke upošteva. Revizijo rokopisa vrne avtor v aplikacijo Editorial Manager. Uredništvo dopušča obravnavo največ treh revizij. Če tretja revizija rokopisa ne upošteva vseh pripomb recenzentov, se rokopis umakne iz uredniškega postopka. Po sprejemu rokopisa sledi jezikovna lektura. Med redakcijskim postopkom je zagotovljena tajnost vsebine rokopisa. Avtor dobi v pogled tudi prve, t. i. krtačne odtise, vendar na tej stopnji upoštevamo le še popravke tiskarskih napak. Krtačne odtise je potrebno vrniti v treh dneh, sicer menimo, da avtor nima pripomb. V uredništvu se trudimo za čim hitrejši uredniški postopek. Avtorji se morajo držati rokov, ki jih dobijo v dopisih, sicer se lahko zgodi, da bo rokopis umaknjen iz postopka. Morebitne pritožbe avtorjev obravnava uredniški odbor revije. Za objavo članka prenesejo avtorji avtorske pravice na založnika, torej na Nacionalni inštitut za javno zdravje in podpišejo Pogodbe o avtorstvu in prenosu avtorskih pravic. Kršenje avtorskih in drugih sorodnih pravic je kaznivo. Člankov in recenzij ne honoriramo. Stroškov obravnave rokopisov in objave člankov avtorjem ne zaračunavamo. Revija Zdravstveno varstvo je na spletu prosto dostopna. Avtorji prejmejo tudi avtorske izvode tiskane revije, v kateri je objavljen njihov članek. Če pri oddajanju rokopisa naletite na nepremostljive težave, se za pomoč prosim obrnite na naslov uredništva zdrav.var@nijz.si. SLOVENIAN JOURNAL OF PUBLIC HEALTH ZDR VARST 2024 • YEAR 63 • No 1 ZDR VARST 2024 • LETNIK 63 • ŠTEVILKA 1 CODEN ZDVAFY • UDK 613 / 614 + 628 • ISSN 0351 - 0026 Zalika KLEMENC-KETIŠ,Uroš ZAFOŠNIK 1 MEDPOKLICNO IZOBRAŽEVANJE S SIMULACIJAMI V PRIMARNEM ZDRAVSTVENEM VARSTVU (1-4) Matic MIHEVC, Črt ZAVRNIK, Majda MORI LUKANČIČ, Tina VIRTIČ POTOČNIK, Marija PETEK ŠTER, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ ANALIZA STROŠKOV TELEMONITORINGA OD SPODAJ NAVZGOR: ŠTUDIJA PRIMERA V SLOVENSKEM PRIMARNEM ZDRAVSTVU (5-13) Andrej PANGERC, Marija PETEK ŠTER, Leja DOLENC GROŠELJ VALIDACIJA SLOVENSKE RAZLIČICE VPRAŠALNIKA STOP-BANG ZA UPORABO V AMBULANTAH NA PRIMARNEM NIVOJU ZDRAVSTVENEGA VARSTVA (14-20) Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ ZAZNAVANJE KULTURE VARNOSTI PACIENTOV NA PRIMARNI RAVNI ZDRAVSTVENEGA VARSTVA: RAZISKAVA V ZDRAVSTVENEM DOMU LJUBLJANA (21-29) Matija AMBROŽ, Candan KENDIR, Wienke BOERMA, Zalika KLEMENC-KETIŠ OCENA IZKUŠENJ IN IZIDOV ZDRAVSTVENE OSKRBE PACIENTOV S KRONIČNIMI NENALEZLJIVIMI BOLEZNIMI: METODOLOŠKI VIDIK PILOTNE IZVEDBE RAZISKAVE V SLOVENIJI (30-37) Črt ZAVRNIK, Nataša STOJNIĆ, Majda MORI LUKANČIČ, Matic MIHEVC, Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ PRILOŽNOSTI IN OVIRE ZA IZBOLJŠANJE CELOSTNE OSKRBE ARTERIJSKE HIPERTENZIJE IN SLADKORNE BOLEZNI TIPA 2 V SLOVENIJI: KVALITATIVNA ŠTUDIJA (38-45) Tadeja HOČEVAR, Tim ANSTISS, Danica ROTAR PAVLIČ VSEBINSKA VELJAVNOST IN KOGNITIVNO TESTIRANJE PRI RAZVOJU SAMO-OCENJEVALNEGA VPRAŠALNIKA O MOTIVACIJSKEM INTERVJUJU (46-54) Špela MIROŠEVIČ, Judith PRINS, Nikola BEŠIĆ , Simona BORŠTNAR , Andreja Cirila ŠKUFCA SMRDEL, Vesna HOMAR, Marko POPOVIĆ, Zalika KLEMENC-KETIŠ SIMPTOMI ANKSIOZNOSTI IN DEPRESIJE PRI SLOVENSKIH BOLNICAH Z RAKOM DOJKE PO ZDRAVLJENJU V ČASU PANDEMIJE COVIDA-19: PRESEČNA ŠTUDIJA (55-62) UVODNIK IZVIRNI ZNANSTVENI ČLANKI Zalika KLEMENC-KETIŠ,Uroš ZAFOŠNIK 1 INTERPROFESSIONAL EDUCATION WITH SIMULATIONS IN PRIMARY CARE (1-4) Matic MIHEVC, Črt ZAVRNIK, Majda MORI LUKANČIČ, Tina VIRTIČ POTOČNIK, Marija PETEK ŠTER, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ BOTTOM-UP ANALYSIS OF TELEMONITORING COSTS: A CASE STUDY IN SLOVENIAN PRIMARY CARE (5-13) Andrej PANGERC, Marija PETEK ŠTER, Leja DOLENC GROŠELJ VALIDATION OF THE SLOVENE VERSION OF THE STOP-BANG QUESTIONNAIRE IN A PRIMARY PRACTICE SETTING (14-20) Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ PERCEPTION OF PATIENT SAFETY CULTURE AT THE PRIMARY CARE LEVEL: THE CASE OF THE COMMUNITY HEALTH CENTRE LJUBLJANA (21-29) Matija AMBROŽ, Candan KENDIR, Wienke BOERMA, Zalika KLEMENC-KETIŠ PATIENT REPORTED INDICATOR SURVEYS (PARIS): METHODOLOGICAL CONSIDERATIONS OF A FIELD TRIAL IN SLOVENIA (30-37) Črt ZAVRNIK, Nataša STOJNIĆ, Majda MORI LUKANČIČ, Matic MIHEVC, Tina VIRTIČ POTOČNIK, Zalika KLEMENC-KETIŠ, Antonija POPLAS SUSIČ FACILITATORS AND BARRIERS TO SCALING-UP INTEGRATED CARE FOR ARTERIAL HYPERTENSION AND TYPE-2-DIABETES IN SLOVENIA: QUALITATIVE STUDY(38-45) Tadeja HOČEVAR, Tim ANSTISS, Danica ROTAR PAVLIČ CONTENT VALIDITY AND COGNITIVE TESTING IN THE DEVELOPMENT OF A MOTIVATIONAL INTERVIEWING SELF-ASSESSMENT QUESTIONNAIRE (46-54) Špela MIROŠEVIČ, Judith PRINS, Nikola BEŠIĆ , Simona BORŠTNAR , Andreja Cirila ŠKUFCA SMRDEL, Vesna HOMAR, Marko POPOVIĆ, Zalika KLEMENC-KETIŠ SYMPTOMS OF ANXIETY AND DEPRESSION AMONG SLOVENIAN BREAST CANCER SURVIVORS POST-TREATMENT DURING THE COVID-19 PANDEMIC: A CROSS-SECTIONAL STUDY (55-62) EDITORIAL ORIGINAL SCIENTIFIC ARTICLES CODEN ZDVAFY • UDK 613 / 614 + 628 • ISSN 0351 - 0026