NEW CHALLENGES IN SCIENTIFIC PUBLICATIONS: REFERENCING, ARTIFICIAL INTELLIGENCE AND CHATGPT Igor ŠVAB 1,4 , Zalika KLEMENC-KETIŠ 1-3* , Saša ZUPANIČ 4 1 Department of Family Medicine, Medical Faculty, University of Ljubljana, Poljanski nasip 58, 1000 Ljubljana, Slovenia 2 Department of Family Medicine, Medical Faculty, University of Maribor, Taborska 8, 2000 Maribor, Slovenia 3 Ljubljana Community Health Centre, Metelkova 9, 1000 Ljubljana, Slovenia 4 National Institute of Public Health, Trubarjeva 2, 1000 Ljubljana, Slovenia Received: Apr 17, 2023 Accepted: May 3, 2023 Invited editorial *Correspondence: zalika.klemenc@um.si 10.2478/sjph-2023-0015 Zdr Varst. 2023;62(3):109-112 109 NOVI IZZIVI V ZNANSTVENIH PUBLIKACIJAH: NAVAJANJE REFERENC, UMETNA INTELIGENCA IN KLEPETALNI ROBOT CHATGPT © National Institute of Public Health, Slovenia. Švab I, Klemenc-Ketiš Z, Zupanič S. New challenges in scientific publications: Referencing, artificial intelligence and ChatGPT. Zdr Varst. 2023;62(3):109-112. doi: 10.2478/Sjph-2023-0015. ABSTRACT Keywords: Scientific articles Referencing Artificial intelligence ChatGPT Peer review Research assessment IZVLEČEK Ključne besede: znanstveni članki navajanje literature umetna inteligenca ChatGPT recenziranje vrednotenje raziskav The COVID-19 pandemic has led to a surge in scientific publications, some of which have bypassed the usual peer-review processes, leading to an increase in unsupported claims being referenced. Therefore, the need for references in scientific articles is increasingly being questioned. The practice of relying solely on quantitative measures, such as impact factor, is also considered inadequate by many experts. This can lead to researchers choosing research ideas that are likely to generate favourable metrics instead of interesting and important topics. Evaluating the quality and scientific value of articles requires a rethinking of current approaches, with a move away from purely quantitative methods. Artificial intelligence (AI)-based tools are making scientific writing easier and less time-consuming, which is likely to further increase the number of scientific publications, potentially leading to higher quality articles. AI tools for searching, analysing, synthesizing, evaluating and writing scientific literature are increasingly being developed. These tools deeply analyse the content of articles, consider their scientific impact, and prioritize the retrieved literature based on this information, presenting it in simple visual graphs. They also help authors to quickly and easily analyse and synthesize knowledge from the literature, prepare summaries of key information, aid in organizing references, and improve manuscript language. The language model ChatGPT has already greatly changed the way people communicate with computers, bringing it closer to human communication. However, while AI tools are helpful, they must be used carefully and ethically. In summary, AI has already changed the way we write articles, and its use in scientific publishing will continue to enhance and streamline the process. Pandemija COVID-19 je povzročila porast znanstvenih publikacij, tudi takšnih, ki so zaobšle običajni recenzentski postopek, kar je povzročilo povečanje sklicevanj na znanstveno nepodprte trditve. Okrepila se je potreba po kritičnem vrednotenju literature. Pričakuje se, da bosta uvodni del in razprava znanstvenih člankov temeljila na sistematičnem pregledu obstoječe literature. Številni strokovnjaki menijo, da je praksa uporabe kvantitativnih meril kakovosti, kot je faktor vpliva, neustrezna. To lahko vodi v izbiro nezanimivih in nepomembnih raziskovalnih tem, ki bodo verjetno ustvarile ugodne meritve. Avtor predlaga ponovni razmislek o tem, kako ocenjevati kakovost člankov in jih znanstveno vrednotiti. Z orodji, ki temeljijo na umetni inteligenci (UI), postaja znanstveno pisanja lažje in manj zamudno. To bo vodilo v povečevanje števila člankov, ki bi ob uporabi teh orodij lahko bili bolj kakovostni. Vse bolj se razvijajo napredna orodja za iskanje, analizo, sintezo, vrednotenje in pisanje znanstvene literature, ki jih poganja UI. Ta orodja globinsko analizirajo vsebino publikacij in upoštevajo njihov znanstveni vpliv ter na podlagi teh informacij prednostno razvrščajo najdeno literaturo, ki jo predstavijo na preprost vizualen način. Avtorjem pomagajo tudi hitro in enostavno analizirati, sintetizirati in oceniti znanje iz literature, pripravijo predloge izvlečkov ključnih informacij, pomagajo organizirati reference in jezikovno izboljšati rokopis. Jezikovni modeli, kot so npr. ChatGPT, so močno spremenili način komunikacije z računalnikom in jo približali človeški komunikaciji. Čeprav so klepetalni roboti koristni, jih je treba uporabljati previdno in etično. UI je torej že spremenila način pisanja člankov, uporaba tovrstnih jezikovnih modelov v znanstvenem objavljanju pa bo še naprej izboljševala in poenostavljala to delo. 1 INTRODUCTION During the COVID-19 pandemic, the scientific community saw a surge in articles submitted to various journals for publication. Some of these journals have bypassed the usual peer-review process to meet the increased demand, resulting in the publication of papers that they would not normally accept (e.g., small, uncontrolled studies, etc.) (1). Other authors have then cited these studies, which has led to almost any claim being supported by a reference, no matter how absurd. At about the same time we have seen the emergence of artificial intelligence tools, especially the language model ChatGPT, which has already changed the way many people write articles (2). Additionally, artificial intelligence (AI) will without doubt enhance the writing of scientific papers, as it makes such writing easier and less time consuming. 2 REFERENCING References are a mandatory part of any scientific and professional article. They serve to prove that the authors’ claims are based on sound prior knowledge (3). However, with the rapid rise in published articles and other sources of information, both the purpose and need for references are being increasingly questioned (4). Science is making more information freely available, but the number of articles published far exceeds the increase in knowledge. New types of quasi-scientific publications have become a serious issue for scientific work (5). All this has led to a situation in which it is now possible to claim anything and provide a reference to support that claim. This problem was particularly evident in the toxic debates during the COVID-19 pandemic (6, 7). The quality of journals is almost exclusively measured by their impact factor. Although the fact that many experts consider this to be an inadequate approach (8), the practice of relying solely on quantitative measures of quality is still widespread. As a consequence, the selection of references by the authors can be done in such a way that it resembles gathering likes on Facebook more than a rigorous review of existing knowledge (9). Quantitative evaluation of the impact of an article and the scientific value of a researcher’s work carries the risk that researchers tend to choose a research idea that will most likely generate favourable metrics. Therefore, rather than choosing an interesting and important topic, the likelihood of being cited in other articles is the main driver for developing new knowledge and insights. This could have serious consequences for the development of humanity, as science will not be focused on improving the world and human life, but on maintaining the artificial evaluation of the impact of research. 10.2478/sjph-2023-0015 Zdr Varst. 2023;62(3):109-112 110 All this requires a rethinking of how to evaluate the quality and scientific value of articles. Most authors, peer reviewers, and editors act in good faith and do their jobs to the best of their abilities. Pre-publication peer review – coordinated by journal editors - tries to separate the wheat from the chaff. A lot of journals are now reducing the number of references and some journals have already limited the number to 30. Moreover, new trends in research assessment show a clear move away from a purely quantitative approach (10). Critical evaluation of cited articles, including the references, is becoming increasingly important. It is no longer enough for a reviewer or editor to limit the assessment to evaluate the research question, methodology, and discussion. Theoretically, the “Introduction and Discussion” part of the manuscript should be based on a systematic review of the existing literature, and therefore only such articles should be cited. 3 AI-POWERED RESEARCH TOOLS FOR SCIENTIFIC LITERATURE SEARCH, ANALYSIS, SYNTHESIS, EVALUATING AND WRITING AI has been around for some time, though many authors may not have been aware of it. Today, however, we are witnessing a sudden surge in AI tools for literature search, analysis, synthesis, and even writing. Just three years ago, we had to manually search PubMed using Boolean operators, prioritize searches, mark phrases, search for synonyms and word roots for truncation, and set search fields. Now, however, AI algorithms do this for us. The newest AI-based tools for finding relevant and authoritative articles are able to search for articles not only by analysing metadata; they use citation information and natural language processing and machine learning algorithms to analyse articles and prioritize them based on their relevance and scientific impact to a specific query. They also visually represent links between connected/ cited articles (and even cited statements; eg. scite) and topics (Semantic Scholar, Connected Papers, Research Rabbit, Litmaps), and generate accurate in-text citations and bibliographies through integrated reference manager tools. Some of these tools (scite) can also be used for research articles assessment (11). AI-based literature search, analysis and synthesis tools (Elicit, Scolarcy, ChatGPT, Bing AI) can help authors to synthesize article knowledge quickly, easily and clearly. Previously, authors would spend days sorting through articles to collect the most important findings. By automatically analysing and summarizing data, AI tools can now generate summaries of key article information, 10.2478/sjph-2023-0015 Zdr Varst. 2023;62(3):109-112 111 present them in easy-to-read tables, generate titles and abstracts (Writefull, Scolarcy, Abstract Generator), and even sections of a manuscript (ChatGPT). Major scientific publishers offer authors the possibility to search for an appropriate journal for manuscript submission (Springer Journal Suggester, Elsevier Journal Finder) by providing the title and abstract of the manuscript. Plagiarism-detection tools help editors and authors by verifying the degree of similarity between a focal manuscript and other publications. Authors can also refine their manuscripts linguistically before submitting them to the chosen journal, as AI-based writing tools can help authors with grammar, spelling, and formatting, suggesting changes in vocabulary and punctuation, as well as more accurate and appropriate words and phrases or paraphrase the text. In the past, authors had to rely on online dictionaries and paid proofreaders, while today computer tools (Grammarly, Writefull) can correct English while authors are writing, not just based on the semantics and syntax of the language but also on the statistics of word-sequence usage in a large collection of texts. With the release of ChatGPT in November 2022, the possibilities of artificial intelligence became even more evident. This large language model makes possible things we had not expected. It is now not necessary to search through document links to find the desired information, as instead authors are able to receive the information they need in their native language and converse about it with a chatbot in order to refine their answers. The model easily analyses and synthesizes our natural language. Often, the model opens our eyes to the breadth and depth of a problem, making it very useful for thinking about new research questions. Within five days, one million users had used ChatGPT, which is something that Facebook took ten months to achieve (12), and within two months some 100 million users had already tried it. There is, of course, a need for caution. A large language model can only be used as personal assistant, and its suggestions should always be verified since they are not always reliable. They can easily mislead the author and reader because the model is known to make factual errors, generate non-existing references and stubbornly and convincingly defend statements that may be false. It is also extremely important to use it ethically and for the benefit of humanity. ChatGPT can also be used for the writing of scientific articles (2). It can provide prompts or topics related to research and generate text based on its understanding of the subject matter. However, it cannot create new, original ideas. It is important to note that as an AI language model, it generates text based on patterns it has learned from its training data. While it can create coherent and grammatically correct sentences, it may not always generate accurate or precise scientific content. Hence, it is essential to review and edit the text carefully to ensure accuracy and clarity. Additionally, it is important to acknowledge any sources or references used in the article to avoid plagiarism. Many AI tools are in the process of development and training. They learn from users also and many of them are currently free. Unfortunately, at the moment there is no general AI tool for researchers or editors. They thus have to use many different AI tools at specific stages of their work and keep abreast of developments to ensure they are using the most appropriate ones. The best advice to authors is not to waste time debating whether to try out AI tools or not, but to take the first step towards the future and learn about their potential and limitations. Currently, optimism is high for what AI tools can offer this community, and the wider world. 4 CONCLUSIONS With AI tools that are available for scientific literature search, analysis, synthesis, writing and assessment, new opportunities are emerging for authors, reviewers and editors that will probably change scientific publishing as we know it. This requires also a rethinking of the research assessment methodology. Researchers and editors of scientific journals should be aware of the opportunities and limitations that modern technology is bringing to this context, and discussions are now vital in order to suggest improvements and develop an appropriate way of publishing and evaluating scientific findings in this new and evolving environment. ACKNOWLEDGEMENTS ChatGPT was used in the development of some parts of this editorial. However, on considering the requirements of a co-author, it came up short in several areas. Although it made a significant contribution to the work, ChatGPT was unable to provide agreement on submission and did not review and approve of the editorial before submission. Perhaps more crucially, it is not able to take responsibility and be accountable for the content of this editorial. CONFLICTS OF INTEREST The authors declare that no conflicts of interest exist. FUNDING The editorial did not receive any funding. 10.2478/sjph-2023-0015 Zdr Varst. 2023;62(3):109-112 112 ETHICAL APPROVAL Not applicable as the article is not based on any human data. AVAILABILITY OF DATA AND MATERIALS Not applicable. REFERENCES 1. Clark J. How Covid-19 bolstered an already perverse publishing system. BMJ. 2023;380:689. doi: 10.1136/bmj.p689. 2. Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Critical Care (London, England). 2023;27(1):75. doi: 10.1186/s13054-023-04380-2. 3. International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals [Internet]. 2023 [cited 2023 Feb 9]. Available from: https://www.icmje.org/recommendations/ 4. Šupak-Smolčić V, Mlinarić A, Antončić D, Horvat M, Omazić J, Šimundić AM. ICMJE authorship criteria are not met in a substantial proportion of manuscripts submitted to Biochemia Medica. Biochem Med. 2015;25(3):324-334. doi: 10.11613/bm.2015.033. 5. Švab I, Makivić I. Predators and hijackers in academic publishing. Eur J Gen Pract. 2015;21(2):95-96. doi: 10.3109/13814788.2015.1037270. 6. Epstein HB. Dispelling Covid-19 misinformation. Med Ref Serv Q. 2022;41(1):80-85. doi: 10.1080/02763869.2022.2021036. 7. Etta G, Galeazzi A, Hutchings JR, James Smith CS, Conti M, Quattrociocchi W, et al. COVID-19 infodemic on Facebook and containment measures in Italy, United Kingdom and New Zealand. PLoS One. 2022;17(5):e0267022. doi: 10.1371/journal.pone.0267022. 8. Simons K. The misused impact factor. Science. 2008;322(5899):165. doi: 10.1126/science.1165316. 9. King MM. Self-publishing is common among academic-journal editors. Nature. 2023;613(7944):445-446. doi: 10.1038/d41586-023-00028-x. 10. Declaration on Research Assessment (DORA) [Internet]. 2012 [cited 2023 May 2]. Available from: https://sfdora.org/ 11. Nicholson JM, Mordaunt M, Lopez P, Uppala A, Rosati D, Rodrigues NP, et al. scite: A smart citation index that displays the context of citations and classifies their intent using deep learning. Quant Sci Studies. 2021;2(3): 882-898. doi: 10.1162/qss_a_00146. 12. ChatGPT sprints to one million users [Internet]. 2023 [cited 2023 Apr 16]. Available from: https://www.digitalinformationworld. com/2023/01/chat-gpt-achieved-one-million-users-in.html