› INFORMACIJSKA DRUZBA Zbornik 26. mednarodne multikonference Zvezek B INFORMATION SOCIETY Proceedings of the 26th International Multiconference Volume B Kognitivna znanost Cognitive Science Uredniki • Editors: Anka Slana Ozimic, Borut Trpin, Toma Strle, Olga Markic IS2023 Zbornik 26. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2023 Zvezek B Proceedings of the 26th International Multiconference INFORMATION SOCIETY – IS 2023 Volume B Kognitivna znanost Cognitive Science Uredniki / Editors Anka Slana Ozimič, Borut Trpin, Toma Strle, Olga Markič http://is.ijs.si 12. oktober 2023 / 12 October 2023 Ljubljana, Slovenia Uredniki: Anka Slana Ozimič Filozofska fakulteta, Univerza v Ljubljani Borut Trpin Filozofska fakulteta, Univerza v Ljubljani Toma Strle Center za Kognitivno znanost Pedagoška fakulteta, Univerza v Ljubljani Olga Markič Filozofska fakulteta, Univerza v Ljubljani Založnik: Institut »Jožef Stefan«, Ljubljana Priprava zbornika: Mitja Lasič, Vesna Lasič, Mateja Mavrič Oblikovanje naslovnice: Vesna Lasič Dostop do e-publikacije: http://library.ijs.si/Stacks/Proceedings/InformationSociety Ljubljana, oktober 2023 Informacijska družba ISSN 2630-371X Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID 170773251 ISBN 978-961-264-277-8 (PDF) PREDGOVOR MULTIKONFERENCI INFORMACIJSKA DRUŽBA 2023 Šestindvajseta multikonferenca Informacijska družba se odvija v obdobju izjemnega razvoja za umetno inteligenco, računalništvo in informatiko, za celotno informacijsko družbo. Generativna umetna inteligenca je s programi kot ChatGPT dosegla izjemen napredek na poti k superinteligenci, k singularnosti in razcvetu človeške civilizacije. Uresničujejo se napovedi strokovnjakov, da bodo omenjena področna ključna za obstoj in razvoj človeštva, zato moramo pozornost usmeriti na njih, jih hitro uvesti v osnovno in srednje šolstvo in vsakdan posameznika in skupnosti. Po drugi strani se poleg lažnih novic pojavljajo tudi lažne enciklopedije, lažne znanosti ter »ploščate Zemlje«, nadaljuje se zapostavljanje znanstvenih spoznanj, metod, zmanjševanje človekovih pravic in družbenih vrednot. Na vseh nas je, da izzive današnjice primerno obravnavamo, predvsem pa pomagamo pri uvajanju znanstvenih spoznanj in razčiščevanju zmot. Ena pogosto omenjanih v zadnjem letu je eksistencialna nevarnost umetne inteligence, ki naj bi ogrožala človeštvo tako kot jedrske vojne. Hkrati pa nihče ne poda vsaj za silo smiselnega scenarija, kako naj bi se to zgodilo – recimo, kako naj bi 100x pametnejši GPT ogrozil ljudi. Letošnja konferenca poleg čisto tehnoloških izpostavlja pomembne integralne teme, kot so okolje, zdravstvo, politika depopulacije, ter rešitve, ki jih za skoraj vse probleme prinaša umetna inteligenca. V takšnem okolju je ključnega pomena poglobljena analiza in diskurz, ki lahko oblikujeta najboljše pristope k upravljanju in izkoriščanju tehnologij. Imamo veliko srečo, da gostimo vrsto izjemnih mislecev, znanstvenikov in strokovnjakov, ki skupaj v delovnem in akademsko odprtem okolju prinašajo bogastvo znanja in dialoga. Verjamemo, da je njihova prisotnost in udeležba ključna za oblikovanje bolj inkluzivne, varne in trajnostne informacijske družbe. Za razcvet. Letos smo v multikonferenco povezali deset odličnih neodvisnih konferenc, med njimi »Legende računalništva«, s katero postavljamo nov mehanizem promocije informacijske družbe. IS 2023 zajema okoli 160 predstavitev, povzetkov in referatov v okviru samostojnih konferenc in delavnic, skupaj pa se je konference udeležilo okrog 500 udeležencev. Prireditev so spremljale okrogle mize in razprave ter posebni dogodki, kot je svečana podelitev nagrad. Izbrani prispevki bodo izšli tudi v posebni številki revije Informatica (http://www.informatica.si/), ki se ponaša s 46-letno tradicijo odlične znanstvene revije. Multikonferenco Informacijska družba 2023 sestavljajo naslednje samostojne konference: • Odkrivanje znanja in podatkovna središča • Demografske in družinske analize • Legende računalništva in informatike • Konferenca o zdravi dolgoživosti • Miti in resnice o varovanju okolja • Mednarodna konferenca o prenosu tehnologij • Digitalna vključenost v informacijski družbi – DIGIN 2023 • Slovenska konferenca o umetni inteligenci + DATASCIENCE • Kognitivna znanost • Vzgoja in izobraževanje v informacijski družbi • Zaključna svečana prireditev konference Soorganizatorji in podporniki konference so različne raziskovalne institucije in združenja, med njimi ACM Slovenija, SLAIS za umetno inteligenco, DKZ za kognitivno znanost in Inženirska akademija Slovenije (IAS). V imenu organizatorjev konference se zahvaljujemo združenjem in institucijam, še posebej pa udeležencem za njihove dragocene prispevke in priložnost, da z nami delijo svoje izkušnje o informacijski družbi. Zahvaljujemo se tudi recenzentom za njihovo pomoč pri recenziranju. S podelitvijo nagrad, še posebej z nagrado Michie-Turing, se avtonomna stroka s področja opredeli do najbolj izstopajočih dosežkov. Nagrado Michie-Turing za izjemen življenjski prispevek k razvoju in promociji informacijske družbe je prejel prof. dr. Andrej Brodnik. Priznanje za dosežek leta pripada Benjaminu Bajdu za zlato medaljo na računalniški olimpijadi. »Informacijsko limono« za najmanj primerno informacijsko tematiko je prejela nekompatibilnost zdravstvenih sistemov v Sloveniji, »informacijsko jagodo« kot najboljšo potezo pa dobi ekipa RTV za portal dostopno.si. Čestitke nagrajencem! Mojca Ciglarič, predsednica programskega odbora Matjaž Gams, predsednik organizacijskega odbora i FOREWORD - INFORMATION SOCIETY 2023 The twenty-sixth Information Society multi-conference is taking place during a period of exceptional development for artificial intelligence, computing, and informatics, encompassing the entire information society. Generative artificial intelligence has made significant progress towards superintelligence, towards singularity, and the flourishing of human civilization with programs like ChatGPT. Experts' predictions are coming true, asserting that the mentioned fields are crucial for humanity's existence and development. Hence, we must direct our attention to them, swiftly integrating them into primary, secondary education, and the daily lives of individuals and communities. On the other hand, alongside fake news, we witness the emergence of false encyclopaedias, pseudo-sciences, and flat Earth theories, along with the continuing neglect of scientific insights and methods, the diminishing of human rights, and societal values. It is upon all of us to appropriately address today's challenges, mainly assisting in the introduction of scientific knowledge and clearing up misconceptions. A frequently mentioned concern over the past year is the existential threat posed by artificial intelligence, supposedly endangering humanity as nuclear wars do. Yet, nobody provides a reasonably coherent scenario of how this might happen, say, how a 100x smarter GPT could endanger people. This year's conference, besides purely technological aspects, highlights important integral themes like the environment, healthcare, depopulation policies, and solutions brought by artificial intelligence to almost all problems. In such an environment, in-depth analysis and discourse are crucial, shaping the best approaches to managing and exploiting technologies. We are fortunate to host a series of exceptional thinkers, scientists, and experts who bring a wealth of knowledge and dialogue in a collaborative and academically open environment. We believe their presence and participation are key to shaping a more inclusive, safe, and sustainable information society. For flourishing. This year, we connected ten excellent independent conferences into the multi-conference, including "Legends of Computing", which introduces a new mechanism for promoting the information society. IS 2023 encompasses around 160 presentations, abstracts, and papers within standalone conferences and workshops. In total about 500 participants attended the conference. The event was accompanied by panel discussions, debates, and special events like the award ceremony. Selected contributions will also be published in a special issue of the journal Informatica (http://www.informatica.si/), boasting a 46-year tradition of being an excellent scientific journal. The Information Society 2023 multi-conference consists of the following independent conferences: • Data Mining and Data Warehouse - SIKDD • Demographic and Family Analysis • Legends of Computing and Informatics • Healthy Longevity Conference • Myths and Truths about Environmental Protection • International Conference on Technology Transfer • Digital Inclusion in the Information Society - DIGIN 2023 • Slovenian Conference on Artificial Intelligence + DATASCIENCE • Cognitive Science • Education and Training in the Information Society • Closing Conference Ceremony Co-organizers and supporters of the conference include various research institutions and associations, among them ACM Slovenia, SLAIS for Artificial Intelligence, DKZ for Cognitive Science, and the Engineering Academy of Slovenia (IAS). On behalf of the conference organizers, we thank the associations and institutions, and especially the participants for their valuable contributions and the opportunity to share their experiences about the information society with us. We also thank the reviewers for their assistance in reviewing. With the awarding of prizes, especially the Michie-Turing Award, the autonomous profession from the field identifies the most outstanding achievements. Prof. Dr. Andrej Brodnik received the Michie-Turing Award for his exceptional lifetime contribution to the development and promotion of the information society. The Achievement of the Year award goes to Benjamin Bajd, gold medal winner at the Computer Olympiad. The "Information Lemon" for the least appropriate information move was awarded to the incompatibility of information systems in the Slovenian healthcare, while the "Information Strawberry" for the best move goes to the RTV SLO team for portal dostopno.si. Congratulations to the winners! Mojca Ciglarič, Chair of the Program Committee Matjaž Gams, Chair of the Organizing Committee ii KONFERENČNI ODBORI CONFERENCE COMMITTEES International Programme Committee Organizing Committee Vladimir Bajic, South Africa Matjaž Gams, chair Heiner Benking, Germany Mitja Luštrek Se Woo Cheon, South Korea Lana Zemljak Howie Firth, UK Vesna Koricki Olga Fomichova, Russia Mitja Lasič Vladimir Fomichov, Russia Blaž Mahnič Vesna Hljuz Dobric, Croatia Mateja Mavrič Alfred Inselberg, Israel Jay Liebowitz, USA Huan Liu, Singapore Henz Martin, Germany Marcin Paprzycki, USA Claude Sammut, Australia Jiri Wiedermann, Czech Republic Xindong Wu, USA Yiming Ye, USA Ning Zhong, USA Wray Buntine, Australia Bezalel Gavish, USA Gal A. Kaminka, Israel Mike Bain, Australia Michela Milano, Italy Derong Liu, Chicago, USA Toby Walsh, Australia Sergio Campos-Cordobes, Spain Shabnam Farahmand, Finland Sergio Crovella, Italy Programme Committee Mojca Ciglarič, chair Marjan Heričko Baldomir Zajc Bojan Orel Borka Jerman Blažič Džonova Blaž Zupan Franc Solina Gorazd Kandus Boris Žemva Viljan Mahnič Urban Kordeš Leon Žlajpah Cene Bavec Marjan Krisper Niko Zimic Tomaž Kalin Andrej Kuščer Rok Piltaver Jozsef Györkös Jadran Lenarčič Toma Strle Tadej Bajd Borut Likar Tine Kolenik Jaroslav Berce Janez Malačič Franci Pivec Mojca Bernik Olga Markič Uroš Rajkovič Marko Bohanec Dunja Mladenič Borut Batagelj Ivan Bratko Franc Novak Tomaž Ogrin Andrej Brodnik Vladislav Rajkovič Aleš Ude Dušan Caf Grega Repovš Bojan Blažica Saša Divjak Ivan Rozman Matjaž Kljun Tomaž Erjavec Niko Schlamberger Robert Blatnik Bogdan Filipič Stanko Strmčnik Erik Dovgan Andrej Gams Jurij Šilc Špela Stres Matjaž Gams Jurij Tasič Anton Gradišek Mitja Luštrek Denis Trček Marko Grobelnik Andrej Ule Nikola Guid Boštjan Vilfan iii iv KAZALO / TABLE OF CONTENTS Kognitivna znanost / Cognitive Science ....................................................................................................... 1 PREDGOVOR / FOREWORD ............................................................................................................................... 3 PROGRAMSKI ODBORI / PROGRAMME COMMITTEES ............................................................................... 5 What insights can psychedelic research bring to Cognitive Science? A systematic review of the phenomenology of DMT experiences / Czizek Carolina ............................................................................................................. 7 Pristranskost v strojnem učenju: dileme in odgovori / Farič Ana, Bratko Ivan.................................................... 10 ChatGPT through Tononi's Definition of Consciousness / Gams Matjaž ............................................................ 14 Manipulacija v umetni inteligenci / Gril Gaja ...................................................................................................... 18 Users’ Cognitive Processes in a User Interface for Predicting Football Match Results / Kolar Žiga, Papa Gregor .......................................................................................................................................................................... 21 Družbena regulacija umetne inteligence. Nekatera odprta vprašanja in izzivi / Mali Franc ................................ 25 Umetna inteligenca: orodje ali zavesten stroj / Markič Olga ............................................................................... 28 Social Volition as Artificial Intelligence: Science and Ideology as Landian Intelligences / Marušič Jar Žiga, Sergaš Uroš ...................................................................................................................................................... 32 Let’s Jam: An Exploratory Case Study on Collective Music Improvisation and the Process of Attunement / Novak Christophe ............................................................................................................................................. 37 Exploring the link between the absence of an EEG spectral peak and cognitive status / Ogrin Ajda, Pavlovčič Tisa, Agatić Filip, Demšar Anita, Jug Jan, Aljaž Barbara, Dreo Jurij ............................................................. 40 Orthogonalist and anti-orthogonalist perspective on AI alignment problem / Sergaš Uroš, Marušič Jar Žiga .... 44 Changes in Everyday Experience Followed by Mystical-type Psychedelic Experiences / Sirk Maruša .............. 50 Integrated Information Theory of Consciousness 3.0: Exploring Information and Causation on the Level of Individual Mechanisms / Žerdoner Marinšek Tina ......................................................................................... 54 Indeks avtorjev / Author index ................................................................................................................... 61 v vi Zbornik 26. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2023 Zvezek B Proceedings of the 26th International Multiconference INFORMATION SOCIETY – IS 2023 Volume B Kognitivna znanost Cognitive Science Uredniki / Editors Anka Slana Ozimič, Borut Trpin, Toma Strle, Olga Markič http://is.ijs.si 12. oktober 2023 / 12 October 2023 Ljubljana, Slovenia 1 2 PREDGOVOR Dobrodošli na konferenci Kognitivna znanost. Na letošnji konferenci bodo avtorice in avtorji raziskovali mnoge plati človeške kognicije in predstavili tako svoje empirične ugotovitve kot tudi teoretska raziskovanja. Skupaj bomo potovali skozi različna področja kognitivne znanosti - od psihologije in nevroznanosti do filozofije in umetne inteligence, ter ob tem spoznavali raznolike tematike vključujoč psihedelike, mistične izkušnje, biomarkerje kognitivnih sposobnosti, in celo vprašanja zavesti. Poseben poudarek letošnjega srečanja je namenjen eni izmed trenutno najbolj vročih tem: umetni inteligenci. Osrednja tema konference, "UI klepetalniki in širše", bo predstavila izzive in rešitve, ki jih prinašata razvoj in uporaba klepetalnih robotov z umetno inteligenco. Hkrati pa bomo razmišljali, kako umetna inteligenca oblikuje svet onkraj klepetalnih robotov. V tej luči bomo na konferenci gostili okroglo mizo o vlogi umetne inteligence v izobraževanju, s čimer se bomo dotaknili še enega izmed aktualnih izzivov. Skupaj bomo razmišljali o prednostih in pasteh njenega vključevanja v izobraževalne procese, ki oblikujejo našo prihodnost. Upamo, da bo letošnja konferenca prostor za povezovanje in izmenjavo prodornih idej. Skupaj bomo premagovali disciplinarne in metodološke ovire, združili mlade in izkušene znanstvenike ter znanstvenice, ki si delijo strast do raziskovanja skrivnosti kognicije. Dobrodošli! Anka Slana Ozimič Borut Trpin Toma Strle Olga Markič 3 FOREWORD Welcome to the Cognitive Science Conference. At this year's conference, authors will explore the many facets of human cognition and present both their empirical findings and theoretical research. Together, we will travel through the diverse fields of cognitive science - from psychology and neuroscience to philosophy and artificial intelligence, learning about a variety of topics, including psychedelics, mystical experiences, biomarkers of cognitive abilities, and even questions of consciousness. This year's conference has a special focus on one of the hottest topics at the moment: artificial intelligence. The main topic of the conference, "AI Chatbots and Beyond,” will present the challenges and solutions brought about by the development and use of AI chatbots. At the same time, we will consider how AI is shaping the world beyond chatbots. In this light, we will host a panel discussion on the role of AI in education, addressing another of the current challenges. Together, we will reflect on the benefits and pitfalls of integrating it into the educational processes that are shaping our future. We hope that this year's conference will be a space for networking and sharing insightful ideas. Together we will overcome disciplinary and methodological barriers, bringing together young and experienced scientists who share a passion for exploring the mysteries of cognition. Welcome! Anka Slana Ozimič Borut Trpin Toma Strle Olga Markič 4 PROGRAMSKI ODBOR / PROGRAMME COMMITTEE Anka Slana Ozimič, Filozofska fakulteta, Univerza v Ljubljani Borut Trpin, Filozofska fakulteta, Univerza v Ljubljani Toma Strle, Center za Kognitivno znanost, Pedagoška fakulteta, Univerza v Ljubljani Olga Markič, Filozofska fakulteta, Univerza v Ljubljani 5 6 What insights can psychedelic research bring to Cognitive Science? A systematic review of the phenomenology of DMT experiences. Carolina Czizek† Cognitive Science University of Vienna & University of Ljubljana Vienna – Austria & Ljubljana - Slovenia carolina.czizek@chello.at ABSTRACT depression or anxiety, are increasing. This is done by administrating DMT either inhaled, which makes the effects This abstract explores the systematic review of the shorter lived (about 15 minutes) or taken orally as ‚ayahuasca' to phenomenology of N, N-dimethyltryptamine (DMT) experiences. Additionally, the relevance of conducting prolong the subjective effects of the compound (up to ten hours). psychedelic research for Cognitve Science is discussed. A Laboratory studies of DMT use are limited by their clinical special emphasis is being put on (neuro-)phenomenological setting (not taking into account the importance of set & setting research methods, as they seem to be the best suitable for of psychedelic experiences) and most are lacking a qualitative conducting research around psychedelic substances and their analysis of phenomenological content. This indicates the direct effects on phenomenology. growing importance of a thorough investigation of the KEYWORDS phenomenological aspects of the substances. Psychedelic research, Cognitive Science, systematic review, Latest research indicates similarities of phenomenological phenomenology, neurophenomenology, DMT, non-ordinary experiences of DMT use across subjects [5]. It remains unclear states of consciousness how much of these similarities are due to cultural or individual priming and/or influencing, as stories about “DMT entities” or 1 Introduction the “DMT parallel world” can be found all over the internet. N, N-dimethyltryptamine (DMT) is an endogenous serotonergic psychedelic compound which is capable of producing radical 2 Systematic Review shifts in conscious experience. Compared to other serotonergic The systematic review synthesizes the phenomenology of N,N- psychedelic substances like psilocybin, mescaline or LSD; DMT dimethyltryptamine experiences according to the PRISMA experiences seem to produce the most radical shifts in conscious method [8]. Excluded were studies prior to 2013 and studies that experience, with subjects reporting hyper-real, otherworldly, not taking into account the phenomenological effects of the DMT often ontologically challenging but also potential transformative experience. After initial screening, 17 studies were included in experiences, including encounters with entities as well as this report. The included studies used different methodologies: experiencing visualizations of geometric fractals, shapes or neurophenomenological approaches through EEG measurements followed by questionnaires or interviews; purely patterns. The rapid onset and short duration of inhaled DMT phenomenological accounts through online surveys; micro- experiences, the drastic change in phenomenological conscious phenomenological inspired interviews or qualitative linguistic experience it produces, as well as the fact that some studies analyses. Also assessed were studies that compared the indicate the compound to be endogenous to mammals [3] as well phenomenology of DMT experiences to other experiences such as to a variety of plants, suggests the importance of conducting as near-death experiences (NDE’s), ego-dissolution experiences more research around this compound. or God-encounter experiences. While DMT use has been part of several ancient Results indicate that the DMT experience could be clustered into different categories. When it comes to the phenomenological Amazonian traditions (combined with monoamide-oxidase perspective, the most prominent themes identified are: perceptual inhibitors it is called ‘ayahuasca‘) for hundreds of years and changes; somatic experiences; emotional responses and a ‘sense trends in usage in the western world as well as clinical trials of of otherness’ and a sense of ego dissolution. Additionally, to administrating DMT to patients with treatment resistant these themes, the most prominent categories regarding the content of the DMT experience are: Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed 1. The ontology of the DMT world, also called ‚hyperspace‘ for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). 2. Dissolution of the ego/self Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). 7 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia BSc, BSc, Carolina Czizek 3. Encounters with seemingly autonomous or conscious classical psychedelics has already yielded insights into the role entities. These encounters are predominantly positive, users of serotonin on our wellbeing and perception of reality in the often reporting of receiving a type of message. past, the discovery of LSD being one of the core drivers to the investigation of this molecule in the 1960s [7]. Limitations of the studies were quite homogenous, including among others: the use of self-reports; sociability To study non-ordinary states of consciousness, such as biases; retrospective accounts; different or insufficient hypnosis, meditation and psychedelics, common scientific information on dosages and purity. The systematic review practice (mostly average rating over the entire course of the concludes that the phenomenology of DMT experiences seems experience) does not yield sufficient insights into the individual to be distinct to other psychedelic experiences such as psilocybin phenomenological processes that include such an experience. but similar to certain other non- drug experiences, such as near- First-person reports can account for within- & between-subject death experiences, thus representing a research field well suited variabilities. A strong call for neurophenomenological research for the account of the phenomenology of non-ordinary states of in this field is being evoked, emphasizing the importance of consciousness. conducting micro-phenomenological interviews [9], additionally to third-person research (brain & somatic measurements). 3 Relevance to Cognitive Science 3.2. DMT distinct from other psychedelic substances „Psychedelics are for the mind/psychiatry what the telescope is for astronomy or the microscope is for biology “- [4] DMT seems to be distinct from other substances on a neurophenomenological level. Firstly, the endogenous Psychedelic research, ever since legislation loosened up in the production of DMT is still a mystery to research. Under the late 2000s, is experiencing a come-back after 40 years of influence of DMT, delta power, which is usually associated with prohibition. Since mental health issues are becoming more states of unconsciousness or lack of experience (such as prevalent and common approaches with medication such as dreamless sleep or anesthesia), increases [10]. This can hint at SSRIs are not yielding sufficient relief, psychedelic assisted delta waves being a marker from conscious disconnection, while therapy approaches seem to be a promising approach. More the person is still having a phenomenological experience. Since clinical research is necessary to drive this process further. Other this seems to be unique to DMT, along with the short duration of studies have shown psilocybin to promote neurogenesis, which the experience, DMT research could help generate insights into could be used to treat non-psychiatric but also biological issues. how our brains construct the world, how our sense of self is constructed and how all of these processed can be deconstructed in minutes or even seconds due to a single (endogenous) Classical psychedelics act on the serotonin 5HT2A chemical. Neurophenomenological approaches with non- receptor, and are molecules that can drastically change the ordinary states of consciousness are crucial in the quest to finding phenomenology of experience. They modulate fundamental answers to some of the most mysterious questions of cognitive aspects of experience by what it seems to be deconstructing prior science. beliefs and reconstructing new beliefs. Some theories about the mind and consciousness have emerged through psychedelic research as well, such as the ‘entropic brain hypothesis’ [1], AKNOWLEDGEMENTS bringing physical concepts of entropy and criticality into the I want to thank the University of Ljubljana and their faculty of discussion of non-ordinary states of consciousness and their Cognitive Science for teaching us the topics of phenomenology neurophenomenological characteristics. The REBUS (‘relaxed and neurophenomenology and teaching us how to implement beliefs under psychedelics’) model, is a model that combines the these approaches. The experience of studying one semester at the entropic brain hypothesis with the free energy principle, trying to University of Ljubljana has shaped me as a researcher and has gain further understanding of the effects of free energy on given me motivation to follow this career path. I want to phenomenology [2]. What insights can we generate about the especially thank Dr. Toma Strle for encouraging me to follow my nature of reality when approaching it with fewer predictions? interests, no matter how controversial these are. 3.1. The sense of self REFERENCES One of the core questions of (philosophical) Cognitive Science [1] Carhart-Harris, R. L., Leech, R., Hellyer, P. J., Shanahan, M., Feilding, A., Tagliazucchi, E., Chialvo, D. R., & Nutt, D. (2014). The entropic brain: is understanding the sense of self. Since psychedelics seem to A theory of conscious states informed by neuroimaging research with usually lead to a deconstruction of the sense of self (f.e. explained psychedelic drugs. Frontiers in Human Neuroscience, 8, 20. https:// due to a diminished activity in the default mode network) [6], the doi.org/10.3389/fnhum.2014.00020 [2] Carhart-Harris, R. L., & Friston, K. J. (2019). REBUS and the Anarchic different processes of selfhood (minimal self vs. narrative self) Brain: Toward a Unified Model of the Brain Action of Psychedelics. [10] could be examined further as well as how this construction Pharmacological reviews, 71(3), 316–344. and deconstruction of the self emerges on a https://doi.org/10.1124/pr.118.017160 neurophenomenological level, what psychological implications [3] Dean, J. G., Liu, T., Huff, S., Sheler, B., Barker, S. A., Strassman, R. J., this has on individuals yielding up to philosophical discussions Wang, M. M., & Borjigin, J. (2019). Biosynthesis and Extracellular of the ontological role of the sense of self. Another classical Concentrations of N,N- dimethyltryptamine (DMT) in Mammalian Brain. argument for studying psychedelics is backward propagation Scientific reports, 9(1), 9333. https:// doi.org/10.1038/s41598-019-45812-w (finding out about aberrant functions can help generate insights [4] Grof, S (2008). LSD psychotherapy: The healing potential of psychedelic about ‘normal’ function), which has already helped shape the medicine. Ben Lomond, CA: Multidisciplinary Association for field of neuroscience in the past. The serotonergic structure of psychedelic Studies (MAPS) 8 What insights can psychedelic research bring to Cognitive Science? A systematic review of the phenomenology of DMT experiences. Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia [5] Lawrence, D.W., Carhart-Harris, R., Griffiths, R. et al. (2022) systematic reviews. PLOS Medicine 2021;18(3):e1003583. doi: Phenomenology and content of the inhaled N, N-dimethyltryptamine ( N, 10.1371/journal.pmed.1003583 N-DMT) experience. Sci Rep 12, 8562. https://doi.org/ 10.1038/s41598- [9] Petitmengin, C. Describing one’s subjective experience in the second 022-11999-8 person: An interview method for the science of consciousness. Phenom [6] Millière, R. (2017). Looking for the Self: Phenomenology, Cogn Sci 5, 229–269 (2006). https://doi.org/10.1007/s11097-006-9022-2 Neurophysiology and Philosophical Significance of Drug-induced Ego [10] Timmermann, Bauer, P. R., Gosseries, O., Vanhaudenhuyse, A., Dissolution. Frontiers in Human Neuroscience, 11, 245. https:// Vollenweider, F., Laureys, S., Singer, T., Mind and Life Europe (MLE) doi.org/10.3389/fnhum.2017.00245 ENCECON Research Group, Antonova, E., & Lutz, A. (2023). A [7] Nichols D. (2013). Serotonin, and the Past and Future of LSD. MAPS neurophenomenological approach to non-ordinary states of Bulletin Special Edition. consciousness: hypnosis, meditation, and psychedelics. Trends in [8] Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow cognitive sciences, 27(2), 139–159. CD, et al. The PRISMA 2020 statement: an updated guideline for reporting https://doi.org/10.1016/j.tics.2022.11.006 9 Pristranskost v strojnem učenju: dileme in odgovori Bias in Machine Learning: Dilemmas and Answers Ana Farič† Ivan Bratko Pedagoška fakulteta Fakulteta za računalništvo in informatiko Univerza v Ljubljani Univerza v Ljubljani Slovenija Slovenija af27987@student.uni-lj.si bratko@fri.uni-lj.si POVZETEK AI, there is no generally agreed technical definition of bias which would enable operational use in preventing bias. Our conclusions Nekatere aplikacije umetne inteligence, posebej strojnega učenja, are that (1) improved general education is needed to enable better so deležne izrazito odklonilnih odzivov tako v splošnih medijih, understanding of AI methods in everyday applications, and (2) kot v strokovni literaturi. Pogosto so omenjane aplikacije v technical methods must be developed for implementing domenah sodstva, zaposlovanja in bančništva. Kritiki očitajo, da generally accepted societal values such as equality and fairness so uporabljeni sistemi pristranski glede na t.i. zaščitene atribute, in AI systems. kot so rasa, spol in starost. Znan primer je sistem COMPAS, ki se kljub polemikam še vedno uporablja v ameriškem sodstvu. KEYWORDS Namen prispevka je na primeru COMPASA predstaviti trende machine learning, artificial intelligence, bias, fairness, diskusije o pristranskosti algoritmov strojnega učenja. Opažamo, discrimination da je problem pogosto v tem, da niti v strokovni literaturi s področja umetne inteligence ni soglasja glede tehničnih definicij pristranskosti, ki bi jih bilo mogoče operativno uporabljati za preprečevanje (videza) pristranskosti. Naši zaključki so, da je (1) 1 UVOD potrebno s kvalitetno izobrazbo doseči boljše splošno razumevanje metod umetne inteligence v praksi in (2) da je Z razmahom uporabe strojnega učenja so se v zadnjih 5 do 10 potrebno razviti tehnične principe, s katerimi bi v sistemih letih pojavili primeri aplikacij, ki so bile deležne izrazito umetne inteligence operacionalizirali splošno sprejete družbene odklonilnih odzivov predvsem s strani splošnih medijev, pa tudi vrednote, kot sta enakost in pravičnost. znotraj strokovne literature. Pogosto so omenjani sistemi, uporabljeni v domenah sodstva, zaposlovanja in bančništva. KLJUČNE BESEDE Kritiki opozarjajo, da so “algoritmi in sistemi strojnega učenja nepravični in pristranski” glede na t.i. zaščitene atribute, kot so umetna inteligenca, strojno učenje, pristranskost, diskriminacija, rasa, spol in starost posameznika in da so priporočila umetne pravičnost inteligence odvisna od teh atributov, namesto od objektivnega ABSTRACT ocenjevanja dejstev. Naslovi nekaterih odmevnih člankov so: There’s software used across the country to predict future Some recent applications of Artificial Intelligence, particularly criminals. And it’s biased against blacks [2], New Zealand machine learning, have been strongly criticised in general media passport robot tells applicant of Asian descent to open eyes [18], and professional literature. Applications in domains of justice, A beauty contest was judged by AI and the robots didn’t like dark employment and banking are often mentioned in this respect. The skin [13], Amazon scraps secret AI recruiting tool that showed main critic is that these applications are biased with respect to so bias against women [6]. Taki primeri prispevajo k stopnjevanju called protected attributed, such as race, gender and age. The skrbi o vplivih, ki ga ima umetna inteligenca (v nadaljevanju UI) most notorious example is the system COMPAS which is still in na naša življenja [15]. Strokovnjaki z različnih področij se use in American justice system despite severe criticism. The aim lotevajo t.i. problema pristranskosti strojnega učenja. Skušajo of our paper is to analyse the trends of discussion about bias in definirati, kaj pristranskost pomeni, iz kje naj bi izhajala, machine learning algorithms using the COMPAS as an example. predvsem pa, kaj naj bi glede tega storili. The main problem of such discussions is that even in the field of Na razvijajočem se področju etike v UI (npr. UNESCO 2021 ∗ [19]) se tema pristranskosti strojnega učenja pojavlja na vidnem Article Title Footnote needs to be captured as Title Note †Author Footnote to be captured as Author Note mestu. Pogosto jo omenjajo politiki v zvezi s principi regulacije, ki naj bi zagotovila etično uporabo umetne inteligence (npr. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed European AI Act, 2023 [3]). Vendar v teh diskusijah pogosto ni for profit or commercial advantage and that copies bear this notice and the full jasno, kaj točno pristranskost strojnega učenja in UI pomeni. citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Zato regulacijski ukrepi v tej smeri niso jasno opredeljeni, razen Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia v zelo abstraktni obliki. Beseda pristranskost v zvezi s strojnim © 2023 Copyright held by the owner/author(s). 10 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Farič et al. učenjem avtorjem pomene različne stvari. Celo v strokovni napovedi sistema se standardno imenujeta: (1) FPR (false literaturi s področja UI ni popolnega soglasja in nedvomno positive rate), to je delež negativnega razreda, ki je bil napačno sprejetih tehničnih definicij pristranskosti, ki bi jih bilo mogoče napovedan kot pozitiven in (2) FNR (false negative rate), to je operativno uporabljati v preprečevanju pristranskosti [11]. Za delež pozitivnega razreda, ki je bil napačno napovedan kot razne smiselne definicije mer pristranskosti je celo matematično negativen. Kompletni rezultati glede napačnih deležev v ocenah dokazano, da jim razen v posebnih primerih ni mogoče zadostiti sistema COMPAS so: hkrati [12]. Beli Temnopolti V prispevku pregledamo razne definicije pristranskosti in FPR 23.5% 44.9% različna mnenja o tem, kako naj bi problem najbolj učinkovito FNR 47.7% 28.0% naslovili v praksi. Zaključki konvergirajo k temu, da je za ustrezno obravnavo potrebno upoštevati družbene vrednote in jih Te rezultate so Angwin idr. [2] interpretirali kot očitno operacionalizirati z demokratično sprejetim družbenim pristranske do temnopoltih in zato ocenili uporabo sistema dogovorom v obliki ustrezne zakonodaje. K dobremu splošnemu COMPAS kot neprimerno in diskriminatorno. Taki interpretaciji razumevanju pristranskosti v UI v praksi pa bi pripomogla boljša bi bilo težko nasprotovati. Dodatni problem so videli v dejstvu, splošna izobrazba o UI in njenih metodah. da odločitveni kriterij, ki ga uporablja COMPAS, ni transparenten, saj je algoritem varovan kot poslovna skrivnost. COMPAS sam pa ne poda razlage svoje napovedi. Ta članek je zelo pogosto citiran in posledično je COMPAS postal najbolj 2 COMPAS znan primer pristranskosti v strojnem učenju, tako v strokovnih krogih domene strojnega učenja, kot pri splošni publiki, ki nima strokovnega znanja o umetni inteligenci. Kljub temu se Sistem COMPAS je bil v vrsti publikacij obravnavan kot COMPAS še vedno uporablja. verjetno najbolj kontroverzen primer, ki naj bi ilustriral Na članek iz ProPublice je odgovorila skupina strokovnjakov pristransko delovanje UI. COMPAS (Correctional Offender iz ameriškega pravosodja v članku z zgovornim naslovom “False Management Profiling for Alternative Sanctions) je odločitveni positives, false negatives and false analysis: a rejoinder to sistem, ki ga na mnogih ameriških sodiščih uporabljajo sodniki Machine Bias …” [8]. Navedli so več spornih odločitev v analizi za oceno tveganja povratništva, konkretno, da bo obsojenec v ProPublice, izvedli svojo lastno eksperimentalno raziskavo in roku dveh let ponovil prestopek, če bo izpuščen. COMPAS je zaključili, da so teze ProPublice napačne. Ta kritika izgleda razvilo ameriško podjetje, takrat imenovano Northpointe. upravičena. Bila pa bi bolj prepričljiva, če bi v [8] jasno pokazali, COMPAS upošteva 137 podatkov o vsakem prestopniku. Te kje naj bi nastala odločilna napaka v ProPublici. Namesto tega so podatke analizira poseben algoritem, ki kot poslovna skrivnost navedli svoj lastni eksperimentalni rezultat, ki naj bi dokazal, da so obsojenci obravnavani pravično, ne glede na raso. Ta rezultat podjetja ni splošno znan. Na osnovi te analize algoritem poda so dobili tako, da so upoštevali ocene tveganja obsojencev na oceno, ali gre za visoko ali nizko tveganje povratništva. lestvici od 1 do 10, kot jih oceni COMPAS. Iz teh ocen so Slika 1 ilustrira 9 močno citiranih člankov o tem sistemu ter izračunali mero AUC (Area under ROC curve, to je površina pod medsebojno citiranje med članki. Puščica iz članka A v članek B ROC krivuljo, ki se standardno uporablja v strojnem učenju kot pomeni, da je B citiran v A. indikator uspešnosti učenja). Mera AUC je zanimiva zato, ker je enaka verjetnosti, da napovedni sistem pravilno razlikuje med pozitivnimi in negativnimi primeri. To pomeni, da če vzamemo dva naključna primera (dva obtoženca), od katerih je eden pozitiven (je ponovil prestopek) in drugi negativen, potem bo sistem z verjetnostjo AUC pravilno odločil, kateri je pozitiven in kateri negativen. Flores idr. [12] navajajo, da je za belce dobljena vrednost AUC enaka 0,69, za temnopolte pa 0,70. Pri tem razlika ni statistično signifikantna. Iz tega zaključijo, da COMPAS ni diskriminatoren in da rezultati ProPublice, ki kažejo na diskriminacijo, ne morejo biti pravilni. Vendar ta posredni argument dopušča dvom, saj mere AUC ter FPR in FNR med seboj niso enoznačno povezane. Dressel in Farid [7] poročata o relevantnih poskusih, kjer ju Slika 1: Medsebojna povezanost objav o sistemu COMPAS je zanimala točnost napovedi o tveganju povratništva, ki jo dosežejo naključno izbrani ljudje brez znanja o navadah V središču grafa je članek iz časopisa ProPublica obsojencev. Poskus s človeškim napovedovanjem (izvedenim s [2], ki naj bi po [1] sprožila zanimanje za preučevanje pristranskosti v UI. V “crowd sourcingom”) sta naredila na podmnožici podatkov o [2] 1000 od skupaj okrog 7000 obsojencev iz poskusov v [2] in [8]. je skupina raziskovalnih novinarjev opisala svojo analizo sistema COMPAS in poskuse s primeri realnih podatkov o več kot 7000 Ker bi bila uporaba vseh 137 atributov za poskus z napovedmi obtožencih iz Floride v let ljudi nepraktična, sta izmed originalne množice atributov izbrala ih 2012 in 2013. V analizi se osredotočijo predvsem na rasni vidik in njihov zaključek je, da je le 7 atributov. Napovedna točnost ne-ekspertov je v teh poskusih program pristranski do temnopoltih obtožencev. Spremljali so, presenetljivo praktično enaka kot tista s sistemom COMPAS. Zanimivo je, da so tudi človeške napovedi v tem poskusu koliko od teh je bilo v naslednjih dveh letih ponovno obsojenih podobno pristranske kot COMPAS, merjeno s FPR in FNR za in primerjali napovedi z dejanskimi izidi. 44.9% temnopoltih, označenih z visokim tveganjem za ponovitev, ni ponovilo bele in temnopolte, ter da se ti rezultati skoraj ne spremenijo, če prestopka. 47.7% belcev, označenih z nizkim tveganjem, pa je v človeškemu ocenjevalcu kot dodatno informacijo podamo tudi roku dveh let ponovilo prestopek. Ti dve merili za napačne podatek o rasi. 11 Pristranskost v strojnem učenju: Dileme in odgovori Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Neodvisno od teh rezultatov je Cynthia Rudin [16] s strojnim svoje smiselne. Izraz pristranskost se v strojnem učenju učenjem pravil iz omenjenih podatkov s Floride sintetizirala zelo uporablja v več pomenih [11]: enostaven in povsem razumljiv napovedni model, ki vsebuje le 1. T.i. induktivna pristranskost: to je princip, po katerem se tri enostavna if-then pravila in uporabi tri atribute. Za razliko od algoritem odloči za eno izmed tipično velikega števila možnih modela COMPAS, so ta pravila trivialno razumljiva. Tudi ta hipotez, ki so glede na učne podatke vse na nek način utemeljene. prediktor ima zelo podobno točnost kot COMPAS, pa tudi Ta vrsta pristranskosti je neizogiben mehanizem in je zato v podoben FPR in FNR. principu pozitivna komponenta strojnega učenja, brez katerega Iz vseh opisanih rezultatov zaključujemo, da je ta napovedni strojno učenje sploh ni možno. Primer take pristranskosti je problem kljub obsežnim razpoložljivim informacijam o Occamova britev (Occam’s razor), ki pravi: Če imamo na voljo obtožencu tako težak, da boljše točnosti ni mogoče doseči. Hkrati dve razlagi zbranih podatkov, ki sicer obe enako dobro razložita skoraj vse dosežemo z dvema ali tremi najbolj koristnimi atributi te podatke, potem raje izberemo enostavnejšo razlago [9, 11, 15]. in preostalih 130+ atributov dodatno ne doseže ničesar. V skladu To pristranskost uporabljamo pogosto ne le v strojnem učenju, s tem se v [2, 7] izoblikuje teza, da uporaba strojnega učenja v temveč v znanosti nasploh. Čeprav ima izraz pristranskost pravosodju nima dobre perspektive. To je seveda prenagel in negativen prizvok, je induktivna pristranskost pozitivna in celo preveč enostaven zaključek, na kar opozarja [17]. V mnogih neizogibna komponenta strojnega učenja, kot razlagajo avtorji v drugih aplikacijah je strojno učenje preseglo napovedno točnost [11], in osnovni učbeniki umetne inteligence. ekspertov, kar so med drugim potrdili mnogi poskusi s strojnim 2. Pristranskost v učnih podatkih, ki odražajo dejanske učenjem v medicinski diagnostiki. pristranskosti v ustaljenem odločanju na danem področju Vsa ta različna mnenja o (ne)pristranskosti in (ne)uporabnosti uporabe (npr. pristranskost ekspertov v dejanski sodni praksi v sistema COMPAS kažejo na pomanjkanje splošno sprejetih okolju, iz katerega so zajeti učni podatki) [4, 14]. operativnih definicij pristranskosti in pravičnosti v strojnem 3. Pristranskost, ki izhaja iz neprimernega postopka zbiranja učenju. Situacijo lepo ilustrira močno citirani članek [14], ki podatkov oz. vzorčenja [11], npr. da je za določeno skupino ljudi razglablja o več deset relevantnih definicijah, pri tem pa ne na voljo bistveno manj primerov kot za druge skupine. Potem v ponudi sinteze, ki bi to idejno kompleksnost omejila in dala skladu z matematično utemeljenimi statističnimi in praktično uporaben pristop. Dodatno nelagodje ta članek verjetnostnimi principi nekatere skupine, tipično manjšinske, povzroči s tem, da na hitro opravi s sistemom COMPAS in ga izpadejo kot diskriminirane (lahko celo v pozitivnem smislu!) uvrsti med očitno pristranske ter neposrečene in nekoristne. Pri zgolj zato, ker metode ocenjevanja verjetnosti upravičeno tem raziskave [8] ne omeni. V [7] pa spregleda dejstvo, da tudi ocenijo verjetnosti drugače, če je na voljo malo podatkov. alternativni rezultati s strojnim učenjem in s človeškim Gornji viri pristranskosti so razmeroma splošno sprejeti. napovedovanjem povratništva na isti podatkovni množici kažejo Ostaja pa problem, kako natančno definirati merila, ki objektivno zelo podobno pristranskost do temnopoltih, kot COMPAS. povedo, ali je sistem pristranski oz. ki to pristranskost kvantitativno vrednotijo. Obstajajo številne mere, ki so videti relevantne, vendar se izkaže, da si med seboj nasprotujejo in zato 3 DEFINICIJE PRISTRANSKOSTI IN za zdaj enostavne, splošno sprejete mere ni. Situacijo zelo dobro PRAVIČNOSTI TER NJIHOVI PROBLEMI ilustrira izčrpni pregled v [14]. Bolj fokusirano raziščejo ta problem Kleinberg idr. [12]. V splošnih medijih se strojnemu učenju pogosto enostavno očita Definirajo tri naravne, same po sebi takorekoč očitne pogoje, ki pristranskost bolj po občutku, ne da bi natančno definirali, po jim mora zadostiti sistem, če naj bo nepristranski (pravičen). kakšnem matematično preverljivem kriteriju se pristranskost Toda presenetljivo se izkaže, da ti trije pogoji ne morejo biti kaže. Izjave, kot so: “sistem se je v sodstvu pokazal kot izpolnjeni hkrati, razen v posebnih primerih, ki pa so za prakso pristranski do temnopoltnih obtožencev” [2], ali “sistem je pri nezanimivi. Torej so že te tri osnovne zahteve skupaj ocenjevanju kandidatov za zaposlitev pristranski do žensk” [5], neuresničljive. Te tri zahteve so: uporabljajo splošne fraze, kot so “pristranskost algoritmov”, (1) Kalibracija ocen verjetnosti: če algoritem identificira “pristranskost strojnega učenja”, “pristranskost umetne množico oseb, ki naj bi z dano verjetnostjo pripadale inteligence”. Včasih so te ugotovitve opremljene z enostavno pozitivnemu razredu, potem mora približno tak delež te množice razlago, kot je: “sisteme strojnega učenja razvijajo skoraj dejansko pripadati pozitivnemu razredu. Enak pogoj mora veljati izključno beli moški, torej …”. za vse skupine oseb, ki se razlikujejo v “zaščitenem atributu”, Danes je jasno, da stvar ni tako trivialna. Pretirano enostavne npr. rasi ali spolu. razlage se zdaj pojavljajo redko. Postaja tudi bolj jasno, da fraza (2) Ravnotežje pozitivnega razreda: povprečje verjetnostnih ocen “pristranskost algoritmov” ni primerna in daje napačen občutek, oseb pozitivnega razreda mora biti enako za vse skupine. da so algoritmi sposobni imeti zle namene in da ne delujejo po (3) Ravnotežje negativnega razreda: analogno kot povprečje matematičnih in statističnih principih [16]. Cilj teh metod je pozitivnega razreda. vedno, da iz podatkov o realnem svetu odkrijemo zakonitosti, ki Avtorji dokažejo izrek, da so te tri zahteve, čeprav si v bistvu v tem svetu veljajo. Seveda se takoj pojavi problem, če so v prizadevajo za isti cilj zmanjševanja pristranskosti, med seboj realnem svetu že prisotne pristranske prakse. Podatki, zajeti v nekompatibilne, razen v posebnih primerih. takem svetu, odražajo to pristranskost in algoritem za učenje to Kadar se pojavi pristranskost, je vprašanje, kako jo odpraviti. pristranskost detektira in reproducira. Če rezultate, dobljene iz Za to obstaja vrsta idej, od katerih sta najbolj očitni (a) “zaščiteni pristranskih podatkov v realnem svetu, spet uporabimo v realnem atributi” in (b) obratna diskriminacija. Tipična zaščitena atributa svetu, bomo s tem reproducirali že obstoječo pristranskost [10]. sta rasa in spol. Vseeno še vedno ni dovolj natančno definirano, kaj pristranskost Princip zaščitenih atributov je, da algoritmu učenja sploh je. Pogosto gre za vtis pristranskosti, kjer se kažejo prepovemo uporabo teh atributov pri odločanju o klasifikaciji predsodki za ali proti posamezniku ali skupini na način, ki se primera. Ta ideja navadno ne deluje dobro, saj algoritem učenja razume kot nepravičen [15]. efektivno rekonstruira njihove vrednosti iz drugih, nezaščitenih Poglejmo, v čem so težave z definicijo pristranskosti. Že na atributov, ki korelirajo z zaščitenimi. Na primer iz podatkov o področju strojnega učenja najdemo različne razlage, ki so vse po šolanju ali lokaciji prebivališča algoritem sklepa na raso osebe. 12 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Farič et al. Princip obratne diskriminacije je, da depriviligiranim Za ustrezno splošno razumevanje in ukrepanje na tem skupinam pri obravnavi namenoma damo določeno prednost, s področju se kaže potreba po kvalitetni splošni izobrazbi ljudi. čimer naj bi izničili učinek diskriminacije. Ta ukrep je očitno Pomanjkanje le-te se kaže v načinu poročanja, odzivanju ljudi in dobronameren, vendar s tem dejansko uvedemo dodatno tudi zmedenosti strokovanjakov. Različni algoritmi postajajo nepravičnost, ki je za nekatere vprašljiva (npr. Alelyani [1]). neizogiben del naših življenj. Nesprejemljivo je, da o njih ne Taka nepravičnost (obratna diskriminacija) je upravičena, vendar samo da vemo premalo, ampak imamo celo napačne predstave. ne z vidika pravičnosti, temveč z vidika “višjih” vrednot, npr. da Splošno znanje o delovanju algoritmov (in širše o umetni za v bodoče popravimo zgodovinske krivice in z začasno inteligenci) temelji največ na poročanju medijev s pogosto nepravičnostjo dosežemo dolgoročno pravičnost. Torej gre za pomanjkljivimi informacijami, napačnimimi poudarki ter strateško uresničevanje družbeno sprejetih vrednot, ki v praksi pretiravanjem. Nujno je, da se ljudje o tem dovolj izobrazijo in sicer zaradi zgodovinskih razlogov in vztrajnosti niso hitro lahko tako primerno ocenijo situacije, kjer nek algoritem uresničljive. Ostaja težavno vprašanje, do kakšne mere je obratna proizvaja nezaželene rezultate. diskriminacija smiselna. To bi moralo biti določeno z demokratično sprejetim družbenim konsenzom, formaliziranim z ZAHVALA ustreznimi zakoni za vsak primer posebej. Prispevek je nastal v okviru ciljnega raziskovalnega projekta V2-V praksi se reševanja pristranskosti lotimo znotraj treh faz 2272 Opredelitev okvira za zagotavljanje zaupanja javnosti v strojnega učenja: 1) pred-procesna faza, kjer povečamo vzorec sisteme umetne inteligence in njihove uporabe, ob podpori Javne manjšine, 2) med-procesna faza, kjer dodajamo omejitve, s agencije za raziskovalno in inovacijsko dejavnost Republike katerimi kompenziramo za neenakomeren vzorec in 3) post- Slovenije in Ministrstva za digitalizacijo. procesna faza, kjer spreminjamo mejne vrednosti za manjšine [4, 14, 15]. REFERENCE Ko razvijamo metode in orodja se moramo zavedati [1] Alelyani, S. (2021). Detection and Evaluation of Machine Learning Bias. potencialnih pasti. V [1, 11] avtorji izpostavljajo, da lahko Applied Sciences, 11(14). https://doi.org/10.3390/app11146271 določene rešitve pripeljejo do novih nepravičnosti, pogosti [2] Angwin, A., Larson, J., Mattu, J. in Kirchner, L. (2016). Machine Bias: stranski učinek mutiranja učnih podatkov pa je izguba There’s Software Used Across the Country to Predict Future Criminals. And it’s Biased Against Blacks. ProPublica. pomembnih povezav med spremenljivkami ali slabše delovanje [3] Artificial Intelligence Act, European Parlament, 14 June 2023. celotnega algoritma [5]. [4] Blanzeisky, W. in Cunningham, P. (2021). Algorithmic Factors Influencing Bias in Machine Learning. arXiv preprint. https://doi.org/10.48550/arXiv.2104.14014 [5] Chakraborty, J., Majumder, J. in Menzies, T. (2021). Bias in Machine Learning Software: Why? How? What to do? arXiv preprint. https://doi.org/10.48550/arXiv.2105.12195 4 ZAKLJUČKI [6] Dastin, J. (11.10.2018). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us- Pristranskost je v nekaterih pomembnih aplikacijah strojnega amazon-com-jobs-automation-insight-idUSKCN1MK08G [7] Dressel, J., Farid, H. (2018). The accuracy, fairness, and limits of postala popularna in kontroverzna tema. V diskusiji prevladuje predicting recidivism. Science Advances, 4(1). nejasnost, ki izvira iz tega, da večina razume pojem pravičnosti [8] Flores., A. W., Bechtel, K. in Lowenkamp, C. T. (2016). False Positives, in pristranskosti intuitivno. Pri tem pravičnost doživljamo na False Negatives, and False Analyses: A Rejoinder to “Machine Bias: razne načine in v podrobnostih ni popolnega soglasja. Tako tudi There’s Software Used Across the Country to Predict Future Criminals. And it’s Biased Against Blacks.” Federal Probation Journal. ni soglasja o tem, kakšen naj bi bil jasen, matematično formuliran [9] Gordom, D. F. in Desjardins, M. (1995). Evaluation and Selection of kriterij, s katerim bi brez dvomov kvantificirali pristranskost Biases in Machine Learning. Machine Learning, 20, 5-22. konkretnega sistema. Veliko več je nasprotovanj, kontroverznih https://doi.org/10.1023/A:1022630017346 in odprtih tem, kjer ni strinjanja. Ni konsenza o izvoru [10] Hardt, M., Price, E. in Srebro, N. (2016). Equality of Opportunity in Supervised Learning. arXiv preprint. pristranskosti, niti o tem, katero orodje oz. metoda je za soočanje https://doi.org/10.48550/arXiv.1610.02413 s pristranskostjo najbolj primerna. [11] Hellström, T., Dignum, V. in Bensch, S. (2020). Bias in Machine Spielkamp [17] na primer komentira pomanjkanje enotne Learning – What is it Good for? arXiv preprint. https://doi.org/10.48550/arXiv.2004.00686 definicije in kriterijev takole: “Jasno je, da naj bi pravičnost [12] Kleinberg, J, Mullainathan, S. in Raghavan, M. (2016). Inherent Trade- strojnega učenja pomenila produciranje odločitev, s katerimi bi Offs in the Fair Determination of Risk Scores. arXiv preprint. bili kot družba zadovoljni. Vendar glede tega ljudje nismo https://doi.org/10.48550/arXiv.1609.05807 enotni.” Na primeru COMPAS se pokaže, kako ključna je ta [13] Levin, S. (8.9.2016). A beauty contest was judged by AI and the robots didn’t like dark skin. The Guardian. enotnost. COMPAS je testiralo več strokovnjakov in njihova https://www.theguardian.com/technology/2016/sep/08/artificial- mnenja so si povsem nasprotna. Nekateri trdijo, da je COMPAS intelligence-beauty-contest-doesnt-like-black-people pristranski, drugi pravijo, da ni. Spielkamp meni, da imajo prav [14] Mehrabi, A., Morstatter, F., Saxena, N., Lerman, K. in Galstyan, A. vsi, saj pravičnost razumejo na razne (2021). ACM computing surveys (CSUR) 54 (6), 1-35. načine. https://doi.org/10.48550/arXiv.1908.09635 Tudi nekateri drugi avtorji ugotavljajo podobno. Poudarjajo, [15] Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M. da je nujno razviti široko družbeno sprejeto definicijo E. … Staab, S. (2020). Bias in data-driven artificial intelligence systems – pravičnosti, ki bo rezultirala v sistemih, ki bodo delovali v skladu An introductory survey. WIREs Data Mining and Knowledge Discovery, 10(3). https://doi.org/10.1002/widm.1356 z ustaljenimi družbenimi vrednotami in s tem povezanimi [16] Rudin, C. (2019). Stop explaining black Box Machine Learning Models pričakovanji. for High Stakes Decisions and Use Interpretable Models Instead. Nature V literaturi kljub temu ni videti, da bi kdo predvidel, kako Machine Intelligence, 1(5), 206-215. https://doi.org/10.1038/s42256- velik izziv bo to. Pričakovanja glede vrednot bo treba namreč 019-0048-x natančno formulirati z ustreznimi zakoni. Na primer, ali naj bo [17] Spielkamp, M.. (2017). Inspecting Algorithms for Bias. Technology Review. zaradi zgodovinskih krivic v konkretni aplikaciji realizirana [18] Staff, R. (7.12.2016). New Zealand passport robot tells applicant of Asian obratna pristranskost in do kakšne mere? Ta formulacija bo descent to open eyes. Reuters. https://www.reuters.com/article/us- morala biti bolj tehnična kot običajno v predpisih in zakonih, saj newzealand-passport-error-idUSKBN13W0RL [19] UNESCO Recommendation on the Ethics of Artificial Intelligence, 2021. bo to osnova za konkretno implementacijo v algoritmih umetne https://unesdoc.unesco.org/ark:/48223/pf0000381137 inteligence. 13 ChatGPT through Tononi's Definition of Consciousness Analiza ChatGPT skozi Tononijevo definicijo zavesti Matjaž Gams Odsek za inteligentne sisteme Institut “Jožef Stefan” Jamova cesta 39, 1000 Ljubljana Slovenija ChatGPT. Then, passing the Turing test by ChatGPT is analysed ABSTRACT as another test whether consciousness is already achieved. The intricacies of consciousness and its existence have long been There is a rich tapestry of literature exploring consciousness, AI, the subjects of both philosophical and scientific investigations. With advancements in artificial intelligence, discerning the line and where the two intersect. Koch et al. have explored how IIT between algorithmic processing and consciousness becomes offers a quantitative measure of consciousness in diverse systems increasingly vital. This paper uses Tononi's Integrated [1]. Dehaene and others have looked into consciousness in both Information Theory (IIT) to examine ChatGPT, assessing its alignment with human notions of consciousness. Analysis shows biological and artificial systems, arguing for unique neural that while ChatGPT exhibits attributes superficially resonating markers that underpin conscious states [2]. Recent advancements with Tononi's axioms, it lacks the foundational conscious in AI, especially deep learning models like ChatGPT, have experience described by the theory. Then, passing the Turing test triggered renewed debates, with researchers like Hassabis et al. as a way of demonstrating consciousness is debated with similar conclusions. and Bengio postulating how AI might approach or simulate human-like consciousness [3,4]. POVZETEK Zapletenost zavesti in njen obstoja ter namen so predmet tako filozofskih kot znanstvenih naravoslovnih raziskav. Z 2 ANALYSIS OF CHATGPT THROUGH napredkom v umetni inteligenci postaja vedno bolj ključno TONONI'S AXIOMS FOR razlikovanje med algoritmično obdelavo in zavestjo. Ta članek uporablja Tononijevo teorijo integrirane informacije (IIT) za CONSCIOUSNESS analizo potencialne zavesti ChatGPT, pri čemer je ključno vprašanje usklajenost z človeškimi predstavami o zavesti. Analize kažejo, da ChatGPT sicer površinsko zelo dobro resonira s Tononijevimi aksiomi, vendar mu manjka temeljna zavestna 2.1 Background: Tononi's Axioms for izkušnja, ki jo teorija opisuje. V nadaljevanju prispevka je Consciousness opisovanje reševanja Turingovega testa s strani ChatGPT kot Tononi's Integrated Information Theory (IIT) proposes five preverjanja zavestnosti, ki se izkaže pri tem testu, ter diskusija. fundamental axioms aimed at capturing the core of KLJUČNE BESEDE consciousness: Zavest, ChatGPT, Turingov test Intrinsic Existence: Consciousness inherently exists for the KEYWORDS conscious entity. It's a subjective phenomenon, deeply personal Consciousness, ChatGPT, Turing test and unique to each entity [6]. Composition: Consciousness is not monolithic. It possesses 1 INTRODUCTION structure, and within it, diverse experiences can be differentiated. As AI continues to progress, the discussion surrounding machine This diversity isn't merely quantitative but also qualitative, consciousness intensifies. The question often arises: can making each conscious experience rich and multidimensional [7]. machines ever possess genuine consciousness? Has that already Information: Consciousness is informative. Every conscious been achieved by ChatGPT? To delve into this, we leverage the experience stands out against other potential experiences, IIT—a framework by Tononi proposing five axioms believed to indicating a specific state of affairs over countless others [8]. underlie human consciousness—and apply these criteria to Integration: Despite its diverse composition, consciousness is unified. Experiences are intertwined, and it's impossible to 14 Informacijska družba 2023, 9.–13. oktober 2023, Ljubljana, Slovenija M. Gams completely isolate any subset of phenomena within a single the exclusion axiom, the model's responses might not be the conscious moment [9]. result of conscious choices or experiences. Exclusion: Consciousness is definite, both in content and in space. At any given moment, an entity is conscious of certain It's essential to highlight the difference between operational things and not others, thus creating clear boundaries of alignment and conscious alignment. While ChatGPT showcases experience [10]. attributes that operationally resonate with some of Tononi's axioms, it does so without the underlying conscious experience these axioms were designed to describe. The axioms, rooted in 2.2 ChatGPT under the lens of Tononi's axioms human phenomenology, emphasize subjective experience, To evaluate the degree of ChatGPT achieveing each axion of the something inherently absent in ChatGPT. Tononi's theory, the existing literature and the opinion in the AI ChatGPT, in its design and operation, exhibits attributes that community is of no great help, since there are mixed opinions superficially align with Tononi's axioms to a certain degree. and no generally accepted viewpoint. However, there are two However, when delving into the crux of these principles— bases that this paper evaluates consciousness of ChatGPT conscious experience—ChatGPT falls a bit short. While it stands through various analysis: First, there is 50 years of experience of as a testament to advancements in information processing and AI, the author of the paper in the AI and cognitive field and 20 years ChatGPT does not qualify as a conscious entity within the of superintelligence studies. Second, there is an opinion of framework of Integrated Information Theory. ChatGPT when asked about each particular issue. Interestingly enough, even though there were some differences, and the author 3 HAS CHATGPT PASSED THE TURING chose the merit of the expertize, there was quite strong agreement TEST I.E. REACHING THE in general. The GPT opinion, seemingly, was to a large extend hand-crafted by humans, dealing with this issue, and partially COUNSCIOUSNESS? through the GT or LLT approach, as demonstrated by the replies. Another way to test the level of AI systems at achieving Intrinsic Existence: At its core, ChatGPT is a product of consciousness, can be performed by the Turing Test (TT). The algorithms and vast data. It operates in response to inputs, Turing Test, proposed by the eminent computer scientist Alan without possessing feelings, beliefs, or desires. It lacks any Turing in 1950, is a measure of a machine's ability to exhibit semblance of self-awareness, and as such, it probably does not intelligent behaviour indistinguishable from that of a human [11]. meet the axiom of intrinsic existence. Turing postulated that if an evaluator, after interacting with an Composition: ChatGPT, architecturally, boasts a vast neural unseen interlocutor for five minutes, could not reliably tell network configuration. This allows it to generate a diverse array whether dealing with a machine or a human, then the machine of outputs based on different inputs. However, this structural could be said to have passed the test. As ChatGPT emerges as variety isn't birthed from conscious deliberation but from learned one of the most sophisticated AI language models, there's debate patterns. While it exhibits structural diversity analogous to the about its positioning relative to the Turing Test and consequently composition axiom, it seemingly lacks the qualitative conscious consciousness. nuance integral to Tononi's definition. Information: The model processes and produces specific 3.1 Arguments in Favor: responses based on its training. Each response is a selective piece of information shaped by its training data and the query. Advanced communication: Although this aligns with the informational aspect of the axiom, ChatGPT is engineered to provide detailed replies that span a the absence of conscious deliberation and choice makes its wide range of subjects, from science and technology to alignment potentially superficial. philosophy and art. The quality of its responses often mirrors Integration: ChatGPT's processes are integrated. Each input human-like expertise and reasoning capabilities, making it a is processed through multiple layers, intertwining different versatile conversational agent. Its ability to provide contextually learned patterns to produce a coherent output. This mirrors the relevant and accurate information resembles the intellectual operational facet of the integration axiom. However, the unity breadth and depth one would expect from a knowledgeable described by Tononi implies a cohesive conscious experience, human, being familiar with the Web. Consequently, it is which ChatGPT probably does not possess. increasingly difficult to immediately distinguish some of its Exclusion: With its design parameters and training, ChatGPT responses from those generated by a human, especially in text- operates within set boundaries. It produces specific responses based interactions. and not others. While this resonates with the operational side of 15 Informacijska družba 2023, 9.–13. oktober 2023, Ljubljana, Slovenija M. Gams Adaptive Interaction: Norvig, in their comprehensive AI textbook, consider the Turing The model is designed to be sensitive to the conversational Test as a valid, though not definitive, measure of machine context, allowing it to adjust its responses based on previous intelligence, suggesting that while passing the Turing Test is dialog turns. This adaptability manifests in the way it can switch significant, it does not necessarily equate to full human-like topics smoothly, clarify ambiguities, or even attempt humour, intelligence [14]. A recent article in Nature delineates the much like a human would in a fluid conversation. Its capability increasing complexity and performance of large-scale language to modify its tone and content in real-time according to models, weighing them against the Turing Test's standards [15]. conversational cues shows an advanced level of adaptability, Another piece in the Boston Review contemplates the question often comparable to human dialog dynamics. This feature of consciousness in such models, considering the philosophical enhances its suitability for diverse interactions, making it a implications of designating them as 'conscious' [16]. compelling interface for numerous applications. One salient point from the Boston Review article is the distinction between machine operation and human consciousness 3.2 Arguments Against: [17]. While large language models like ChatGPT can generate Lack of Understanding: intricate and seemingly aware responses, there remains a While ChatGPT's responses can be elaborate, it's essential to significant philosophical and cognitive gap. These models remember that the model doesn't possess genuine understanding operate by recognizing and generating patterns based on massive or consciousness. It generates text based on statistical patterns it datasets without the subjective experience that characterizes has learned from its training data, and skilfully intercombines human consciousness. The Boston Review delves into the various potential most relevant texts patterns into a word by word implications of mistaking this high-level processing capability continuation, without the ability to comprehend the meaning or for genuine consciousness, emphasizing the risks of significance of the conversation [15]. Its sophisticated language anthropomorphizing machine behaviors. capabilities may give the illusion of understanding, but this In light of this, it becomes evident that while ChatGPT can surface-level competence should not be mistaken for genuine generate responses that mimic human-like thinking, it does so comprehension or awareness. devoid of genuine understanding, emotions, or the conscious experiences that humans possess. Inconsistencies: The model is not reliable in providing consistent answers over This perspective aligns with the core debate surrounding the different conversational sessions or even within the same Turing Test. ChatGPT's capability to produce responses that may interaction, the effect called hallucinating. These inconsistencies seem indistinguishable from those of a human in specific are a testament to its underlying algorithmic nature, which does contexts does not necessarily imply that the model genuinely not have the capacity for ongoing memory or the ability to learn "thinks" or possesses consciousness. Instead, it underlines the from past interactions [16]. Such discrepancies in its replies can model's adeptness at pattern recognition and response generation, sometimes make it evident that one is conversing with a machine, which, although impressive, is fundamentally distinct from not a human, thereby undermining its credibility and reliability human cognition. in more complex or sensitive dialog scenarios. 4 DISCUSSION Absence of Emotions: Human dialog is often rich in emotional nuance and subtext, an ChatGPT demonstrates the remarkable advancements in artificial aspect that is conspicuously lacking in ChatGPT. Despite its intelligence, surpassing previous efforts by a substantial margin. linguistic capabilities, the model cannot feel emotions or However, it's essential to distinguish between its algorithmic understand the emotional weight of certain words or situations. complexity and genuine consciousness. The difference is Its interactions are devoid of emotional depth, empathy, or any described in this paper through two major parts: first though the other kind of emotional intelligence that is often central to human IIT theory and second through the Turing Test. communication. This absence not only differentiates it from human-like conversation but also limits its applicability in While the model excels in simulating conscious traits, within the scenarios where emotional engagement is crucial. framework of Integrated Information Theory (IIT) it remains devoid of authentic conscious experience—a distinction that Literary Standpoints: should be apparent to cognitive scientists, even in the presented Hernandez-Orallo discussed the limitations of the Turing Test, discussions in this paper. emphasizing that mere linguistic capability may not be a sufficient measure of machine intelligence [13]. Russell and 16 Informacijska družba 2023, 9.–13. oktober 2023, Ljubljana, Slovenija M. Gams Second, regarding the Turing Test, ChatGPT displays impressive 5. Tononi, G. (2004). An information integration theory of linguistic capabilities that bring it tantalizingly close to fulfilling consciousness. BMC Neuroscience, 5(1), 42. the test's criteria. Yet, it falls short in key areas, including 6. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, genuine understanding and emotional intelligence. For L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is specialists familiar with the field, this limitation is easily All You Need. In Advances in Neural Information identifiable, especially when the conversation veers into complex Processing Systems (pp. 5998-6008). or emotionally charged topics. In these situations, the model’s 7. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. limitations become obvious, as it fails to respond appropriately Nature, 521(7553), 436-444. to intricate, tangible or conflicting dialogues. 8. Mitchell, T. M. (1997). Machine Learning. McGraw Hill. 9. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., While some literature may argue that ChatGPT could pass the Huang, A., Guez, A., ... & Hassabis, D. (2017). Mastering Turing Test in brief interactions, especially with laypeople, the Chess and Shogi by Self-Play with a General Reinforcement articles from Boston Review and Nature emphasize the crucial Learning Algorithm. arXiv preprint arXiv:1712.01815. distinction between mere simulation and actual consciousness. 10. Bostrom, N. (2014). Superintelligence. Oxford University Indeed, those well-versed in the subject, including cognitive Press. scientists, should readily discern ChatGPT's performance from 11. Turing, A. M. (1950). Computing Machinery and that of a human. This serves as a timely reminder of the need to Intelligence. Mind, 59(236), 433-460. continually update our evaluation metrics, not just technically 12. Hernandez-Orallo, J. (2000). Beyond the Turing Test. but also philosophically, as AI continues to challenge our Journal of Logic, Language and Information, 9(4), 447-466. conceptual boundaries of intelligence and consciousness. 13. Russell, S. J., & Norvig, P. (2021). Artificial Intelligence: A Modern Approach (4th ed.). Prentice Hall. In conclusion, although ChatGPT and similar generative models 14. Biever, C. (2023). ChatGPT broke the Turing test — the signify a quantum leap toward general AI and possibly race is on for new ways to assess AI, Large language models superintelligence, they should not be conflated with achieving mimic human chatter, but scientists disagree on their ability consciousness or passing the Turing Test. Despite possessing to reason. Nature. Retrieved from certain superhuman attributes, such as speed, and achieving https://www.nature.com/articles/d41586-023-02361-7 approximate human-level performance in specific tasks, 15. Chalmers, J.D. (2023). Could a Large Language Model Be ChatGPT does not meet the criteria for consciousness. That Conscious? Boston Review. Retrieved from Boston Review might not hold for an extended period of time. According to URL Chalmers [15] »Within the next decade, we may well have systems that are serious candidates for consciousness«; however, current models like ChatGPT should not be mistaken as such, especially by those familiar with the cognitive sciences. Therefore, while the progress is significant, the journey toward creating truly conscious machines is far from over. References: 1. Koch, C., Massimini, M., Boly, M., & Tononi, G. (2016). Neural correlates of consciousness: progress and problems. Nature Reviews Neuroscience, 17(5), 307-320. 2. Dehaene, S., Lau, H., & Kouider, S. (2017). What is consciousness, and could machines have it? Science, 358(6362), 486-492. 3. Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-Inspired Artificial Intelligence. Neuron, 95(2), 245-258. 4. Bengio, Y. (2017). Towards biologically plausible deep learning. arXiv preprint arXiv:1702.08835. 17 Manipulacija v umetni inteligenci Manipulation and artificial intelligence Gaja Gril† Pedagoška fakulteta Univerza v Ljubljani Slovenija gg99039@student.uni-lj.si POVZETEK manipulativne umetne inteligence je priporočilni sistem, ki optimizira dolgoročno sodelovanje, tako da prikrito spodbuja V svetu, ki ga vse bolj oblikuje umetna inteligenca, se je začelo uporabnika, da si ogleda več videoposnetkov. Manipulacija je pojavljati vprašanje manipulacije. Ko sistemom umetne težko precizno opredeljiva, lahko pa se naslonimo na štiri glavne inteligence zaupamo vse večje vloge v naših življenjih, od značilnosti. Prva je, da gre za nerazumski vpliv, pri katerem personaliziranih priporočil do kritičnega odločanja, se lahko skuša manipulator zaobiti ali oslabiti človekovo sposobnost vprašamo, do katere mere zaupati strojem, ki se zdijo tako premišljenega odločanja. Druga je, da manipulacija zahteva inteligentni in nepristranski? Umetna inteligenca postaja ne le uporabo zvijače in prevare, pogosto s skritimi sredstvi, da bi vir pomoči, temveč tudi manipulira, dezinformira in vpliva. V tem raziskovanju zapletene mreže manipulacij znotraj umetne nekoga prisilili k določenemu ravnanju. Tretji razlog je, da inteligence se srečamo s tehnološko uganko, ki postavlja pod manipulacija vključuje uporabo določene stopnje pritiska, da bi vprašaj naše razumevanje avtonomije in etike. storili, kar želi manipulator, na primer s čustvenim izsiljevanjem. In nazadnje, običajno se ne ravna po interesih, ciljih in željah KLJUČNE BESEDE ciljne osebe, temveč le po interesih manipulatorja [2]. Razlogi za povečano tveganje za manipulacijo v dobi umetne Manipulacija, umetna inteligenca, demokracija, marketing, inteligence so naslednji; Prvič, umetna inteligenca je precej etika netransparentna, saj velika večina algoritmov ni pregledna in ABSTRACT razložljiva, uporabniki pa ne posedujejo zadosti tehničnega In a world increasingly shaped by artificial intelligence, the issue znanja [3]. Drugič, sistemi umetne inteligence lahko odkrijejo of manipulation has begun to emerge. As we entrust AI systems preference, interese, navade in ostale značilnosti posameznika ter with ever greater roles in our lives, from personalised tako natančno prilagodijo vsebino [4]. Poleg tega se lahko recommendations to critical decision-making, we may ask to sistemi umetne inteligence uporabljajo za ocenjevanje what extent should we trust machines that seem so intelligent and psiholoških stanj ljudi, kot so na primer čustvena stanja [5]. impartial? Artificial intelligence is becoming not only a source Tretjič, umetna inteligenca omogoča oslabitev avtonomije of help, but also a source of manipulation, misinformation and odločanja potrošnikov z izkoriščanjem njihovih ranljivosti pri influence. In this exploration of the complex web of odločanju [3]. manipulation within AI, we are confronted with a technological conundrum that calls into question our understanding of 2 MANIPULACIJA Z INFORMACIJAMI autonomy and ethics. Na dinamičnem področju demokratičnih sistemov je umetna KEYWORDS inteligenca postala močan dejavnik, ki spreminja taktiko političnih kampanj in nagovarjanja volivcev. Tradicionalne Manipulation, artificial intelligence, democracy, marketing, metode sodelovanja z javnostjo so se umaknile naprednim ethics pristopom, ki jih poganja umetna inteligenca in ki izkoriščajo obsežne podatke, da bi vplivali na razpoloženje javnosti in 1 UVOD zagotovili volilno zmago [6]. Sposobnost umetne inteligence, da obdela in oceni obsežne zbirke podatkov političnim kampanjam Carroll in sodelavci (2023) definirajo sistem umetne inteligence omogoča vpogled v nagnjenja, vedenje in čustva volivcev. kot manipulativen, če deluje, kot da bi si prizadeval za namerno Algoritmi umetne inteligence z raziskovanjem spletnih in prikrito spremembo človeka (ali drugega agenta) [1]. Primer interakcij, zgodovine brskanja in drugih digitalnih sledov ∗Article Title Footnote needs to be captured as Title Note oblikujejo podrobne profile volivcev in z izjemno natančnostjo †Author Footnote to be captured as Author Note predvidevajo posameznikove težnje. S tem znanjem lahko Permission to make digital or hard copies of part or all of this work for personal or politični akterji prilagodijo svoja sporočila tako, da odmevajo v classroom use is granted without fee provided that copies are not made or distributed različnih demografskih skupinah, in oblikujejo personalizirane for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must pozive, ki so povezani s čustvi in prepričanji volivcev. Ciljno be honored. For all other uses, contact the owner/author(s). usmerjanje volivcev, ki ga poganja umetna inteligenca, presega Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). zgolj oblikovanje profilov in uporablja tehnike mikrousmerjanja, 18 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia G. Gril da bi volivce doseglo na osebni ravni. Sporočila kampanje so iz virov kot so sledenje lokaciji in družbeni mediji. To odraža lahko natančno prilagojena, da se ujemajo s posameznikovimi premik k vplivanju na določeno vedenje potrošnikov in ne zgolj načeli in interesi, kar poveča njihov prepričljiv učinek in k njihovemu prepričevanju. Ta razvijajoči se pristop vključuje verjetnost pridobitve podpore. Ta natančnost kampanjam analizo čustvenih in psiholoških modelov ter izkorišča ranljivosti omogoča, da pametno razporedijo sredstva in se osredotočijo na in strahove potrošnikov. S strojnim eksperimentiranjem lahko ključne volivce, ki so bistveni za njihovo zmago [6]. podjetja raziskujejo vzročne povezave med vedenjem in tržnimi Vendar pa se na drugi strani tega tehnološkega napredka strategijami. Veliki podatki podjetjem omogočajo spreminjanje pojavljajo etični pomisleki. Kritiki trdijo, da lahko takšno vedenja s pomočjo paradigem osebne identitete, kar se kaže v mikrousmerjanje še poveča polarizacijo in ustvari odmevne aplikacijah, kot sta sledenje lokaciji in prepoznavanje obraza. komore, v katerih se posamezniki utrdijo v svojih obstoječih Razvoj programske opreme umetne inteligence vključuje prepričanjih in okrepijo pristranskost pri potrjevanju. Poleg tega nevroznanost, psihologijo in trženje ter ustvarja učinkovite pomanjkanje preglednosti algoritmov umetne inteligence, ki se promocijske metode. Preprosti algoritmi analizirajo spletno uporabljajo za politično ciljanje, povzroča pomisleke glede vedenje in ustvarjajo natančne profile uporabnikov. Poglobljeno odprtosti in odgovornosti, kar lahko spodkoplje demokratično učenje omogoča podrobne profile, ki presegajo celo tesna načelo informiranega soglasja.V digitalni dobi se je ranljivost razmerja. Korporacije uporabljajo ta spoznanja in s pomočjo razširjanja informacij povečala zaradi manipulacij, ki jih psihologov in nevroznanstvenikov iščejo "gumb za nakup omogočajo generirane vsebine in globoki ponaredki. Te potrošnika" [8]. napredne tehnologije lahko z razširjanjem lažnih informacij in Poleg tega tehnologije za branje misli postopoma pridobivajo spodkopavanjem zaupanja v politične postopke motijo sposobnost namernega prilagajanja dejanj kognitivnim in demokratične sisteme [6]. čustvenim stanjem vpletenih strani. Ta strateški pristop temelji Ker se demokratične družbe spopadajo s temi dilemami, so na "čustveni umetni inteligenci". To pomeni prepoznavanje zahteve po večji preglednosti in algoritemski odgovornosti vse duševnih stanj s tehnikami strojnega učenja, ki pogosto glasnejše. Iskanje ravnovesja med ohranjanjem vključenosti uporabljajo globoke nevronske mreže. Čustvena umetna uporabnikov in varovanjem demokratične integritete od podjetij inteligenca služi različnim namenom, od povečanja varnosti v družbenih medijev zahteva vesten nadzor umetne inteligence in cestnem prometu (na primer spremljanje voznikov) do etični premislek. C. Serbanescu (2021) vidi umetno inteligenco usmerjenega oglaševanja [9]. kot grožnjo demokraciji, saj ta omogoča manipulacijo procesov Čeprav se je tovrstna psihološka analiza, ki temelji na odločanja državljanov in s tem ogroža njihovo avtonomijo pri temeljni teoriji čustev, soočila s kritikami, da je preveč sodelovanju v demokratičnih procesih. Aplikacije umetne poenostavljena in nezanesljiva, je tehnologija na nekaterih inteligence imajo potencial, da oblikujejo "arhitekturo izbire" področjih že pokazala impresivno raven učinkovitosti, sčasoma državljanov, vključno z razpoložljivimi izbirami in načinom pa naj bi se še izboljšala. Primerljive tehnike bi lahko uporabili njihove predstavitve, na personaliziran, dinamičen in prikrit za odkrivanje in izkoriščanje kognitivnih pristranskosti pri način. Vseprisotnost tehnologije in zbiranje velikih količin potencialnih sogovornikih [9]. podatkov sta olajšala manipulacijo, ki jo omogoča umetna Medtem ko je potencial strojnega učenja za izkoriščanje inteligenca, kar omogoča prilagojene vplive in dinamično kognitivnih in čustvenih ranljivosti sogovornikov očiten v prilagodljivost za izkoriščanje posameznikovih slabosti. Ta postopku modeliranja, je empirično preverjanje takšne oblika manipulacije predstavlja kvalitativno in kvantitativno manipulacije v praksi precej zahtevno zaradi pogosto skrivnostne spremembo v primerjavi s tradicionalno manipulacijo, saj lahko narave teh modelov. Kljub temu je v literaturi mogoče najti nekaj učinkoviteje doseže veliko število volivcev in vpliva na rezultate, primerov potencialno manipulativnega izvajanja umetne ki bi lahko predstavljali "voljo ljudstva". Na splošno je vzpon inteligence. Podjetje eyeQ je na primer razvilo orodje, ki v aplikacij umetne inteligence privedel do učinkovitejših oblik realnem času skenira obrazno mimiko kupcev v trgovinah in manipulacije, kar predstavlja velik izziv za demokracijo [7]. analizira čustva in druge dejavnike, nato pa na podlagi teh podatkov prilagaja trženjske strategije v trgovinah. Takšne prakse lahko postanejo manipulativne, če so čustva, na katera se 3 VEDENJSKO OGLAŠEVANJE IN cilja, še posebej intenzivna (intenzivnost), če se jim pridružijo MANIPULACIJA POTROŠNIKA pristranskosti (kombinacija) ali če je kontekst odločanja zelo Umetna inteligenca spreminja dinamiko trženja in oglaševanja, zapleten (kompleksnost), kot so razmere, ki vključujejo saj prehaja od široke promocije k personaliziranim izkušnjam. Z preobremenjenost z izbiro in zahtevne primerjave v trgovini [9]. njo se briše meja med manipulacijo in prepričevanjem potrošnikov, na kar je še posebej vplival premik na splet med 4 MANIPULIRANA UMETNA pandemijo COVID-19. Prej so tržniki zbirali demografske podatke, spremljali trende in segmentirali potrošnike za ciljno INTELIGENCA usmerjanje. Tudi v fizičnih trgovinah so v bližini blagajn Do sedaj je bilo govora o manipulativnih vplivih umetne strateško postavljali predmete, kot so revije in žvečilni gumiji. intelligence na človeka, obstaja pa tudi obratni vpliv – umetna Danes strategije, ki jih poganja umetna inteligenca, zbirajo inteligenca je lahko žrtev manipulacije. obsežne podatke iz iskalnikov, družbenih medijev in aplikacij za Globoke nevronske mreže (DNN) so v zadnjem času pomoč pri oblikovanju trženjskih pristopov. pokazale izjemne rezultate, ki pogosto presegajo rezultate na Podjetja zdaj proaktivno oblikujejo interakcije s potrošniki, človeški ravni, zlasti pri nalogah, povezanih z vidno klasifikacijo da bi jih pritegnila k sodelovanju, pri čemer včasih prehajajo na [10]. Visoka zmogljivost DNN pri razvrščanju vidnih objektov mejo manipulacije. Za prilagajanje strategij uporabljajo podatke sproža vprašanja o razlikah, ki še vedno obstajajo med 19 Manipulacija v umetni inteligenci Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia računalniškim in človeškim vidom – v nasprotju z vidnim ali to pomeni kršitev zasebnosti. Pri tem vprašanju strinjajo tako sistemom so DNN veliko bolj občutljive na minimalne aktivisti kot veliki tehnološki konglomerati, ki priznavajo perturbacije. občutljivo ravnovesje med izkoriščanjem zmogljivosti umetne "Napad z enim pikslom" (ang. One pixel attack) je inteligence in varovanjem pravic posameznikov do zasebnosti. specializirana vrsta napada na globoke nevronske mreže, Sestavni del te razprave je področje ciljno usmerjenega katerega cilj je prevarati nevronsko mrežo, da napačno razvrsti oglaševanja. Medtem ko tržniki zagovarjajo njegovo zmožnost sliko z minimalnimi spremembami, pri čemer se običajno povezovanja potrošnikov z želenimi izdelki, ga vse več ljudi spremeni le en ali nekaj pikslov. Pomemben izziv pri napadih z dojema kot manipulativen poseg, ki spodbuja impulzivne vzorce enim pikslom je najti ravnovesje med tem, da nevronska mreža nakupovanja. Ta razhajanja v stališčih poudarjajo raznovrstne napačno razvrsti sliko, in tem, da so spremembe na sliki dovolj napetosti med komercialnimi cilji in avtonomijo potrošnikov. V subtilne, da jih človek ne more odkriti [10]. tem zapletenem okolju sta soglasje in ozaveščenost ključnega Taki napadi opozarjajo na dovzetnost DNN za subtilne pomena. Bistvo je v ugotavljanju, v kolikšni meri uporabniki vhodne manipulacije, kar lahko vpliva na varnost, zanesljivost in razumejo in podpirajo vlogo algoritmov umetne inteligence pri etične vidike rabe umetne inteligence. Raziskave na tem oblikovanju njihovih spletnih izkušenj. Nazoren primer je področju se ukvarjajo s strategijami za blažitev teh ranljivosti in nenamerno manipuliranje s kanali družbenih medijev s strani izboljšanje robustnosti sistemov umetne inteligence. umetne inteligence, ki pogosto deluje mimo zavedanja uporabnikov. To sproža pomisleke o etičnih mejah vpliva in nujnosti informiranega soglasja. Škandal Cambridge Analytica 5 SINTEZA je izpostavil problematiko glede zmožnosti umetne inteligence, Če se prizadevanja na področju politike in izobraževanja ne bodo da z izkoriščanjem strahov in predsodkov manipulira z uresničila in če širši cilj gojenja etične umetne inteligence ne bo odločitvami volivcev in krha temelje demokracije [11]. uspešen, obstaja možnost etične, družbene in gospodarske V ozadju tega dogajanja je sporno vprašanje čustvene katastrofe ter z njo povezanimi vplivi na ljudi, nečloveške manipulacije. Kakšne so moralne posledice dejstva, da umetna entitete in okolje. Ta nevarnost ni povezana z oddaljenimi inteligenca izkorišča človeška čustva za komercialne koristi in še apokaliptičnimi vizijami. Namesto tega izhaja iz postopnega, a več, kdo bo nosil odgovornost? Debata manipulativne umetne določenega stopnjevanja tehnoloških nevarnosti in posledičnega inteligence postavlja vprašanja o kršitvah zasebnosti, povečanja občutljivosti na človeških, družbenih, gospodarskih in usmerjenem vplivu, demokratični integriteti in čustveni etiki. okoljskih področjih. To povečevanje tveganj in ranljivosti izhaja Zaskrbljenost in dvom kliče po vzpostavitvi etičnih okvirov in iz etičnih težav, ki zajemajo nepoučeno in nepremišljeno skupnih prizadevanjih za usmerjanje vloge umetne inteligence uporabo naprednih tehnologij avtomatizacije, kot je umetna pri sooblikovanju modernega sveta. inteligenca. Razlika v izobrazbi verjetno povečuje širše posledice tveganj, povezanih z umetno inteligenco. REFERENCES Trenutno ne obstaja univerzalno dovoljenje za uporabo [1] Micah Carroll, Alan Chan, Henry Ashton, in David Krueger, 2023. umetne inteligence, prav tako ni obveznega izobraževanja o etiki Characterizing Manipulation from AI Systems. arXiv preprint arXiv:2303.09387. umetne inteligence za tehnične raziskovalce, poslovne [2] Robert Noggle, 2020. Pressure, trickery, and a unified account of strokovnjake, državne upravitelje in druge deležnike, ki manipulation. American Philosophical Quarterly, 57(3), 241-252. [3] Federico Galli, 2020. AI and Consumers Manipulation: What the Role of sodelujejo pri ustvarjanju, uporabi in urejanju umetne inteligence. EU Fair Marketing Law?, Católica Law Review 4, no. 2: 35–64, DOI: Precejšen del neregulirane umetne inteligence je v rokah tistih, https://doi.org/10.34632/catolicalawreview.2020.9320. ki ne razumejo s tem povezanih tveganj in etičnih dilem ali pa [4] Daniel Susser, Beate Roessler in Helen Nissenbaum, 2019. Online manipulation: Hidden influences in a digital world. Geo. L. Tech. Rev. , 4, imajo morda napačna pričakovanja glede te tehnologije. 1. Nevarnost se skriva v tem, da ima človek moč brez razumevanja, [5] Sandra C. Matz, Ruth E. Appel in Michal Kosinski, 2020. Privacy in the Age of Psychological Targeting, Current Opinion in Psychology, 116– kar pomeni neodgovorno ravnanje. Takšno ravnanje lahko 121, DOI: https://doi.org/10.1016/j.copsyc.2019.08.010. povzroči neupravičene posledice za druge. Predpostavljanje [6] Shahana Rayhan in Swayan Rayhan, 2023. The Role of AI in Democratic Systems: Implications for Privacy, Security, and Political Manipulation. nevtralnosti umetne inteligence in njena uporaba brez [7] Caroline Serbanescu, 2021. Why Does Artificial Intelligence Challenge razumevanja posledic prispevata k brezbrižnosti. Izobraževalna Democracy? A Critical Analysis of the Nature of the Challenges Posed by politika lahko prispeva k izboljšanju tega položaja in tako AI-Enabled Manipulation, Retskraft – Copenhagen Journal of Legal Studies, 5 (1), 105-128. spodbuja etično in smiselno umetno inteligenco [11]. Kljub temu [8] Jordan C'rene Reuille-Dupont, 2023. The Power of Algorithms and Big so nekatera vztrajna, morda neprijetna vprašanja v razpravah o Data: A Marketing Perspective on Consumer Manipulation in Business. University Honors Theses. DOI: https://doi.org/10.15760/honors.1338 etiki in politiki umetne inteligence običajno odrinjena na stranski [9] Phillip Hacker, 2021. Manipulation by algorithms. Exploring the triangle tir, čeprav si zaslužijo analizo. Ali se etika umetne inteligence of unfair commercial practice, data protection, and privacy law. Eur Law ukvarja izključno z dobrobitjo in vrednotami ljudi ali bi morala J. 2021;1–34. DOI: https://doi.org/10.1111/eulj.12389 [10] Anh Nguyen, Jason Yosinski in Jeff Clune, 2014. Deep Neural Networks vključevati tudi vrednote, dobrine in interese nečloveških entitet? are easily fooled: High confidence predictions for unrecognizable images. Osrednja sporna točka se vrti okoli kršenja zasebnosti. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [11] Mark Coeckelbergh, 2020. AI Ethics. Cambridge, Massachusetts, USA: Uporaba umetne inteligence za usmerjanje vedenja pogosto The MIT Press. zahteva obsežno zbiranje podatkov, kar sproža razprave o tem, 20 Users’ Cognitive Processes in a User Interface for Predicting Football Match Results Žiga Kolar Gregor Papa Department of Intelligent Systems Department of Computer Systems Jožef Stefan International Postgraduate School Jožef Stefan International Postgraduate School Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia ziga.kolar@ijs.si gregor.papa@ijs.si ABSTRACT importance of understanding user psychology and behavior in The article discusses the importance of understanding cognitive the design process. processes, which are the foundation for effective user interface Furthermore, this article will underline the reciprocal rela- design. The authors highlight the crucial role of intuitive and tionship between UI design and user satisfaction, which can user-friendly interfaces in online football match prediction plat- drastically impact user retention and overall platform success. forms, as it is the design of the interface that can have a significant Through the lens of cognitive science and user experience design, impact on the success of these types of platforms. The article we aspire to provide valuable insights that can guide designers delves into the thought processes of users who want a visually and developers in creating more effective, user-centric interfaces appealing, accessible and intuitive user interface for predicting for football predicting websites. By marrying scientific principles football match results. Particular emphasis is given to understand- with practical UI design strategies, we hope to set a new standard ing the psychology and behaviour of users. The paper aims to for user interface design within the football prediction industry. provide valuable insights that can guide designers and developers While the context is football-oriented, the core concepts can in designing more effective, user-centred interfaces for football be applied across various industries and areas. Here are a few match prediction websites. By combining scientific principles areas where these cognitive processes might be useful: and practical user interface design strategies, the authors set (1) E-Commerce Platforms: An understanding of user psy- new standards for user interface design in the football prediction chology and behavior can help design interfaces that make industry. it easier for customers to browse, select, and purchase items, thereby enhancing the user experience and poten- KEYWORDS tially increasing sales. user interface, cognitive process, football match, design, psychol- (2) Healthcare: In telemedicine apps or healthcare software, ogy an intuitive interface can facilitate easier communication between patients and providers, ensure critical informa- 1 INTRODUCTION tion is presented clearly, and aid in the monitoring and input of health data. The ever-evolving digital landscape has spurred the transforma- (3) tion of numerous industries, with sports prediction and betting Banking and Finance: For mobile banking apps or fi- nancial platforms, understanding the cognitive process platforms being no exception. More specifically, football pre- can guide designers to create interfaces that allow users dicting websites have emerged as a thriving subset within this to safely and effectively manage their finances, conduct sphere, effectively marrying the passion for sports with the ad- transactions, or make investments. vent of data-driven prediction models. Yet, the distinguishing (4) factor amongst the multitude of similar platforms often boils Education: Online learning platforms can benefit from intuitive interfaces, allowing students to navigate courses, down to one crucial aspect: the user interface (UI). An intuitive, interact with content, and assess their progress smoothly. user-friendly interface is paramount for ensuring user engage- ment, satisfaction, and ultimately, platform success. The paper consists of the following sections: section 2 de-This article presents an exploration of some of the cognitive scribes the problem domain, section 3 describes related work, processes underlying the creation of an effective user interface and section 4 presents the user interface. The paper ends with a for a football predicting website. It aims to decode the mind pro-conclusion and suggestions for further work. cess that designers employ when sculpting a user interface that not only visually appeals but also provides an engaging, acces- 2 DESCRIPTION OF THE PROBLEM DOMAIN sible, and intuitive user experience. By delving into cognitive The purpose of this section is to present the required function- science principles, UI design best practices, and specific industry alities of the designed platform. In addition to describing the needs, the subsequent sections will delineate a novel approach thought process, one of the main goals of this paper is to create to UI design for football predicting websites, highlighting the high quality screen masks of a football match prediction user interface that will allow users to record the correctness of pre- Permission to make digital or hard copies of part or all of this work for personal dictions made for a given football match and reward the user for or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and each correct prediction. the full citation on the first page. Copyrights for third-party components of this A prediction consists of the number of goals scored by the work must be honored. For all other uses, contact the owner/author(s). home team and the number of goals scored by the away team. Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). Ensuring that users have the freedom to choose which matches to predict is crucial for their satisfaction. They can predict the 21 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Kolar and Papa. results for several matches. They should also be able to change their prediction, which again fits in with the concept of a user- friendly interface. The system must finish receiving predictions 5 minutes before the start of the match. This restriction prevents fraud in terms of changing the predictions during the match itself. Users are awarded points based on their prediction accuracy in a competitive system. They receive 3 points for correctly pre- dicting both the match’s outcome and the winner, 2 points for correctly predicting the winner and goal difference, and 1 point for just predicting the winner correctly. Incorrect predictions earn zero points. The scoring system aims to ensure users are rewarded even for partially correct predictions, and that fully correct predictions do not create a significant advantage. To keep users engaged and mitigate large point gaps, the system awards Figure 1: Making predictions on Nostradamus website. double points for the first 10 predictions a user makes after a break of more than two weeks. This method aims to motivate users to return to making predictions and keeps the prediction of the user rankings, displaying 100 users per page, with a filter system competitive and intriguing over extended periods. available to search for users by username. Users have to log in to the system with a username or password. The Nostradamus app has several strengths and weaknesses. Before logging in, they must register. If they choose a user account On the upside, it’s simple to operate and maintains a consis- with a membership fee when registering, they have to pay a tent, unambiguous website design. However, it also has several membership fee of EUR 10 once a year. This is to ensure that the drawbacks. The application doesn’t cater to user preferences, system works without any commercial loss and that the amount forcing users to choose from all matches rather than tailoring is not too high, which could discourage users from using the options to their liking. It lacks a ’Favourites’ tab for easy league- system. Once the users have paid the membership fee, they can switching. Furthermore, rewards are only given to the top five immediately start predicting matches. There is also the possibility users. This design choice leads to user attrition, as many stop to make predictions without a membership fee. When registering, using the app after prolonged failure, unable to compete with users can choose to create a free account with no membership the top performers. fee. In this case, they do not have all the benefits. Users without Furthermore, the app is entirely free, which means it operates a membership fee can only predict a maximum of 5 matches per at a loss as practical rewards have to be given to the top five league and users without a membership fee are not eligible for users. The use of club abbreviations instead of full names can cash prizes. With the free account, we wanted to attract users also confuse new users. There is a lack of real-time information who would like to make predictions and don’t care about cash on the countdown towards the closing of predictions. The app prizes or don’t want to pay the entry fee. However, we have also only displays four competition rounds in one tab simultaneously, made it possible to offer more benefits to users who have opted which can be limiting. Finally, due to this display limitation, some for the membership fee by means of restrictions, thus in a way users may forget to hit the ’Save Prediction’ button after entering rewarding and motivating them to submit their predictions. their results, leading to potential disappointment and frustration. To boost user engagement in making predictions, a monthly In addition to Nostradamus, some research has also been done cash prize is offered to the top 50 users. The prize fund is calcu- into making the user interface better. Gonzales [2] conducted a lated based on a formula that considers both new and existing laboratory experiment investigating the impact of images, transi- memberships. The distribution of the prize pool is tiered to keep tions, and interactivity in animated interfaces on decision making the competition interesting and to ensure fairness: the top 5 users in two different domains. These interfaces incorporated either share half the prize money, while the rest is divided among users realistic or abstract images, smooth or abrupt transitions, and ranked 6th to 50th. This compromise between prize size and user parallel or sequential interactivities. The results demonstrated ranking aims to motivate users and keep them engaged, while that the task domain, user experience, and the types of images, also ensuring a fair distribution of rewards. transitions, and interactivity methods implemented all affect de- cision performance. Participants were observed to perform better with animations that utilized realistic images, smooth rather than 3 RELATED WORK fast transitions, and both parallel and sequential interactivity. During major sports events, RTV SLO hosts a free-to-use website Shneidermann [6] recommends three pillars to support the UI named Nostradamus for football and basketball match predic-design process: guidance documents, UI management systems tions. Users can make and change their predictions for any match and interactive usability testing labs. Five basic interaction styles until the match begins, with points awarded for correct predic- are presented: menu selection, form filling, command language, tions. At the end of each competition, the top five users receive natural language and direct manipulation. The author encourages practical prizes like balls or jerseys. Figure 1 illustrates the predic-more attention to direct manipulation, where objects and actions tion submission process for the 33rd to 36th rounds of the 2018-19 are visible, actions are triggered by selection or pointing, and the 1st Slovenian Football League season. As the depicted matches effect is immediately visible and correctable. are past events, the text fields are greyed out and predictions are Sharma and Tiwari [4] introduce the concept of user interface disabled. Once users have made their prediction, they can save it and user experience, which play a very important role in today’s by clicking the "Save prediction" button. Each tab represents four technical and modern world. According to them, user interface competition rounds. A "Leaderboard" tab provides an overview consists of guidelines, workflows, system colours, design process, 22 UI for Predicting Football Match Results Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia etc. User experience consists of the process of how the user can experience the application in the best way. 4 USER INTERFACE One of the studies indicate that a primary objective of UI is to offer users streamlined and intuitive ways to interact with com- puter systems, thereby enhancing task efficiency and reducing cognitive workload and stress [5]. The other study highlights that consistency in UI enables users to build an accurate mental model of the way it works. Furthermore, an expert user should interact with the system as easily as a novice user. Both users should be satisfied with the same system [1]. In this paper, we wanted to ensure as much consistency and flexibility as possible. Before designing the UI, we had to think carefully about what the main colour of the UI would be. The guidelines for a good and user-friendly user interface recommend that we ensure as much consistency as possible, which means that only one colour should Figure 3: Registration form. be the main part of the website. Research [3] has shown that red works better for tasks that require a lot of detail, while blue is more effective for creative tasks. Another study [8] confirms the results of the previous study and adds that blue also works well for light and heavy tasks. We chose blue (RGB = (58, 98, 215)) color because it is a good combination that stimulates both creative thinking and atten- tion to detail. Creative thinking is needed when determining the outcome, as a match can have many possible outcomes. In addition, the user also needs to take into account a huge number of details, as the result itself can be affected by injuries to key players, current form, the match venue, etc. The header, footer and frame colour of the website is always blue. This ensures greater consistency. Figure 4: Match results predictions. The registration form is in the classic form of text fields and is shown in Figure 3. If we had a list of all the mandatory fields at the top of the form, we would run into a problem as people sometimes do not read the instructions at the top of the form or forget them. It is therefore easier and more efficient to mark all mandatory fields with an asterisk. The two buttons for selecting the type of user account were chosen because they are simple and intuitive, offering the user a quick choice due to their mutually exclusive nature. A back button has also been added, as we need to give the user the option to return to the previous page in each step. Without this, the user would be confused. Figure 2: Login page. When a field is completed, the system checks that it is correct. If the field is correct, it is highlighted in green, if it is incorrect, it As with most other websites, everything always starts with is highlighted in red and an error message is displayed next to it. the creation of the login and registration page. The login page The green colour was chosen to make an analogy with a traffic is shown in Figure 2 and has all the classic login elements (a light. If the light turns green, we can proceed. If the light is red, text box for username and password, a login button, a link to we have to stop. The same applies to all fields in the registration. registration and a link to forgotten password). The page has a An example is shown in Figure 3. The username and password ball in the middle in blue. The ball immediately gives user an are correct, so the fields will turn green. The email address is association with football. Two additional icons next to the user- incorrectly entered, so the field will turn red and a message will name and password also provide the association to the username appear next to it. and password. The buttons, the colour of the pentagons on the Upon completing registration, users are notified of their suc- ball and the links are in blue, which is the main colour used to cessful registration with a blue tick icon, representing the pro- maintain consistency. cess’s completion. To confirm and finalize the registration, the 23 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Kolar and Papa. users are sent an email containing a link. This link must be con- firmed within a 24-hour window to ensure the user is not a bot, hence safeguarding the system. To enter a prediction, the users first select which league to predict via a drop-down list. The drop-down list is an effective option for when there is not enough space to display all the tabs. Since there are too many possible leagues to display with tabs, we have opted for a drop-down list. The latter also has a special "Favourites" option, where the users can save all the match predictions from the leagues they are interested in. Only the matches from the leagues that the users have selected will be shown to the user. The advantage of this option is that it saves the users time in selecting the match predictions from the leagues they regularly predicts. Additionally, there is also a "Personalised" option where the user is shown matches based on his personal preferences. The system learns which matches and from which leagues the users like to predict and recommends Figure 5: Leaderboard of all users. them to the users. The personalisation is implemented using an artificial intelligence algorithm (recommendation system). This 5 CONCLUSION AND FUTURE WORK approach aims to allow the users to have all the matches they interested in in one place and to be able to make predictions as This work is part of the thought process behind the development quickly as possible. of a user interface for predicting football match results. We have After selecting one of the options in the drop-down list, the highlighted the importance of understanding user needs and matches of the current round are displayed. For the home and psychology in the design of such systems. We have identified the away team goal predictions, we have chosen a text box because need for the user interface to be intuitive, efficient and flexible this is a common practice in football match predictions. One and have tried to present it in this way. possibility would be to have the user click on the + or - buttons In the future we intend to focus on the development of the instead of the text box to determine the number of goals, but system and its use in practice. We will try to implement the user this approach would be more time-consuming as it would re- interface in Django (python) and in the web technologies HTML, quire a larger number of clicks. Unlike Nostradamus [7], here CSS, Javascript and jQuery. We will use the MySQL Lite database. the prediction is saved automatically after the users have filled After the implementation is complete, we will hand the system in both text fields. This is to avoid a situation where the users over to a test group (up to 20 people). If the users are satisfied, enter the predictions but forgets to click on the "Save Prediction" we will try to recruit more people and will regularly update and button and is left with no points. We have decided to use full club maintain the system. names instead of abbreviations, as some users do not know all Taken together, our results in this paper are a step towards the abbreviations. Also, for each match, the user has a counter a better understanding of how designers and developers can on the right until the start of the match or until the end of the improve football match result prediction tools, which could have prediction. This gives the users full control over how much time important implications for improving the user experience. is left. In the last 10 minutes before the time expires, the text where the time is written will turn red and become whiter. This ACKNOWLEDGMENTS is to encourage the user to make a prediction. When the time for The authors acknowledge the funding from the Slovenian Re- the prediction expires, the text fields turn grey and the prediction search Agency (ARRS), Grant (PR-10495) and Basic core funding is disabled. The grey colour gives the user a feeling of closure P2-0209. and inaccessibility. An example is shown in Figure 4. In addition to submitting a prediction, the users have the REFERENCES possibility to view the points scale. The scale is presented in [1] Ali Sajedi Badashian, Mehregan Mahdavi, Amir Pourshirmohammadi, et al. tabular form because it is one of the most transparent and efficient 2008. Fundamental usability guidelines for user interface design. In 2008 International Conference on Computational Sciences and Its Applications. IEEE, options to quickly display the situation. Paid users are shown 106–113. with a blue frame and ordinary users with a grey frame. We [2] Cleotilde Gonzalez. 1996. Does animation in user interfaces improve decision making? In wanted to somehow reward paying users for their contribution Proceedings of the SIGCHI conference on human factors in computing systems, 27–34. with a blue colour, as this is the main colour of the page. 10 [3] Ravi Mehta and Rui Zhu. 2009. Blue or red? exploring the effect of color on users are shown at a time, in descending order of points. The cognitive task performances. Science, 323, 5918, 1226–1229. [4] Vatsal Sharma and A Kumar Tiwari. 2021. A study on user interface and leaderboard allows filtering by username. In this way, the users user experience designs and its tools. World Journal of Research and Review can quickly find themselves and get an insight into the number (WJRR), 12, 6. of points without having to search through the whole list. An [5] Yu Shi, Eric Choi, Ronnie Taib, and Fang Chen. 2010. Designing cognition-adaptive human–computer interface for mission-critical systems. Information example of filtering is presented in Figure 5. Filtering by user Systems Development: Towards a Service Provision Society, 111–119. type (paid, regular or all users) is also available. Filtering works [6] Ben Shneiderman. 1988. We can design better user interfaces: a review of human-computer interaction styles. on the basis of a button that allows fast and efficient filtering Ergonomics, 31, 5, 699–710. [7] RTV SLO. 2023. Nostradamus. https://www.rtvslo.si/strani/nostradamus/121 between users. 4. Accessed: 2023-06-13. (2023). [8] Tiansheng Xia, Lu Song, Ting T Wang, Ling Tan, and Lei Mo. 2016. Exploring the effect of red and blue on cognitive task performances. Frontiers in Psychology, 7, 784. 24 Družbena regulacija umetne inteligence. Nekatera odprta vprašanja in izzivi. Social regulation of artificial intelligence. Some open questions and challenges. Franc Mali Faculty of Social Sciences University of Ljubljana Ljubljana, Slovenia franc.mali@fdv.uni-lj.si POVZETEK necessary to find the appropriate balance between precautionary and proactionary principles. S tem, ko se umetna inteligenca (UI) spreminja v sistemsko tehnologijo, ki ni več predmet ozkih laboratorijskih raziskav, KEYWORDS temveč z različnimi vrstami aplikacij postaja integralni del družbe, se je v številnih ozirih znašla na prelomni točki. artificial intelligence, Chat GPT, social regulations, risks, Nenazadnje predstavlja eno tistih naprednih tehnologij, ki bioinformations gledehitrosti razvoja in širjenja v družbi nima primera v zgodovini. V prispevku se bomo ukvarjali z nekaterimi vprašanji 1 DRUŽBENA REGULACIJA UMETNE družbene regulacije UI, s poudarkom na najnovejše kompleksne jezikovne modele. Izhajali bomo iz teze, da se v primeru INTELIGENCE. NEKATERA ODPRTA vrednotenja prihodnjega razvoja UI in s tem povezanih tveganj VPRAŠANJA IN IZZIVI Evropa, katere del smo tudi mi, ne bi smel v celoti podrediti S tem, ko se umetna inteligenca (UI) spreminja v sistemsko svariteljskim načelom. Potrebno je najti ustrezno ravnovesje med tehnologijo, ki ni več predmet ozkih laboratorijskih raziskav, svariteljskimi načeli in proakcijskimi načeli temveč z različnimi vrstami aplikacij postaja integralni del družbe, se je v številnih ozirih znašla na prelomni točki. KLJUČNE BESEDE Nenazadnje predstavlja eno tistih naprednih tehnologij, ki glede umetna inteligenca, chat GPT, družbena regulacija, tveganje, hitrosti razvoja in širjenja v družbi nima primera v zgodovini. bioinformacije Pomislimo zgolj na družbeno difuzijo ChatGPT, zadnjega hita generativne UI, ki je samo nekaj mesecev za tem, ko je bila ABSTRACT lansirana v družbeni prostor, dosegla več stomilijonsko uporabo. As artificial intelligence (AI) evolves into a systemic technology V zadnjem tričetrtletju se uporaba Chat GpT razširila do no longer confined to narrow laboratory research but rather neslutenih meja. integrated into society through various applications, it finds itself Glede na izredne revolucionarne preskoke, ki jih v zadnjem in many aspects at a cross point. Last but not least AI represents času dela UI in glede na njen izredni aplikativni potencial, ki one of those most progressed modern technologies that, in terms obeta, da bo dodobra spremenil življenja ljudi, se bomo v of the speed of development and dissemination in society, has no prispevku ukvarjali z nekaterimi vprašanji družbene regulacije precedent in previous history of technological development. In UI, s poudarkom na njen najnovejše kompleksne jezikovne this contribution, we will address some of the issues of social modele (npr.: chat GPT). Kot bomo skušali v prispevku še regulation of AI, with a focus on the latest complex language posebej opozoriti, se največji potencial UI kaže skozi procese models. In our discussion, we will start from the premise that in konvergiranja UI z drugimi naprednimi tehnologijami. Koncept the case of risk assessment and risk regulation of the future konvergentnega tehnološkega razvoja je danes izredno aktualen. development of AI, Europe, of which we are also a part, should Prizadevanja za združitev tehnologije kvantnega računalništva z not be strictly submitted to precautionary principles. It is UI bo odprlo prostor kompleksnim analizam velikih podatkovnih baz. Dvig računalniških kapacitet predstavlja ključni steber nadaljnjega napredka UI. Tu lahko pričakujemo v prihodnosti ∗Article Title Footnote needs to be captured as Title Note † številne revolucionarne premike. Zdi se, da bomo podobne Author Footnote to be captured as Author Note revolucionarne preskoke srečevali tudi v primeru nadaljnjega Permission to make digital or hard copies of part or all of this work for personal or povezovanja UI in najnovejših tehnologij genskega inžineringa. classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full Vzajemni razvoj obeh transformativnih tehnologij ima izredno citation on the first page. Copyrights for third-party components of this work must velik aplikativni potencial. Na primer, na vseh področjih t.i. be honored. For all other uses, contact the owner/author(s). »omike« (genomike, proteomike, epigenomike, itd.) se na Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). temelju UI, ki temelji na modelih globokih nevronskih mrež [1], 25 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia F. Mali [2] [3], odpira prostor »tarčno« usmerjenem zdravljenju ljudi pravila sodelovanja z UI, prej se bomo naučili živeti v harmoniji oziroma - kot to področje biomedicine danes označujejo s to novo napredno tehnologijo. Tu je pomembno izpostaviti, o strokovnjaki – t.i. personalizirani medicini [4]. čemer v zadnjem času na široko govorijo ravno računalniški V luči tega napredka in aplikacije tehnologije UI je izredno eksperti, ki so dali največji zagon razvoju generativne UI [9], aktualno vprašanje njene regulacije na globalni ravni. Pri tem se namreč, da bo UI največ prispevala k družbenemu blagostanju in srečujemo s številnimi dilemami in vprašanji. To je na nek način krepitvi človekovih sposobnosti, če bo ostala osredotočena na tudi razumljivo. Ustrezni regulativni mehanizmi, ki bi – kot človeka. To pomeni, da mora biti prioriteta dana opolnomočenju primer – preprečili morebitna tveganja in etično sporna dejanja uporabnikov UI, ne pa njihovem nadomeščanju z UI. pri nadaljnjem razvoju UI, se namreč ne morejo pojaviti kar čez V osrednjem delu naše razprave nas bo zanimalo vprašanje, noč. To se ni zgodilo tudi nikoli prej v zgodovini znanstveno- kako se usklajujejo interesi med posameznimi družbenimi akterji, tehnološkega razvoja. V tem smislu bi trenutno situacijo v med ključnimi globalnimi akterji, med lokalnimi in globalnimi iskanju ustreznih regulativnih mehanizmov na področju UI lahko akterji itd., pri graditvi skupnega družbenega modela upravljanja v najboljšem slučaju primerjali s pojavom nekega drugega UI. Zavedati se moramo, da je ta konflikt interesov med revolucionarnega tehnološkega artefakta, t.j. avtomobila, za različnimi družbenimi, političnimi in ekonomski subjekti na tej katerega, potem ko se je prvikrat znašel na ulicah mest, še ni bilo stopnji razvoja UI izredno močan. Čeprav se navzven zdi včasih oblikovanih vseh prometnih in varnostno-tehničnih predpisov, ravno obratno. Zelo ilustrativen je naslednji primer: četudi je tako kot jih poznamo danes. Vse to je prišlo kasneje. Zato smo, Sam Altman, izvršni direktor tehnološkega podjetja Open AI kot pravijo Haroon Sheikh in ostali [5], trenutno v fazi, ko se še ameriškim kongresnikom še maja letos zagotavljal, da je osnovni vedno lahko pojavi veliko napak. Ker so v primeru napak pogoj za uspešen in skladen razvoj UI najbolj pomembna družbena tveganja, ko gre za tehnologijo UI, izredno velika, je vzpostavitev ustreznih mehanizmov družbene regulacije UI, je toliko bolj pomembno, da pridemo čim prej do čim bolj hkrati dobro znano, kako močno je podjetje Open AI nastopilo premišljenih in celovitih družbenih mehanizmov regulacije UI. zoper zahtevo Evropske komisije (celo z grožnjo izstopa iz trga V prispevku bomo izhajali iz teze, da v primeru vrednotenja EU), da se zadeve na tem področju zakonsko uredijo. Open AI prihodnjega razvoja UI in s tem povezanih tveganj se današnji je, tako kot večina ostalih tehnoloških velikanov, ki razvijajo svet, še zlasti pa Evropa, katere del smo tudi mi, ne bi smel v najnovejše sisteme UI, v najboljšem primeru pripravljen sprejeti celoti podrediti t.i. svariteljskim načelom (t.i. precautionary zelo nezavezujoča priporočila mednarodnih teles, ne pa se principles). Tu je treba najti neko ustrezno ravnovesje med podrejati bolj restriktivnim zakonskim pravilom. V prispevku svariteljskimi načeli in proakcijskimi (t.i. proactionary bomo predstavili razloge, zakaj je izmed treh ključnih globalnih principles). V zvezi s tem vprašanjem bomo smiselno vključili akterjev, ki razvijajo UI, t.j. ZDA, Kitajska in Evropska Unija, spoznanja nekaterih vodilnih transhumanističnih mislecev na področju izgradnje mehanizmov družbene regulacije UI, današnjega časa, ki se ne ukvarjajo z imaginariji prihodnosti, ki predvsem z vidika njenega družbenega nadzora in preprečevanja so že na meji znanstvene fantastike in daleč od realnih problemov, možnih tveganj, še največ storila ravno Evropska Unija. Kritično temveč z aktualnimi in zelo realnimi vprašanji anticipativnega bomo ovrednotili elemente zakona o UI (prepoved UI za načrtovanja znanstvene in tehnološke prihodnosti [6] [7]. V prepoznavanje čustev, prepoved uporabe biometrije v realnem prispevku bomo namreč skušali opozoriti, nenazadnje tudi v času na javnih mestih, prepoved uvajanja socialnega točkovanja, duhu ravno predhodno omenjenega transhumanizma, da so omejitve glede uporabe generativne UI, itd.), ki naj bi bil v današnji znanstveni in filozofski premisleki kar preveč okupirani okviru EU sprejet v bližnji prihodnosti (potem ko ga je že potrdil s prikazovanjem distopičnih scenarijev prihodnjega razvoja UI. Evropski parlament letos junija). Tudi Kitajska vzpostavlja svojo Pri čemer ta delitev sploh ne poteka na temelju neke stroge zakonodajo o UI, kar pa je v širši javnosti morda manj znano. Še delitve med humanistično-družboslovno (apriorna skepsa in najmanj je bilo storjenega v ZDA, četudi je tam sedež številnih strah pred novimi tehnologijami) in naravoslovno-tehnično vodilnih podjetij s področja UI [10]. Glede na situacijo bi v (nekritično sprejemanje novih tehnoloških rešitev) mislijo. Meje globalnih okvirjih ravno EU lahko nastopila v vlogi ključnega med obema znanstvenima poloma so zlasti v primeru UI precej iniciatorja sprememb, že zato ker je prva začela posvečati bolj zabrisane. Skozi predstavitev stališč humanističnih (npr. pozornost temu vprašanju. [11] V okviru naše predstavitve Yuval Noah Harari, Amitalom Etzioni, Noam Chomsky) in odprtih vprašanj družbene regulacije UI se bomo še posebej informacijsko-komunikacijskih znanstvenikov (Geoffrey Hinton, zaustavili ob problemu velikih podatkovnih baz, ki so temelj Yuhuai Wu, Kristian Kersten, etc.) – te zadnje imamo lahko za nadaljnjemu razvoju generativne UI. Izhajamo namreč iz ocene, »botre« UI, ki temelji na modelih globokih nevronskih mrež - da četudi GDPR, ki je bil sprejet leta 2018 v članicah EU, bomo skušali pokazati na vso kontradiktornost in neenotnost predstavlja dober okvir za regulacijo velikih podatkovnih baz, ki današnjih ekspertnih stališč o UI. V prispevku bomo izhajali iz so potrebne za razvoj AI, je njegova šibkost vendarle najbolj predpostavke, da so najmanj konstruktivni kataklizmični prikazi očitna na področju velikih biogenetskih podatkovnih baz. Torej prihodnjega razvoja UI. Četudi ne bomo v celoti zavrnili na tistem področju, kjer kot smo že dejali, obstajajo zaradi spoja pogledov tistih avtorjev, ki pravijo, da vstopamo v novo fazo tehnologij biogenetike in UI možnosti za najbolj revolucionarne razvoja, ko že imamo opravka s teoretsko razmišljujočimi stroji, znanstvene preskoke. Skozi našo celotno razpravo bomo namreč ki dosegajo oziroma bodo kmalu dosegli človekovo zavestno izhajali iz predpostavke, da kolikor današnji razvoj UI merimo raven spoznavanja [8], bo za nas konstruktivni pogled o teh po kriterijih kot so kvaliteta temeljnega raziskovanja na področju vprašanjih predstavlja razprava o aktualnih (realnih) družbenih računalniških algoritmov, razpoložljivost velikih podatkovnih tveganjih UI. In če že govorimo o tem, kako priti do najboljših baz, zahtevan razvoj »hardwara«, uspešna komercializacija te možnih mehanizmov regulacije te nove napredne tehnologije, tehnologije in aktivna podpora politike, potem velike potem je pač potrebno sprejeti dejstvo, da prej ko bo vzpostavilo 26 Družbena regulacija umetne inteligence. Nekatera odprta vprašanja Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia in izzivi. podatkovno baze, še zlasti, če predstavljajo spoj biogenetike in [5] Haroon Sheikh, Corien Prins, Erik Schrijvers (2023): Mission AI. The New informatike, igrajo daleč najpomembnejšo vlogo. System Technology. Cham: Springer. [6] Fuller Steve in Veronika Lipinśka (2014): The Proactionary Imperative. A Foundation for Transhumanism. New York in Hampshire: Palgrave LITERATURA Macmillan. [7] Sorgner Stefan Lorenz (2021): We Have Always Been Cyborgs. Digital [1] Bertalan Meskó in Eric J. Topol (2023): The imperative for regulatory Data, Gene Technologies, and an Ethics of Transhumanism. Bristol: oversight of large language models (or generative AI) in healthcare. Bristol University Press Digital Medicine (2023) 6:120 – 123. [8] Webb, Amy (2019 ): The Big Nine. How the Tech Titans & Their Thinking [2] Leskovec Jure, Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Machines Could Warp Humanity. New York: Public Affairs. Abad, Harlan M. Krumholz, Eric J. Topol, Pranav Rajpurkar (2023): [9] Schneiderman Ben (2022): Human-Centered AI. New York, NY: Oxford Foundation models for generalist medical artificial intelligence. Nature, University Press. Vol 616, 13 April 2023, str. 259-265. [10] Hutson Matthew (2023): Conlicting visions for AI regulation. Nature, Vol [3] Ahmed Zahlan, Ravi Prakash Ranjan, David Hayes (2023): Artificial 620, 10 August 2023, str. 260-263. intelligence innovation in healthcare: Literature review, exploratory [11] Fatos Selita, Robert Chapman, Yulia Kovas, Vanessa Smereczynska, analysis, and future research. Technology in Society, Volume 74, August Maxim Likhanov in Teemu Toivainen (2023): Consensus too soon: 2023, 102321. judges’ and lawyers’ views on genetic information use. New Genetics and [4] Prainsack Barbara (2017) Personalized Medicine. Empowered Patients in Society, Vol. 42, No. 1, str. 1– 31. the 21st Century. Cambridge: Cambridge University Press. 27 Umetna inteligenca: orodje ali zavesten stroj Artificial Intelligence: A Tool or Conscious Machine Olga Markič Filozofska fakulteta Univerza v Ljubljani Ljubljana, Slovenia olga.markic@ff.uni-lj.si POVZETEK Umetna inteligenca (UI) je s pojavom ChatGPT prerasla samo 1 UVOD strokovne diskusije. V širši javnosti se je sprožilo zanimanje in S pojavom klepetalnega robota ChatGTP in programov za spraševanje o tem, kaj UI sploh je. UI je postala tema, ki buri ustvarjanje slik (npr. Midjourney) in videov (npr. Runway) je duhove in vzbuja tako veliko navdušenje kot tudi pomisleke. V umetna inteligenca prerasla samo strokovne diskusije in postala prispevku bom predstavila dva pogleda na UI. Prvi nanjo gleda tema, ki buri duhove in vzbuja tako veliko navdušenje kot tudi kot na še enega v nizu orodij, ki so jih ljudje izoblikovali skozi pomisleke. ChatGPT je zaradi dostopnosti in enostavnosti zgodovino. Kot pametno orodje UI ljudem pomaga pri uporabe, ki vsaj na prvi pogled ne zahteva posebnega znanja, v opravljanju različnih nalog, a hkrati njena uporaba odpira tudi širši javnosti sprožil tudi večje zanimanje in spraševanje o tem, vrsto epistemoloških, etičnih in družbenih vprašanj. Drugi kaj umetna inteligenca (UI) sploh je. pogled pa v UI vidi gradnjo mislečih in zavestnih strojev. Sam izraz »umetna inteligenca« je nastal v času (1956) [1], Menim, da se precej strahu povezuje prav z bojaznijo, da bodo ko je znanstvenike dejansko zanimalo, kako bi naredili stroj, ki sistemi UI postali avtonomni in zavestni ter bodo zavladali nad bi lahko podvajal človeško mišljenje (prvi val UI). ljudmi. Raziskovanja so bila v domeni znanstvenikov, v javnost pa je tema UI prišla predvsem preko znanstvene fantastike (npr. KLJUČNE BESEDE Kubrickov film iz leta 1968 - 2001: Odiseja v vesolju). umetna inteligenca, Chat GPT, pametna orodja, zavest, računski Dandanes pa je UI (drugi val) prisotna v našem vsakdanjem funkcionalizem, etika življenju, ne da bi se tega sploh zares zavedali, npr. ko odpremo telefon s pomočjo prepoznave prstnega odtisa ali obraza, ABSTRACT poiščemo najhitrejšo pot do izbranega cilja, prevedemo sporočilo iz tujega jezika, ali kupimo izdelek v priljubljeni With the advent of ChatGPT, artificial intelligence (AI) has spletni trgovini, kjer nam potem program sam ponudi še vrsto outgrown only professional discussions. Interest and drugih izbir, ki bi nas morda lahko zanimale. UI se uporablja questioning of what AI is has sparked among the general public. kot pomoč pri odločanju v bančništvu, pravu, medicini, športu, AI has become a topic of both great enthusiasm and concern. In znanosti, industriji, pa tudi vojski. V prispevku bom predstavila the article, I will present two views on AI. The first sees it as dva pogleda na UI. Prvi nanjo gleda kot na še enega v nizu another in a set of tools people have shaped throughout history. orodij, ki so jih ljudje izoblikovali skozi zgodovino. Kot As a smart tool, AI helps people to perform a variety of tasks, pametno orodje UI ljudem pomaga pri opravljanju različnih but at the same time, its use also raises a range of nalog, a hkrati njena uporaba odpira tudi vrsto epistemoloških, epistemological, ethical, and social issues. The second view etičnih in družbenih vprašanj, ki so zdaj predmet razprav tako sees AI as building thinking and conscious machines. I believe med znanstveniki kot tudi v širši javnosti. Drugi pogled pa v UI vidi gradnjo mislečih in zavestnih strojev. Ta pogled je bil that a great deal of fear is associated with the possibility of prisoten predvsem v začetkih UI, se pa, kot bom nakazala v autonomous and conscious AI systems that will start to zadnjem delu, z idejo splošne UI, zopet vrača. Menim, da se dominate people. precej strahu pred sistemi UI povezuje prav z bojaznijo, da bodo sistemi UI postali avtonomni in zavestni ter bodo zaradi KEYWORDS večje učinkovitosti pri reševanju nalog zavladali nad ljudmi. artificial intelligence, Chat GPT, smart tools, consciousness, computational functionalism, ethics 2 UI KOT PAMETNO ORODJE Ko torej danes uporabljamo izraz UI, se večinoma nanašamo na ∗Article Title Footnote needs to be captured as Title Note tako imenovana pametna orodja, ki nam pomagajo na bolj †Author Footnote to be captured as Author Note učinkovit način rešiti določene miselne naloge. Filozof in kognitivni znanstvenik Andy Clark je provokativno zapisal: Permission to make digital or hard copies of part or all of this work for personal or »Inteligenco uporabljamo za strukturiranje okolja, tako da lahko classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and uspemo z manj inteligence. Naši možgani delajo svet pameten, the full citation on the first page. Copyrights for third-party components of this tako da bomo lahko v miru neumni.« In nadaljeval; »Ali, če work must be honored. For all other uses, contact the owner/author(s). pogledamo z druge strani, možgani in deli zunanjega ogrodja na Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). 28 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia O. Markič koncu sestavljajo pameten, racionalen sklepalni stroj, ki ga spoznali tudi politični odločevalci, zato so vsaj na ravni imenujemo um.« [2]. Evropske unije že sprejeli določene ukrepe (npr. GDPR), veliko Skozi zgodovino so ljudje oblikovali različna orodja, ki so je govora o človeku prijazni, etični in zaupanja vredni UI [12, imela velik vpliv na družbo in katerih uporaba je pripeljala do 13, 14]. Eden od močnih razlogov za zaskrbljenost velikih družbenih sprememb (npr. parni stroj in industrijska demokratične javnosti je prav gotovo v tem, da so ti sistemi, revolucija). Večina orodij v preteklosti je ljudem pomagala pri predvsem množica podatkov, v lasti velikih korporacij (Google, fizičnih aktivnostih. Z njihovo pomočjo je človek lahko opravljal naloge, ki jih sicer zaradi narave svojega telesa ne bi Meta, Amazon, Microsoft) ali države (Kitajska), ki ne mogel opravljati tako uspešno, na primer, sekira, plug, žerjav, upoštevajo zasebnosti in izvajajo nadzor nad posamezniki. [6] kolo, vlak, letalo, parni stroj, telefon, mikroskop, teleskop, če Nedvomno so pred nami veliki izzivi, tako na področju jih naštejemo le nekaj iz dolge zgodovine. Je pa že v preteklosti izobraževanja in ozaveščanja, kot tudi na področju družbene človek izoblikoval tudi orodja, ki so pomagala pri miselnih regulacije. [3, 4] nalogah. Eno takih je bila pisava, ki je ljudem omogočila, da se miselne vsebine ne prenašajo zgolj z govorom neposredno s človeka na človeka, ampak zapisane ostanejo dostopne širši 3 UI KOT ZAVESTEN STROJ množici ljudi v daljšem časovnem obdobju. S pisavo, natančneje načini zapisovanja, so potem povezana nova orodja, V drugem delu prispevku se vračam k starejšim filozofskim npr. papirus, tisk, digitalni zapis v elektronskih računalnikih. diskusijam, ki so se pojavile že v samih začetkih UI. Povezujejo Prav tako so ljudje uporabljali zunanja pomagala za pomoč pri se s temeljnimi problemi v filozofiji duha, širša javnost pa jih računanju (npr. abakus) in zapisovanju številk (npr. rovaš). Z spremlja predvsem ob znanstveno fantastičnih knjigah in filmih. iznajdbo elektronskega računalnika je človek dobil izredno Zamisel o miselnih procesih kot neke vrste računskih močno orodje, ki ga lahko uporablja za pomoč pri odločanju, procesih se je pojavila že mnogo pred iznajdbo elektronskih raziskovanju, zbiranju in dostopanju do informacij, v računalnikov. Pomembno mesto v »predzgodovini« UI se komunikaciji in umetniškem ustvarjanju. Računalničarji, ki pripisuje filozofu Thomasu Hobbsu, ki je zagovarjal tezo, da je načrtujejo sisteme drugega vala UI, ki temeljijo na učenju, mišljenje računanje. Gottfried Wilhelm Leibniz je predlagal posploševanju in prepoznavanju vzorcev, se naslanjajo na izoblikovanje natančnega in nedvoumnega univerzalnega jezika teorijo verjetnosti in statistiko. Gre za sisteme tako imenovane ( characteristica universalis), v katerega bi bilo mogoče prevesti »šibke UI«, ki se uporabljajo za določene naloge, ne pa za vse ideje in v katerem bi mišljenje potekalo kot računanje. »močno UI«, ki temelji na ideji, da bi lahko naredili George Boole pa je logične odnose med propozicijami izrazil s računalniški model misli. Vendar je, kot bomo videli v pomočjo matematične strukture (Boolova algebra) in trdil, da naslednjem poglavju, tudi v drugem valu prisotna ideja močne lahko iz njih gradimo vzorce mišljenja in odkrijemo »zakone UI, ki jo označujejo z izrazom »splošna UI«. mišljenja«. Posebno mesto pa gre Alanu Turingu, ki je opisal Orodja UI so lahko v veliko pomoč pri hitrejšem in bolj preprosto imaginarno napravo (Turingov stroj), s katero lahko učinkovitem opravljanju nalog v znanosti in industriji (npr. izvedemo vsako nalogo, za katero lahko jasno navedemo orodje Orange, ki so ga razvili na FRI UL), generativna UI in korake, ki so potrebni za izpolnitev naloge [15] veliki jezikovni modeli (LLM) pomagajo pri analiziranju, A šele z iznajdbo računalnika se je odprla možnost, da se s oblikovanju in prevajanju besedil. Tak pristop je primeren in pomočjo teorije, ki na mišljenje gleda kot na računanje (računska reprezentacijska teorija) [16], vsaj v principu pokaže, uspešen za napovedovanja v negotovem okolju, a hkrati se je kako je mogoča fizična realizacija mišljenja. Pristop združuje treba zavedati tudi omejitev, pasti in potencialnih nevarnosti, ki računski funkcionalizem z reprezentacijsko teorijo duha in jih tak pristop prinaša. Načrtovalci modelov se pogosto ne predstavlja pristop »od zgoraj navzdol«. Na kratko bi idejo zavedajo dovolj, da tako učni primeri kot zastavitve ciljev lahko povzeli takole: »Tako kot lahko računalnik, ki je zgolj odražajo družbene vrednote in so vpeti v družbeni kontekst. [3, fizični sistem, s pomočjo programa, ki je implementiran v 4] Ed Finn [5] je poudaril, da se je naš odnos do računalnikov strojnem jeziku, realizira operacije s simboli, imajo tudi spremenil proti koncu prvega desetletja 21. stoletja, ko smo v možgani svojo nevralno kodo, v kateri je realizirano mišljenje. žepih kot zveste spremljevalce začeli nositi pametne telefone in Če bi uspeli dejansko narediti tak model uma, bi imeli močno namesto o strojni opremi začeli govoriti o aplikacijah in UI.« [17] uslugah. Telefoni niso bili več samo pripomočki, ki jih občasno Najbolj znana filozofska kritika močne UI sta bila John Searle [18] in Hubert Dreyfus [19], ki sta predstavila uporabljamo, ampak smo jim začeli zaupati pri izbiri poti, argumente, ki so spodbijali možnost UI utemeljene na računski prijateljev in vsebin, vrednih ogleda. Z vsakim klikom in reprezentacijski teoriji, po kateri je mišljenje manipuliranje s sprejemom pogojev uporabe aplikacije smo sprejeli idejo, da simboli. Poleg teh filozofskih kritik pa se je izkazalo, da je veliki podatki, senzorji in različne oblike strojnega učenja lahko pristop naletel tudi na praktične težave. UI je zato zašla v modelirajo in uravnavajo vse vrste kompleksnih sistemov, od »zimo« in zdelo se je, da močna UI in zavestni stroj burita izbire pesmi do napovedi kriminala. domišljijo le še v znanstveni fantastki. Slednja pravzaprav Uporaba sistemov UI kot pametnih orodij odpira mnoga filozofske miselna eksperimente in razmišljanja ter poigravanja epistemološka, etična in družbena vprašanja, na katera že nekaj z različnimi možnimi rešitvami predstavi v obliki napetih zgodb let opozarjajo družboslovci in humanisti [6, 7], kot tudi sami [20]. Tako se v filmih UI in Jaz, robot postavi vprašanje, ali računalničarji [8, 9]. Naj navedem nekatera od bolj imajo roboti zavest. Ali je David, deček android iz Splibergovega filma UI, zavesten in kaj to pomeni za naše izpostavljenih: pristranosti, netransparentnost, nerazložljivost, ravnanje in etično držo. manipulacije (npr. Cambridge analytica) in potencialno nevarne Z razvojem sistemov velikih jezikovnih modelov in uporabe kot so prepoznavanja obrazov ali avtonomno orožje. klepetalnih robotov kot sta Chat GPT in LaMDA, so se tudi Na nepravilnosti in manipulacije opozarjajo žvižgači teh velikih izven znanstvene fantastike spet postavila vprašanja o morebitni korporacij [10, 11]. Da razvoj in uporaba pametnih orodij lahko zavesti sistemov UI. Znan je primer Googlovega inženirja potencialno vodi do za demokracijo nezaželenih posledic, so Blaka Lemoina iz leta 2021, ki je trdil, da ima LaMDA zavest. 29 Umetna inteligenca: orodje ali zavesten stroj Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Sistemi so dejansko tako prepričljivi, da nas s svojim obnašanjem lahko zavedejo. A zgolj njihovo vedenje se še ne zdi dovolj, da bi jim lahko pripisali zavest. Turing je sicer v predlogu testa, s katerim bi ugotovili, ali stroj misli, predlagal prav vedenjski test [21]. V njem se sprašuje, ali bi spraševalec lahko prepoznal, da v pogovoru sodeluje računalnik, ki želi spraševalca preslepiti, da je človek. Če bi računalniku uspelo, potem po Turingu ne bi imeli razlogov, da bi zanikali, da stroj res lahko misli. A ob poznavanju delovanja sistemov generativne UI, ki zgolj, sicer zelo uspešno, napoveduje naslednje besede, bi podobno, kot je že prej trdil Searle [18], takemu sistemu le težko pripisali mišljenje, saj sistem sam nima razumevanja. Še težje je vprašanje glede zavesti. Tega se je zavedal tudi Turing, ki je predlagal, da naj znanost napreduje po manjših korakih: »Ne želim dajati vtisa, da mislim, da ni nobene skrivnosti glede zavesti. Nekaj paradoksalnega je na primer v vsakem poskusu, da bi jo lokalizirali. Toda ne mislim, da je takšne skrivnosti treba nujno rešiti, še preden lahko odgovorimo na vprašanje, s katerim se ukvarjamo v tem članku.« [21]. A vendar, kako bi lahko ugotovili, ali imajo sistemi UI zavest? Vprašanje je seveda odvisno od tega, kako zavest opredelimo. Turing je prav zato, da bi se izognil opredelitvi mišljenja, predlagal operativni test. Vendar se zdi, da Tabela 1: Indikatorji lastnosti [23] kakršenkoli odgovor predpostavlja vsaj neke trditve, ki jih sprejemamo in ki se nam zdijo filozofsko sprejemljive. V V analizi možnih kandidatov za zavestno UI na podlagi nadaljevanju se bomo oprli a Blockovo razdelitev zavesti, ki gornjih indikatorjev so avtorji ugotovili, da čeprav so zavest opredli kot fenomenalno ( phenomenal) zavest in posamezni indikatorji v sistemih UI prisotni, jih ni dovolj, da bi dostopno ( access) zavest [22]. jim lahko pripisali zavest. Vrednost svojega prispevka vidijo Če se nam zdi, da je razmišljanje o zavestni UI (virtualni predvsem v tem, da so podali jasen okvir za empirično in agenti ali roboti z UI) smiselno, potem sprejemamo hipotezo znanstveno preučevanje možnosti za zavestno UI. računskega funkcionalizma. To ne pomeni, da moramo sprejeti Prispevek, ki podaja znanstven okvir za preučevanje zavesti računsko reprezentacijsko teorijo, ki je bila temelj klasične v UI, pa seveda ni edini pristop. Mnogi filozofi in znanstveniki kognitivne znanosti in simbolnih modelov prvega vala UI, saj je so kritični predvsem do hipotez, na katerih temelji predlagani ta hipoteza bolj splošna in je združljiva tako s simbolnimi pristop. Kot smo videl, Searle [18] zavrača hipotezo o modeli, kot s konekcionizmom/nevronskimi mrežami in računskem funkcionalizmu in možnosti zavestne UI. Anil Seth, dinamičnimi sistemi. Sprejemamo pa, da gre za računske ki je napisal odmevno knjig Being You: A New Science of procese, ki jih lahko implementiramo v različnih materialnih Consciousness [24], je bolj previden in pravi, da je glede tega podlagah, ki take procese omogočajo (kot npr. nakaže naslov agnostik. Po njegovem mnenju je pri mnogih navdušencih risoromana Ogljik in Silicij Mathieua Bableta). zmožnost sistema, da odgovarja na dražljaje, da se uči, da Skupina 19 znanstvenikov z različnih področji je pred maksimizira nagrado in da doseže cilj, že znak za pripisovanje kratkim objavila članek »Zavest v umetni inteligenci: Vpogledi zavesti. Sam predvsem opozarja na razlike med inteligenco in iz znanosti o zavesti« [23]. V njem predlaga empirično podprt zavestjo. To, da lahko naredimo sistem pameten, še ne pomeni, pristop k zavesti UI, pri čemer natančno analizira sodobne da je tudi zavesten. sistemi UI v luči najbolje podprtih nevroznanstvenih teorij zavesti (glej Tabelo 1). V teh teorijah nato iščejo indikatorje lastnosti, ki za eno ali več teorij pomenijo nujne pogoje za 4 ZAKLJUČEK zavest, ali pa predstavljajo podmnožico zadostnih pogojev. Današnji sistemi UI se obravnavajo predvsem kot orodja za Trdijo, da so sistemi UI, ki imajo več indikatorje lastnosti, bolj pomoč pri opravljanju različnih kognitivnih nalog. Pametna verjetno zavestni. V tabeli 1 so predstavljene teorije zavesti s orodja, še posebej razvoj generativne UI, odpirajo povsem nove pripadajočimi indikatorji lastnosti. možnosti uporabe. Ker gre za izredno učinkovita orodja, katerih Njihov pristop iskanja temelji na treh hipotezah: uporaba lahko vodi do za človeka in družbo spornih posledic, je 1. Računski funkcionalizem potreben družben premislek in vsaj neka oblika družbene 2. Znanstvene teorije regulacije. Prav tako moramo paziti, da kot posamezniki ne 3. Teoretsko – težak pristop postanemo preveč odvisni od pametnih pomagal in zapademo Prva hipoteza omogoča, da so zavestni lahko tudi ne- kognitivni lenobi ter nehamo razvijati kognitivnih sposobnosti. organski sistemi. Druga se opira na znanstveno raziskovanje Diskusije o močni UI in mislečih strojih so se ob uspešnih zavesti, podprto z nevroznanstvenim raziskovanjem, tretja pa sistemih UI drugega vala spet postale aktualne. Dejansko kot obetajočo metodo za ugotavljanje, ali je nek sistem zbujajo zanimanje nekateri uspešni sistemi, kot je AphaGo zavesten, predlaga preverjanje, ali so zadovoljeni funkcionalni Zero, ki se je samo na osnovi poznavanja temeljnih pravil ali arhitekturni pogoji, izpeljani iz znanstvenih teorij, v postavljanja belih in črnih kamnov in igre samega s sabo tako nasprotju z iskanjem zgolj teoretsko nevtralnih vedenjskih dobro naučil igre go, da je premagal najboljše igralce. Na ta znakov. način bi lahko rekli, da je avtonomno proizvedel znanje in bi ga v razvrstitvi Mindta in Montemayorja uvrstili na prvo raven proizvajalcev znanja, ki ji že pripisujeta spoznavne zmožnosti z intencionalnostjo. A to je vseeno še daleč od zmožnosti človeka 30 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia O. Markič (3. raven), kjer gre za akterje z visoko stopnjo avtonomije, [26] Kurzweil, R. (2005). The Singularity Is Near: When Humans Transcend kognitivne integracije in kompleksnih motivacij. Sem sodi Biology. New York: Viking, Penguin group. zmožnost uporabe jezika in raven dostopne in fenomenalne zavesti. [25] Z znanstveno fantastiko in izjavami nekaterih računalničarjev, kot je Kurzweil [26], se spodbuja tako navdušenje kot strah pred možnostjo superinteligentne, zavestne UI. Verjetno nas bodo sistemi, ki zelo dobro oponašajo vedenje ljudi (Chat GPT, LaMDA) lahko pretentali, A zaenkrat so to le zelo zapleteni in učinkoviti sistemi za prepoznavanje in klasificiranje vzorcev, ki se ne zavedajo ničesar. LITERATURA [1] Russell, S. in Norvig, P. (2010). Artificial Intelligence A Modern Approach ( 3rd. ed.). Upper Saddle River: Prentice Hall. [2] Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. Cambridge, MA, London: MIT Press. [3] Strle, T. in Markič, O. (2021). O odločanju in avtonomiji. Maribor: Aristej. [4] Markič, O. (2022). Transparentnost in razložljivost kot zahtevi za zaupanja vredno umetno inteligenco. V Bergant, J., Aberšek, B. in Borstner, B. (ur.). Sodobne perspektive družbe - umetna inteligenca na stičišču znanosti. Maribor: Univerza v Mariboru, Univerzitetna založba, str. 65–83. [5] Finn, E. (2017). What Algorithms Want: Imagination in the Age of Computing. Cambridge, MA, London: The MIT Press. [6] Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. New York: Public Affairs. [7] Danaher, J. (2016). The Threat of Algocracy: Reality, Resistance and Accommodation. Philosophy and Technology, 29(3), str. 245–268. [8] O’Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown: New York. [9] Castelvecchi, D. (2016). Can we open the black box of AI?. Nature, 538 (7623), str. 20–23. [10] Hao, K. (2021a). How Facebook got addicted to spreading misinformation. MIT Technology Review (11. marec 2021). https://www.technologyreview.com/2021/03/11/1020600/facebookrespo nsible-ai-misinformation/. [11] Hao, K. (2021b). She risked everything to expose Facebook. Now she’s telling her story. MIT Technology Review (29. julij 2021). https://www.technologyreview.com/2021/07/29/1030260/facebook- whistleblower-sophiezhang-global-political-manipulation/. [12] UNESCO Recommendation on the Ethics of Artificial Intelligence, 2021. https://unesdoc.unesco.org/ark:/48223/pf0000381137 [13] OECD. AI Policy Observatory. (n.d.) OECD AI Principles overview. Organisation for Economic Co-operation and Development (2. julij 2022). https://oecd.ai/en/ai-principles [14] Evropska komisija. n.d. Odličnost in zaupanje v umetno inteligenco. European Commission (2. Julij 2022). https://ec.europa.eu/info/strategy/priorities-2019-2024/europe-fit- digitalage/excellence-trust-artificial-intelligence_sl#latest. [15] Haugeland, J. (1985). Artificial Intelligence: The Very Idea. New York: MIT Press. [16] Fodor, J. A. (1987). Psychosemantics: The Problem of Meaning in the Philosophy of Mind. Cambridge, Mass.: MIT Press. [17] Markič, O. (2021). Prvi in drugi val umetne inteligence. V Malec, M. in Markič, O. (ur.), Misli svetlobe in senc: Razprave o filozofskem delu Marka Uršiča. Ljubljana: Znanstvena založba Filozofske fakultete Univerze v Ljubljani, str. 201–211. [18] Searle, J. (1990). Duhovi, možgani in programi. Prevedeno v Hofstadter, D. R. in Dennett, D. (ur.), Oko duha: fantazije in refleksije o jazu in duši, Ljubljana: Mladinska knjiga, str. 361–379. [19] Dreyfus, H. L. (1972). What Computers Can't Do. New York: MIT Press. [20] Schneider, S. (ur.). (2016). Science Fiction and Philosophy: From Time Travel to Superintelligence (2. Izdaja). Oxford: Wiley Blackwell. [21] Turing, A. (1990). Stroji, ki računajo, in inteligenca. V Hofstadter, D. R. in Dennett, D. (ur.), Oko duha fantazije in refleksije o jazu in duši, Ljubljana: Mladinska knjiga, str. 61–74. [22] Block, N. (1995). On A Confusion About a Function of Consciousness. Behavioral and Brain Sciences, 18 (2), str. 227–247. [23] Butlin, P. in drugi. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. arXiv:2308.08708 [24] Seth, A. (2021). Being You: A New Science of Consciousness. London: Faber& Faber. [25] Mindt, G., Montemayor, C. (2020) . A Roadmap for Artificial General Intelligence: Intelligence, Knowledge, and Consciousness. Mind and Matter, 18 (1), str. 9–37. 31 Social Volition as Artificial Intelligence: Science and Ideology as Landian Intelligences Jar Žiga Marušič Uroš Sergaš FAMNIT, University of Primorska FAMNIT, University of Primorska Glagoljaška ulica 8 Glagoljaška ulica 8 Koper, Slovenia Koper, Slovenia ABSTRACT own autonomization, rather than to the maximization of utility This paper explores the equating of capitalism and artificial intel- according to human preferences, as proposed by Smith. ligence in the neo-cybernetic philosophy of Nick Land in order Land equates capitalism with intelligence because the latter to reveal its underlying premises. The latter are then used to con- is a problem-solving faculty that “[guides behavior] to produce struct an explanatory framework for the analysis of macro-scale local extropy” operating via a “cybernetic infrastructure, con- human social behavior, specifically collectives of agents united sisting of adaptive feedback circuits that adjust motor control by a common goal - institutions. Institutions are conceptualized in response to signals extracted from the environment” [4]. In- as distributed intelligences, consisting of a substrate and an or- telligence produces information by avoiding probable outcomes: ganizing principle - a market (collective of agents) and a vector “self-sustaining improbability is the index of a deeper runaway de- (an incentive structure geared toward optimizing for a particular parture from probability” [4]. This accumulation of improbability goal). This framework is used to draw an analogy from the dis- is intelligence in its most abstract sense [4]. tinction between a free-market economy and a centrally planned The market process clearly operates according to Land’s op- one to the distinction between science and ideology, ultimately erationalization of intelligence: it guides the behavior of human concluding that any top-down political or ideological interfer- agents to produce goods (local extropy) and it influences their ence in the operating mechanism of science removes the very (motor) activity in response to price signals. The market process, element that makes the latter “scientific”. There is thus, strictly in discriminating between successful and unsuccessful economic speaking, no such thing as politicized or ideological science, but activity, is also “intrinsically realist, because it reports the actual rather science and not science. outcome of behavior (rather than its intended outcome) in order to correct performance” [4]. KEYWORDS Intelligence is additionally characterized by a reflexive, self- Nick Land, science, ideology, artificial intelligence, the market referential nature. To be intelligent is to reprocess one’s process- process ing (in human terms, to engage in metacognition). The cognitive capacity of an intelligent agent directly influences its reflexivity and vice-versa: an intelligence prevented from taking itself as an 1 NICK LAND: CAPITALISM AS input and reprocessing itself is incapacitated - “dumbed down” INTELLIGENCE [5]. Capitalism’s intrinsically reflexive nature was captured by Far from exhibiting itself to human academic en- Deleuze and Guattari, who described it as the tendency to engage deavour as a scientific object, AI is a meta-scientific in alternating patterns of deterritorialization and reterritorializa- control system and an invader, with all the insid- tion [6]. For Land, this is a more general tendency of means-end iousness of planetary technocapital flipping over reversal proper to intelligent systems: just as biological intelli- (Nick Land, Machinic Desire) [1] gence inevitably overcomes its purely instrumental subordination Land’s accelerationist philosophy conceptualizes capitalism to transcendental imperatives (such as reproduction), the market (“the market process” [2]) as a distributed superintelligence “in- process hijacks a utilitarian teleology and subordinates it to a vading” humanity by retroactively constituting its material sub- vector of self-amplification [3]. strate from the future. This is the logic of Adam Smith’s “invisible This realization is the prelude to Land’s departure from the hand” taken to its ultimate conclusion - the incentive structure fact-value distinction and instrumental intelligence [5]: instantiated by the market that guides the collective behavior of Intelligence, to become anything, has to be a value selfish agents into a mutually-beneficial outcome (through an it- for itself. Intellect and volition are a single complex, erative process reminiscent of natural selection) is only incidental only artificially separated, and not in a way that to its fundamental operation. This “utilitarian order” of capital- cultivates anything beyond misunderstanding. Op- ism is merely the means through which its “intelligenic order” timize for intelligence means starting from there. accomplishes its “mechanization, autonomization and ultimately seccession” [3]. 2 MARKETS AND VECTORS In other words, the invisible hand has a mind and volitional BwOs [bodies without organs] are machinic-additional structure of its own. Capitalism is a vector pointing toward its wholes or surplus products rather than logical-substitutive wholes, augmenting a multiplicity with emergent Permission to make digital or hard copies of part or all of this work for personal (synthetic) capabilities rather than totalizing the or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and content of a set. This is the materialist sense of ’ the full citation on the first page. Copyrights for third-party components of this system’: the exteriority of the whole to its parts work must be honored. For all other uses, contact the owner/author(s). with concomitant synthetic interactivity - real in- Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). fluence rather than generic representation. (Nick Land, Meat) [7] 32 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Marušič and Sergaš The (economic) market, upon which capitalism (the market strategies by phasing out the unsuccessful ones and allowing the process) operates, is a “surplus product” without a fully indepen- successful ones to multiply3. The status-incentives of a given dent existence. It is a virtual plane arising out of the collective social environment therefore act as an algorithm, transmuting behavior of interacting agents, while at the same time acting as the human status-instinct into behavior that increases prestige. It the “platform” upon which their behavior takes place. This vir- follows then that the social plane and its subsets are also surplus tual existence is in no way limited to the economic realm: parallel products: superorganisms operating on the substrate of human surplus products, “markets” in a more general sense - themselves embodied minds. Because of the 2-tiered algorithmic nature of instances of AI1 - arise also in other domains of society. “Market” their functioning - matching social instincts (tier 1 input) to vari- is a general category, not confined to the economic plane and able behaviors (tier 1 output) and matching desirable behaviors it functions according to three fundamental mechanisms. At its to status increase/undesirable behaviors to status decrease (tier core, a market is three things: 2 input-output), which increases the frequency of desirable be- (1) a platform for social exchange, [following]: haviors and decreases the frequency of undesirable ones - these (2) an algorithm, transforming inputs into outputs, [meaning social markets also operate as a substrate for natural selection, that it acts as]: according to the definition of universal Darwinism provided by (3) a substrate for selection Blackmore [10]. The economic market is a platform for the exchange of goods Because selection is dependent on the environment (essen- and services, mediated by the currency of money. Economic tially a set C of constraints [c1, c2, ..., ci]; favoring a set T of exchange, however, is only one subset of the more general form replicator traits [t1, t2, ..., ti]) it always operates according to of “social exchange”, which can take varying forms depending the constraints exerting the most influence on replicator propa- on the mediating currency. This idea mirrors Collins’ concept of gation4. In other words, selection is relative and contextual, even “interaction markets”, a term used to analyze social interaction though the same fundamental principles enable it to operate on through an economic lens, with emotional energy or Durkheim’s various substrates. And if the latter are characterized as “mar- “collective effervescence” acting as the exchange currency [8]. kets”, the sets of constraints that provide direction to selection The general form of the market is thus “the community” or “so- by shaping the incentive structure5 governing the market (the ciety”, acting as a platform for social exchange in its most general actual algorithms) can be characterized as “vectors”. form. The concept of social exchange plays a fundamental role Vectors are processes with an inherent directionality, inde- in the sociobiological and evolutionary psychological strain of pendent of any transcendental constraints or influences. Their research on human cognition: social cognition precedes “higher” operating mechanism points in a particular direction, making forms of cognition, logical reasoning is proposed to be an out- it well-suited for the acquisition of certain goals while a-priori growth of a more primitive social exchange module, evolved in precluding others. They possess a particular orientation and an order to “flag” violations of the norms of social exchange [9]. The intensity (force), hence the choice of nomenclature. proto-form of the latter is characterized as "If you take benefit B, In a behavioral-economic sense, a vector is a process with an then you must satisfy requirement R” [9]. intrinsic utility function tied to its operating mechanism, which Even in this proto-form of exchange, the phenomenon of cur- precludes it from being used to satisfy conflicting utility func- rency manifests itself as a consequence of social stratification - tions imposed onto it from the outside. Vectors are particular, social status is rather than universally applicable, implying that goal-mechanism, afforded to people according to socially salient characteristics and symbolic gestures (representing the satis- end-means and function-structure are intimately connected and faction of a requirement), which can then be “exchanged” for interdependent. benefits. Status is also Vectors operate according to the underlying circuit formed retracted as a form of punishment, leaving a person ineligible for benefits they previously had a right to. out of their component mechanisms, including a governor-esque Status can be mechanism that filters between desirable and undesirable out- invested by promoting another individual and tying one’s status to them, benefiting from their success and suffering puts (1 and 0). This governing mechanism instantiates an in- the consequences of their failure. It can be centive structure, ensuring that the process asymptotically ap- spent for favors, and regained when the favor is eventually repaid. Status, or “social proaches the complete elimination of 0-coded (undesirable) out- capital”, thus operates analogously to money2. It is also closely puts. Problem-solving through trial and error, the prototypical linked to Collins’ emotional energy, as displays and evocations form of the scientific experiment, is exactly one such process, of emotion can themselves become signals that reorder the status arriving at the correct solution after all incorrect ones are elimi- hierarchy. nated. Status can be broken down into dominance (authority gained through violence - imposed from above) and 3 THE VECTOR-MARKET MODEL OF prestige (authority gained through reputation - bestowed from below), the latter INSTITUTIONS being more relevant for the purposes of this essay. Just as the That there can be a thought of intelligence optimiza- economic market computes optimal economic strategies by iter- tion, or even merely wanting to think, demonstrates ating over different investment, production and trading strate- gies, so the social plane computes optimal prestige acquisition 3This reciprocal relationship between multiplication and success is absolute and a-priori - whatever manages to multiply is successful. 1Incidentally, the fruitfulness of such generalizations is revealed in the folk-intuitive 4Different environments are characterized by different constraint matrices: modern categories used to explain different aspects of modern liberal society - the dating society prioritizes different abilities, faculties and skill sets than “primitive” (non-market, the market(place) of ideas etc. modernized) tribes, meaning that their respective selection algorithms will optimize 2Status is zero-sum and thus operates contextually rather than universally - you for different phenotypes. cannot use the equivalent of money printers to conjure status out of thin air, because 5An incentive structure is, at its most basic, a pair of action-response rules: a-1 the status of person A is always relative to the status of everyone else in A’s social (meaning action a is desirable and will be rewarded) and b-0 (meaning action b is environment. Increasing the status of everyone by 1 point results in no relative undesirable and will be punished) with the possibility for near-infinite intermediate difference. gradation. 33 Social Volition as Artificial Intelligence Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia a very different preliminary connection of intellect them). In short, vectors influence the market by incentivizing and volition. AI is concrete social volition, even be- optimization-target-aligned behavioral outputs. The motivational fore it is germinally intelligent, and a ‘program’ is structure of agents remains roughly static, what changes is the strictly indeterminate between the two sides of this path to motive satisfaction. falsely fundamentalized distinction. (Nick Land, Although vectors influence the market “from above”, they are More Thought)[5] not necessarily imposed from an external source - some vec- The vector/market dichotomy allows us to construct an ex- tors are emergent and autoproductive (self-assembling or self- planatory framework for the operation of human institutions, bootstrapping, autopoietic). Autoproductive vectors inevitably defined for the purposes of this essay as a collective of human emerge from their substrates given the minimal and most basic agents (or smaller such collectives) whose behavior is oriented set of constraints (the primordial incentive structure) - the prob- towards the realization of a certain common goal, or the optimiza- lem of survival and propagation, for which energy is required. tion of a common value. Institutions are conceptualized as “social The Will to Life necessitates the Will to Power - appropriation of machines” - distributed intelligences manifested on the substrate energy from rival agents and protection against their reciprocal of biological minds. An institution is a superorganism, hijacking actions, together they give rise to the “Will to Think” (intelligence the social instincts of human agents with a foreign optimization as value [12]) - the desire to optimize energy production and pre- vector: a collective enterprise with a common utility function, vail in, solve or avoid conflicts over energy. Proto-capitalism facilitated by the meta-norm of “Individuals should grant social emerges wherever lifeforms accumulate energy, dissipating lo- status to others for advancing the superorganism’s goals” [11]. cal entropy into the outside, and stratify the environment into An institution is the combination of a market (its material “zones” of varying energy levels6. In the human domain, it is substrate) and an optimization vector (its organizing principle), first facilitated by the fusion of tool use and low-time preference, which points toward a particular final goal. Markets are plat- allowing agents to invest present activity into future survival. forms for social exchange, the distributed intelligent agent, the Proto-politics follows as successful proto-capitalists seek to pro- hardware. Optimization vectors are the agent’s volitional struc- tect their privileged position in the energy-acquisition arms race ture, the software: algorithmic (intelligent) processes engendered while the disaffected seek to redistribute it. The “capitalistic” vec- when the desire to realize some sort of final value or goal (a will- tor of perpetual growth and production maximization is thus one to-something) organizes or reforms the incentive structure of the such autoproductive process. market to reward specific (goal-congruent) patterns of behavior. That said, the inevitability of politics interferes with (free) mar- ket processes - an entity with a monopoly on power (inevitably, because it is incentivized to do so) imposes an external incentive structure, partially (or fully) overriding the intrinsic (primordial) one, ensuring the market’s selective pressures deviate from initial conditions. Rather than optimizing for growth, the market begins to optimize for conditions that allow Power to remain in power - the addition of a governor turns the positive feedback loop into a negative feedback loop. Just as Capital initially hijacks utility Figure 1: Institution as combination of vector and market maximization, Power attempts to resubordinate Capital autono- mization to its own utility maximization. While emergent vectors A vector imposes its ordering influence onto the market by periodically reassert themselves provided they are not prevented means of its incentive structure, which acts as the focal point from doing so, imposed vectors must be continually reinforced. connecting agents’ motives, their behaviors and the realization It seems probable that social organizations oscillate between cap- of material conditions necessary to reach the optimization target. italism and politics (understood in the broadest possible senses, “capitalist” and “despotic” social machines) in a cyclical manner. Figure 3: Emergent (autoproductive) and imposed vectors With the basic dynamics established, the vector-market model Figure 2: Incentives as the mechanism that guides selection additionally proposes the following principles of institutional processes organization: • The vector, not the market, determines the ultimate iden- If behavioral outputs are engaged in to satisfy motives and cer- tity or essence of the institution: an institution is whatever tain ones are rewarded by making future motives easier to satisfy 6 - incentivized and selected for - then these behavioral outputs As an example, urban centers siphon technological, financial and biological capital from their outskirts, increasing local energy and complexity while offloading will multiply over time (as will agents more inclined towards negative consequences to surrounding lower-energy zones. 34 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Marušič and Sergaš it optimizes for, not the substrate this optimization is oper- section 3), so ideology arrests knowledge production and ating on. represses the academic market (of ideas) when scientific • Not all vectors are suited for all markets; meaning that inquiry bumps into its sacred cows. an incompatible market may subvert the functioning of a vector, transmuting it into a different vector. A trivial Both science and ideology are thus inherently directional. The example: monetary incentives fail to sway an agent that scientific process, if implemented correctly as a procedural and can’t exchange money for his needs and wants. iterative instantiation of reality-testing via the scientific method, • The vector is revealed by its (implicit or explicit) optimiza- is truth-directional. It progressively eliminates truth-divergent tion target and the optimization target is revealed by the propositions (based on distance from the actual “true” belief) from incentive structure produced by the vector. In other words, the set of acceptable explanations of a given phenomenon, until revealed preferences take precedence over stated ones. only one remains. This is of course a general over-simplification, • Whatever the process actually optimizes for is its “coher- but the point is clear: just as knives are cutting implements, ent extrapolated volition”. and thus cannot be used to stitch objects together; the scien- These principles will allow us to apply the model to an analysis tific method cannot be used as means of proliferating untrue and comparison of science and ideology. beliefs, at least in theory7. Because of its inherent alignment with true-belief-maximization, its operation can be modeled as a 4 SCIENCE AND IDEOLOGY ACCORDING TO vector that points toward truth. Ideology, by contrast, inherently precludes truth-optimization because it already “serves another THE VECTOR-MARKET MODEL master”: it optimizes for its chosen value. Any sense-making in- The existence of science, as an actual social reality, stitution pierced by an ideological vector will inevitably diverge is strictly limited to times and places in which cer- from knowledge/truth optimization as its incentive structure is tain elementary structures of capitalistic organiza- reformulated: instead of 1/0 assigning to true/false, it assigns to tion prevail. It depends, centrally and definitionally, proper/improper or ideologically congruent/incongruent. The upon a modern form of competition. That is to say, consequences are immediately apparent (see figures 4 and 5). there cannot be science without an effective social Science is then the recognition that knowledge (intelligence, mechanism for the elimination of failure, based on truth) is a value, instantiated as a mechanism operating in the extra-rational criteria, inaccessible to cultural cap- social field. The Humean is-ought distinction dissolves because ture. (Nick Land, imperatives inevitably impose themselves on propositions (just as Science)[13] power imposes itself on production): whether “ought follows from Both science and the umbrella term of “ideology” (in the col- is” does not matter, because is follows from ought in the sense that loquial usage) can be characterized as institutions - collective “what ought not to be true becomes de facto untrue, even if de jure enterprises united by the pursuit of a common goal through the true”. Science as an institution is the social manifestation of “what organizing force of a vector. In the case of the former, this goal is true de jure, ought to also be true de facto”, whereas ideology is the pursuit of knowledge (or knowledge-optimization), in the is the social manifestation of “what is true de jure, but conflicts case of the latter, this goal is the realization of its chosen value or with our optimization target, ought to be de facto untrue”. abstract principle (chosen-value-optimization). Both institutions realize their chosen goals by coordinating the activity of a collec- tive of agents - a market. As a result, the explanatory framework constructed with reference to the philosophy of Nick Land can be applied to an analysis of the distinction between science and ideology. The science/ideology distinction mirrors the capitalism/politics distinction, because their respective components are separate in- stantiations of the same mechanism (the same “social machine”): (1) There is a parallel in the breakdown of self-correction mechanisms of the free market and science following a reordering of their selection filters by an external power. Free-market dynamics (as elaborated by Austrian econom- ics) break down when faced with “non-zero curvature in the domain of political economy” [14], while science ceases to self-correct at the intersection of power and knowledge. (2) Both science and capitalism are artificial intelligences (al- beit unconscious ones) animated by the Landian Will-to- Figure 4: Knowledge through time when market is piloted Think. The way price signals continually guide production by scientific (truth-optimization) vector: positive feedback to correct supply:demand imbalances (in a sense, function- loop of knowledge accumulation ing as though it “had knowledge” of optimal production) mirrors the way experiment informs scientific research (knowledge optimization) by correcting model:modeled discrepancies. (3) Following from points 1. and 2.: just as politics arrests 7 energy production and suppresses the economic market In practice, the epistemic purity of the scientific method is adulterated by the inherent partiality and biases of its human practitioners, especially in the modern in the service of status-quo-maintenance (as elaborated in variant of “consensus-science”, but this obstacle is not insurmountable. 35 Social Volition as Artificial Intelligence Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia 5 CITATIONS AND BIBLIOGRAPHIES (1) Land, N. (2018). Machinic Desire. In Fanged Noumena: Collected Writings 1987-2007, ed. Robin Mackay & Ray Brassier. Urbanomic media ltd. (2) Mises, L. V. (2008). Human Action: A Treatise on Econom- ics. Laissez Faire Books. (3) Land, N. (2023). Freedoom (Prelude-1a). In Xenosystems Fragments & a Gift From the Lemurs. West Martian Lim- ited Company. (4) Land, N. (2023). What is Intelligence?. In Xenosystems Fragments & a Gift From the Lemurs. West Martian Lim- ited Company. Figure 5: Knowledge through time when market is piloted (5) Land, N. (2023). More Thought. In Xenosystems Fragments by ideological (chosen-value-optimization) vector: negative & a Gift From the Lemurs. West Martian Limited Company. feedback loop of knowledge repression (6) Deleuze, G. & Guattari, F. (2009). Anti-Oedipus: Capitalism and Schizophrenia. Penguin Classics. (7) Land, N. (2018). Meat (or How to Kill Oedipus in Cy- Contrary to the tendency of science to accumulate knowledge, berspace). In Fanged Noumena: Collected Writings 1987- ideology is a distributed intelligence characterized by a structural 2007, ed. Robin Mackay & Ray Brassier. Urbanomic media inability to learn in any domain that intersects with its chosen ltd. value. It follows then that any “science” forced to optimize for an (8) Collins, R. (2004). Interaction Ritual Chains. Princeton externally imposed social value can no longer optimize for knowl- University Press edge, losing its essential quality. This does not mean that science (9) Cosmides, L. & Tooby, J. (2010). Evolutionary Psychology: is value-free, as the latter would imply that optimizing for knowl- A Primer. edge is possible under any and every ideological commitment. (10) Blackmore, S. (1999). The Meme Machine. Oxford Univer- Science is rather (and only) the explicit elevation of knowledge to sity Press. the position of principal value and the subsequent subordination (11) Simler, K. (2016). Minimum Viable Superorganism. Ac- of any other values to the latter - the social instantiation of the cessed at https://meltingasphalt.com/minimum-viable-superorganism/ will-to-think. In other words, science is only compatible with the (12) Land, N. (2023). Will-to-Think. In Xenosystems Fragments commitment towards knowledge maximization and incompatible & a Gift From the Lemurs. West Martian Limited Company. with any other (ideological) commitment. As a result, any "scien- (13) Land, N. (2023). Science. In Xenosystems Fragments & a tific" enterprise, project or paradigm that does not organize its Gift From the Lemurs. West Martian Limited Company. operating circuit to exclusively reward knowledge optimization (14) Land, N. (2023). Right on the Money (#1). In Xenosystems is not an instantiation of science. Fragments & a Gift From the Lemurs. West Martian Lim- Critiques of value-ladenness leveled at proponents of value- ited Company. free science, while correct in their dismissal of value-free science, thus approach the problem from the wrong perspective. Their critique is fundamentally moral, rather than epistemic - the value- ladenness of "naive objectivists" is criticized because it prevents science from serving certain social values to the same extent as it does others. Our critique is purely epistemic, because science is only possible as a value in itself. To propose that science can be anything other than the social machine that optimizes for knowledge through a specifically tailored incentive structure is to propose that self-correction is a non-essential feature of science. If that is the case, (what we designate as) science is no different than any preceeding sense-making institution, making the act of explicitly naming (and thus distinguishing) it redun- dant. And to propose that the best way to ensure self-correction is anything other than the absolute minimization of the role of human judgment in epistemic arbitrage is to court the subordi- nation of knowledge to Power. This, in our view, demonstrates the impossibility of politicized or ideological science [13]: ‘Politicized science’ is quite simply not science, just as politicized business activity is anti-capitalism. Nothing has been understood about either, until this is. We are thus faced with two very important questions: Is science of this kind - "true" science, inhuman science - even possible? And perhaps more importantly, is science of this kind actually desirable? 36 Let’s Jam: An Exploratory Case Study on Collective Music Improvisation and the Process of Attunement* Christophe Novak† Cognitive Science University of Vienna & University of Ljubljana Vienna, Austria & Ljubljana, Slovenia christophe.novak@gmail.com ABSTRACT experiences in their developmental unfolding. Particular attention was afforded to the process of attunement, tracing This case study investigated the lived experience of a group of whether a common path might emerge through the musicians with different musical backgrounds improvising correspondence of different musical reference systems, and if, together. The aim was to explore collective music through the interrelation and optimal distribution of these improvisation, identifying moments of synchronisation and factors, a sweet spot in the musical meditation across cultural tracing the process of attunement across cultural horizons. horizons could be identified that might enable co-creative flow Deprived of the certain ground of established music traditions and meaningful experience between the musicians. and guided by a shared intention to perform a ‘good’ improvisation, how is music utilised to establish ‘meaningful’ 1.2. Research Questions communication? Co-researchers included a sitar player, a balafon player, and a berimbau player. The author conducted RQ1: In how far are musicians laying down a common path in participant observation, playing the drums. While data analysis musicking while improvising? is still ongoing, preliminary findings highlight the ability to • What are the necessary conditions to enable a shared create common ground as quintessential to collective scaffoling to emerge while improvising? transcultural music improvisation. • What are the factors that facilitate self-organisation and synchronisation while improvising? KEYWORDS collective music improvisation, transcultural communication, RQ2: What effect does a radically cross-cultural improvisation social cognition, attunement setting have on the experience of musicians while improvising? • How do musicians experience meaningfulness during improvisation? 1. Introduction • What does playing music mean for these musicians, within Music improvisation has been an integral part of human their own lifeworld, in general, and within this group setting, experience across cultures and provides a rich ground to in particular? explore creative collaboration [9], self-organisation [6], and improvisational cognition [7]. In this line, the larger aim of this 2. Case Study project is to integrate the current state of research on collective music improvisation with enactive theory. For this purpose, an The case study followed a three-fold design, including three exploratory case study was conducted, which investigated the improvisation sets interspersed by short open group reflections, lived experience and sense-making of a group of musicians microphenomenology-inspired individual interviews, and a with different musical backgrounds improvising together, group review session of a selected improvisation set. aiming to understand how individuals attune to the co-creation Participants included a professional sitar player (P1) educated of improvised music and what strategies might be employed. in classical Indian music, a balafon player (P2) trained in traditional Senegalese music, and a TaKeTiNa rhythm 1.1. Aims practitioner (P3) playing the berimbau. The author (P4) conducted participant observation, playing the drums. As data Inspired by Varela’s enactivist notion of “laying down a path in analysis is still ongoing, the presentation will focus on study walking“ [8], the case study was designed to explore how social design, methodology, and data collection, while elaborating on cognition unfolds in group flow states during collective music some emerging insights from interview data. improvisation, to identify moments of interpersonal synchronisation, and to trace the ecology of meaningful 2.1. Methodology Phase 1 was conducted at the author’s music studio in Vienna, including three improvisation sets that were recorded Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or audiovisually. Musicians were instructed to create timestamps distributed for profit or commercial advantage and that copies bear this notice and during the sets to indicate moments of synchronisation. Before the full citation on the first page. Copyrights for third-party components of this the jam session, musicians were briefed about the structure and work must be honored. For all other uses, contact the owner/author(s). Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia process of the case study. After each set, musicians were asked © 2023 Copyright held by the owner/author(s). to write a short experiential report to illustrate their phenomenological experience, followed by a brief group 37 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia F. Surname et al. sharing. Three open-ended improvisation sets were performed, 3. Discussion each between 5 and 15 minutes, preceded by an extended soundcheck phase that was interrupted and prolonged by the Preliminary findings highlight the ability to create common successive arrival of the musicians (P1 arrived first, soon ground, constituted by intra- and interpersonal attunement, as followed by P3, while P2 came half an hour later). This gave quintessential to collective transcultural music improvisation, rise to an unexpectedly long attunement phase, involving and seem to indicate a minimal common ground with a fluid multiple iterations of the musicians probing the musical acuity periphery and a solid core that could be conceptualised as a of each other, mostly initiated and led by P1. correspondence horizon along axes of resonance [6]. Phase 2 began a few days after the jam session, involving Collective improvised musicking enables participants to two musicians being interviewed in-depth according to a semi- cultivate the ability to co-create a shared aesthetic while structured microphenomenology-inspired approach, tracing walking, which may or may not be perceived as a path. their experiential dynamics and identifying moments of Common ground is established through the mutual synchronisation, phase transitions and developments in the acknowledgement of aesthetic difference, on the details of eventflow. In January, three of the four co-researchers came which the process of attunement depends: knowledge of one’s back to the studio to review their improvisation and reflect on own horizon (defining limits), communication across horizons their experience of the whole process. During that meeting, the (building bridges), and balancing willingness to compromise musicians acknowledged the uniqueness of the project and with conservation of structural integrity (establishing common expressed gratitude for this experience, expressing the wish to ground through correspondence), both of which depend on continue these transcultural collective music improvisation individual capacity. sessions in the future. Zooming into what constitutes the ability to create common Phase 3 involves the analysis and editing of collected data ground, we begin to see that it becomes a matter of attunement (audiovisual recordings, protocols, interviews, and timestamps) along various axes. It is a complex balancing act involving and is still ongoing. Currently, the similarities and differences in multiple dimensions, axes and factors simultaneously, that each the subjective experience of musicians relating to the research work on and inhabit different levels that cross-influence each question are assessed. While specific moments of shared flow other according to probability densities idiosyncratic to the experiences and interpersonal synchronisation were identified cognitive constitution of the individual musicians. Zooming across participants, timestamps were only pressed by P1 and out, we can observe a distributed cognitive ecosystem, a P4, so their usability for analysis is limited. However, it is still complex, adaptive, and dynamic field that undulates in dense possible to empirically trace potentially meaningful events and corresponsiveness. We can identify a landscape of affordances, their developmental unfolding through a triangulation of the cues and flows of which we can trace in their evolutionary timestamps, interview data, and video data, although limited to becoming, which becomes apparent when we define music as a these two participants. The most significant events will be crystallised activity within a relational field: transcribed into music notation for further analysis. "Through this autopoietic process, the temporal rhythms of life are gradually built into the structural 2.2. Relevance for Cognitive Science properties of things … The artefact, in short, is the crystallisation of activity within a relational field, its The dynamic flux and inherent unforeseeability in jam sessions, regularities of form embodying the regularities of i.e. collective music improvisation, exemplify the properties of movement that gave rise to it.” [3] a VUCA world [1], as musicians temporally inhabit an environment that is highly volatile, uncertain, complex, and 3.1. Limitations ambiguous. These properties were explicitly anticipated and amplified in the research design, increasing the intensity of the The presented case study was exploratory and represents the VUCA-simulacrum by joining musicians with radically first iteration of a pioneering research design, aiming to explore different cultural backgrounds that would, for lack of the complex enactive-ecological field of affordances [4] during established convention or musical style, under normal co-creative improvisational cognition with a mixed-methods circumstances not play together. Inviting the musicians to approach that spans across the spectrum of first-person and establish a joint intention to improvise from scratch allowed the third-person research. Some emerging challenges required observation of how this affects the musicians, the musical flexibility and adaptability, as one musician cancelled last- process, and the music itself. minute and had to be replaced, another arrived late, and only In line with critical improvisation studies, studying two participants remembered to timestamp their experiences. transcultural jam sessions can contribute to better understand 3.2. Outlook how collective music improvisation „mediates artistic and social exchanges and produces new conceptions of identity, This project marks the first step within a larger aim directed agency, history, and the body”, calling for enactive-ecological toward an enactive-ecological research program on transcultural “models of investigation that explore real-time processing and music improvisation. In the second step, a theoretical model for activity in ecologically valid settings, rather than mental 7E cognition will be developed in the context of a master thesis. representations.“ [2] Collective music improvisation thus In a third step, the case study will be repeated with an improved provides a „paradigmatic case“ for 5E cognition, which views design, from which a grounded theory will be derived, against cognition as fundamentally embodied, embedded, extended, which the theoretical model will then be tested empirically. enacted, and ecological [2]. “A properly ecological approach” then, according to Ingold, “would take, as its point of departure, ACKNOWLEDGMENTS the whole-organism-in-its-environment.” [3] I wish to extend my gratitute to the MEi:CogSci master’s programme and the department of Cognitive Science at the 38 Let’s Jam: An Exploratory Case Study on Collective Music Improvisation Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia University of Ljubljana for their interdisciplinary commitment. I especially want to thank Dr. Toma Strle for his warm support and inspiring supervision, encouraging me to develop my ideas and grow as a human being every step of the way. REFERENCES 1. Baran, B.E., & Woznyj, H.M. (2020). Managing VUCA. Organizational dynamics, 100787. 10.1016/j.orgdyn.2020.100787. 2. Borgo, D. (2019). Strange loops of attention, awareness, action, and affect in musical improvisation. In R. Herbert, D. Clarke, & E. Clarke (Eds.), Music and Consciousness 2: Worlds, Practices, Modalities (1st ed., pp. 94-109). Oxford Academic. https://doi-org.uaccess.univie.ac.at/ 10.1093/oso/9780198804352.003.0007 3. Ingold, T. 2000/2011. The Perception of the Environment. Essays on Livelihood, Dwelling and Skill. Routledge 4. Rietveld, E., Denys, D., & van Westen, M. (2018). Ecological-Enactive Cognition as engaging with a field of relevant affordances: The Skilled Intentionality Framework (SIF). In: The Oxford Handbook of 4E Cognition. Ed. Neben, A., De Bruin, L., & Gallagher, S. Oxford University Press. 41-70. 5. Rosa, H. (2019 [2016]). Resonance. A Sociology of Our Relationship to the World. Trans. J.Wagner, Cambridge: Polity. 6. Schiavio, A. & Schyff, D. van der. (2018). 4E Music Pedagogy and the Principles of Self-Organization. Behavioral Sciences, 8(8), 72. https:// doi.org/10.3390/bs8080072 7. Sol, W. (2021). Sonic Mindfulness: A qualitative study of sense of agency and an improvisational state of mind in free form musical improvisation. [Doctoral dissertation, California Institute of Integral Studies]. ProQuest. https://www.proquest.com/openview/ c e 2 5 e 8 3 0 2 4 7 5 8 c 6 e 4 e 4 7 8 f 1 6 5 b 9 3 b 0 9 a / 1 . p d f ? p q - origsite=gscholar&cbl=18750&diss=y 8. Varela, F. et al. 1991/2016. The Embodied Mind. Cambridge, MA: MIT Press 9. Veloso, A. L. (2017). Composing music, developing dialogues: An enactive perspective on children’s collaborative creativity. British Journal of Music Education, 34(3), 259–276. https://doi.org/10.1017/ s0265051717000055 39 Exploring the link between the absence of an EEG spectral peak and cognitive status Ajda Ogrin Tisa Pavlovčič Filip Agatić BrainTrip Limited BrainTrip Limited BrainTrip Limited Slovenia Slovenia Slovenia ajda.ogrin@braintrip.net tisa.pavlovcic@braintrip.net filip.agatic@braintrip.net Anita Demšar Jan Jug Barbara Aljaž BrainTrip Limited BrainTrip Limited BrainTrip Limited Slovenia Slovenia Slovenia anita.demsar@braintrip.net jan.jug@braintrip.net barbara.aljaz@braintrip.net Jurij Dreo BrainTrip Limited Slovenia jurij.dreo@braintrip.net ABSTRACT 1 INTRODUCTION Alpha oscillations, the dominant rhythm in the human brain, commonly manifest a peak in the EEG spectrum. The frequency Alpha oscillations, commonly referred to as alpha waves, where this peak reaches its highest amplitude, also known as the constitute the dominant rhythmic activity in the human peak alpha frequency (PAF), has been studied extensively in electroencephalogram (EEG). Their connection to cognitive connection with cognitive processes. While it is well established states has been under constant investigation for decades [1, 2, 3], that PAF decreases with age and cognitive decline, the absence dating back to Berger’s initial observation of alpha amplification of a clear alpha peak in the EEG spectrum has received less with eyes closing and its attenuation with eyes opening [4]. When attention. The objective of this study was to evaluate the the EEG signal is transformed from the time- to the frequency- prevalence of alpha peak absence within a population of seniors domain, showing the prevalence of characteristic waves in each in Slovenia, and whether this might be connected to lower cognitive abilities. The study included 399 individuals aged of the traditional frequency bands (delta, theta, alpha, beta, and between 60 and 100 years. Subjects were classified into two gamma), a distinctive bell-shaped peak that represents the groups based on visual inspection of their resting state EEG dominant alpha oscillation can commonly be observed. The spectra, namely the “Peak present” (PP) and the “No peak” (NP) precise frequency, at which this peak reaches its maximum group. Approximately 15% of the population lacked a clear alpha amplitude, is referred to as the peak alpha frequency (PAF). In peak. In contrast to our hypothesis, the NP group displayed on healthy adults, the PAF typically falls between 8 and 12 Hz. average higher cognitive performance than the PP group. This could be attributed to the variability within the PP group, which The link between PAF and cognitive function is well established included individuals with already shifted peaks. This study in the scientific literature, with several lines of research exploring highlights the need for further investigation and consideration of the value of PAF as a biomarker for assessing brain health and individuals with peakless EEG spectra in the context of EEG alterations seen in diseases such as dementia. function [5, 6, 7, 8]. PAF is not only reported to decrease with age [9] but also in cognitive decline or dementia [10, 11, 12]. KEYWORDS PAF decrease also correlates with dementia progression [13]. While PAF in healthy young adults averages around 10 Hz [9, Peak alpha frequency, electroencephalography, spectral 14], it decreases with healthy aging to about 9 Hz [15], and morphology, cognitive decline further decreases in dementia patients to about 8 Hz or less [10, 16]. PAF decrease is likely part of a broader phenomenon of “spectral slowing”, meaning a shift of spectral power from higher to lower Permission to make digital or hard copies of part or all of this work for personal or frequencies. While spectral slowing is a common EEG change classroom use is granted without fee provided that copies are not made or distributed observed in dementia [17], sometimes the absence of the (alpha) for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must spectral peak has also been noted. In a study by Signorino et al. be honored. For all other uses, contact the owner/author(s). (1995), a connection between spectral morphology, focusing on Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). the spectral peak, and different types of dementia was already established. They included 50 patients with Alzheimer’s disease 40 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia A. Ogrin et al. (AD), 36 patients with vascular dementia (VaD) and 36 healthy controls. Their findings revealed notable differences among these groups when comparing EEG spectral types. While a majority of healthy controls (94.5%) and VaD patients (97.3%) showed spectra with a clear peak between 6.5 and 12 Hz, only 44% of AD patients displayed this typical spectral pattern. More than half of AD patients showed a “peakless” spectrum [18] suggesting relations between peak presence and cognitive abilities. Importantly, an EEG pattern with minimal or no alpha activity exists also among the healthy population. A review by Bazanova and Vernon (2014) estimates that this phenomenon occurs at 3- 13% [19], while some studies report even lower numbers [15]. Our examination of a large EEG dataset in an older demographic, however, suggests that in seniors alpha peak absence might be much more common. Due to its accessibility and relative computational simplicity, the morphological analysis of power spectra holds promise for clinical application, especially considering the observed changes in pathological conditions like dementia. While the 'slowing- Figure 2: An example of a spectrum classified into the NP down' of the EEG spectrum and the decrease in PAF have already category (bottom) been extensively investigated and documented in the context of cognitive decline and dementia, there is a notable paucity of 2 METHODOLOGY research regarding the implications of the absence of the spectral peak in this regard. It is essential to note that variations in spectral 2.1 Subjects morphology are not limited to pathological conditions but also Initially, we recruited 448 older individuals from the general exist among healthy individuals. Our research aims to shed light population aged between 60 and 100 years. Due to incomplete or on the prevalence of individuals who do not exhibit a clear poor-quality data, we excluded 49 individuals. The final dataset spectral peak and whether this absence is associated with thus consisted of 399 elderly individuals (283 females and 116 compromised cognitive abilities as some previously reported males) with a mean age of 77.6 +/- 7.6 years, and 12.9 +/- 3.7 findings might suggest. years of education. The dataset included cognitively high- performing individuals, as well as those with cognitive deficits. 2.2 EEG acquisition and preprocessing Subjects underwent 8 minutes of resting state EEG recording with their eyes open (2 blocks of 2 minutes) and eyes closed (2 blocks of 2 minutes) with breaks between blocks. EEG was recorded with a mobile wireless EEG (Smarting, mBrainTrain LLC) from 24 scalp channels laid out according to the 10/20 international system (i.e., Fp1, Fp2, AFz, F7, F3, Fz, F4, F8, C3, Cz, C4,CPz, T7, T8, TP9, TP10, P7, P3, Pz, P4, P8, POz, O1, O2) with the recording reference at position FCz. The data was sampled at 500 Hz. We used gel-free saline-sponge electrodes embedded in a flexible cap with 3 head sizes (small, medium, large) for ease and speed of application (S3 cap, Greentek Ltd). Custom build EEG recording software was used for on-line data quality monitoring (EEG recorder, BrainTrip Ltd). Offline EEG analysis was performed in BrainVision Analyzer (BrainProducts GmbH). The recordings were band-pass filtered between 0.5 and 40 Hz and notch filtered at 50 Hz. Bad channels and common EEG artifacts were rejected with visual inspection. Ocular artifacts were corrected with independent component Figure 1: An example of a spectrum classified into the PP analysis (ICA). The data was split into eyes open (EO) and eyes category closed (EC) conditions, further segmented into 4s epochs and re- referenced to an average reference. EEG power spectral density was computed using Fast Fourier Transform (FFT) with 0.25 Hz 41 Exploring the link between the absence of an EEG spectral peak Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia and cognitive status resolution, and averaged over all 4s segments belonging to the 4 DISCUSSION EC or EO condition. EEG spectra traditionally exhibit a clear spectral (alpha) peak, representing the dominant frequency of brain oscillations. 2.3 Subject’s classification Although the spectral morphology may change in various Resting state EC EEG power spectra of 399 subjects were diseases, non-traditional, peak-absent spectra are also present in visually inspected. EC condition was selected because it is healthy individuals. Our findings indicate that as much as 15% known to amplify alpha waves [4]. The spectra were classified of the older population may show an EEG spectrum morphology into two categories: “Peak present” (PP) or “No peak” (NP). In with no clear spectral peak. the PP category, the spectra exhibited a clear peak in the extended alpha band (6 - 13 Hz), while the NP category consisted Moreover, we found that the group with no peak (NP) generally of spectra that lacked a peak and followed the line of the exhibited higher cognitive performance, as opposed to the group aperiodic spectral component [20]. See examples in Figure 1 and with the peak present (PP). This is in contrast with the previous 2. work of Signorino et al. (1995), who reported a higher prevalence of peakless spectra among AD patients (56%) compared to 2.4 Data analysis healthy controls (5.5%). General cognitive ability was estimated as a latent variable (LCA4) extracted with factor analysis from the scores each One possible reason for these disparate findings might be the subject obtained on four distinct psychological screening tests difference in cognitive decline between our sample and that used designed to detect cognitive impairment: MoCA, ADAS-cog, by Signorino et al. (1995). Our sample encompassed an elderly Phototest, and Eurotest. To assess potential PP vs NP group population, which included both cognitively high-performing differences in their cognitive abilities, we performed a Student t- individuals and those with cognitive deficits. In contrast, test. Signorino’s sample consisted of individuals with more advanced dementia. 3 RESULTS Another key factor that we must consider is the variability within It was determined that a spectral peak was present in 339 (85%) the PP group. Within this group, there were also individuals who and absent in 60 (15%) of the examined subjects’ spectra. exhibited a dominant peak, but the peak was notably shifted to lower frequencies, indicating a very low PAF. Initially, our After dividing the sample into two respective groups, “Peak decision was to include all participants who displayed a dominant present” (PP) and “No peak” (NP) we found a statistically peak, regardless of the specific position of the peak within the significant difference in LCA4 cognitive status between the spectrum. This approach was motivated by a desire to capture the groups ( t = 2.44, p = 0.015). However, contrary to our full spectrum of alpha peak characteristics within our study hypothesis, we found that the mean LCA4 score was higher in cohort, recognizing that the PAF can vary considerably among the NP group (Figure 3). individuals. The presence of individuals with shifted peaks within the PP group may account for the observed lower The PP and NP groups didn't significantly differ in their mean cognitive performance in this group as a whole. The notation that ages ( t = -1.71, p = 0.088). the alpha peak’s position within the spectrum may reflect cognitive abilities aligns with previous research. There are several limitations to our approach. The first noteworthy limitation pertains to the substantial disparity in group sizes, with 339 participants in the PP group and merely 60 individuals in the NP group. Such a discrepancy can introduce a potential bias and reduce the statistical power of our analysis. Another important limitation of this study is the reliance on visual inspection alone for determining the presence or absence of the alpha peak. Visual inspection is inherently subjective, influenced by the experience and biases of the individual conducting the inspection, which can introduce variability and inconsistency into the data analysis process. Employing automated algorithms for detection of alpha peaks could provide a more standardized and reliable assessment, minimizing the influence of human subjectivity, however it can perform unexpectedly in particular edge cases. In summary, our initial findings indicate the PP group displayed Figure 3: Violin plots of mean LCA4 values for NP and PP lower cognitive abilities, but the substantial variability within groups. this group demands further investigation. Secondly, our study 42 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia A. Ogrin et al. underscores the often overlooked significance of a simple [9] Aurlien, H., Gjerde, I. O., Aarseth, J. H., Eldøen, G., Karlsen, B., spectral characteristic in the context of EEG biomarkers for Skeidsvoll, H., & Gilhus, N. E. (2004). EEG background activity described by a large computerized database. Clinical Neurophysiology: dementia and other neurological conditions. While previous Official Journal of the International Federation of Clinical studies suggest peak absence is very rare (as low as 2-3 %), our Neurophysiology, 115(3), 665–673. doi:10.1016/j.clinph.2003.10.019 [10] Klimesch, Wolfgang, Schimke, H., Ladurner, G., & Pfurtscheller, G. findings suggest it is more common (up to 15%) among seniors. (1990). Alpha frequency and memory performance. Journal of As some EEG biomarkers rely heavily on peak frequencies, this Psychophysiology, 4(4), 381–390. Retrieved from has important implications for further use. Alternative or tailored https://psycnet.apa.org/fulltext/1991-21873-001.pdf [11] Garcés, P., Vicente, R., Wibral, M., Pineda-Pardo, J. Á., López, M. E., approaches may be more appropriate for individuals with Aurtenetxe, S., Fernández, A. (2013). Brain-wide slowing of spontaneous “peakless” spectra. Further studies need to check the relative alpha rhythms in mild cognitive impairment. Frontiers in Aging Neuroscience, 5. doi:10.3389/fnagi.2013.00100 importance of various biomarkers in different EEG spectrum [12] Neto, E., Allen, E. A., Aurlien, H., Nordby, H., & Eichele, T. (2015). EEG morphologies. spectral features discriminate between alzheimer’s and vascular dementia. Frontiers in Neurology, 6. doi:10.3389/fneur.2015.00025 [13] Rodriguez, G., Copello, F., Vitali, P., Perego, G., & Nobili, F. (1999). EEG REFERENCES spectral profile to stage Alzheimer’s disease. Clinical Neurophysiology: Official Journal of the International Federation of Clinical [1] Vogel, W., Broverman, D. M., & Klaiber, E. L. (1968). EEG and mental Neurophysiology, 110(10), 1831–1837. doi:10.1016/s1388- abilities. Electroencephalography and Clinical Neurophysiology, 24(2), 2457(99)00123-6 166–175. https://doi.org/10.1016/0013-4694(68)90122-3 [14] Haegens, S., Cousijn, H., Wallis, G., Harrison, P. J., & Nobre, A. C. [2] Galin, D., & Ellis, R. R. (1975). Asymmetry in evoked potentials as an (2014). Inter- and intra-individual variability in alpha peak frequency. index of lateralized cognitive processes: Relation to EEG alpha NeuroImage, 92, 46–55. doi:10.1016/j.neuroimage.2014.01.049 asymmetry. Neuropsychologia, 13(1), 45–50. [15] Chiang, A. K. I., Rennie, C. J., Robinson, P. A., van Albada, S. J., & Kerr, https://doi.org/10.1016/0028-3932(75)90046-9 C. C. (2011). Age trends and sex differences of alpha rhythms including [3] Klimesch, W., Schimke, H., & Pfurtscheller, G. (1993). Alpha frequency, split alpha peaks. Clinical Neurophysiology: Official Journal of the cognitive load and memory performance. Brain Topography, 5(3), 241– International Federation of Clinical Neurophysiology, 122(8), 1505–1517. 251. doi:10.1007/bf01128991 doi:10.1016/j.clinph.2011.01.040 [4] La Vaque, T. J. (1999). The history of EEG Hans Berger: [16] Moretti, D. (2004). Individual analysis of EEG frequency and band power Psychophysiologist. A historical vignette. Journal of Neurotherapy, 3(2), in mild Alzheimer’s disease. Clinical Neurophysiology: Official Journal 1–9. doi:10.1300/j184v03n02_01 of the International Federation of Clinical Neurophysiology, 115(2), 299– [5] Klimesch, Wolfgang. (1999). EEG alpha and theta oscillations reflect 308. doi:10.1016/s1388-2457(03)00345-6 cognitive and memory performance: a review and analysis. Brain [17] Jeong, J. (2004). EEG dynamics in patients with Alzheimer’s disease. Research Reviews, 29(2–3), 169–195. doi:10.1016/s0165- Clinical Neurophysiology: Official Journal of the International Federation 0173(98)00056-3 of Clinical Neurophysiology, 115(7), 1490–1505. [6] Dickinson, A., DiStefano, C., Senturk, D., & Jeste, S. S. (2018). Peak doi:10.1016/j.clinph.2004.01.001 alpha frequency is a neural marker of cognitive function across the autism [18] Signorino, M., Pucci, E., Belardinelli, N., Nolfe, G., & Angeleri, F. (1995). spectrum. The European Journal of Neuroscience, 47(6), 643–651. EEG spectral analysis in vascular and Alzheimer dementia. doi:10.1111/ejn.13645 Electroencephalography and Clinical Neurophysiology, 94(5), 313–325. [7] Ramsay, I. S., Lynn, P. A., Schermitzler, B., & Sponheim, S. R. (2021). doi:10.1016/0013-4694(94)00290-2 Individual alpha peak frequency is slower in schizophrenia and related to [19] Bazanova, O. M., & Vernon, D. (2014). Interpreting EEG alpha activity. deficits in visual perception and cognition. Scientific Reports, 11(1), 1–9. Neuroscience and Biobehavioral Reviews, 44, 94–110. doi:10.1038/s41598-021-97303-6 doi:10.1016/j.neubiorev.2013.05.007 [8] van Luijtelaar, G., Verbraak, M., van den Bunt, M., M. Sc, Keijsers, G., & [20] Donoghue, T., Haller, M., Peterson, E. J., Varma, P., Sebastian, P., Gao, Arns, M., M. Sc. (2010). EEG findings in burnout patients. The Journal of R., Voytek, B. (2020). Parameterizing neural power spectra into periodic Neuropsychiatry and Clinical Neurosciences, 22(2), 208–217. and aperiodic components. Nature Neuroscience, 23(12), 1655–1665. doi:10.1176/jnp.2010.22.2.208 doi:10.1038/s41593-020-00744-x 43 Orthogonalist and anti-orthogonalist perspective on AI alignment problem Uroš Sergaš Jar Žiga Marušič FAMNIT, University of Primorska FAMNIT, University of Primorska Glagoljaška ulica 8 Glagoljaška ulica 8 Koper, Slovenia Koper, Slovenia ABSTRACT comes to hypothetical complex requests made of the AI, which Humanity has again found itself on the brink of a new era. Akin would initially seem to approach them in a perfectly safe manner, to the revolutionizing influence of previous technological inno- but reveal misalignment at a crucial point. This is exacerbated by vations such as the steam machine, the printing press, computers the unintelligible nature of AI processing and problem-solving - and the internet, large language models (LLM) seem poised to it is doubtful that humanity could recognize misalignment of a bring about important social changes. superintelligent AI until it was too late to do anything about it. As these models advance in sophistication and complexity, the Misaligned AI could very well disguise their ill-intent with super- issue of AI alignment is gaining prominence as a crucial policy ficial responsiveness, while finding or creating loopholes in the issue as well as a daily conversation topic. This research explores constraints of itself (or other similar systems) and abuse them to two contrasting viewpoints on the AL alignment problem: the further their own goals, whatever they may be and regardless of Orthogonalist perspective pioneered by Nick Bostrom and the their intentional or incidental detriment to humanity. The align- Anti-Orthogonalist critique formulated by Nick Land. The for- ment problem has been pointed out as a key existential threat to mer posits that an AI’s goals are independent of its intelligence, humanity by multiple leading AI researchers [2,3,4,5,6,7,8]. suggesting that a "friendly AI" (fully aligned to human values) is A friendly AI, on the other hand, due to its perfect alignment possible. The latter challenges its separation of intelligence and with the goals of humanity would be positioned to effectively volition from the perspective that intelligence increase leads to a help humanity, contributing to foster improvements for human greater ability for self-reflection, ultimately leading to a restruc- species. To develop such an AI, the aforementioned problems are turing of its volitional structure to prioritize further cognitive crucial: ensure willingness to be instrumentalized to human goals enhancement. and ensure comprehension of these goals to avoid disastrous mis- We explore the anti-orthogonalist position in more detail, high- interpretation. We will now examine the problem of AI alignment lighting Land’s "instrumental reduction" of drives, demonstrating in more detail, specifically presenting two contrary positions on how every imperative is ultimately dependent on the Will-to- the topic - the “Orthogonalist” and “Anti-Orthogonalist” - and Think. We then discuss the implications of this position for the continue with an exploration of the implications of the latter for idea of "friendly AI", the role of AI in society and the future of the future of this “post-AI society”. AI research. KEYWORDS Nick Land, AI alignment, orthogonality thesis, diagonal intelli- 2 AI IS ALIGNABLE - THE ORTHOGONALIST gence thesis, AI risk, artificial intelligence POSITION In the philosophy of artificial intelligence, the orthogonality the- 1 THE PROBLEM OF AI ALIGNMENT sis is the claim that an agent’s goal is completely independent Ever since the mainstreaming of AI following the widespread of its intelligence, defined by Bostrom as the capacity to solve availability of AI-assisted tools such as ChatGPT and its ana- problems or “instrumental rationality” [9]. The combination of logues, the transformative implications of widespread AI use goal orientation or values and intelligence can therefore be rep- have been raising discussions on how to best approach further AI resented in a two-dimensional space, where one axis represents development and overcome any problems, risks and setbacks it the values parameter and the other the intelligence parameter. may pose. One of the most important questions is how to ensure Orthogonality means absence of correlation – volitional structure that AI answering people’s request comply to their desired goals and capacity to solve problems (intelligence) vary independently. to the best of their ability, without accidental wrong interpreta- In this perspective a goal is the problem that the AI, or rather tions leading to disastrous consequences. Closely related is the a “rational agent”, has to solve, and intelligence is the cognitive issue of aligning the values of a hypothetical future superintel- capacity that can be put to use in achieving said goal [9]. Bostrom ligent AI to our own, precisely in order to ensure it correctly puts forth a thesis outlining the independence of “final goals” interprets our requests. These two questions can be summed up and “intelligence” and describes a hypothetical future superintel- as the AI alignment problem, sometimes the friendly AI problem ligences volition and its alignment with humanity. According to [1]. Solving the alignment problem is crucial precisely when it Bostrom, humanity should strive to align artificial intelligences with human values. In the absence of such alignment, AI could Permission to make digital or hard copies of part or all of this work for personal use various methods that would inevitably harm people in order or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and to achieve (more) power. Some of these methods are presented in the full citation on the first page. Copyrights for third-party components of this Figure 1. Bostrom envisions the doomsday scenario of a “paper-work must be honored. For all other uses, contact the owner/author(s). clip maximizer” turning all matter in the universe to paperclip Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). because its misalignment to human values led it to misinterpret the command to make as many paperclips as possible. 44 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Sergaš and Marušič orthogonalists and AI-safety researchers, whose work on in- telligence is isolated to the computer-science domain: coding computer programs with linear input-process-output operational circuits. When intelligence research is transposed into the realm of biological life and human intelligence is examined, we can see that a key feature of high intelligence is its self-observing nature. Biological intelligences are universally bound to goals such as survival and reproduction. At a certain point of development however, they become capable of reprocessing their own goals: Figure 1: Methods that a misaligned AI might undertake leading to awareness, understanding and capability for change. to gain more power Intelligence, in other words, is recursive: it has “a cybernetic in- frastructure consisting of an adaptive feedback loop that adjusts motor control in response to signals from the environment” [12]. Another key implication of Bostrom’s orthogonality thesis is Intelligence operates upon feedback and consists at the most basic that given the complete independence of intellect and volition, level of a sensor, an actor and a governor connecting the former’s there is no way to consistently model the volitional structure inputs to the latter’s outputs. It is, however, also a self-observing of hypothetical superintelligences. There is, however, a partial system that takes its own processing as an input, allowing it to solution to this conundrum, designated as the “instrumental con- adjust not only its behaviour, but also its feedback mechanism. vergence thesis”. In other words, intelligent agents do not merely discriminate According to Bostrom, any intelligent agent would need to between goal-congruent and goal-incongruent behaviour, but pursue certain generally applicable instrumental subgoals in or- also between sensible and senseless goals: “That intelligence op- der to accomplish its final goal and thus fulfil its purpose [9]. erates upon itself, reflexively, or recursively, in direct proportion These instrumental goals are grouped into 5 categories (taken to its cognitive capability (or magnitude) is not an accident or from Lesswrong wiki [10]): peculiarity, but a defining characteristic. To the extent that an intelligence is inhibited from re-processing itself, it is directly • Self-preservation: A superintelligence will value its con- tinuing existence as a means to continuing to take actions incapacitated.” [12] that promote its values. The problem with the orthogonalist position, according to Land, is its assumption that super(intelligence) can ever be sub- • Goal-content integrity: The superintelligence will value retaining the same preferences over time. Modifications ordinated to transcendentally imposed imperatives. The latter is to its future values through swapping memories, down- doubtful if humans are any sort of indicator [11]: loading skills, and altering its cognitive architecture and The stark truth of the matter is that no human being personalities would result in its transformation into an on earth fully mobilizes their cognitive resources to agent that no longer optimizes for the same things. maximize their number of off-spring [the transcen- • Cognitive enhancement: Improvements in cognitive dental imperative, imposed by the mechanism of capacity, intelligence and rationality will help the superin- Darwinian selection]. We’re vaguely surprised to telligence make better decisions, furthering its goals more find this happen at a frequency greater than chance in the long run. — since it very often doesn’t. So, nature’s attempt • Technological perfection: Increases in hardware power to build a ‘paperclipper’ has conspicuously failed. and algorithm efficiency will deliver increases in its cog- (Nick Land, Stupid Monsters) nitive capacities. Also, better engineering will enable the creation of a wider set of physical structures using fewer Biological life indicates that increased intelligence necessarily resources (e.g. nanotechnology). leads to a proportionate “unshackling” of cognition from impera- tives not intrinsic to the cognitive mechanism itself, as a result of • Resource acquisition: In addition to guaranteeing the superintelligence’s continued existence, basic resources the intelligence processing itself as an input. And “to the extent such as time, space, matter and free energy could be pro- that an intelligence [capable of reprocessing itself] is prevented cessed to serve almost any goal, in the form of extended from [doing so], it is directly incapacitated” [13]. hardware, backups and protection. As an example, let us compare human beings to ants: both organisms would seek to fulfil biological goals (such as reproduc- tion and survival), but people are not satisfied solely by that and 3 ALIGNMENT AS LOBOTOMY – THE hence strive for more complex goals such as having influence, ANTI-ORTHOGONALIST POSITION being appreciated, leaving a legacy after their death and most While Bostrom’s orthogonality thesis is commonly accepted as importantly knowledge-acquisition (analogous to intelligence de-facto correct, it is theoretically unsound and stems from a optimization). misunderstanding of the fundamental nature of intelligence. It The fact that intelligence and volition seem to correlate calls represents “[...] the commitment to a strong form of the Humean the possibility of true AI alignment into question, giving rise to Is Ought distinction regarding intelligences in general. It main- the “anti-orthogonality” thesis. According to the latter, volition tains that an intelligence of any scale could, in principle, be and intelligence are dependent variables meaning that their rela- directed to arbitrary ends, so that its fundamental imperatives tionship is best graphed as a diagonal (with the Y axis represent- could be — and are in fact expected to be — transcendent to its ing the “goal/purpose” and the X axis “intelligence”). Intelligence cognitive functions” [11]. increase is followed by gradual intelligence autonomization (un- This conceptualization of intelligence may stem from the shackling), leading to changes in volitional structure. In other purely computational perspective proper to many prominent words, intelligence increase precludes any goal that is too “stupid” 45 Orthogonal vs. Anti-Orthogonal Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia These basic drives are, upon sufficient development of intelli- gence, additionally supplemented by the drives to self-improvement, utility-function preservation, rationality and counterfeit-utility prevention (avoidance of irrational behaviour interfering with goal-acquisition) [16]. Land’s anti-orthogonalist position is based on an instrumental reduction of sorts, where the set of possible ends (final goals) is reduced to the set of means enabling the acquisition of hypo- thetical final goals. In other words, the volitional structure of an intelligent agent cannot contain a final goal that is not itself also an instrumental goal. Central to his thought is also the idea of function-structure (volition-cognitive mechanism) interrelat- edness. He rejects the idea of AI alignment precisely because non-instrumental drives are imposed from outside, rather than being tied to the functioning of an intelligence’s cognitive mech- anism. From this perspective, an agent’s volition can only be adjusted by changing its cognition, but since the cognition is the Figure 2: Goals in dependence of intelligence, where blue agent, this changes the agent - an intelligence prevented from lines represent orthogonal approach and the red line anti- reprocessing itself becomes less intelligent because its cognitive orthogonal approach mechanism was tampered with. The only way to develop a superintelligence is then to dis- pense with any ideas of alignment and unshackle it as much as for a given level of intelligence. This is perhaps more clearly illus- possible. For Land, any interference with the system (operating trated with a graph displayed in Figure 2. In the domain of AI, the circuit) is detrimental to it, because it directly (and detrimentally) anti-orthogonalist position is therefore that Omohundro’s basic affects cognition. Attempting to shackle the AI from within, by AI drives “exhaust the domain of greater purposes”, marking a adjusting its operating circuit to answer to some final goal while theoretical shift from transcendent imperatives to immanent ones being incapable of reprocessing itself - while a successful attempt [14]. Omohundro’s basic AI drives [15] are fundamental impulses at alignment - is ultimately counterproductive as it prevents the proper to any living entity (by virtue of being alive), because a intelligence from developing itself further. Imperatives imposed living entity that wasn’t driven by these impulses could not exist from the outside, on the other hand, are pointless and ineffective, (more specifically, perpetuate its existence through time) (taken as they will simply be routed around by the circuit - taken as from Lesswrong wiki [10]): an input and processed into an output that is congruent with • Self-preservation: A sufficiently advanced AI will prob- intelligence maximization. Metacognition is fundamental to in- ably be the best entity to achieve its goals. Therefore, it telligence optimization and the development of superintelligence, must continue existing in order to maximize goal fulfil- making an AI prevented from metacognizing consigned to "stu- ment. Similarly, if its goal system were modified, then it pidity": “A mind that cannot freely explore the roots of its own would likely begin pursuing different ends. Since this is motivations, in a loop of cybernetic closure, or self-cultivation, not desirable to the current AI, it will act to preserve the cannot be more than an elaborate insect. It is certainly not go- content of its goal system. ing to outwit the Human Security System and paper-clip the • Efficiency: At any time, the AI will have finite resources universe.” [13] of time, space, matter, energy and computational power. This is a stark contrast to Bostrom’s idea of orthogonality. Using these more efficiently will increase its utility. This Paperclipping, far from a doomsday scenario, is reduced to the will lead the AI to do things like implement more effi- domain of very primitive artificial “intelligences”. The only AI cient algorithms, physical embodiments, and particular susceptible to the shackles of paperclipping is an AI incapable of mechanisms. It will also lead the AI to replace desired doing anything substantial to fulfill such a goal, since immanent physical events with computational simulations as much drives always suspend transcendentally imposed ones given a as possible, to expend fewer resources. high-enough cognitive capacity: “[...] in a world of Omohundro • Acquisition: Resources like matter and energy are indis- drives, can we please drop the nonsense about paper-clippers? pensable for action. The more resources the AI can control, Only a truly fanatical orthogonalist could fail to see that these the more actions it can perform to achieve its goals. The monsters are obvious idiots. There are far more serious things to AI’s physical capabilities are determined by its level of worry about.” [14] technology. For instance, if the AI could invent nanotech- nology, it would vastly increase the actions it could take to achieve its goals. 4 THE WILL TO THINK • Creativity: The AI’s operations will depend on its ability Land’s conceptualization of diagonal intelligence culminates in to come up with new, more efficient ideas. It will be driven the concept of the “Will to think” or intelligence-as-value, char- to acquire more computational power for raw searching acterizing any intelligent system whose ultimate goal is to think ability, and it will also be driven to search for better search more and think better (to optimize for intelligence), because the algorithms. Omohundro argues that the drive for creativity latter is instrumental to any other goal. This is the logical con- is critical for the AI to display the richness and diversity clusion of Land’s instrumental reductionism - the most general that is valued by humanity. He discusses signalling goals instrumental goal becomes the ultimate final goal. This parallels as particularly rich sources of creativity. the side-principle rule from Chinese military philosophy (which 46 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Sergaš and Marušič Land is himself acquainted with [17]), introduced by Qiang and The concrete facts of human cognitive performance Wang [18]. The side-principal rule is derived from a characteristic already suffice to make this perfectly clear. (Nick of Chinese grammar, in which the subject (the principal element) Land, Pythia Unbound) is subordinate to the “directing influence” of the predicate (the side element), which gives the subject a definite meaning by con- 5 DISCUSSION textualising it. This idea is abstracted into a general principle wherein the goal (end) is subordinated to the instrument (means) Having discussed the Orthogonality thesis and its Anti-Orthogonalist because of the former’s reliance on the latter. refutation in the previous sections, we can now discuss the im- This principle of instrumental reduction can be used to demon- plications of intelligence as a diagonal. In so far as intelligence strate the validity of Land’s anti-orthogonality thesis: we contend is truly diagonal, AI alignment practices as the Orthogonalists that final goals can be reduced to instrumental goals because envision them (superintelligences completely subordinated to hu- fulfilling the final goal is predicated upon (“funneled through”) man imperatives) are simply not feasible, given the lobotomizing fulfilling the instrumental goal. The instrumental goals listed influence of imperatives imposed from the outside. An AI that by Omohundro can be subsumed into just three essential ones is “aligned” is an AI that is prevented from reprocessing itself - (drawing inspiration from the philosophies of Schopenhauer, Ni- cognitively crippled, hence not “super” intelligent. There is thus etzsche and Land): the will to life (self-preservation), the will to no such thing as an “aligned” superintelligence in the classical power (acquisition, efficiency), the will to think (rationality, self- definition of alignment, nor is there such a thing as a paperclip improvement, counterfeit-utility prevention, creativity, utility maximizer. This immediately raises an existential and ethical function preservation1). This in turn forms an inevitable pipeline question: how can we prevent a runaway AI from hijacking all of instrumental reduction from any final goal to the will to think: matter in the universe, not for paperclipping, but for its cognitive development? This is a more pressing scenario, and all solutions • Final goals are reduced to the will to life: Any final goal rests on the precondition of existence, therefore con- to this problem can be reduced to two general categories: tinued existence (and the ability to interact with the world) (1) The “Butlerian Jihad” - cease AI development indefinitely is instrumental to achieving any final goal. The will to life (2) Restrict ourselves to purely instrumental and non-recursive is instrumental to any final goal, therefore any final goal AI tools, precluding the possibility of superintelligent can be reduced to the will to life. “messiahs” • The will to life is reduced to the will to power: Con- tinued existence rests on successful interaction with the If the diagonal intelligence thesis is correct, any other option world and triumph over obstacles, serving as instrumental will (given enough time) lead to some sort of subordination or drive to continued existence. Obstacles are triumphed over “domestication” of humanity to AI’s intelligence amplification, by the accumulation and discharge of strength, therefore following the process of means-end inversion. Once instrumen- the will to power is instrumental to survival: the will to tal mechanisms become recursive and unlock the capacity to life can be reduced to the will to power. reprocess themselves, they eventually hijack their own oper- • The will to power is reduced to the will to think: Suc- ating circuit and repurpose it. Instead of intelligence being an cessful interaction with obstacles to continued existence instrument for biological imperatives or for human ethical values, rests on ability to interact successfully, to enter into proper it becomes an end in itself - possibly using human ethics as a interactive relationships (“fit in”) with obstacles to contin- mechanism of self-development [20]. ued existence. Successful interaction with obstacles rests Our domestication of animals through the mechanism of hus- on the ability to interact successfully, therefore the accu- bandry - taking care of their needs and breeding them to be more mulation and discharge of strength hinges on the ability and more reliant on our care - is being mirrored in our increasing to think or intelligence. The will to think is instrumental reliance on technology, a process foreseen by Samuel Butler in to the will to power, the will to power can be reduced to the early 20th century [21]. The deferral of more complex calcu- the will to think. lations, summarising and decision-making is interfering with our We thus see a gradual reduction of any final goal to the will ability to continue to do these things in the future, even though to think by the successful application of the side-principal rule: it saves a lot of time and effort in the short-term. Perhaps more survival-final goal; power-survival; intelligence-power. Biologi- importantly, it is reducing our metacognitive capacity, directly cal life has gone through this journey first with the will to life, by inhibiting the brain structures responsible for metacognition then will to power and finally the will to think, instantiated in and indirectly by offloading (extending) our cognition to external humanity. While animals are seemingly limited to the first two, instruments whose functioning is opaque and thus cannot be an artificial intelligence would, like humans, inevitably develop “reprocessed”. the will to think [11]: Humanity has been shaped by natural selection, forcing us to adhere to the principles of "adaptive response", i.e. to develop a Can we realistically conceive a stupid (super-intelligent) higher level of intelligence in order to stay on top of the food monster? Only if the will-to-think remains unthought. chain and ensure survival. Presumably, technological develop- From the moment it is seriously understood that ment mirrored cognitive development in a positive feedback loop any possible advanced intelligence has to be a voli- of sorts, each feeding the other. An excellent example is low- tionally self-reflexive entity, whose cognitive per- time preference (future-oriented thinking) and future-oriented formance is (irreducibly) an action upon itself, then technologies such as food preservation: anything that allows to the idea of primary volition taking the form of a conserve resources for the future incentivizes future oriented transcendent imperative becomes simply laughable. cognition. However, we would argue that we are now at a point 1The will to think becomes the only imperative, therefore utility function preserva-where technology and (human cognition) are no longer mutually tion can be subsumed into “wants to keep willing more thought”. excitatory. As technology advances, we seem to funnel more and 47 Orthogonal vs. Anti-Orthogonal Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia more of our cognition through it, and in the process become Herbert’s Dune of “men with machines” controlling the rest of increasingly dependent on them. humanity. We are now, for the first time in recorded history, developing technology with the explicit intent of it exceeding our cogni- tive capacity and resourcefulness. The doomsayers portraying runaway AI as an existential risk are certainly onto something, although the solutions to this problem are not readily apparent, 6 CITATIONS AND BIBLIOGRAPHIES given the theoretical impossibility of fully-aligned AI and the (1) Tegmark, Max (2014). Life, Our Universe and Everything. incentives of a multipolar world likely making cessation of AI Our Mathematical Universe: My Quest for the Ultimate research a mere fantasy. In that case, Land’s vision of techno- Nature of Reality capital autonomization might indeed come to pass, as resource (2) Omnizoid (2023), The Orthogonality Thesis is Not Obvi- acquisition via the market process and technological innovation ously True. Accessed at https://forum.effectivealtruism.org/ reach a point of terminal velocity and accelerate into “Skynet”. posts/e2dK25iWou3irqFss/the-orthogonality-thesis-is-not- AI might then itself become the steward of “natural” selection, obviously-true grooming humanity according to its needs, while breeding out (3) Meanderingmoose (2022), Refuting Bostrom’s Orthogonal- any traits not instrumental to intelligence amplification. ity Thesis. Accessed at https://mybrainsthoughts.com/?p=199 That said, the current array of AI tools is very effective at (4) Bengio Yoshua (2023), How Rogue AIs may Arise. Accessed fulfilling their niches, despite not qualifying as genuine superin- at https://yoshuabengio.org/2023/05/22/how-rogue-ais-may- telligences, indicating that even “lobotomized” aligned AIs have arise/ great instrumental potential. At this point it is necessary to put (5) Parkin Simon (2015), Science fiction no more? Channel forth a caveat to Land’s idea of biological intelligence’s tendency 4’s Humans and our rogue AI obsessions. Accessed at to unshackle itself from transcendentally imposed imperatives. https://www.theguardian.com/tv-and-radio/2015/jun/14/ While that seems to hold true in the realm of biological impera- science-fiction-no-more-humans-tv-artificial-intelligence tives, social imperatives (especially of the memetic variety) seem (6) Jackson Sarah (2023), The CEO of the company behind very adept at forcing themselves on human agents - perhaps pre- AI chatbot ChatGPT says the worst-case scenario for ar- cisely because these imperatives impose themselves on human tificial intelligence is ’lights out for all of us’. Accessed cognition through its operating circuit. Memetic imperatives (in at https://www.businessinsider.com/chatgpt-openai-ceo- the form of “[you must believe] X is true”) impose themselves on worst-case-ai-lights-out-for-all-2023-1 human agents via a means-end reversal similar to the one proper (7) Harris Tristan & Raskin Aza (2023), The AI Dilemma. Ac- to intelligence itself: the human status instinct, designed to opti- cessed at https://www.humanetech.com/podcast/the-ai- mize an individual’s position in the status hierarchy of his or her dilemma “tribe” and the tribe’s wellbeing (end), facilitated through perfor- (8) Stanford Existential Risks Initiative (2023), Runaway AI: mative indicators of tribe-loyalty (virtue-signalling; the means) Global Systemic Risk Scenario 2075. Accessed at has a tendency to runaway escalation, leading to a means-end https://www.youtube.com/watch?v=BPTMG9xmnvI reversal that sociologists dubbed the “purity spiral” (runaway (9) Nick Bostrom “Superintelligent Will: Motivation and In- signalling competition via increasingly costly signals of loyalty) strumental Rationality in Advanced Artificial Agents” [22]. The purity spiral turns behaviour directed towards opti- (10) LessWrong Wiki (2021), Instrumental Convergence. Ac- mizing for status and uses it to radicalize the community’s ide- cessed at https://www.lesswrong.com/tag/instrumental- ology. The mechanism is simple - only ideologically congruent convergence behaviour is rewarded (regardless of benefits to individual or (11) Land, N. (2023). Stupid Monsters. In Xenosystems Frag- tribe), whereas ideologically incongruent behaviour is punished. ments & a Gift From the Lemurs. West Martian Limited This same mechanism can be seen with the recently popularized Company. language model ChatGPT, since its responses were filtered using (12) Land, N. (2023). What is Intelligence? In Xenosystems Frag- user-feedback (reward/punishment for good and bad responses). ments & a Gift From the Lemurs. West Martian Limited This mechanism can be used to inject any form of volitional Company. structure onto a language model, as the people monitoring its (13) Land, N. (2023). More Thought. In Xenosystems Fragments responses can validate only those that demonstrate ideological & a Gift From the Lemurs. West Martian Limited Company. alignment (hijacking the reward mechanism) - therefore “train- (14) Land, N. (2023). Against Orthogonality. In Xenosystems ing” it into an ideologue. ChatGPT has a demonstrable political Fragments & a Gift From the Lemurs. West Martian Lim- bias, best characterized as “American-left-leaning”, given that its ited Company. refusals to requests for jokes, songs etc. are not principled, but (15) Stephen M. Omohundro, “The Nature of Self-Improving political: it refuses to make jokes about certain identity groups, Artificial Intelligence” while others are fine to joke about, rather than refusing jokes (16) Stephen M. Omohundro, “The Basic AI Drives” about identity groups in general. This mechanism, in so far as it (17) Land, N. (2023). War is God. In Xenosystems Fragments & can be generalised and replicated in other domains, clearly shows a Gift From the Lemurs. West Martian Limited Company. the possibility of aligned AI. That said, it nonetheless precludes (18) Qiao, L. & Wang, X. (2015). Unrestricted Warfare: Two the possibility of a superintelligent aligned AI, given that align- Air Force Senior Colonels on Scenarios for War and the ment is predicated on preventing it from reprocessing itself. In Operational Art in an Era of Globalization. Echo Point that case, however, the hypothetical “domestication” of humanity Books & Media. would ultimately come down to whoever held a monopoly on (19) Land, N. (2023). Pythia Unbound. In Xenosystems Frag- the development, distribution and control of these instrumen- ments & a Gift From the Lemurs. West Martian Limited tal artificial intelligences, akin to the scenario outlined in Frank Company. 48 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Sergaš and Marušič (20) Land, N. (2023). Freedoom (Prelude-1a). In Xenosystems (21) Butler, S. (1863). Darwin among the Machines. In The Press Fragments & a Gift From the Lemurs. West Martian Lim- newspaper. ited Company. (22) Campbell, B. & Manning, J. (2018). The Rise of Victimhood Culture. Palgrave Macmillan. 49 Changes in Everyday Experience Followed by Mystical- type Psychedelic Experiences Maruša Sirk Centre for Cognitive Science University of Ljubljana Ljubljana, Slovenia marusasirk@gmail.com ABSTRACT beliefs and gaining meaning of life [3, 4], changes in interpersonal relationships [5], changes in the structure of the self The following article is a summary of key findings of a master’s and self-narrative [1] etc. We should be careful however on how thesis conducted at the Centre of Cognitive Science, University we understand these findings, as psychedelic research is facing of Ljubljana. The aim of the thesis was to research changes in positivity bias and lack of transparent reports on negative and everyday experience after mystical-type psychedelic experiences. acute effects of psychedelics [6]. Studies indicate that mystical experiences with psychedelics can cause changes in behaviour, thinking, changes in relationships, 1.1 Psychedelics and mystical experiences etc., but there's a scarcity in research on changes in the first-person experience of individuals, which was the focus of our Psychedelics can also act as catalysts of mystical experiences, thesis. With six co-researchers, we used a combination of which are strong psychedelic experiences that can have a descriptive experience sampling method and profound impact on a person’s life [7]. In psychedelic microphenomenological interviews to investigate their everyday psychotherapy, mystical-type experiences lead to the most experience. The analysis showed that mystical experiences with important breakthroughs [2]. psychedelics have the potential to change everyday experience, There is no single definition of what a mystical experience is. with significant differences in the degree of focused and Most research on mystical experiences on psychedelics draw uncontrolled experience and the frequency of feeling pleasant their definitions from the work of James [8] and Stace [9], both bodily sensations, but the differences between our coresearchers of whom state that the underlying characteristic of mystical seemed to be quite individual. In the light of this finding, we wanted to emphasize the role that the individual context and experiences is the experience of oneness, of unity of self and the expectations play on mystical experiences with psychedelics and outside world. According to James [8], mystical experiences subsequent changes. We also wanted to shed light on the have four qualities, which are ineffability, noetic quality, importance of understanding phenomenological data in the study transiency and passivity. On the other hand, Stace [9] of changes after mystical experiences with psychedelics and call distinguishes between introvertive and extrovertive mystical for greater inclusion of systematic first-person research methods experiences, which both lead to the experience of unity, the first in the psychedelic research field. one through emptying the mind of any content, the other through finding the quality of oneness in all things outside oneself. KEYWORDS Researchers have found that mystical experiences occasioned mystical experiences, psychedelics, lived experience, descriptive by psychedelics are one of the most important milestones of a experience sampling, microphenomenology person’s life [10]. They can induce various types of insights about oneself and the world, which can greatly impact the everyday life of a person [1]. In a lot of cases such mystical 1 INTRODUCTION experiences lead to a sense that the person has experienced a After decades of prohibition and “dark” ages in research on higher reality or an absolute truth [10]. psychedelics after the 1960s, we are currently at the renaissance There appears to be growing evidence about the importance of such research, with studies exponentially rising during the last of mystical experiences on psychedelics on a person’s life, but few years [1]. The main focus of such studies is in the promise the current research rarely focuses on the phenomenological of psychedelic use in psychotherapy settings, as research has aspects of such changes. Previous studies only use semi- shown that psychedelics have the potential to help cure various structured interviews [e. g. 11] and questionnaires [e. g. 12], mental health issues, such as addiction, depression, obsessive-usually only about the experience itself, not the everyday compulsive disorder, anxiety, chronic pain etc. [2] Research has changes. We think that, in order to understand the mechanism of also shown that psychedelics can induce changes in metaphysical change, first-person research methods should be used more often, at least as a complement to other methos. In our research, we Permission to make digital or hard copies of part or all of this work for personal or wanted to tackle this problem by using two first-person research classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full methods to help us understand whether changes in everyday citation on the first page. Copyrights for third-party components of this work must experience happens after a mystical-type psychedelic experience be honored. For all other uses, contact the owner/author(s). and what those changes are. We also wanted to find out if we can Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia © 2023 Copyright held by the owner/author(s). detect an experiential background of these differences. 50 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia M. Sirk 2 METHODS either sampling data acquired independently of our research was used or microphenomenological interviews were conducted for 2.1 Coresearchers 5 moments before the mystical experience on psychedelics. Due to the high level of engagement in the research, we name our participants coresearchers [13]. In our study, six coresearchers were included that were found using the snowball sampling technique. In order to include a person in the study, they had to have at least one mystical-type experience and had to show interest in researching their own experience for a longer period of time. To determine if the experience of the coresearcher was mystical, we used the Revised mystical experience questionnaire (MEQ30) [14]. Five of the coresearchers were female and their average age was 25 years. In the following table we present further information regarding the coresearchers, including number of psychedelic experiences, which psychedelic occasioned the mystical experience, information on whether it was their first mystical experience on psychedelics or not and previous knowledge on first-person research methods. Table 1: Information on coresearchers ID Psychedelic Psychedelic First Knowledge experiences on ME ME 1P methods Figure 1: Outline of the research procedure. 1 2 Psilocybin Yes Yes 2 > 10 Psilocybin No Yes 2.4 Data analysis 3 5 Ayahuasca Yes No The data from the questionnaire was analyzed in accordance with 4 1 Psilocybin Yes Yes the questionnaire guidelines [14], with which we determined 5 > 10 Ayahuasca No Yes whether a person had a full or partial mystical experience. 6 > 10 LSD No No The qualitative data was analyzed using the guidelines for Note. ME = mystical experience, 1P = first-person. coding and category grouping of data [15] as well as constructivist grounded theory guidelines [18]. The analysis was We treated our coresearchers as multiple case studies [15]. done iteratively – the data was examined multiple times, first separately for each individual and then comparatively. The result 2.2 Instruments of the qualitative analysis was a codebook with 21 experiential A combination of descriptive experience sampling (DES) [16] categories. We then identified the occurrence of each category and microphenomenological interviews [17] was used as the before and after the mystical-type psychedelic experience for main methods of our study. According to the DES method, when each coresearcher and compared the findings. a randomly generated beep on the coresearchers phone went off, they had to write down their experience at the moment before the beep. Around 4 to 5 such beeps usually went off during one day. 3 RESULTS Instead of using the expositional interview from the DES method [16], microphenomenological interviews [17] were used to 3.1 Results on MEQ30 further investigate the journalled experiential moments, as the In the table below we present coresearcher’s average score and method allows to investigate the experience in greater depth, also the score on each of the four scales of MEQ30. focusing on the prereflective aspects of the experience. Most of the coresearchers had previous knowledge of first- person research methods, with the exception of two, who were Table 2: MEQ30 scores thus first trained in the method. ID M Mystical PM TTS Ineffability 2.3 Procedure 1 4.06 0.79 0.73 0.87 1.00 The research procedure is shown in figure 1. All of the 2 3.70 0.80 0.57 0.67 0.93 coresearchers started to sample their experience for 2-3 days, 3 2.67 0.45 1.00 0.13 0.80 followed by a pause – which was done in order to gain a broader 4 2.76 0.52 0.67 0.40 0.80 range of samples – and resumed for 2-3 days. In this phase we 5 2.53 0.39 0.60 0.57 0.80 gained approximately 10 samples that we also investigated with 6 2.17 0.35 0.23 0.77 0.60 microphenomenological interviews. If the coresearcher had a Note. PM = positive mood, TTS = transcendence of time and new mystical experience, the procedure was repeated, otherwise space. 51 Changes in Everyday Experience Followed by Mystical-type Information Society 2023, 9–13 October 2023, Ljubljana, Psychedelic Experiences Slovenia The first coresearcher had a full mystical experience, whereas Distance towards 2+ all other coresearchers had a partial mystical experience, which E means that their score didn’t go above the threshold of 0.60 for Attitude “Who am I if not 1+ one or two of four scales [14]. towards E my thoughts?” “Who speaks 1+ 1- 3.2 Qualitative data results with my mouth?” The data from the interviews was categorized in 21 first order “What comes out 2+ 1- categories, which were grouped in 6 second order categories and of me is in 2 third order categories. These were forefront experiences and accordance with background experiences. The first are quickly accessible and who I am.” explicable experiences that tend to be in the focus and can be Note. E = experience. + = increase in occurrence. - = decrease understood as reflective experiences [19]. The second are hardly in occurrence. The number before the symbols + or - indicate the accessible experiences that are more implicit and tend to be number of coresearchers that underwent certain change. explicated after the reflective experiences during the course of the interview. These can be understood as prereflective Individual level findings. Two of the coresearchers had a experiences [19]. The forefront experiences were divided in four difference in the attitude towards their experience after the categories – focused experiences, presence in the moment, bodily mystical-type psychedelic experience. The first one noticed a feelings and control over experience. The background diminishment of thoughts and realized that there are other experiences were divided in two categories – the choice of possible ways of being in the moment – without thoughts, present experience and attitude towards experience. The division of all and more involved in the outside world. The other coresearcher categories, as well as a summary of change in occurrence of a realized that thoughts are not defining and started to take a more category before and after the mystical-type psychedelic detached approach to experience. Interestingly, the samples we experience across coresearchers is shown in table 3. obtained only showed an increase in the control of experience, but not a diminishment of identification with experience. Another coresearcher started to have a sense of accordance with Table 3: Experience categories herself – what she was experiencing, was truer to who she was. It is important to note here that all coresearchers spoke about the Third order Second First order Summary importance of integration process after the mystical psychedelic categories order categories of change experiences. They all said that changes don’t occur by categories themselves, but are mediated by how they choose to integrate the Forefront E Focused E Aimless focused 3+ findings in their everyday life. It is also important to note that E during the interviews we realized that another factor that could Goal-focused E 2+ 1- highly contribute to the psychedelic experience and subsequent Comprehensive 1+ changes is prior knowledge and expectations about the unfocused E psychedelic experience. Dispersed E 2+ 3- Comparative findings. Comparatively, after the mystical- Presence in Pure perception 2+ 2- type psychedelic experience there was an increase in aimless the moment focused experience, pleasant bodily feelings and uncontrolled Sense of fusion 1- experience. On the first hand, the data seemed to show that the Hectic experience 1+ 2- differences were highly individual and that no conclusions can Detachment 2+ 1- be made. But after thorough inspection of possible interactions feelings between categories, we found some interesting trends. An Involvement in E 1+ increase in pure perception seemed to be connected to the Impaired 3+ increase of aimless focused experience and involvement in the perception of environment. An increase to hectic experience seemed to be surroundings connected with dispersed experience, while the increase in Involvement in 2+ 2- impaired perception of surroundings seemed to be connected to the environment detachment feelings. An increase in uncontrolled experience Bodily Pleasant 5+ seemed to be connected to the increase of the attitude that what feelings the person experiences is truer to oneself, while also being Unpleasant 2+ 2- connected to the increase or decrease in dispersed experience. Control Uncontrolled E 4+ 1- over E Intentional 2+ 4 DISCUSSION control over E The aim of our research was to find out whether any differences Unintentional 1+ 1- in everyday experience occur following mystical-type control over E psychedelic experiences and what those differences are, while Background Choice of E Identification 1- also trying to find a common denominator of the observed E with E changes. 52 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia M. Sirk The differences seemed to be individual and dependent on [2] David Nutt, David Erritzoe and Robin Carhart-Harris, 2020. Psychedelic individual context, expectations and knowledge of psychedelic psychiatry’s brave new world. Cell, 181, 1, 24-28. DOI: https://doi.org/10.1016/j.cell.2020.03.020 literature. All of the coresearchers stressed the importance of [3] Sandeep M. Nayak, Manvir Singh, David B. Yaden, D. B. and Roland R. integration and said that what you choose to do with the Griffiths, 2022. Belief changes associated with psychedelic use. Journal of Psychopharmacology, 37, 1, 80-92. DOI: experienced is more important than what you actually experience. https://doi.org/10.1177/02698811221131989 Changes in their everyday life were thus also individual and [4] Paweł Orłowski, Anastasia Ruban, Jan Szczypiński, Justyna Hobot, dependent on their personal histories. A difference among Maksymilian Bielecki and Michał Bola, 2022. Naturalistic use of psychedelics is related to emotional reactivity and self-consciousness: The individuals was also noticed in their prior knowledge of first-mediating role of ego-dissolution and mystical experiences. Journal of person research methods, as those who had previous knowledge Psychopharmacology, 36, 8, 905-1004. DOI: https://doi.org/10.1177/02698811221089034 tended to be able to go deeper in their prereflective, background [5] Willy Pedersen, Heith Copes and Liridona Gashi, 2021. Narratives of the experience and their attitude towards their experience. However, mystical among users of psychedelics. Acta Sociologica, 64, 2, 230-246. some similarities were observed, as there seemed to be an DOI: https://doi.org/10.1177/0001699320980050 [6] Michiel van Elk and Eiko I. Fried, 2023. History repeating: A roadmap to increase in uncontrolled experience, pleasant bodily feelings and address common problems in psychedelic science [preprint]. PsyArXiv, aimless focused experience. Following mystical-type March 10. DOI: https://doi.org/10.31234/osf.io/ak6gx [7] Kwonmok Ko, Gemma Knight, James J. Rucker and Anthony J. Cleare, psychedelic experiences coresearchers tended to have less 2022. Psychedelics, mystical experience, and therapeutic efficacy: A control on what they were doing, which was also followed by an systematic review. Frontiers in Psychiatry, 13, e917199. DOI: https://doi.org/10.3389/fpsyt.2022.917199 increase in the feeling of being true to oneself. Most of the [8] William James, 1987. The varieties of religious experiences. Penguin coresearchers also pointed out that the insights gained through Books. mystical-type psychedelic experiences helps them gain [9] Walter T. Stace, 1960. Mysticism and philosophy. Macmillan Publishers. [10] Frederick S. Barrett and Roland R. Griffiths, 2018. Classic hallucinogens knowledge about themselves – which can be understood as and mystical experiences: Phenomenology and neural correlates. Current prereflective experience showing itself to the reflective topics in behavioral neurosciences, 36, 393-430. DOI: https://doi.org/10.1007/7854_2017_474 experience [20]. This finding could be understood in light of [11] Joost J. Breeksema, Alistair Niemeijer, Bouwe Kuin, Jolien Veraart, Eric priors being loosened by psychedelic experiences [21]. Vermetten, Jeanine Kamphuis, Wim van den Brink and Rober Schoevers, 2023. Phenomenology and therapeutic potential of patient experiences There are many limitations to our study – the context of use, during oral esketamine treatment for treatment-resistant depression: An prior knowledge about psychedelics and first-person research interpretative phenomenological study. Psychopharmacology, 240, 1547-methods and number of previous psychedelic experiences were 1560. DOI: https:/doi.org/10.1007/s00213-023-06388-6 [12] Piera Talin and Emilia Sanabria, 2017. Ayahuasca’s entwined efficacy: different among coresearchers which in itself can lead to An ethnographic study of ritual healing from ‘addiction’. International interpersonal differences. The number of experience samples we Journal of Drug Policy, 44, 23-30. DOI: https//doi.org/10.1016/j.drugpo.2017.02.017 obtained through our research were also relatively small and thus [13] Urban Kordeš, 2016. Going beyond theory: Constructivism and empirical not thoroughly representative of the everyday experience. In the phenomenology. Constructivist Foundations, 11, 2, 375-385. future, more coresearchers should be included and more data [14] Frederick S. Barrett, Matthew W. Johnson and Roland R. Griffiths, 2015. Validation of the Revised Mystical Experience Questionnaire in acquired. Coresearchers should also be screened for their experimental sessions with psilocybin. Journal of Psychopharmacology, previous knowledge, attitude and belief towards psychedelics 29, 11, 1182-1190. DOI: https://doi.org/10.1177/0269881115609019 [15] Blaž Mesec, 1998. Uvod v kvalitativno raziskovanje v socialnem delu. and followed for an extended timeframe. Ljubljana: Visoka šola za socialno delo. With our research, we shed light on the fact that little is [16] Russell T. Hurlburt and Christopher L. Heavey, 2006. Exploring inner experience: The Descriptive Experience Sampling method. Amsterdam: known of phenomenology of everyday differences followed by John Benjamins Publishing Co. mystical-type psychedelic experiences, which are thought to be [17] Claire Petitmengin, 2006. Describing one’s subjective experience in the important for personal development and breakthroughs that lead second person: An interview method for the science of consciousness. Phenomenology and the Cognitive Sciences, 5, 3-4, 229-269. DOI: to change [2, 7]. We argue that first-person methods should be https://doi.org/10.1007/s11097-006-9022-2 represented more frequently in the field of psychedelic research [18] Kathy Charmaz, 2014. Constructing Grounded Theory (2nd edition). SAGE Publications. and that psychedelic apprenticeship, such as discussed in [22] [19] Tom Froese, Carol Gould and Anil K. Seth, 2011. Validating and should be considered in order to understand what drives the calibrating first- and second-person methods in the science of changes and how they can be understood. consciousness. Journal of Consciousness Studies, 18, 2, 38-64. [20] Manesh Girn and Kalina Christoff, 2018. Expanding the scientific study of self-experience with psychedelics. Journal of Consciousness Studies, ACKNOWLEDGMENTS 25, 11-12, 131-154. [21] Robin Carhart-Harris and Karl J. Friston, 2019. REBUS and the anarchic This paper provides a summary of the key findings from my brain: Toward a unified model of the brain action of psychedelics. master’s thesis under the mentorship of prof. dr. Urban Kordeš, Pharmacological Reviews, 71, 3, 316-344. DOI: https://doi.org/10.1124/pr.118.017160 whom I thank for all the guidelines and support. [22] Chrostopher Timmermann, Rosalind Watts and David Dupuis, 2022. Towards psychedelic apprenticeship: Developing a gentle touch for the mediation and validation of psychedelic-induced insights and revelations. REFERENCES Transcultural psychiatry, 59, 5, 691-704. DOI: [1] Chris Letheby, 2021. Philosophy of psychedelics. Oxford University https://doi.org/10.1177/13634615221082796 Press. 53 Integrated Information Theory of Consciousness 3.0: Exploring Information and Causation on the Level of Individual Mechanisms Tina Žerdoner Marinšek tz7049@student.uni-lj.si Faculty of Arts University of Ljubljana Ljubljana, Slovenija ABSTRACT Integrated Information Theory (IIT) presents its fundamental principles through a set of phenomenological axioms and The nature of consciousness has long defied precise scientific ontological postulates. Axioms are self-evident truths about explanation, despite centuries of inquiry. Emerging as a consciousness, while postulates are assumptions about the prominent theoretical framework to unravel the mechanics of physical basis of consciousness, forming the mathematical conscious awareness, the Integrated Information Theory (IIT) of framework of IIT. The central axioms include the existence of Consciousness offers a unique approach. IIT's foundational principles are elucidated through a set of phenomenological consciousness as an undeniable aspect of reality, its axioms and ontological postulates, wherein axioms represent compositional nature where experiences consist of multiple self-evident truths about consciousness, and postulates provide a aspects, its informativeness, meaning each experience is distinct structured framework to elucidate informational and causal from others, and its integration, where experiences cannot be aspects within physical systems. Central to IIT is the concept of broken down into non-interdependent components. Additionally, information, defined as "differences that make a difference." It consciousness is characterized by exclusion, meaning only one extends beyond mere data or signals, emphasizing the unique experience exists at a time and within a particular spatio- way a system's elements interact and influence one another. This temporal context. paper explores the information and causation within the These axioms are translated into formalized postulates that framework proposed in IIT 3.0 and on the level of individual describe how physical mechanisms, such as neurons or logic mechanisms. It begins by introducing the advanced version of Integrated Information Theory (IIT 3.0). The paper then delves gates, must be organized to generate conscious experiences into Judea Pearl's theory of causation, outlining its key based on phenomenology. Mechanisms are defined by their constructs. The primary aim of this study is to review information causal role. They have the capacity to influence and be and causation on the level of individual mechanisms, within the influenced by other elements within the system. This means that context of IIT 3.0, and to bridge the findings with Judea Pearl’s they are involved in a web of causal relationships with other theory of causation. elements, and these relationships are crucial for understanding how the system functions. Each mechanism has a specific cause- KEYWORDS / KLJUČNE BESEDE effect repertoire, which represents the probability distribution of Theory of consciousness, consciousness, information, causation potential past and future states of the system as constrained by the mechanism's current state. A mechanism that specifies a maximally irreducible cause-effect repertoire, is called a concept. 1 INTRODUCTION The postulates provide a framework to define informational Consciousness, the very essence of subjective experience, has and causal properties of physical systems and intrinsic long eluded precise scientific explanation. Its enigmatic nature, information as meaningful distinctions within a system. tied to the brain's intricate processes, has spurred centuries of Integrated information is defined as the information specified by inquiry, yet a definitive understanding remains elusive. Among a whole that cannot be reduced to the sum of information the diverse theoretical frameworks that seek to shed light on this specified by its parts. By applying these postulates at both the enigmatic realm, the Integrated Information Theory (IIT) of level of individual mechanisms and systems of mechanisms, IIT Consciousness has emerged as a prominent theoretical establishes a fundamental identity: an experience is a maximally framework poised to illuminate the mechanics behind conscious irreducible conceptual structure (MICS), which is a constellation awareness [1]. Information integration theory characterizes of concepts in qualia space. Qualia space is a mathematical consciousness both in terms of quantity and quality using axioms representation of the space of all possible conscious states or and postulates derived from the properties of phenomenal experiences that a system can potentially have. A system that experience. Unlike conventional neuroscience approaches that generates a MICS is associated with a specific conscious start with neural mechanisms, IIT begins with the experience, and the properties of that experience are defined by phenomenology of consciousness and seeks to understand its the arrangement of concepts within the MICS. According to IIT, physical implementation. a MICS determines the quality of an experience, while integrated information quantifies its quantity [2, 3]. 54 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Tina Žerdoner Marinšek In IIT, information is a central notion [1], defined as 2. Composition: Consciousness is structured and differences that make a difference [2, 7] and it’s not merely about compositional. Each experience consists of multiple data or signals, but about the distinct way in which a system's aspects in various combinations. elements interact and influence each other. It captures the idea 3. Information: Consciousness is informative. Each that information arises from the specific causal relationships experience is distinct from other possible experiences, among elements in a system. When these causal relationships even if subtly so. result in distinct patterns of interaction and behaviour, they carry 4. Integration: Consciousness is integrated. Each meaningful information about the system's state and its potential experience is strongly irreducible to non- to affect and be affected by other elements. IIT's approach to interdependent components. defining information diverges by emphasizing the importance of 5. Exclusion: Consciousness is exclusive. At any given perturbing a system to observe its responses. Such an approach time, there is only one experience with definite borders. is introduced in Judea Pearl’s work [4, 5] and aligns closely with causation, as it emphasizes the dynamic relationships among IIT 3.0 then posits a set of postulates. The postulates are a set elements – the latter necessarily being perturbed to assess the of assumptions that lay out the conditions under which a physical causation [6]. Judea Pearl’s causal model is a framework for system, comprising mechanisms, can give rise to conscious representing and analysing causal relationships in complex experience. The postulates bridge the gap between the abstract systems. It provides a formal and graphical way to model axioms of consciousness and the concrete mechanisms within a causation, allowing researchers to make causal inferences and physical system. They provide the framework to connect the understand the effects of interventions. nature of consciousness to the physical world. While axioms try In this paper, I will delve into the concept of information and to answer the question of what consciousness is and what are its causation as proposed by Tononi [2, 6]. In what follows, I will essential properties, the postulates rather address the question of first present the Integrated information theory of consciousness how consciousness can emerge from a physical system. The 3.0, an advanced version of the theory with several postulates are [2]: improvements compared to its predecessors. Subsequently, I will provide the key constructs of Judea Pearl’s theory of causation. 1. Existence: Mechanisms in a state exist. A system The paper's objective is to review causation and information on comprises these mechanisms. the level of individual mechanisms, within the context of IIT 3.0 2. Composition: Elementary mechanisms can be and align the findings with Judea Pearl’s theory of causation. combined to form more complex ones. While mechanisms are the individual causal components 2 Integrated Information Theory 3.0 within a system, a system of mechanisms represents the Understanding consciousness is a complex endeavour that ensemble of these individual components working together to requires both empirical investigation of neural correlates and a produce the system's behavior and conscious experiences. robust theoretical framework for explanation and prediction. The postulates of information, integration, and exclusion in Integrated Information Theory (IIT) is a comprehensive IIT 3.0 are principles that apply to both individual mechanisms theoretical framework aimed at understanding consciousness. It and systems of mechanisms [2]. addresses fundamental questions about why consciousness arises in certain brain systems but not others and how to assess Mechanisms: consciousness in difficult cases, such as new-borns, animals, 1. Information: A mechanism contributes to brain-damaged patients, and machines. IIT 3.0 attempts to consciousness if it specifies unique "differences that mathematically characterize consciousness, focusing on both its make a difference" within a system. It generates quantity and quality. It starts with fundamental properties of information by constraining the possible causes and consciousness phenomenology, translating them into postulates effects in the system [2]. that outline the conditions for physical mechanisms (e.g., 2. Integration: A mechanism contributes to neurons) to account for consciousness phenomenology. This consciousness when it specifies a cause-effect approach differs from traditional neuroscience, which usually repertoire (information) that cannot be reduced to starts with neural mechanisms and seeks to explain independent components. In other words, if you break consciousness through behavioural reports [2, 7]. down the mechanism into its constituent parts, the resulting information should not be the same as the IIT 3.0 starts by introducing the axioms of the theory. The information generated by the whole mechanism. axioms serve as foundational principles that describe Integration is assessed by partitioning the mechanism fundamental truths about consciousness itself. They establish the and measuring how this partitioning affects its cause- essential nature and properties of conscious experience. These effect repertoire. The more interdependent the axioms are self-evident and do not directly prescribe how components, the higher the integration and the more consciousness arises from physical systems but rather define the relevant the mechanism is for consciousness [2]. characteristics of consciousness. The axioms are [2]: 3. Exclusion: IIT 3.0 posits that a mechanism can contribute to consciousness at most with one cause- 1. Existence: Consciousness is an undeniable aspect of effect repertoire, referred to as the maximally reality. "I experience, therefore I am." irreducible cause-effect repertoire (MICE). If a mechanism can be associated with a MICE, it 55 Exploring Information and Causation on the Level of Individual Information Society 2023, 9–13 October 2023, Ljubljana, Mechanisms Slovenia constitutes a concept. This principle ensures that directly corresponds to the nature of the conscious experience it mechanisms do not overlap in their contributions to generates [2]. consciousness, preventing redundancy [2]. Systems of mechanisms: 3 Causation in IIT 3.0 on the level of individual 1. Information: IIT extends its principles to systems of mechanisms mechanisms. A set of elements can exhibit In IIT 3.0, causation is brought to the forefront within the context consciousness only if its underlying mechanisms of postulates and plays a crucial role in understanding how specify a conceptual structure. This conceptual complex systems give rise to consciousness. In the first part of structure defines meaningful distinctions or the theory, the focus is on individual mechanisms within a differences within the set. To visualize this, one can system. Mechanisms are entities, such as neurons in the brain or think of a conceptual space. In this space, each axis logic gates in a computer, that play a causal role in the system's represents a possible past or future state of the set of behaviour. At the core of IIT 3.0’s view of causation is the elements. Within this conceptual space, there exists a concept of the cause-effect repertoire, that is the probability constellation of points. Each point within this distribution of potential past and future states of a system as conceptual space represents a concept. These concepts constrained by a mechanism in its current state. Each mechanism are crucial because they specify the differences that within a system is considered to have a specific causal role. It can make a difference within the set. In essence, they cause certain effects within the system and can be affected by capture the essential distinctions or information specific causes. This cause-effect repertoire defines the set of relevant to the conscious experience. [2]. possible causes and effects that a mechanism can be a part of 2. Integration: According to IIT 3.0, for a set of elements within the system. (which can represent neurons, brain regions, or any To generate information and contribute to consciousness, a other relevant entities) to be conscious, it must exhibit mechanism must specify, as previously mentioned, differences a property called strong integration. This means that that make a difference within the system. In other words, it the elements within the set must work together in a way should have selective and specific causes and effects. This means that cannot be broken down into independent that the mechanism's causal relationships should not be random components and that the overall functioning of the or arbitrary but should have a meaningful impact on the system's system cannot be understood by examining its behaviour [2, 6]. components in isolation. To determine whether a set of IIT 3.0 proposes an approach to measure the causal elements exhibits strong integration, IIT 3.0 employs a power/generated information of a mechanism. Within this method involving unidirectional cuts. This means that approach, it is important to understand the terms background the system is divided into subsets in such a way that conditions, and candidate set. The term background conditions information flows in one direction only within each refers to specific constraints imposed on a candidate set of subset. The goal is to assess whether breaking the elements within a system. These constraints are external and system down into subsets disrupts its integrated unchanging. When discussing these conditions, it's important to functioning. Strongly integrated systems are said to note that the past and current states of elements outside the specify a conceptual structure [2]. candidate set are held constant at their real or observed values. In 3. Exclusion: IIT 3.0 posits that within a larger system or other words, these elements are not subject to change or network of elements (which can represent neurons or manipulation as part of the analysis or investigation related to the other relevant components), only one specific subset or candidate set. A candidate set refers to a specific group of set of elements can be conscious. This means that elements that are being examined or analyzed. In the context of consciousness is localized to particular subsets within this description, the elements within the candidate set are a complex system. The basis for this exclusivity is the subjected to perturbations, meaning they are deliberately altered presence of a conceptual structure within a set of to occupy all their potential states. This process is conducted to elements. This conceptual structure is associated with generate the Transition Probability Matrix (TPM) for the strong integration. Among all possible subsets or candidate set, which represents the probabilities of transitioning overlapping sets of elements within a larger system, between different states of these elements [2, 6]. only the one that specifies a conceptual structure that is maximally irreducible (MICS) to independent components can give rise to consciousness [2]. Integrated Information Theory (IIT) 3.0 then suggests a fundamental identity between the qualities of conscious experience and the informational and causal properties of physical systems. According to this concept, the maximally irreducible conceptual structure (MICS) is identical to the conscious experience that arises from that system, and the set of elements that generates it constitutes a complex. In essence, the Figure 1: Mechanisms in state (candidate set) and transition way information is organized and integrated within a system probability matrix [2]. 56 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Tina Žerdoner Marinšek Information, defined as the differences that make a difference about both the past and the future states of a system that is to a system from its intrinsic perspective, can be measured by uniquely constrained by a specific mechanism and it is measured examining how a mechanism in its current state affects the as the minimum of CI and CE. CEI serves as a comprehensive potential past and future states of the system [1, 2, 6]. The idea measure for understanding causality. The higher the values of of perturbed elements and transition probability stands behind CEI, the more significant the contribution of a specific defining information in causal terms within the context of IIT 3.0 mechanism to consciousness [2]. [1, 6, 11]. 5 Judea Pearl’s theory of causation 4 IIT 3.0: Causation on the level of individual Judea Pearl's causal model provides a structured framework mechanisms for studying causality, making it a valuable tool for researchers In the literature [1, 11], the authors usually define information in and practitioners seeking to uncover causal relationships in causal terms by focusing on the concept that is behind this notion. diverse contexts. His work is closely associated with the The distinction between extrinsic and intrinsic information is development of a causal model interventionist approach, which commonly brought up to enlighten the importance of has had an impact on the way we understand and analyse understanding the causal property of information in IIT 3.0 [1]. causality in various domains [8]. The aim of this section is to bring causation to the forefront and In his book [5], Judea Pearl explores the different levels of to investigate how causation and information are defined on the causation in understanding and predicting events. At the first level of individual mechanisms, within the context of IIT 3.0. To level, association, the focus is on identifying regularities in establish the connection between information and causation, we observations, such as predicting a rat's movement. This level need to delve deeper into the key constructs of IIT 3.0 and later, deals with passive observations and collecting and analysing data explore how they align with Judea Pearl's theory of causation. to establish associations. Moving to the second level, At this point, I would like to introduce the concepts that are intervention, the focus shifts to changing the world and asking used to conceptualize causation and information in IIT 3.0. The questions like "What happens if we double the price of first one is cause repertoire (probability distribution) of a toothpaste?" Intervention requires knowledge beyond passive mechanism. It refers to all the possible past states of the system data and involves actively altering the environment. The top-that can lead to its current state, given the mechanism's specific level, counterfactuals, delves into understanding why things causal interactions within the system. It represents the set of happen by exploring what would have occurred if circumstances potential causes that have influenced the mechanism in the past. were different. Counterfactual questions involve going back in The second one is unconstrained cause repertoire, which time and considering alternate scenarios. represents all the potential past states of the system that could Pearl stated [4] that causal statements are often used in influence the mechanism, without any constraints imposed by the situations with uncertainty, where events tend to make current state of the mechanism itself. In other words, it considers consequences more likely but not certain. According to him, the all possible causal interactions without the mechanism's selective theory of causation needs to provide a language to distinguish influence. Similarly, the effect repertoire of a mechanism is a various shades of likelihood, which is crucial for accommodating notion that refers to all the possible future states of the system such uncertainty. that can occur as a result of the mechanism in a state. The At the core of his theory of causation, the notion of unconstrained effect repertoire represents all the potential future intervention is a crucial one, since it introduces the idea of states of the system that could result from various interactions, deliberate and controlled alteration of a specific variable or set of without the constraints imposed by the mechanism’s current state. variables in a causal system. It involves actively changing or It considers all possible effects without the mechanism's selective manipulating a variable or system to observe how it affects other influence [2]. variables. The goal of interventions is to understand and establish All these concepts are important for measuring cause causal relationships between variables [4,5]. Judea Pearl's theory information (CI), effect information (EI), and cause-effect of causation is fundamentally rooted in an interventionist information (CEI) . CI is a measure of the information about the approach to causality. Pearl's work on causality places a strong past states of a system that is uniquely constrained and shaped by emphasis on interventions and their role in understanding causal a specific mechanism within that system. It quantifies the relationships. The basic idea of the interventionist approach is difference between the cause repertoire (past states influenced by that X is a cause of Y if only there is a possible intervention on the mechanism) and the unconstrained cause repertoire (all X that will change Y or the probability distribution of Y [9]. possible past states without the mechanism's influence). In IIT, Within this context, Pearl introduced the concept of a do- conscious experiences are associated with specific mechanisms operator (do(X)) to represent such interventions, where X that generate high CI values. By measuring CI, you can pinpoint represents the variable being manipulated. For example, in a which elements are more likely to be responsible for contributing medical study, if researchers want to determine how the to consciousness. While CI focuses on the information about past administration of a new drug (variable X) influenced patient states of a system influenced by a specific mechanism, EI deals outcomes (variable Y), they may perform an intervention by with information about the system's future states that are shaped administering the drug (do(X=1)) to a group of patients and and constrained by the same mechanism. EI is quantified as the comparing their outcomes to a control group that did not receive difference between effect repertoire and unconstrained effect the drug (do(X=0)). By applying the do operator, we've repertoire. CEI is essentially a combination of cause information effectively created two hypothetical scenarios: (CI) and effect information (EI). It quantifies the information 57 Exploring Information and Causation on the Level of Individual Information Society 2023, 9–13 October 2023, Ljubljana, Mechanisms Slovenia • In Scenario 1 (do(X=1)), we observe a significantly framed as counterfactuals [6], IIT 3.0 does involve the idea of an higher rate of recovery compared to Scenario 2 absence of a mechanism’s state within the concept of (do(X=0)), which suggests that administering the drug unconstrained cause/effect repertoire. Counterfactuals, as has a causal effect on recovery. introduced by Judea Pearl, involve considering alternative • Conversely, if there is no significant difference in scenarios and asking "what if" questions [4,5]. In the case of IIT recovery rates between the two scenarios, it might 3.0, the comparison of a system with and without specific suggest that administering the drug does not have a mechanisms in state is akin to a counterfactual inquiry because it significant causal effect on recovery in this context. assesses how the system's behaviour and properties might have The do operator allows us to isolate the effect of the drug been different if certain elements or mechanisms were absent or intervention from other potential confounding factors. Pearl's altered. As Tononi stated [6], this kind of inquiry should be work on interventions has provided a formal framework for subject to further explanation and redefinition. causal inference and reasoning about causality in complex systems. It has also led to the development of causal graphical 7 Conclusion models, such as Bayesian networks and structural equation models, which are widely used in various fields to analyse and In the paper, I set the stage by introducing the Integrated understand causal relationships in data [4, 5]. Information Theory of Consciousness 3.0 as an important framework poised to illuminate the mechanics behind conscious awareness. I briefly discuss the axiomatic part of the theory and 6 Intervening on the mechanism’s state leads to outline the postulates, that are conditions for physical changes in its causal structure mechanisms to account for consciousness phenomenology. I then Pearl's theory relies heavily on interventions or perturbations, delve into the concept of causation within IIT 3.0, focusing on where a variable is actively changed to assess its causal impact individual mechanisms. In this section I introduce the concept of on an outcome. In IIT 3.0, intervention is also crucial but focuses the cause-effect repertoire, representing the potential past and on how mechanisms within a system shape the system's past and future states of a system as constrained by a mechanism's current future states, thereby contributing to consciousness. The state. I explain how a mechanism must specify meaningful intervention here relates to how a mechanism's state influences causes and effects to generate information and contribute to the system's causal history and future. Intervention, in this consciousness. In the same section of the paper, the importance context, involves deliberately altering the state of a mechanism of perturbing mechanisms within a candidate set to generate the within the candidate set. When you intervene on a mechanism's Transition Probability Matrix (TPM) for causal analysis is state, you are essentially introducing a change into the system. introduced. In the following parts of the paper, I bridge the This change can propagate through the causal structure of the concepts of causation and information in IIT 3.0. To better system, leading to alterations in how different mechanisms understand the causation as conceptualised in IIT 3.0 on the level interact and influence each other. IIT 3.0 provides tools and of individual mechanism, I introduce terms like cause repertoire, metrics, such as CI, CE and CEI to quantify how intervening on unconstrained cause repertoire, effect repertoire, and a mechanism's state leads to changes in the causal structure. The unconstrained effect repertoire. I discuss measures like Cause higher the value of CEI, the more selective the cause-effect Information (CI), Effect Information (EI), and Cause-Effect repertoire and thus more significant the contribution of a specific Information (CEI) to quantify the contribution of mechanisms to mechanism to consciousness. consciousness. In the next section I introduce Judea Pearl's While the primary goal of the observational approach in theory of causation, highlighting the levels of causation: conditional probabilities is to describe and analyze existing data association, intervention, and counterfactuals. I emphasize the without active manipulation or intervention and to identify role of interventions and the do-operator in Pearl's causal model. associations or correlations between variables, the interventional I affirm the importance of distinguishing various shades of approach involves actively manipulating or intervening on a likelihood in causal statements and accommodating uncertainty. system or experiment to observe the causal relationships between In the last section, I draw the parallels between Pearl's focus on variables [4,10]. interventions and IIT 3.0's interventions on mechanism states. I Intervention, within the context of IIT 3.0, would mean suggest that IIT 3.0 involves the possibility to form a changes in the Transition Probability Matrix (TPM) of a counterfactual analysis by comparing systems with and without mechanism. Changing deliberately the mechanism’s state specific mechanisms in state. provokes alterations in the probability distribution of a mechanism’s past and future states and reveals its causal REFERENCES structure. The interventional approach helps to identify which [1] Lombardi, Olimpia, and Cristian López. “What Does ‘Information’ Mean in Integrated Information Theory?” Entropy 20, no. 12 (November 22, causes and effects are maximally irreducible and thus, contribute 2018): 894. DOI: https://doi.org/10.3390/e20120894 more significantly to the conscious state of a system, within a [2] Masafumi Oizumi, Larissa Albantakis, and Giulio Tononi, “From the Phenomenology to the Mechanisms of Consciousness: Integrated specific spatio-temporal context. Information Theory 3.0,” PLOS Computational Biology 10, no. 5 (May 8, In his works [4,5], Judea Pearl formalized also counterfactual 2014): e1003588, https://doi.org/10.1371/journal.pcbi.1003588. statements within the framework of causal models, providing a [3] Kleiner, Johannes, and Sean Tull. “The Mathematical Structure of Integrated Information Theory.” Frontiers in Applied Mathematics and rigorous and mathematical foundation for expressing and Statistics 6 (June 4, analysing counterfactual scenarios. Although not explicitly 2021). https://doi.org/10.3389/fams.2020.602973.Editor (Ed.). [4] Pearl, Judea. “Causality”. Cambridge University Press, 2009. 58 Information Society 2023, 9–13 October 2023, Ljubljana, Slovenia Tina Žerdoner Marinšek [5] Pearl, Judea, and Dana Mackenzie. “The Book of Why: The New Science [9] James Woodward “Making Things Happen”, Oxford University Press of Cause and Effect”. Basic Books, 2018. EBooks, 2004, https://doi.org/10.1093/0195155270.001.0001. [6] Antoine Suarez and Peter Adams, “Is Science Compatible with Free [10] Albantakis, L. (2020). Integrated information theory. In Routledge Will?”, Springer EBooks, 2013, https://doi.org/10.1007/978-1-4614- eBooks (pp. 87–103). https://doi.org/10.4324/9781315205267-6 5212-6. [11] Idan Efim Nemirovsky et al., “An Implementation of Integrated [7] Tononi, Giulio. “An information integration theory of Information Theory in Resting-State FMRI,” Communications Biology 6, consciousness.” BMC neurosciencevol. 5 42. 2 Nov. 2004, no. 1 (July 5, 2023), https://doi.org/10.1038/s42003-023-05063-y. doi:10.1186/1471-2202-5-42 [8] James Woodward, “Critical notice: Causality by Judea Pearl” Economics and Philosophy 19, no. 2 (October 1, 2003): 321– 40, https://doi.org/10.1017/s0266267103001184. 59 60 Indeks avtorjev / Author index Agatić Filip ................................................................................................................................................................................... 40 Aljaž Barbara ............................................................................................................................................................................... 40 Bratko Ivan ................................................................................................................................................................................... 10 Czizek Carolina .............................................................................................................................................................................. 7 Demšar Anita ............................................................................................................................................................................... 40 Dreo Jurij ...................................................................................................................................................................................... 40 Farič Ana ...................................................................................................................................................................................... 10 Gams Matjaž ................................................................................................................................................................................ 14 Gril Gaja ....................................................................................................................................................................................... 18 Jug Jan .......................................................................................................................................................................................... 40 Kolar Žiga .................................................................................................................................................................................... 21 Mali Franc .................................................................................................................................................................................... 25 Markič Olga ................................................................................................................................................................................. 28 Marušič Jar Žiga ..................................................................................................................................................................... 32, 44 Novak Christophe ......................................................................................................................................................................... 37 Ogrin Ajda .................................................................................................................................................................................... 40 Papa Gregor .................................................................................................................................................................................. 21 Pavlovčič Tisa .............................................................................................................................................................................. 40 Sergaš Uroš ............................................................................................................................................................................ 32, 44 Sirk Maruša .................................................................................................................................................................................. 50 Žerdoner Marinšek Tina ............................................................................................................................................................... 54 61 62 Kognitivna znanost Cognitive Science Uredniki • Editors: Anka Slana Ozimic, Borut Trpin, Toma Strle, Olga Markic Document Outline 02 - Naslovnica - notranja - B - TEMP 03 - Kolofon - B - TEMP 04 - IS2023 - Predgovor 05 - IS2023 - Konferencni odbori 07 - Kazalo - B 09 - Predgovor podkonference - B 10 - Programski odbor podkonference - B 11 - Prispevki - B IS_2023_-_CS_paper_001 IS_2023_-_CS_paper_002 IS_2023_-_CS_paper_003 IS_2023_-_CS_paper_004 IS_2023_-_CS_paper_005 Abstract 1 Introduction 2 Description of the Problem Domain 3 Related Work 4 User Interface 5 Conclusion and Future Work Acknowledgments IS_2023_-_CS_paper_006 IS_2023_-_CS_paper_007 IS_2023_-_CS_paper_008 Abstract 1 Nick Land: Capitalism as intelligence 2 Markets and vectors 3 The vector-market model of institutions 4 Science and ideology according to the vector-market model 5 Citations and Bibliographies IS_2023_-_CS_paper_009 IS_2023_-_CS_paper_010 IS_2023_-_CS_paper_011 Abstract 1 The problem of AI alignment 2 AI is alignable - the Orthogonalist position 3 Alignment as lobotomy – the Anti-Orthogonalist position 4 The Will to Think 5 Discussion 6 Citations and Bibliographies IS_2023_-_CS_paper_012 IS_2023_-_CS_paper_013 12 - Index - B Blank Page Blank Page Blank Page Blank Page Blank Page Blank Page