7 2591-2259 / This is an open access article under the CC-BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/ DOI: 10.17573/cepar.2023.1.01 1.01 Original scientific article Public Administration Digitalization and Government Effectiveness in the EU Countries Armenia Androniceanu Bucharest University of Economic Studies, Faculty of Administration and Public Management, Romania armenia.androniceanu@man.ase.ro https://orcid.org/0000-0001-7307-5597 Irina Georgescu Bucharest University of Economic Studies, Faculty of Cybernetics and Economic Informatics, Romania irina.georgescu@csie.ase.ro https://orcid.org/0000-0002-8536-5636 Received: 16. 3. 2023 Revised: 4. 5. 2023 Accepted: 30. 5. 2023 Published: 30. 5. 2023 ABSTRACT Purpose: This study continues the series of previous research carried out by the authors on digital transformation in the EU countries. It aims to identify and analyse the progress made by EU administrations in terms of digitization and effective governance in the period 2019–2021. Design/methodology/approach: Based on selected variables from Eu- rostat and the World Bank databases, the article provides insights into the dynamic changes that occurred in the EU in the period under consid- eration. The research employs Principal Component Analysis (PCA) and K-means clustering as the primary research methods to analyse the varia- tions and correlations among 10 numerical variables selected from Euro- stat and World Bank databases. Findings and Practical Implications: The results obtained through PCA show the impact of digitalization on government effectiveness. A com- prehensive analysis of government effectiveness has been conducted us- ing PCA, which reveals that the first two principal components account for 77.05% of the initial variance. A K-means clustering with four clusters reveals that the most significant and noteworthy increase in digitaliza- tion and government effectiveness in the EU is achieved by the Nordic countries, which exhibit high levels of digitalization and government ef- ficiency. Androniceanu, A., Georgescu, I. (2023). Public Administration Digitalization and Government Effectiveness in the EU Countries. Central European Public Administration Review, 21(1), pp. 7–30 Central European Public Administration Review, Vol. 21, No. 1/20238 Armenia Androniceanu, Irina Georgescu Originality: In this ranking, clusters 2 and 3 comprise well-developed countries with a positive relation between governmental digitalization and efficiency, while cluster 4 consists of emerging countries where the correlation between digitization and governmental efficiency is low or very low, which is primarily attributed to the limited degree of digitiza- tion within government institutions. Keywords: public administration, digitalization, digital transformation, government effectiveness, principal component analysis JEL: I28, I38 1 Introduction Digitalization and digital transformation reduce bureaucracy and increase citizens' access to public services, offering better transparency to documents, activities, and internal processes (Strafford & Schindlinger, 2019). The main objective of digitalization and digital transformation is to contribute to a pro- found change in a state's public administration, economy, and society. Digital- ization and digital transformation also contribute to increased performance and efficiency in the public sector. Studies show that European countries have different speeds in their digital transformation process. The research that is the subject of this paper is based on an analysis using a set of variables that highlight the differences between the EU states grouped in representative clusters, which reflect the influence of digitization and digital transformation on governance effectiveness. The digital transformation of public processes and services requires public in- vestments in infrastructure and equipment, but it makes the work of officials more efficient and faster and significantly influences the quality of public ser- vices for citizens and businesses. During the digital transformation process, a wide variety of cumbersome bureaucratic administrative procedures are re- placed by simple digital applications. Repetitive and bureaucratic administra- tive procedures carried out manually are replaced by simplified digital opera- tions that are particularly accessible and easy. By using computer applications that centralize electronic documents, data, and information in different databases with simultaneous access, communi- cation is more efficient, decisions are made more easily and public services are provided more quickly. In the European states where the digitization and digital transformation process started a few decades ago, civil servants and citizens are using successfully information technologies and digital tools to communicate and solve their internal problems or those of the citizens (Euro- pean Union, 2019). Digital technologies enable innovation and keep public administrations adapt- able and fit for purpose, contributing to major improvements in the govern- Central European Public Administration Review, Vol. 21, No. 1/2023 9 Public Administration Digitalization and Government Effectiveness in EU Countries ment’s effectiveness by offering innovative solutions for delivering public ser- vices to citizens and the business environment (European Commission, 2020). "Digital first" is the principle according to which some public administrations in the EU states design their policies and strategies. A unified approach at the national level regarding the reuse of public data can help the public adminis- tration to deliver better quality services for citizens and private companies. Public administration can use the existing data to support an effective deci- sion-making process in key strategic areas of the public sector. At the same time, administrations produce the necessary data and infrastructure to facili- tate the reuse of data by public and private sector organizations to increase economic competitiveness. State administrations have the role of driving the demand for digital tech- nologies, and promoting confidence in the use of new technologies in provid- ing public services and government effectiveness (Bratu, 2019). New applica- tions such as artificial intelligence and big data are integrated into different activities already (Mura and Machyniak, 2014; Harrover, 2019; Kovacova et al., 2019; Ionescu, 2019). Digitalization and digital transformation influence the effectiveness of gov- ernments in their process of correct management of public money to meet the needs of citizens and economic agents. Digital transformation of government activities is a real way to increase gov- ernment efficiency and effectiveness. In general, efficiency is correlated with rentability and productivity, being the expression of the relationship between the results recorded and the resources introduced into the public system. In other words, efficiency consists in maximizing outputs or results by using the minimum of inputs or resources. The effectiveness of government means the capability of it to deliver expected outputs. However, there was a wide debate that sometimes has created confusion about good governance and government efficiency and effectiveness (Koivisto, 2014). The conceptual confusion can also be increased due to other expressions that have delimited different debates, such as democratic governance, multi-level governance, new governance, good governance, effective governance, administrative jus- tice, global governance, global administrative law, new public management and similar others (Rhodes, 1996). The different theoretical, historical and contextual roots of all these expressions can be brought to the same com- mon denominator, namely: they can all be arguments to refer to government effectiveness. In our research, we refer to government effectiveness in the EU states, reflected by several relevant indicators: government effectiveness index, digital economy, and society index and total government expenditure (European Commission, 2021). The variables selected for this research con- tribute to the knowledge of the effects of digitalization and digital transfor- mation in the administrations of the EU states. The correlations identified and analysed, as well as the grouping of the EU member states into clusters according to government effectiveness, show the effectiveness of govern- ment decisions and policies in the EU states recorded in recent years. Practi- Central European Public Administration Review, Vol. 21, No. 1/202310 Armenia Androniceanu, Irina Georgescu cally, the considered variables measure not only the digital dimension of the changes but also their intensity and effectiveness reflected by the correla- tions between the research variables. The governments of democratic states need information regarding the meas- urement and evaluation of the economy and the effectiveness of the pro- grams or activities carried out in order to adopt decisions and measures that contribute to the improvement of government performance. If the objectives of a government were not met or were partially met, the causes that gener- ated this situation should be identified and at the same time, recommenda- tions should be formulated. Government effectiveness can be known follow- ing a process of monitoring, evaluation, and reporting of the way in which the objectives assumed by the government institutions at the central, regional, or local level of a country are achieved. The monitoring process must be continu- ous. Thus, deviations and causes can be identified, which determine concrete changes necessary to increase government effectiveness. Governmental ef- fectiveness can be the result of the effective development of the activities and processes brought together in the fields of activity of government insti- tutions. Therefore, it becomes necessary to evaluate the efficiency and ef- fectiveness of the subfields, activities, and necessary processes carried out by the state governments. The scholars (Kaufmann et al., 1999, 2011; Korv- isto, 2014) propose the development of specialized audits by governmental fields and activities. The evaluation of the performance of programs and ac- tivities should be done to determine the impact of government decisions. In the evaluation process of programs and activities, a wide range of data and information regarding their performance are examined. The efficiency of the monitoring, evaluation, and change processes is reflected in governmental ef- ficiency and effectiveness, which can increase if the internal evaluation mech- anisms along the way work efficiently and governments can make the nec- essary changes during the governance process (Rosenbloom, 2014). In this research, the correlations between the governmental effectiveness of the EU states in the digital transformation process were analysed. Our research re- sults show the impact of digital transformation on government effectiveness in the EU states grouped into four relevant clusters. In this research, the correlations between digitization and efficient govern- ance in the EU states were analysed. In the research process that is the basis of this work, the correlation between the digitization of institutions in the admin- istrations of European states and their governmental effectiveness reflected by ten specific variables selected from international databases were analysed. Next, the paper contains a literature overview. Then, the paper continues with the presentation of the variables included in the research and methodologi- cal frame. The following sections contain Principal Component Analysis (PCA) and K-means clustering, obtaining correlations that show that digitization has positively and significantly influenced government efficiency in most EU states. The cluster analysis facilitated the grouping of the EU states according to the intensity of the correlations between the variables that reflect digitiza- Central European Public Administration Review, Vol. 21, No. 1/2023 11 Public Administration Digitalization and Government Effectiveness in EU Countries tion and those that reflect government efficiency. The data analysis revealed that a cluster of Nordic countries was placed on top, followed by a cluster of well-developed countries and two clusters of emerging states. Then, some of their most important and significant elements of impact in 2021-2022 are presented. The last part of this work contains the main conclusions and con- tributions to the specialized literature. 2 Literature overview In most EU member states, there has been an extremely rapid increase of the digital transformation processes. Four strategic axes were identified for the digital transformation of public services for the citizens and the private sector (Rymarczyk, 2021). These four strategic axes are: (1) digital public ad- ministration that consists of a process of digital transformation of public ad- ministration to that of smart government; (2) digital economy through the digital transformation of economies, in general and of small and medium- sized companies; (3) digital education through the digital transformation of the educational process through the transversal inclusion of technology in all educational processes (Stoyanova et al., 2022), but especially through the im- plementation of policies to acquire/increase/improve digital skills at the level of the entire population; (4) digital society meaning that citizens are leaving in a new digital social frame based on a variety of accessibilities both for work and different public services like health, social assistance, public transport, re- laxation, and recreation (Lindgren et al., 2019). The implementation of these strategic changes is carried out on four levels of representation: digital infra- structure (level 1); hardware and software digital tools (level 2); digital com- petencies and skills (level 3); digital ecosystems (level 4 – companies, public in- stitutions and citizens). The four strategic axes and levels reflect the fact that digital transformation is a process that requires a systemic and integrated ap- proach, which includes the economy, administration and society as a whole (Abraham et al., 2020; Berman, 2012). The reduction of digital transforma- tions in one or another of these areas can significantly influence the efficiency of the whole process. If in the economic fields, and especially in the business field, the effects of investments in the implementation of digital transforma- tions have already demonstrated efficiency and effectiveness, in the fields of administration and public services, the digital transformation continues at an accelerated speed. The researchers (Avram, 2020; Novak et al., 2018) show that there are correlations and gaps between the three categories of digi- tal transformations (economic, administrative, and social) in different states, mainly due to government strategies and investment policies that differ in content (Drechsler, 2022; Demir, 2022). In these areas, the major challenge that the government assumes, is to overcome in the next 4 years several stag- es of the digital transformation of government and public administration, so that it acts based on the principles of smart government characterized by the administration based on data (data-centric) and able to use analytical and pre- dictive tools such as artificial intelligence and data mining (Yigitcanlar et al., 2021; Shank et al., 2019). Government effectiveness can be influenced by the Central European Public Administration Review, Vol. 21, No. 1/202312 Armenia Androniceanu, Irina Georgescu speed with which digital transformations take place, and in turn, it can influ- ence the speed with which digital changes occur in the economy, administra- tion and society. This perspective is analyzed in the research that is the basis of this work. Our objective is to demonstrate that government effectiveness is significantly influenced by digital transformations in state administrations and contributes to the delivery of better public services (Demircioglu, 2017). For this, we considered a set of specific variables, including the public ex- penditures made by the governments of the European states, which means total costs that also include the specific costs necessary for digital changes in administration and society. Practically, the ability of the governments of the member states to effectively manage the digital transformations in the administration in order to be able to offer better administrative and social services means governmental effectiveness, which is one of our key variables. Digital transformation involves the use of digital technologies and data to change the way of working and interaction between public administration stakeholders (Dunleavy et al., 2006). Examples of digitalization projects include automating processes or training human resources to use new technologies in the work environment. To consider that digitalization, in the sense described above, is sufficient to ensure the transition to a digital society is a strategic er- ror (Kitchin, 2014). In the field of e-government, for example, a few states stop at digitization meaning downloading forms from the Internet or digitalization by filling in these forms online, instead of making the transition to a full digital transformation by moving services of e-government in the digital environment (Șandor et al., 2020; Munoz & Bolivar, 2018; Mura & Vlacseková, 2018). Digital transformation is progressing in the UE member states (Micheli et al., 2018). Studies (Szostak, 2022; Taburchak et al., 2022; Craglia et al., 2018) in the field highlight the fact that Estonia is globally recognized as the most digi- talized country in the world, with a rate of digitization of interactions between the state and citizens of 99% (Sabatini et al., 2022; Stępień & Światowiec- Szczepańska, 2022). Estonia is a country where the only non-digital interac- tions citizens have with state officials are at marriage or divorce. All this is due to a digitization process developed over the last 20 years. From a statistical point of view, 98% of Estonian citizens have an identity card based on which they can access 99% of the services the state provides, such as online vot- ing. In other words, for the release of any document needed by citizens from the Estonian state, they use the method of online requests or even reduce to an agreement for public institutions to collaborate and access their informa- tion from each other's archives, without having to circulate any paper from side to side. The various types of data that can circulate in the network of state databases can be stored in only one of them, by one of the agencies responsible for example for personal data of citizens, land, or taxes, and can be accessed by other agencies only with the consent of the citizen and the in- stitution holding the information. Each public institution has its own database, created according to the interactions it has had with certain citizens. Public institutions access an interface to communicate with a national network of databases (Micheli et al., 2020). In recent years, Estonia's digital system has Central European Public Administration Review, Vol. 21, No. 1/2023 13 Public Administration Digitalization and Government Effectiveness in EU Countries expanded to collaborate with other countries such as Finland, Portugal, and Croatia, especially for the interoperability of prescriptions received by citizens of any of these countries. For example, a drug prescription received by an Es- tonian is also valid at a pharmacy in Portugal, if the Estonian is in its territory. Contrary to expectations, interacting with digital services is not difficult at all. Each person has a digital ID card and even a mobile ID with the same functions. Specialists (Sun et al., 2022) consider that the main risk of digitization is data security. Regarding this, the Estonian authorities have ensured the existence of a "digital embassy", outside the country, a huge archive in which the entire national network is regularly backed up. Data stored in this "digital embassy" in Luxembourg can be safely accessed even when possible problems arise with the network and databases in Estonia. The digital system in this country is accessible and efficient. Statistical data show that the digitization of the ad- ministration and de-bureaucratization saves almost 2% of the GDP (European Commission, 2021b). In addition, each citizen saves time with each necessary document that they issue online, instead of going to a counter, saving annu- ally, at the country level, approximately 1345 years of work, collected at the level of all citizens (International Labour Organization, 2022). Another example of successful digital applications for public administration works in France. Statistics show that in recent years the French government has announced that it has decided to adopt and use the Office package from Microsoft 365 at the national level, as well as several services with Microsoft Azure. However, they will not be delivered from the Data Centers that Mi- crosoft has in 13 European countries - Austria, Denmark, France, Germany, Greece, Ireland, Italy, the Netherlands, Norway, Poland, Spain, Sweden and Switzerland - but from centers owned and operated by French companies. Similarly, in Germany, to comply with the federal government's "Cloud Sover- eignty" requirements, Microsoft has proposed a model whereby the owner- ship and management of the cloud platform rest with the companies that own and manage the data centers from which they are delivered. Analysts esti- mate that the German and French models can be replicated by other countries (Kohler & Weisz, 2018). Thus, the Cloud services delivered by Microsoft could become the de facto standard in the field of public administration in the EU. Most EU states have not stopped at digitalization. They continued the process of digital transformations. First, it requires a new cultural orientation that em- phasizes challenging the status quo and continually testing the best solutions. Also, digital transformation is characterized by customer/beneficiary-oriented organizational changes, supported by leadership and driven by significant challenges from the environment. Digitalization and digital transformation need a legislative framework and a governance framework for digital public services, where relevant actors can collaborate and co-create solutions with other private partners, such as citizens, and the academic or business sector. Proponents of governance for the digital age of public services refer to two characteristics that must be found in the provision of digital public services. The authors of this governance model (UNDP, 1997) do not support the idea of Central European Public Administration Review, Vol. 21, No. 1/202314 Armenia Androniceanu, Irina Georgescu the supremacy of technology in administration but use it as a means for reor- ganization and for encouraging the participation of citizens in the co-creation of public policies. In fact, they emphasize the need for a fundamentally differ- ent way of organizing the processes in the administration that will lead to the rethinking of services from the perspective of the needs of citizens, but also to the elimination of redundant processes from the old administrative culture. Two of the principles enunciated by them are also found in European public policy documents, namely Government as a Platform (GaaP) and the once-only principle. If once only refers to the reintegration of services, GaaP refers to the creation of a broad, national integrated framework of digital services. Faced with such challenges, the city halls of the Member States saw the con- cept of a "smart city" as way to use technology for a better life for citizens in cities and municipalities (Calzada, 2017). Unfortunately, in the rush for tech- nological progress, city halls have lost sight of the desire to improve citizens' experience with the city, and technology has become an end in itself, not a means, as it was intended (Eom et al., 2016; Howell & Higgins,1990). So, in a short time, the expensive investments in smart traffic lights, for example, were outclassed by Waze-type applications or other alternative urban mobil- ity applications (Gutschow, 2019; Lee et al., 2019; Torugsa & Arundel, 2016). By the same token, parks with free Wi-Fi have proven difficult to maintain in a cyber-insecure area, and the examples can go on (Meijer & Thaens, 2021; Barabashev et al., 2022). Town halls in EU states collect and hold significant volumes of data. Even so, in some EU states, including Romania, these vol- umes are reduced in relation to what a city could obtain and use (Avram, 2020; Șandor et al., 2020). Regardless of the stage of digital transformation in the local administrations of the EU states, public investments in data infra- structure and their best possible analysis and decision-making capacity are essential (Pollitt & Bouckaert, 2011; Margetts, 2009). According to the European Framework of Competences (DigiComp), there are 21 competencies grouped into five key categories necessary for the digi- tization of public administrations in the EU states. Therefore, digitalization has a major impact both on public administration, the economy, and the well-being of citizens in general (Androniceanu A-M. et al., 2020). In a previous paper (Androniceanu et al., 2022) we studied the impact of digitalization on public administration, economic development and well-being in EU countries during 2019-2021, by Principal Component Analy- sis and cluster analysis. Digitalization is influencing the efficiency of public ad- ministration. The origins of effective governance and modern good govern- ance lie in the structural adjustment and development support programs of the late 1980s. For the first time, the World Bank called crises in sub-Saharan Africa governance crises (World Bank, 1992). As it follows from the official documents of several international organizations, International Monetary Fund, United Nations Development Programme and the World Bank (1997) initially good governance was focused on solving the problems generated by the poor functioning of economic mechanisms (Privara, 2022). Later, the con- Central European Public Administration Review, Vol. 21, No. 1/2023 15 Public Administration Digitalization and Government Effectiveness in EU Countries cept also extended to political and public administration aspects (Duong et al., 2022). Therefore, the key aspects of good governance can be expressed as follows: a predictable and transparent policy and administration, an ethi- cal and socially responsible government, a strong civil society involved in the management of public institutions, the existence of a rule of law, respect for democracy and human right, a functioning market economy (Androniceanu, 2019). A trans-disciplinary approach to the concept and practices of good governance can ensure a better understanding from a multi-disciplinary per- spective. Also, such an approach can provide a better understanding of the challenges involved in identifying good governance solutions. Thus, we can have a better analysis and appreciate good governance practices (Chottray & Stoker, 2009, p. 214). Chottray and Stoker argue that the cross-disciplinary approach to good governance can be achieved through an investigative ap- proach rather than drawing up a list of normative principles against which the system is checked. Governmental efficiency is directly dependent on the abil- ity of public institutions to manage resources to solve social problems (Trettin et al., 2019). Policies and public services cannot be reformed and delivered efficiently without someone thinking about them, applying them, and solving them. The link between the effective functioning of governments and digiti- zation depends on good governance and is reflected in government efficien- cy. Over time, EU governments have focused their resources on governance as a key factor that reflects variations in government efficiency. Thus, in the World Bank Report on Governance and Development, from 1997, governance means that the government deal with the national resources for effective de- velopment (World Bank, 1997). Later, in 2007, the definition of effective gov- ernance also included the way in which public officials and institutions pro- vide public goods and services (World Bank, 2007). Specific indicators were developed by UNDP (1997) through which good governance and its impact on government efficiency could be evaluated. However, there is a growing demand for a wider range of indicators, with some countries even calling for newer or better good governance laws (Trapnell, 2011). 3 Research variables and methodological frame In this section, the main variables together with the methodological frame are discussed. The selected variables have been chosen with respect to the purpose of the analysis, to analyse the progress towards digitization and effective governance that the administrations of the EU states have made during 2021-2022. By applying Principal Component Analysis and K-means clustering, we also examine the digital divide existing in the public adminis- trations of EU countries. The combination of variables also used in previous research (Georgescu et al., 2023; Georgescu and Kinnunen, 2021) is impor- tant in revealing regions or countries with similar digital inequality and similar levels of government effectiveness. Also, the data were collected according to the three pillars of this research: digitalization, government effectiveness, and public administration infrastructure. The main research variables used are presented in Table 1. Central European Public Administration Review, Vol. 21, No. 1/202316 Armenia Androniceanu, Irina Georgescu Table 1. The main research variables Variable Variable Label Measurement Unit Source OSI Online services index percentage publicadministration.un.org GEF Government effectiveness index [-2.5 weak; 2.5 strong] www.maxinomics.com EGDI E-Government index [0,1] publicadministration.un.org GOVEX Total general government expenditure percentage of GDP Eurostat ICT Employed ICT specialists - total percentage Eurostat IT Internet use by individuals percentage Eurostat CON Connectivity [0,100] https://digital-strategy. ec.europa.e INT Integration of digital technology [0,100] https://digital-strategy. ec.europa.eu DPS Digital public services [0,100] https://digital-strategy. ec.europa.eu DESI The Digital Economy and Society Index [0,100] https://digital-strategy. ec.europa.eu Source: Our selection is based on various databases The data set composed of 10 numerical variables was gathered from vari- ous databases for 2021 and 2022. PCA is a data reduction technique used to analyze highly dimensional datasets (Kassambara, 2017; Jolliffe, 2002). By a linear transformation, the initial data are transformed into fewer variables called principal components (PCs) which retain the maximum amount of in- formation. The steps of PCA are the following: 1. Data standardization. 2. Computing the covariance matrix of the dataset. 3. Obtaining the eigendecomposition according to the covariance matrix. 4. Ranking the eigenvalues decreasingly. 5. Selecting the first k PCs. 6. Determination of the k-dimensional feature space. Each PC retains a certain amount of variation represented by the eigenvalues, as seen in Table 2. Central European Public Administration Review, Vol. 21, No. 1/2023 17 Public Administration Digitalization and Government Effectiveness in EU Countries Table 2. Eigenvalues and the cumulative variance Source: Authors own computation The first column of Table 2 displays the variation represented by the eigenval- ues. Eigenvalues and eigenvectors are computed from the covariance matrix to determine the PCs. The eigenvectors are the directions of axes (PCs) while the eigenvalues are the variances captured by PCs. The other columns of Ta- ble 2 represent the individual variance and the cumulative variance associated with each dimension. After the PCA application, the analysis continues with K-Means clustering (MacQueen, 1967) in Section 3. K-means clustering is one of the most effi- cient clustering techniques. The distance between the objects is the Euclid- ean distance and the cluster scatter is measured by variance. 4 Research results and discussions This section contains the main results of the applied methods, namely PCA and K-Means clustering. The section ends with a comparison of our results with previous similar studies. For the application of PCA, we use the scree plot and the principle of cumulative variance to determine the optimal number of PCs. Then we use K-means to cluster the 27 EU countries into 4 clusters. The first PC captures 66.43% of the initial variability. The second PC captures 12.61% of the initial variability. The cumulative variance is shown in the third column. The first two PCs explain 77.05% of the total variance. By Borcard et al. (2018), we retain a number of PCs which account for around 75% of the total variation. Thus the first two PCs will be retained, at time same time in accordance with Kaiser’s rule (Kaiser, 1960), which says that we retain the PCs with an eigenvalue greater than 1. This is also confirmed by the scree plot in Figure 1. Central European Public Administration Review, Vol. 21, No. 1/202318 Armenia Androniceanu, Irina Georgescu Figure 1. Scree plot explained by principal components Source: Authors’ determination The biplot in Figure 2 displays the correlations between variables and their contributions to PCs. One can see in Figure 2 that all variables are grouped together, being positively correlated. Countries with similar features are po- sitioned closer to each other. Figure 2. PCA biplot with countries and variables Source: Authors’ determination Central European Public Administration Review, Vol. 21, No. 1/2023 19 Public Administration Digitalization and Government Effectiveness in EU Countries On the biplot from Figure 2, the objects are represented as points (here coun- tries) and the vectors denote the variables (Kroonenberg, 2008). On the biplot, the variables are characterized by their correlations (Abdi and Williams, 2010). If the angle between two vectors is small, then the variables represented by vectors are strongly associated. The strong correlation between variables and PCs is shown by high cos values. High cos values indicate that the variables are located near the circumference of the correlation circle. Small cos values indi- cate that the variables are positioned near the circle centre. Small cos values belong to the variable CON. Middle cos values belong to ICT, IT, and OSI. High cos values belong to variables GOVEX, EGDI, INT, GEF, DESI and DPS. Variables situated closer to the correlation circle centre do not present so much importance for the first PCs (Kassambara, 2017). Countries like Denmark, Finland, Sweden, and Netherlands are mainly deter- mined by OSI, EGDI, INT, GEF, DESI, DOS and ICT. Luxembourg and Estonia are mostly characterized by ICT, DPS and IT. Malta is mostly dominated by GEF, DESI and DPD. France, Germany, Italy, and Austria are mostly dominated by GOVEX. Slovenia and Spain are mainly determined by CON, OSI, EDGI and INT. On the opposite quadrant are Romania, Bulgaria, Poland, Czech Republic, Hungary, Croatia, Slovakia, Greece, Italy. A similar representation on the fac- tor map is obtained by Androniceanu et al. (2022), whose results confirmed that there is a digital divide between the countries situated on opposite quad- rants. The countries in the right quadrant are mainly developed economies, for which digitalization, ICT and e-government index are on an increasing trend. For these countries, ICT improves the communication between government and citizens, the government services are more accessible, more transparent and more democratic. Dobrolyubova et al. (2019) confirm this assertion, that government digitalization makes more efficient public services. On the left quadrant on the biplot in Figure 2 are positioned mainly emerging economies, between whom and the developed economies the digital divide still exists (Androniceanu et al., 2022; Mesa, 2023). During the COVID-19 pan- demic, internet usage increased, making the digital gap less wide. The governments were spurred to accelerate digital transformations. The countries lying on the left quadrant need important investments in infrastruc- ture and experimenting with new technologies. Central European Public Administration Review, Vol. 21, No. 1/202320 Armenia Androniceanu, Irina Georgescu Figure 3. Correlation matrix Source: Authors’ determination Figure 3 displays the histograms, density functions and smoothed regression lines together with correlation coefficients at different levels of significance (no stars mean not significant, *, ** and *** signify 10%, 5% and 1% levels of significance). As we can also notice from the correlation matrix in Figure 3, the strongest as- sociations are between EGDI and OSI (0.89), GEF and DESI (0.9), DESI and INT (0.89), DESI and DPS (0.89), EGDI and DESI (0.87), GEF and IT (0.85). GOVEX is weakly correlated with most variables. GOVEX are fiscal instruments by which the state intervenes in the economy. GOVEX is meant to be a living standard indicator, with a role in investments in innovation and in new technology. Its weak correlation with most of the indicators shows that public spending has a minimal influence in stimulating economic growth, unlike private spending (Pop Silaghi et al., 2014). In Figure 4 are displayed the square cosines of variables on the first 5 PCs. The first 5 PCs are not correlated. Central European Public Administration Review, Vol. 21, No. 1/2023 21 Public Administration Digitalization and Government Effectiveness in EU Countries Figure 4. The cos2 of variables on the first 5 PCs Source: Authors’ determination PC1 is dominated by GEF, OSI, EGDI, GOVEX, ICT, IT, DESI, CON, INT and DPS. The first direction will be named the impact of digitalization on government ef- fectiveness. PC2 is dominated by government expenses GOVEX. It follows that the total expenditures are not affecting any other variable, which is what this PC separation suggests. This is because GOVEX is weakly correlated with the other variables, as shown by the correlation matrix. PC3 is dominated by CON. The next step of the research was the application of the K-Means clustering algorithm. The steps of the K-means algorithm are the following: 1. Determination of the number of centroids. The initial centroids are selec- ted randomly. 2. The objects are assigned to the nearest centroid. 3. The centroids are computed again and a new assignment of objects to clu- sters takes place. 4. The algorithm is repeated until the centroids become unchanged or until the objects cannot be allocated to other clusters. According to the elbow method (Thorndike, 1953), the optimal number of clusters is 4. The clusters composition is the following: Cluster 1: Estonia, Ireland, Luxembourg, Malta Cluster 2: Netherlands, Denmark, Sweden, Finland Cluster 3: Belgium, Czechia, Germany, Spain, France, Italy, Cyprus, Latvia, Lit- huania, Austria, Slovenia, Portugal Cluster 4: Bulgaria, Greece, Croatia, Hungary, Poland, Romania, Slovak Repu- blic. Central European Public Administration Review, Vol. 21, No. 1/202322 Armenia Androniceanu, Irina Georgescu In the cluster plot from Figure 5, we can see that the four clusters are well separated in the plane determined by the first two PCs. Figure 5. Cluster Plot Source: Authors’ determination Table 2 contains the cluster means of scaled data. The countries in cluster 2 have the highest values for all variables. The Nordic countries together with the Netherlands are the most digitalized, the implementation of IT services in public administration increases transparency and makes citizens more in- volved in administrative and political activities (Bernhard et. al, 2018). Citi- zens can benefit from e-government in three directions according to Bern- hard et al. (2018): first, the living conditions such as education, health, and infrastructure, secondly the digitization of services for citizens and third, the decision-making processes made available for public scrutiny. Pérez-Morote et al. (2020) analyze an EU panel during 2010-2018 and their results confirm the hypothesis that the use of e-government services is positively correlated with the digital divide caused by income and education. Their analysis contin- ues with a 3-cluster analysis, in which the low cluster contains most Eastern European countries together with Italy and Greece, the medium cluster is het- erogeneous while the high cluster comprises the Nordic countries, France and Luxembourg. This cluster is characterized by a high level per capita, higher use of e-government, more investments in education, and a younger urban population. Ma and Zheng (2018) explore a survey of 28000 EU citizens and find that e-government performance is negatively correlated with the use of e-information and e-services. Central European Public Administration Review, Vol. 21, No. 1/2023 23 Public Administration Digitalization and Government Effectiveness in EU Countries Table 2. Cluster Means Cluster GEF OSI EGDI GOVEX ICT IT DESI CON INT DPS 1 0.666 0.56 0.486 -1.216 0.907 0.874 0.731 -0.233 0.316 1.065 2 1.389 1.325 1.619 0.452 1.574 1.181 1.555 1.221 1.645 0.963 3 0.038 -0.144 -0.069 0.322 -0.413 -0.065 -0.041 0.033 0.002 0.055 4 -1.24 -0.829 -1.083 -0.116 -0.709 -1.062 -1.235 -0.627 -1.126 -1.254 Source: Authors’ determination As a result, the countries in cluster 2 also have the highest DESI index. Coun- tries in cluster 1 are positioned second, with regard to most of the indicators. In this group of countries, Estonia is an important frontrunner in digital pub- lic services, being in the 7th place in the DESI 2021 hierarchy. Estonia is one of the most advanced digital economies, keeping online 99% of government services during the pandemic. The last position in this ranking is occupied by the countries in cluster 4, where Romania, Hungary, Slovakia, Poland and Bul- garia are positioned. Wandaogo (2022) conducted a panel data study for 139 countries during 2006-2016 and concludes that the positive effect of digitali- zation on public administration is higher in developed countries than in de- veloping countries. The reverse causal effect may also happen, an effective government will induce the use of digitalization to provide online services or to improve their quality. Pakhnenko and Kuan (2023) group world countries with respect to their digital governance development. From this dataset, 69 countries have a very high level of digital governance, followed by a cluster of 46 countries with a high technological level in public administration. In this analysis, the GovTech Maturity index and its components are included. Among EU countries, Romania, Bulgaria, Slovakia, Poland, and Ireland have a GoVTech Maturity index varying between 0.5 and 0.75. The three countries at the top of GovTech Maturity index for 2022 are Estonia, France, and Lithuania. Park et al. (2022) introduce the Digital Transformation Index (DTI) to study the digital divide and its causes. The study delimits three phases of digital transforma- tion: foundation, adoption, and acceleration. 106 world countries are included in the DTI framework and grouped into five clusters, according to the five in- tervals of DTI scores. In the DTI ranking, USA places first, followed by Swit- zerland and UK & Northern Ireland. Romania places in the second interval of DTI scores, together with 31 world countries, among which: Hungary, Bulgaria, Latvia, Poland, Slovakia and Greece. DTI’s significance consists in the positive correlation between income and the digital transformation level of countries. 5 Conclusions In this paper, we applied PCA and cluster analysis to study the panel of 27 EU member states and 10 variables collected for the period 2021-2022. PCA revealed that two PCs are retained, explaining 77.05% of the original variabil- ity. The first dimension is dominated by most of the variables included in the study, therefore it was called the impact of digitalization on public administra- Central European Public Administration Review, Vol. 21, No. 1/202324 Armenia Androniceanu, Irina Georgescu tion. The second dimension is dominated by government expenses. The com- bination of these indicators in the composition of the first PC measures the short-term effects of digitalization on public administration and government effectiveness. In a K-means clustering with 4 clusters, we obtain that cluster 2 which contains Nordic countries places on top, followed by cluster 1, cluster 3 of well-developed countries and cluster 4 of mainly emerging economies. The results of this paper have certain policy implications, such as the enhance- ment of e-government could contribute to improved government effectiveness. Other policy implications could be oriented to reduce corruption and more trans- parent institutions and public services (Sadik-Zada et al., 2022; Osborn, 2006). The research results obtained about the digital transformation in the Euro- pean public administration proved the fact that it increases the quality of the governance process as a whole. It must be understood that digital solutions mean more than the transfer of decision-making processes from paper to digi- tal format. In addition, these improve the quality of governance by minimizing the influence of corruption factors, provide equal chances and opportunities to all members of society, and much greater transparency of public services and expenditures (Głodowska et al., 2023). Digital transformation of public ad- ministration makes everyone’s life much simpler, more efficient, and harmoni- ous. In some cases, in addition to improving some processes, it fundamentally changes them (Korzynski et al., 2023). Effective e-government leads to good governance. The digital divide at the country level can be reduced by urgent technological and financial support for countries with a lower level of digital transformation or in cases when this cannot be assessed. Müller et al. (2022) discuss the potential EU-added value in different directions of social and la- bour policy, with the purpose to move to a convergence among EU countries. The results of our research can be used in the extensive transformation process of European administrations until 2030. As the European Commis- sion shows, a mechanism for the digital transformation of Europe is being launched in 2030. The European Parliament, the member states, and the Commission have jointly established objectives and targets in the four key areas namely digital skills, infrastructure including connectivity, digitalization of businesses, and online public services, in line with the Declaration on Euro- pean Digital Rights and Principles. In this context, government effectiveness can become a necessary and relevant indicator. In our future research, the current research results could be completed with an increased number of research variables and with different panel data tech- niques, for longer time series. Alternatively, we may tackle these issues by qualitative methods or other statistical approaches. Acknowledgment The paper has been prepared under the research project with the title Digitiza- tion, digital transformation and artificial intelligence in public administration – 2023 financed by the Bucharest University of Economic Studies and is part of the research strategy of the International Centre for Public Management. Central European Public Administration Review, Vol. 21, No. 1/2023 25 Public Administration Digitalization and Government Effectiveness in EU Countries References Abdi, H. and Williams, L. J. (2010). Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4), pp. 433–459. doi:10.1002/wics.101 Abraham, R., Schneider, J. and Vom Brocke, J. (2019). Data governance: A conceptual framework, structured review, and research agenda. International Journal of Information Management, 49, pp. 424–438. Androniceanu, A., Georgescu, I. and Sabie, O.-M. (2022). The Impact of Digitalization on Public Administration, Economic Development, and Well- Being in the EU Countries. Central European Public Administration Review, 20(2), pp. 9–31. Androniceanu, A. and Georgescu, I. (2021). E-Government in European countries, a comparative approach using the Principal Component Analysis. NISPAcee Journal of Public Administration and Policy. XIV(2), pp. 65–86, 2021, DOI: https://doi.org/10.2478/nispa-2021-0015 Androniceanu, A. (2019). The social sustainability of smart cities: urban technological innovation, big data management, and the cognitive Internet of Things. Geopolitics, History, and International Relations 11(1), 110–115. doi:10.22381/GHIR11120197 Androniceanu, A. et al. (2020). A Multidimensional Approach to Competitiveness, Innovation and Well-Being in the EU using Canonical Correlation Analysis. Journal of Competitiveness, 12(4), pp. 5–21. https://doi.org/10.7441/ joc.2020.04.01 Avram, A. (2020). Digitization of services in the member states of the European Union. Journal of World Economy of the Romanian Academy, 12(1), 49-52. Barabashev, A., Makarov, I. and Zarochintcev, S. (2022). How to shape government policies on high-technology development using the indicative evaluation of risks? Administratie si Management Public, 38, pp. 70–89. Doi:10.24818/amp/2022.38-04 Berman, S.J. (2012). Digital transformation: opportunities to create new business models. Strategy & Leadership, 40(2), pp. 16–24. https://doi. org/10.1108/10878571211209314. Bernhard, I. et al. (2018). Degree of Digitalization and Citizen Satisfaction: A Study of the Role of Local e-Government in Sweden. The Electronic Journal of e-Government, 16(1), pp. 59–71, Available online at www.ejeg.com. Accessed on 8 December 2022. Borcard, D., Gillet, F., and Legendre, P. (2018). Numerical ecology with R. 2nd edition. Springer, New York, NY. Bratu, S. (2019). Algorithmically constructed identities: networked digital technologies, dynamic behavioral big data collection, and automated decision-making. Contemporary Readings in Law and Social Justice, 11(2), 49–55. doi:10.22381/CRLSJ11220197 Calzada, I. (2017). From smart cities to experimental cities? In: Giorgino VM and Walsh ZD (eds) Co-Designing Economies in Transition: Radical Approaches in Dialogue with Contemplative Social Sciences. London: Palgrave Macmillan, pp. 191–217. Craglia, M, Annoni, A., and Benczur, P. (2018). Artificial Intelligence: A European Perspective. Report, Joint Research Centre. Publications Office, Luxembourg. Demir, F. (2022). Innovation in the public sector. Smarter states, services and citizens, Cham: Springer. Central European Public Administration Review, Vol. 21, No. 1/202326 Armenia Androniceanu, Irina Georgescu Demircioglu, M.A. (2017). Reinventing the Wheel? Public Sector Innovation in the Age of Governance’. Public Administration Review, 77(5), pp. 800–805. Dobrolyubova E., Klochkova E., and Alexandrov. O. (2019). Digitalization and Effective Government: What Is the Cause and What Is the Effect? In D. Alex-androv, A. Boukhanovsky, A. Chugunov, Y. Kabanov, O. Koltsova and I. Musabirov (eds). Digital Transformation and Global Society. DTGS 2019. Communications in Computer and Information Science 1038. Cham: Springer, pp. 55–67. doi:10.1007/978-3-030-37858-5_5. Dolan-Canning, R. (2019). Sensor-based and cognitive assistance systems in industry 4.0: big data analytics, smart production, and sustainable value creation. Economics, Management, and Financial Markets 14(3), 16–22. doi:10.22381/EMFM14320192 Drechsler, W. (2022). Public administration studies: the digital trajectory. Transylvanian Review of Administrative Sciences, [S.l.], pp. 40–51, dec. 2022. ISSN 1842-2845. Available at: . Date accessed: 19 May 2023. doi:http://dx.doi.org/10.24193/tras. SI2022.4. Dunleavy, P., H. Margetts, S. Bastow and Tinkler J. (2006). Digital era governance: IT corporations, the state, and E-government. Oxford: Oxford University Press. Duong, C. et al. (2022). Entrepreneurial education, government policies and programmes, and entrepreneurial behaviour: A serial moderated mediation model. Entrepreneurial Business and Economics Review, 10(4), pp. 37–54. https://doi.org/10.15678/EBER.2022.100403 Eom, S., Choi, N. and Sung, W. (2016). The use of smart work in government: empirical analysis of Korean experiences. Government Information Quarterly, 33(3), pp. 562–571. European Commission. (2019). Digital Economy and Society Index (DESI). Retrieved from: https://digital-strategy.ec.europa.eu/en/library/digital- economy-and-society-indexdesi-2019. Accessed on 11 January 2022. European Commission. (2020). The Digital Europe Programme. Retrieved from https://digital-strategy.ec.europa.eu/en/activities/digital-programme. Accessed on 7 January 2022. European Commission. (2021a). The Digital Economy and Society Index (DESI). Retrieved from: . Accessed on 8 January 2022. European Commission (2021b). Digital transformation of public administration and services. Retrieved from: https://knowledge4policy.ec.europa.eu/ foresight/digital-transformation-public-administration-services_en. Accessed on 19.05.2023 Georgescu, I., Androniceanu, A. M. and Drăgulănescu, I.V. (2023). A Deep Learning Approach to Digitalization and Economic Growth. In: Busu, M. (eds) Digital Economy and the Green Revolution. ICBE 2022. Springer Proceedings in Business and Economics. Springer, Cham. https://doi.org/10.1007/978-3- 031-19886-1_6 Georgescu, I. and Kinnunen, J. (2021). The Digital Effectiveness on Economic Inequality: A Computational Approach. In: Dima, A.M., D'Ascenzo, F. (eds) Business Revolution in a Digital Era. Springer Proceedings in Business and Economics. Springer, Cham. https://doi.org/10.1007/978-3-030-59972-0_16 Central European Public Administration Review, Vol. 21, No. 1/2023 27 Public Administration Digitalization and Government Effectiveness in EU Countries Głodowska, A., Maciejewski, M. and Wach, K. (2023). Digitalization, digital transformation and technology-based entrepreneurship in contemporary business. International Entrepreneurship Review, 9(3), ahead of print. Gutschow, E. (2019). Big data-driven smart cities: computationally networked urbanism, real-time decision-making, and the cognitive internet of things. Geopolitics, History, and International Relations 11(2), pp. 48–54. doi:10.22381/GHIR11220197 Harrower, K. (2019). Algorithmic decision-making in organizations: network data mining, measuring and monitoring work performance, and managerial control. Psychosociological Issues in Human Resource Management, 7(2), pp. 7–12. doi:10.22381/PIHRM7220191 Howell, J. and Higgins, C. (1990). Champions of technological innovation. Administrative Science Quarterly, 35(2), pp. 317–341. International Labour Organization (2022). Digitalization and Employment – A review. At , accessed on 20 March 2022. Ionescu, L. (2019). Big data, blockchain, and artificial intelligence in cloud-based accounting information systems. Analysis and Metaphysics, 18, pp. 44–49. doi:10.22381/AM1820196 Jolliffe, I. T. (2002). Principal Component Analysis. Second Edition, Springer. Kaiser, H. F. (1960). The Application of Electronic Computers to Factor Analysis. Educational and Psychological Measurement 20, pp. 141–151. Kassambara, A. (2017). Practical Guide to Principal Component Methods in R, Sthda. Kaufmann, D., Kraay, A. and Zoido-Lobaton, P. (1999). Governance Matters. World Bank Policy Research Working Paper, No. 2196.39. 5. Kaufmann, D., Kraay. A. and Mastruzzi, M., (2011). The Worldwide Governance Indicators: Methodology and Analytical Issue, 3, Hague Journal on the Rule of Law, p. 220. Kohler, D. and Weisz. J-D. (2018). The Digital Transformation of the Industry: A Franco-German Issue. At , accessed on 1 May 2023. Kitchin, R. (2014). The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. London: Sage. Koivisto, I., (2014). Varieties of Good Governance: A Suggestion of Discursive Plurality. Int J Semiot Law, [e-journal] 27, pp. 587–611. DOI 10.1007/s11196- 013-9329-6. Korzynski, P. et al. (2023). Generative artificial intelligence as a new context for management theories: analysis of ChatGPT, Central European Management Journal, 31(1), ahead of print. https://doi.org/10.1108/CEMJ-02-2023-0091 Kovacova, M. et al. (2019). Big data governance of automated algorithmic decision-making processes. Review of Contemporary Philosophy, 18, pp. 126–132. doi:10.22381/RCP1820196 Kroonenberg, P. M. (2008). Applied Multiway Data Analysis. John Wiley & Sons, Inc. Lee, E., Li, X. and Jung, K. (2019). Policy innovation across Korean local governments after democratization: community safety committee and municipal police system. Korea Observer, 50(3), pp. 443–478. Central European Public Administration Review, Vol. 21, No. 1/202328 Armenia Androniceanu, Irina Georgescu Lindgren, I. et al. (2019). Close encounters of the digital kind: A research agenda for the digitalization of public services. Government Information Quarterly, 36(3), pp. 427–436. Ma L. and Zheng Y. (2018). Does e-government performance actually boost citizen use? Evidence from European countries. Public Management Review, 20(10), pp. 1513–1532. MacQueen, J. B. (1967). Some Methods for classification and Analysis of Multivariate Observations. Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, no.1. University of California Press, pp. 281–297. Margetts, H. (2009). Public management change and e-government: The emergence of digital era governance, in A. Chadwick and P. Howard (eds.) Routledge Handbook of Internet Politics. London: Routledge. Meijer, A. and Thaens, M. (2021). The dark side of public innovation. Public Performance and Management Review, 44(1), pp. 136–154. Micheli, M. et al. (2020). Emerging models of data governance in the age of datafication. Big Data & Society, July–December, pp. 1–15. Doi: 10.1177/2053951720948087 Micheli, M, Blakemore, M. and Ponti, M. (2018).The governance of data in a digitally transformed European Society. Second Workshop of the DigiTranScope Project, European Commission, JRC114711. Mesa D. (2023) Digital divide, e-government and trust in public service: The key role of education. Front. Sociol. 8(1140416). doi: 10.3389/fsoc.2023.1140416 Morozov, E. and Bria, F. (2018). Rethinking the Smart City: Democratizing Urban Technology. New York, NY: Rosa Luxemburg Stiftung. Müller, K., Navarra, C. and Jančová, L. (2022). Improving the quality of public spending in Europe. Study. European Parliamentary Research Service PE 699.487. At , accessed on 1 May 2023. Munoz, L.A. and Bolivar, M. (2018). International E-Government Development. Policy, Implementation and Best Practice, Londra: Palgrave Macmillan. Mura, L. and Vlacseková, D. (2018). Motivation of public employees: case study of Slovak teaching and professional staff. Administratie si Management Public, (31), pp. 67-80. Doi: 10.24818/amp/2018.31-05 Mura, L. and Machyniak, J. (2014). Ethical aspects of public administration. Hradec Economic Days 2014: Economic Development and Management of Regions, PT V, pp. 59-65. Novak, J. et al. (2018). The rise of digital challengers. How digitalization can become the next growth engine for Central and Eastern Europe. Perspective on Romania. In the McKinsey Group Digital Challengers Report, pp. 7–31. Osborn, E., (2006). Corruption and Technological Progress: A Take-off Theory of Good Governance. Atlantic Economic Journal, 34, pp. 289-302. Doi: 10.1007/ s11293-006-9017-y. Pakhnenko, O. and Kuan, Z. (2023). Ethics of Digital Innovation in Public Administration. BusinessEthics and Leadership, 7(1), 113–121. Park, J., Jun, S. and Kim J. Y. (2022). Methodology for Data Analysis of Digital Transformation (version 1). United Nations ESCAP, Information and Communications Technology and Disaster Risk Reduction Division, May 2022. Bangkok. Central European Public Administration Review, Vol. 21, No. 1/2023 29 Public Administration Digitalization and Government Effectiveness in EU Countries Pérez-Morote R., Pontones-Rosa C. and Núñez-Chicharro M. (2020). The effects of e-government evaluation, trust and the digital divide in the levels of e-government use in European countries. Technological Forecasting & Social Change, 154. Pollitt, C. and Bouckaert, G. (2011). Public management reform: A comparative analysis. New York: Oxford University Press. Pop Silaghi, M. I. et al. (2014). Do Business and Public Sector Research and Development Expenditures Contribute to Economic Growth in Central and Eastern European Countries? A Dynamic Panel Estimation. Economic Modelling, 36, pp. 108–119. Privara, A. (2022). Economic growth and labour market in the European Union: lessons from COVID-19. Oeconomia Copernicana, 13(2), pp. 355–377. Rhodes, R. (1996). The new governance: Governing without government. Political Studies 44(4), pp. 652–667. Rosenbloom, D. (2014). Public Administration: Understanding Management, Politics and Law in the Public Sector, New York: McGraw-Hill Education. Rymarczyk, J. (2021). The impact of industrial revolution 4.0 on international trade. Entrepreneurial Business and Economics Review, 9(1), pp. 105–117. Sabatini, A., Cucculelli, M. and Gregori, G. L. (2022). Business model innovation and digital technology: The perspective of incumbent Italian small and medium-sized firms. Entrepreneurial Business and Economics Review, 10(3), pp. 23–35. Sadik-Zada E. R., Gatto A. and Niftiyev I. (2022). E-government and petty corruption in public sector service delivery. Technology Analysis & Strategic Management, pp. 1-17. Doi:10.1080/09537325.2022.2067037. Shank, D. B. et al. (2019). Feeling Our Way to Machine Minds: People's Emotions when Perceiving Mind in Artificial Intelligence. Computers in Human Behavior, 98, pp. 256–266. https://doi.org/10.1016/j.chb.2019.04.001. Stępień, B. and Światowiec-Szczepańska, J. (2022). The role of public aid and restrictions’ circumvention in SMEs’ pandemic survival strategies. Entrepreneurial Business and Economics Review, 10(4), pp. 23–36. https:// doi.org/10.15678/EBER.2022.100402 Stoyanova, T. Angelova, M. and Parushev, D. (2022). Peculiarities of Bulgarian universities' digitalization during the Covid-19 pandemic, Entrepreneurship and Sustainability Issues, 10(2), pp. 160–175. https://doi.org/10.9770/ jesi.2022.10.2(10) Strafford, B. and Schindlinger, D. (2019). Governance in the Digital Age, San Francisco: Wiley. Sun, C. et al. (2022). Digital transformation and corporate cash holdings in China’s A-share listed companies. Oeconomia Copernicana, 13(4), pp. 1081– 1116. Doi: 10.24136/oc.2022.031 Szostak, M. (2022). Peculiarities of values transfer among countries via arts digitalization. Entrepreneurship and Sustainability Issues, 10(1), pp. 10–49. Șandor, S., Creța, S. and Fechete, T. (2020). Digital competences among the Romanian population. Transylvanian Journal of Administrative Sciences, 2, 132–147. Taburchak, A. P. Bychkova, S. M. and Butina, A. A. (2022). Analysis of risk factors for applied projects in a digital economy, Entrepreneurship and Sustainability Issues, 9(3), pp. 152–172. Central European Public Administration Review, Vol. 21, No. 1/202330 Armenia Androniceanu, Irina Georgescu Thorndike R. L. (1953). Who Belongs in the Family? Psychometrika. 18(4), pp. 267–276. Torugsa, N. and Arundel, A. (2016). Complexity of innovation in the public sector: a workgroup-level analysis of related factors and outcomes. Public Management Review, 18(3), pp. 392–416. Trapnell, S. (2011). Actionable Governance Indicators: Turning Measurement into Reform. Hague Journal on the Rule of Law, 3, pp. 317–348. Doi:10.1017/ S1876404511200095. Trettin, C. et al. (2019). The social sustainability of citizen-centered urban governance networks: sensor-based big data applications and real-time decision-making. Geopolitics, History, and International Relations, 11(2), pp. 27–33. doi:10.22381/GHIR11220194 United Nations Development Programme. (1997). Governance for sustainable human development. A UNDP policy document. At , accessed 14 February 2023. United Nations Development Programme. (1997). Governance for sustainable human development. A UNDP policy document. January 1997. Yigitcanlar, T., Mehmood, R. and Corchado, J. (2021). Green Artificial Intelligence: Towards an efficient, sustainable and equitable technology for smart cities and futures. Sustainability, 13(16), 8952. Wandaogo, A.-A. (2022). Does digitalization improve government effectiveness? Evidence from developing and developed countries. Applied Economics, 54(33), pp. 3840-3860 https://doi.org/10.1080/00036846.2021.2016590 World Bank (1992). Governance and development I. Washington D.C. World Bank. (1997). Governance and Development. Washington DC: World Bank; World Bank. World Development Report. (1997). The State in a Changing World. World Bank. (2007). Strengthening World Bank Group Engagement on Governance and Anticorruption.