https://doi.org/10.31449/inf.v45i1.3104 Informatica 45 (2021) 133–141 133 Risks Analyzing and Management in Software Project Management Using Fuzzy Cognitive Maps with Reinforcement Learning Ahmed Tlili and Salim Chikhi MISC Labs, Abdelhamid Mahri University of Science and Technology, Constantine, Algeria E-mail: a.tlili@univ-emir.dz, salim.chikhi@univ-constantine2.dz Keywords: Fuzzy Cognitive maps (FCM), complex system, reinforcement learning, project risk management Received: March 31, 2020 Many projects fail each year simply because a risk has been misjudged, ignored or unidentified. An essential motivation for analyzing the risk of a project is to inform managers in order to reduce the risk, and therefore the loss of the project. Risk analysis can help identify the best actions that would reduce the risk and assess by how much. In the last decades, the Fuzzy Cognitive Map emerged as a powerful tool for modeling and supervising dynamic interactions in complex systems. There is two ways to construct them, the first way by experts of domain and the second way by learning method based on the historical of data. In this paper, we develop a new learning fuzzy cognitive maps based on a reinforcement learning algorithm so called Q-learning and we propose here a new formulation of kosko causality principle. This connection between fuzzy cognitive maps and reinforcement learning allows us to choose based on the historical of data learning process the best and the most important connections between concepts. In this work, we illustrate the effectiveness of the proposed approach by modeling and studying the analysis of project risk management as an economic decision support system. Povzetek: Spodbujevalno učenje in metode mehke logike so uporabljene za analizo tveganj pri razvoju programskih sistemov. 1 Introduction Risks represent a major challenge for organizations and more particularly for organizations developing applications. All activities in general, present risks. The objective of risk management is to better understanding of the factors that contribute to software project risk and to propose an approach to deal them. This approach is no longer reserved for the space or nuclear fields; it has become one of the crucial elements of project management, as well as the management of people, resources, planning and performance. Today, the success of a project is strongly conditioned by the way its leaders know how to recognize the risks. Risk prevention and risk analysis is an important task of the managers that threaten it, to study and overcome them. The information’s absorbed by humans; quite complex processes are usually imprecise or approximate [1]. The strategy adopted is usually imprecise in nature with no or partial knowledge of the problem, and generally possible to be expressed in linguistic terms. Thus the main problem with risk estimation is that the input data is imprecise or uncertain in nature and it is difficult to accurately represent them in mathematical models [2]. Usually and naturally, the risk analyst is specified in language terms as high, very high, medium, low… etc., rather than in exact statistical terminology. To this end, the application of the Fuzzy Inference System (FIS) theory to risk analysis seems appropriate because it deals with inaccurate and ambiguous information and the basic idea of this approach is to allow an element to belong to a set with membership degrees within the continuous real interval [0,1], rather than in the set {0,1}. In risk analysis and management RAM, the most important factors contributing to the risk of failure for any type of socio-economic organization are related to the different criteria as: time constraints, high cost, weak operating resources, poor performance of supervisors…etc., and the identification of the relationships between the risks and the ones that causes’ them remains a major challenge for experts in this field because they are in most cases very complex [3]. In this work we propose an approach for risks analysis and management to managing software projects using Koskos' fuzzy cognitive maps FCM improved with reinforcement learning Q-Learning algorithm. This work is implemented and validated on Matlab R2014a platform. The proposed method is summarized by the framework shown on Figure 1. Figure 1: Proposed method’s background. 134 Informatica 45 (2021) 133–141 A. Tlili et al. 2 Research method 2.1 Literature review Several methods can be found in literature review for the risks management mainly classified in deterministic and stochastic approaches: what-if analysis, task analysis, Hazard and Operability (HAZOP), Quantitative Risk Assessment (QRA), the Critical Risk and Error Analysis (CREA), Fault Tree Analysis (FTA), the Event Tree Analysis (ETA), Failure Mode and Effects Analysis (FMEA), Probability Distribution of Failure and Reliability (PDEA), Petri networks, Bayesian networks, … etc. In [4] Samantra et al., explain that the risk associated with a specific risk factor is expressed as a combination of two parameters: the probability of occurrence and the effect. The concept of risk matrix is here to categorize different risk factors at each levels of occurrence to create a plan of actions. A case study of a metropolitan construction project for the construction of an underground metro station was carried out and demonstrated the efficiency of the steps of the procedure for applying the proposed methodology. Taylan et al. in [5] illustrated risk assessment using AHP and fuzzy TOPSIS where many construction projects were studied according to these main criteria: time, cost, quality, safety, and environmental sustainability. Authors showed that these methods are able of evaluating the overall risk factors of projects and selecting a project with the lowest risk with a relative weight matrix. The results showed that these novel methodologies are able to assess the overall risks of projects, select the project that has the lowest risk with the contribution of relative importance index. In the work of Dziadosz & Rejment [6] , risk and risk factor are a measurable part of uncertainty and can be estimated from the probability of occurrence. This risk and risk factor represent a deviation from the desired level, which can be positive or negative. Consequently, risk analysis is very important for selecting a win project. the main result of this approach concerns cases in which the schedule, costs and requirements of the project must be defined in the planning phase and deviations will be detected automatically in the progress phase. The main idea in the paper related by Muriana & Vizzini [7] is that total weight method is used to calculate the current risk level of the project and the risk of the whole project is reduced taken preventive measures. 2.2 Risk analysis and management Rik is an uncertain event that may have positive or negative impact on project and risk management is the process of identifying and migrating risk. Risk management is more important because it affects all aspects of the project as schedule, budget, delay…etc. One of the main difficulties of risk management is that it is not "an exact science", in this way: • It is not possible to predict in the long term without admitting a part of the uncertainty, • Risks are present at all stages of a project and can take a variety of forms with internal and / or external origins, • We can reduce the risks of a project, but we cannot eliminate them completely, • Due to the diversity of the risks and their management, in particular according to the size of the project, the mobilized resources and the sector of activity concerned, there is a difficulty in invariant identifications. Research in risk analysis and management (RAM) using fuzzy systems [8] have provided several models in recent years. However, to the extent that we have found, there are very few sufficiently representative approaches to be used for complex problems in this area. Quantifying or assessing risk and its factors consists in measuring the (linguistic) probability of occurrence and the estimated or the staggered by defining a scale of (linguistic) values associated with it as follows: • Frequent risk with high probabilities of realization, very high. • Occasional or average risk, can be realized • Rare, unlikely or low • Very unlikely or high. 2.3 What-If Analysis method What–If Analysis is defined as a structured brainstorming method of determining what things can go wrong and estimate the likelihood and consequences of those situations occurring. The answers to these questions are not evident and form the basis for determining a recommended course of action for those risks or risk factor. our proposed method here constitute an automatic alternative to expert review team and can effectively and productively discern major issues concerning a software project or with any other risks project. Lead by an energetic and focused facilitator, each member of the review team participates in assessing what can go wrong based on their past experiences and knowledge of similar situations.After the “What-If” answers are generated by different simulation, the review manager then makes judgments regarding the probability and severity of the risk. If the risk is judged unacceptable then a recommendation is made by the manager for further action. The completed analysis is then summarized as mentioned below: Risks Analyzing and Management in Software Project... Informatica 45 (2021) 133–141 135 3 Theory background 3.1 Fuzzy Cognitive Maps The cognitive maps were studied by computer scientists from the 80s when Bart Kosko [9] chooses to provide a new formalization of Axelrod's cognitive maps [10]. Kosko notes that Axelrod's cognitive maps applied to fields such as politics, history, international relations, contain concepts and influences between concepts that are by nature fuzzy. He thus formalizes the model of fuzzy cognitive maps using the theory of fuzzy sets [9]. Fuzzy cognitive map is a directed graph in the form < X,W > where X = [X 1, ...,X n] is the set of the concepts, W is the connection matrix describing weights of the connections, w j,i is the weight of the direct influence between the j-th concept and the i-th concept, taking on the values from the range [−1, 1]. A positive weight of the connection w j,i means X j causally increases X i. A negative weight of the connection wj,i means X j causally decreases X i and A nul weight of the connection w j,i means there is no causality between X j and X i. Fuzzy cognitive map can be used for modeling behavior of dynamic systems. The state of the FCM model is determined by the values of the concepts at the t-th iteration. The simulation of the FCM behavior requires an initial state vector. Next, the values of the concepts can be calculated according to the selected dynamic model. Simulations show the effect of the changes in the state maps and can be used in a what-if analysis [11]. ) . ( X 1 i ji k j k X f   = + () Where X i(k) is the value of the i-th concept at the k- th iteration, i = 1, 2, ..., n, n is the number of concepts. Transformation function f ( x ) normalizes values of the concepts to a proper range. A logistic function is most often used [12]: x e x f  − + = 1 1 ) ( (2) Where  > 0 is a parameter. Other alternatives are taking into account the past history of concepts and jointly proposed a popular dynamic model which was used in this work summarized in the following equation [10]: ) . ( X 1 i ji k j k i k X X f   + = + (3) 3.2 Reinforcement Learning Reinforcement Learning (RL) is one effective method in the solution of multi stage decision making problems. For a comprehensive study of the subject, refer [13][143][15]. The Markov Decision Processes (MDP) defines the formal framework of reinforcement learning [13]. More formally, an MDP process is defined by: • S, a finite set of states. s Є S • A, a finite set of actions in state s. a Є A(s) • r, a reward function. r(s, a) Є R • P, the probability of transition from one state to another depending on the selected action. P (s '| s, a) = P a(s, s'). The problem is to find an optimal policy of actions that achieves the goal by maximizing the rewards, starting from any initial state. At each iteration, the agent being in the state chooses an action, according to these outputs the environment sends either award or a penalty to the agent shown by the following formula: r k = h (s k, a k, s k+1). To find the total cost, which is represented by the formula Σ h(s k, a k, s k+1), the costs are accumulated at each iteration of the system. In [15] the expected reward is weighted by the parameter γ and becomes Σ γ h (s i, a i,s i+1) with 0 ≤ γ ≤ 1. The RL is to find a policy or an optimal strategy π*, among the different π possible strategies in the selection of the action. Considering that an optimal policy π exists, and then the Bellman [16] optimality equation is satisfied:   ) (s a)V , ( ( a) , R(s max ) (s V V 1 i * 1 i i * * + +  → + = = i i s s P   S s   (4) Equation (4) sets the value function of the optimal policy that reinforcement learning will seek to assess: (s) maxV (s) V * /  = (5) In Q-Learning algorithm technique [14], the agent, For any policy π and any state s ∈ S, the value of taking action a in state s under policy π, denoted Q π (s, a), is the expected discounted future reward starting in s, taking a, and henceforth following π. In this case the function (4) can also be expressed for a state-action pair: a) (s, Q max a) (s, Q *  = (6) Q-learning is one of the most popular reinforcement learning methods developed by Watkins [17] in 1989 What-if? Answer Likelihood Consequences Recommendations What-if the high cost risk immerses? scheduling process or technological aspects are deficient possible Very serious 1 - Include inspection in scheduling procedure. 2- Check the technological aspects in terms of equipment and software plate form. Table 1: What-If Analysis Form. 136 Informatica 45 (2021) 133–141 A. Tlili et al. years and is based on TD (0). It involves finding state- action qualities rather than just state values. Q-Learning algorithm technique is to introduce a quality function Q represents a value for each state-action pair and Q π (s, a) is to strengthen estimate when starting from state s, executing action a by following a policy π: Q π (s, a) = E Σγr i and Q*(s, a) is the optimal state-action pair by following policy π* if a) (s, Q max a) (s, Q *  = and if we reach the Q*(s i, a i) for each pair state-action then we say that the agent can reach the goal starting from any initial state. The value of Q is updated by the following equation:   ) , (s Q - a) , (s max( arg ) s , a , h(s ) a , (s Q ) a , (s Q i k 1 i 1 i i i i i k i i 1 k i k a Q + + + + + =   (7) 4 Software risks and risk management perceptions Recent perceptions about risk management from majority of software project organizations contribute to the lack of project stability. in addition to the inherent challenges posed by the nature of software projects. Ibbs and Kwak [18] identified risk management as the least practiced discipline among different project management knowledge areas. Boehm and DeMarco [19] mentioned that “our culture has evolved such that owning up to risks is often confused with defeatism”. In many organizations, the tendency to ‘shoot the messenger’ often discourages people from bringing imminent problems to the attention of management. This attitude is the result of a misunderstanding of risk management. Boehm [20] identified 10 software risk items to be addressed by software development projects: 1. Developing the wrong user interface 2. Personnel shortfalls. 3. Real-time performance shortfalls 4. Unrealistic schedules and budgets. 5. Developing the wrong functions and properties. 6. Gold plating (adding more functionality/features than is necessary). 7. Straining computer-science capabilities. 8. Shortfalls in externally furnished components. 9. Shortfalls in externally performed tasks. 10. Continuing stream of requirements changes. Jones [21] further presented three key software risk factors and concerns of both executives and software managers. Risk factors always generate a loss, i.e. an event or situation that causes the occurrence of a loss. The risk factor therefore constitutes the origin of a risk or a set of risks. 1. Risks associated with inaccurate estimating and schedule planning. 2. Risks associated with incorrect and optimistic status reporting. 3. Risks associated with external pressures, which damage software projects. However, most software developers and project managers perceive risk management processes and activities as extra work, not part of their job, and more expense. Risk management tasks are therefore to be removed from project activities when the project schedule is operational. G.F. Jones in [22] mentioned that "complex computer systems can be built with a very low level of control by intelligent and motivated people." Many software development professionals believe that risk management and control prevent creativity. 5 Modeling Software Project Management In the software project management (SPM), one of the main issues is the consistency of the project in terms of cost, completion time, quality, performance, etc. However, the most significant risk factors (causes) are of external natures that are part of the third point of the risk factors cited by [21]. Among these, there are five main risk factors: • Bad task scheduling. • Deficient developers. • Technological aspect. • Budget limitation. • Fuzzy objectives. In Figure 2, the rectangles are used to represent the risks, the circles to represent the risk factors and the different arcs to represent the different links. As we can see on Figure 2, we have one link with delay, two conditional links, and two non-linear links. Below, we will discuss in more detail. • Weighted links with duration: the Technological Aspect risk factor, generally, will not necessarily have an immediate effect on the Time Delay risk concept, but it will affect it after a certain time or duration. Indeed, if, for example, the computers that are used to develop software are old, the immediate effect on the Time Delay concept will not be so obvious, but in the long run, it will certainly cause an increase in the risk of time delay. Note that the same observation also applies to performance, cost and quality. • Non-linear links: If we increase the risk factor deficient developers, initially it can help meet deadlines, but if we increase more than necessary, it Figure 2: The main different risks, risk factors and influence links of the SPM model [23]. Risks Analyzing and Management in Software Project... Informatica 45 (2021) 133–141 137 might not help anymore, and could even lead to the opposite result. Therefore, the relationship here must be non-linear. • Conditional Links: If initially there is a bad scheduling with a lack of management skills, they will affect the Time Delay risk as well as the High Cost risk. We categorize them as conditional links, because they affect only if they both occur. For example, if the scheduling of tasks is not optimal, but on the other hand, the organization is very experienced in its field to handle this type of frequent situations, the effect would certainly be different. Once the influences between the risks and the factors are identified, we move on to the second stage, which consists in defining the fuzzy rules by considering the three attributes of the prototype schematized in Figure 2, namely the temporal delay and its conditional links. It remains to be noted here that the construction of fuzzy rules in a general way requires a detailed and complete knowledge of the field studied. The three fuzzy rules above reflect an influence or a linear link between the time delay risk and the risk factor deficient developers. ▪ If the risk factor deficient developer is low Then the time delay risk is low. ▪ If the risk factor deficient developer is Medium Then the time delay risk is medium. ▪ If the risk factor deficient developer is high Then the time delay risk is high. For relationships with time weights, we define an additional input delay variable parameter in fuzzy inference rules. For our example application, two fuzzy rules indicating the existence of the delay parameter can be as follows: • If the technological aspect risk factor is high and the delay is short then the high cost risk is Low. • If the technological aspect risk factor is high and the delay is long then the high cost risk is high. Table 2 summarizes the differentiation of concepts into sensory and motor concepts of model associated with SPM. The without learning FCM that model the SPM system of figure 2 is shown in figure 3. As can be seen on Figure 3, the risk factors C 1, C 4 and C 5 that activated the high cost concept C 8 and time delay concept C 9 risks are still active despite the convergence of the non-learning FCM after 46 step. for the organization this implies that the risk remains active. Among the concepts mentioned in Figure 2, we will discuss the concept of high cost risk and see how based on the proposed approach the organization adapts to its environment by treating this risk. The High Cost concept is affected by risk factor concepts bad schedule and fuzzy objectives. Adaptation is translated here by the action or actions (decisions) undertaken by the organization to deal with this type of risk. One can imagine that in order to stabilize costs, we must act on the risk factors that directly affect this concept. In other words, either improve the scheduling of tasks, or seek to clarify objectives related to its field or both in parallel. This search is guided by, on the one hand, the values associated with the pairs (state, action) found in the table of the function Q, and on the other hand by the probabilities of the actions as mentioned above. If the possible or permissible actions are no longer able to meet the needs of the organization, it is called upon to look for other mechanisms that allow it to meet its needs. For example, in our case, the organization can play on the risk factor deficient developers with which C1 C2 C3 C4 C5 C6 C7 C8 C9 C1 0 0 0 0 0 0 0 +0.7 0 C2 0 0 0 0 0 0 0 0 +0.6 C3 0 0 0 0 0 0 0 +0.4 +0.4 C4 0 0 0 0 0 0 +0.5 0 +0.8 C5 0 0 0 0 0 +0.2 0 0 +0.2 C6 0 0 0 0 0 0 0 0 0 C7 0 0 0 0 0 0 0 0 0 C8 0 0 0 0 0 0 0 0 0 C9 0 0 0 0 0 0 0 0 0 Table 2: SPM sensory and motor concepts model. Figure 3: Without learning FCM SPM model. Concepts Description Type C 1 Bad schedule Sensory concept C 2 Budget Limitation Sensory Concept C 3 Fuzzy objectives Sensory Concept C 4 Technological aspects Sensory concept C 5 Deficient developers Sensory concept C 6 Low Quality Motor Concept C 7 Low Performance Motor Concept C 8 High Cost Motor Concept C 9 Time Delay Motor Concept Table 3: Without learning fuzzy cognitive maps initial matrix. 138 Informatica 45 (2021) 133–141 A. Tlili et al. the concept High Cost has no direct influence link, this action results in the creation of a connection between concept risk high cost and the concept risk factor deficient developers. This last case is represented by figure 4. The rules that go along with the organization in the search for the optimal actions or decisions allowing it to adapt to the new environmental data in the proposed approach are of the form: - If High Cost Risk is Active Then // depending on the factor that triggered the risk. If state Q (state, a i) already visited Then execute action a i where action a i is represented here by increase or decrease weight. Otherwise select the action that has the highest probability or choose any other actions. The two links ( increase, decrease) from C 8 to C 5, figure 4, shematise that in complex systems it is difficult to know if a concept causes or decreases another concept only after several simulation of the model. It also happens that a concept can under certain conditions cause one concept and inhib it in others. Taking into account this characteristics of complex systems, we give an another equivalent formulation of the Kosko principle of causality mentioned in [9] that is applyed in our cases study. Defintion 1 : (C i causes C j) OR (C i causally decreases C j) Iff ( Q i  Q j and Q i Q j) OR ( Q i Q j and Q i  Q j). Were  stands for fuzzy set inclusion. The logical operator OR is used here with the reward received from the environment, which allows to select the best link, attributed to each applied weight of the two links that connect the concept C i and C j and it is defined as follows: ) , ( max decrease increase r r Max r = therefore, definition 1 is written in our case study as follows: Defintion 2 : (C i causally increases C j) iff (Q i  Q j and Q i Q j) and r Max= r increase (C i causally decreases C j) Iff (Q i  Q j and Q i Q j) and r Max=r decrease Based on the theoretical aspects described above, the pseudo code of Algorithm 1 summarizes our aproach [24]. Algorithm 1: Pseudo code of the proposed approach Step 1: Read the vector  (k ) and weight matrix W Step 2: Calculate the output vector  (k +) : ) ( A 1 W A A f k k k  + = + Step 3: Apply the transfer function  to the output vector  (k +) Step 4: Among active concepts, choose the one that has the highest value of the Q function, if not the highest in probability. Step 5: calculate the new output vector (output concepts)  (k +) Step 6: Depending on the response of the environment: If r = 1 / / Award (Updating the probability P ij and the Q value) )] a , (s Q - [1 ) a , (s Q ) a , (s Q i i k i i k i i 1 k  + = + ) , (C W ) C , (C W i k j i 1 k j C = + )] a ( P - [1 ) a ( ) a ( P i k i k i 1 k  + = + P If r = o / / Penalty (Updating the probability P ij, the weight of the connection and the value of Q) ) , ( Q ) - (1 ) a , (s Q k i i 1 k i i a s  = + ) , (C W ) C , (C W i k j i 1 k j C = + Figure 4: Evolution of activation values of FCM concepts without learning (Matlab R2014a). Concept Initial Value Final Values Activation Function Tran sfert Func tion Numb . of Iterati on 1. C1 1 1.00 A .W A + Sigm oid 46 2. C2 0 0,65904607 3. C3 0 0,65904607 4. C4 1 1.00 5. C5 1 1.00 6. C6 0 0,69586237 7. C7 0 0,83569675 8. C8 1 0,72975341 9. C9 0 0,90204315 Table 4: Concepts’ final values without learning fuzzy cognitive maps. Figure 5: The SPM Reinforcement learning fuzzy cognitive maps model. Risks Analyzing and Management in Software Project... Informatica 45 (2021) 133–141 139 ) (a P ) - (1 ) a ( i k i 1  = + k P Step 7: If the termination conditions are realized Stop. Otherwise go to Step 2. Thereafter, the organization evaluates its actions towards its environment by the feedbacks of the latter (in the form of positive or negative answers) by updating its decision-making policy that allows it to adapt and improve its behavior towards its economic and social partner. In the next paragraph 6, we discuss the results obtained after simulation of the SPM model in the proposed approach and in the conventional FCMs approach in order to make a comparison between the two approaches to show the effectiveness of the approach proposed in this paper. 6 Experimental results The simulation of the prototype associated with the SPM model of figure 4 is carried out under MATLAB R2014a. The two scenarios are represented by the results obtained in table 7 in the case where the concept C 8 decreases the concept C 5 and in table 8 where the concept C 8 increases the concept C 5. It can be seen that the best result is obtained in the case where the C 8 concept decreases the C 5 concept. The following table 5 represents the initial matrix of the reinforcement learning fuzzy cognitive maps RL- FCM that model the software project studied in this paper and summarizes the different weights between the concepts of the map especially the links that express the behavioral adaptation, in particular the concept High cost C 8 and its links with the concepts Bad schedule C 1 with w 81=-0.25, Technological aspects C4 with w 84=-0.25 and deficient developers C 5 with w 85incfrease=+0.50 in case if C 8 increases C 5 and with w 85decrease=─0.50 in case where C 8 decreases C 5. Table 6 gives the probability and the function Q quantity values before the simulation, the initial values, and after the simulation, the final values, obtained by application of our algorithms 1 while taking into account the natures of the different weights described above paragraph 5. In the next simulation, figure 6, our simulator will consider the model with the weight that will decrease the concept of deficient developers C5 from high cost concept C8 as being the action taken by the organization to adapt to its environment. 7 Conclusion In this paper we presented an approach in which there is a connection between reinforcement learning and fuzzy cognitive maps for studying risk analysis and management in software projects. The nature of software projects generates many risks that must be managed carefully to avoid the project’s loss. In this work the What-If scenario analysis technique is automated, used and has been effectively applied to a variety of processes. It can be useful in other processes per example in job shop scheduling with mechanical systems such as production machines. The results of the analysis are immediately available for managers and usually can be applied quickly. On behalf the firm to be able to make an adequate decision, it has to compare the simulation of its SPM model, in our case study, with two links (increase, decrease) from concept C 8 to concept C 5. Similarly, in our proposed approach, another's situations can arise, in which concept influences another concept with two weights (increase, increase) or with two weights (decrease, decrease). In Also we have presented a new formulation of Kosko causality principle in which one concept increases or decreases another concept according to environmental conditions. The work is realized with MATLAB R2014 platform. 8 Acknowledgments This paper is the result of a research project titled "Modeling and implementing of complex systems" which was supported by the MISC laboratory of abdelhamid mehri University, Algeria. We thank all the laboratory teams and responsible for their cooperation and supports. C 1 C 2 C 3 C 4 C 5 C 6 C 7 C 8 C 9 C 1 0 0 0 0 0 0 0 +0.7 0 C 2 0 0 0 0 0 0 0 0 +0.6 C 3 0 0 0 0 0 0 0 +0.4 +0.4 C 4 0 0 0 0 0 0 +0.5 0 +0.8 C 5 0 0 0 0 0 +0.2 0 0 +0.2 C 6 0 0 0 0 0 0 0 0 0 C 7 0 0 0 0 0 0 0 0 0 C 8 -0.25 0 0 -0.25 ±0.5 0 0 0 0 C 9 0 0 0 0 0 0 0 0 0 Table 5: Reinforcement learning FCM (RL-FCM) Initial matrix. Figure 6: Concept values evolution, the reinforcement learning FCM converge in 27 steps. 140 Informatica 45 (2021) 133–141 A. Tlili et al. References [1] L. Zadeh, (1965). Fuzzy sets. Inf. Contr., vol. 3, no. 8, pp. 338–353. https://doi.org/10.2307/2272014 [2] Angeline, P.J., Fogel, D.B. (1997). An evolutionary program for the identification of dynamical systems. SPIE Aerosence 97, Symp. On Neural Networks, S.K. Rogers and D. Ruck (eds.), Vol. 3077, 409-417. https://doi.org/10.1117/12.271503 [3] R.T Futrell, L.I Shafer & D.F Shafer (2001). Quality software project management. Prentice Hall PTR. January 24, 2002. ISBN: 0-13-091297-2 [4] C. Samantra, S. Datta., S.S Mahapatra. (2017) Fuzzy based risk assessment module for metropolitan construction project: an empirical study. Eng. Appl. Artif. Intell.; 65:449–464. https://doi.org/10.1016/j.engappai.2017.04.019 [5] O.Taylan, A.O Bafail., R.M Abdulaal., M.R Kabli. (2014) Construction projects selection and risk assessment by fuzzy AHP and fuzzy TOPSIS methodologies. Appl. Soft Comput.; 17:105–116. https://doi.org/10.1016/j.asoc.2014.01.003 [6] A.Dziadosz, M Rejment. (2015) Risk analysis in construction project-chosen methods. Proc. Eng.; 122:258–265. https://doi.org/10.1016/j.proeng.2015.10.034 [7] C. Muriana, G. Vizzini, (2017) Project risk management: a deterministic quantitative technique for assessment and mitigation. Int. J. Proj. Manag.; 35(3):320–340. https://doi.org/10.1016/j.ijproman.2017.01.010 [8] B.W. Boehm and T. DeMarco, Software risk management. IEEE Software 14 (3), (1997), 17–19. https://doi.org/10.1109/ms.1997.589225 Action ai Initial Probability P(ai) Final Probability P(ai) Q(si, ai) Initial Value Final Value (C8,C1) 0,25 0,25 Q(C8,C1) 0 0,25 (C8,C4) 0,25 0,25 Q(C8,C4) 0 0,25 (C8,C5) increase 0,25 0,125 Q(C8,C5) increase 0 0 (C8,C5) decrease 0,25 0,625 Q(C8,C5) decrease 0 0,50 Table 6: Values of the actions probabilities and Q-function with α, β=0.5. Concept Initial Values Final Values Activation Function Transfert Function Number of Itérations 1. C 1 1 0,59937409 A+AW Sigmoid 24 2. C 2 0 0,65904607 3. C 3 0 0,65904607 4. C 4 1 0,59937409 5. C 5 1 0,69329384 6. C 6 0 0,74556292 7. C 7 0 0,76593465 8. C 8 1 0,78606504 9. C 9 0 0,65904607 Table 7: Simulation Results with decrease weight from C 8 to C 5 as the best weight. Concept Initial Values Final Values Activation Function Transfert Function Number of Itérations 1. C 1 1 1 A+AW Sigmoid 32 2. C 2 0 0,65904607 3. C 3 0 0,65904607 4. C 4 1 0,97252654 5. C 5 1 0,85343066 6. C 6 0 0,74556292 7. C 7 0 0,76593465 8. C 8 1 0,78606504 9. C 9 0 0,65904607 Table 8: Simulation Results with increase weight from C 8 to C 5. Risks Analyzing and Management in Software Project... Informatica 45 (2021) 133–141 141 Kosko B. (1986), Fuzzy Cognitive Maps, International Journal Man-Machine Studies, 24:65- 75. https://doi.org/10.1016/s0020-7373(86)80040-2 [9] Axelrod Robert (1976). Structure of decision. Princeton university press, Princeton, NewJersy. M. Carr, S. Konda, I. Monarch, C. Walker and F. Ulrich, Taxonomy-Based Risk Identification, (1993). https://doi.org/10.21236/ada266992 [10] Maikel Leon1, Ciro Rodriguez1, Maria M. Garcia1, Rafael Bello1 and Koen Vanhoof (2010). Fuzzy Cognitive Maps For Modeling Complex Systems. G. Sidorov et al. (Eds.): MICAI 2010, Part I, LNAI 6437, pp. 166 – 174, © Springer-Verlag Berlin Heidelberg 2010 [11] R. Sutton & A.G. Barto, (2015). Reinforcement Learning: An Introduction. A Bradford Book. Second Edition. 2014-2015 The MIT press Cambridge, Massachusetts London, England. [12] E.A. Jasmin, T.P. Imthias Ahamed, V.P. Jagathy Raj (2011). Reinforcement Learning approaches to Economic Dispatch problem. Elsevier. https://doi.org/10.1016/j.ijepes.2010.12.008 [13] L.P. Martin (2014). Markov Decision Processes: Discrete Stochastic and Dynamic Programming. Wiley series in probability and statistics. ISBN: 978-1-118-62587-3 [14] Thomas J. Sargent, (2004). Recursive Macroeconomic Theory. Second edition Lars Ljungqvist Stockholm School of Economics. New York University and Hoover Institution. The MIT Press Cambridge, Massachusetts London, England. ISBN: 9780262122740. [15] C. Watkins (1989). Learning from Delayed Rewards. Ph.D. thesis, King's College, Cambridge, England. [16] Ibbs, C. W., and Kwak, Y. H. (2000). Assessing project management maturity. Project Management Journal, pp 32–43. https://doi.org/10.1177/875697280003100106 [17] Boehm, B.W., DeMarco, T. (1997). Software risk management. IEEE Software 14 (3), 17–19. https://doi.org/10.1109/ms.1997.589225 [18] Boehm, B.W., 1991. Software risk management principles and practices. In IEEE Software 8 no 1, pp 32–41. Jan. 1991, Doi:10.1109/52.62930. [19] C. Jones, (1998). Minimizing the risks of software development. Cutter IT Journal 11 (6), 13–21. [20] Jones, G.F., (2001). What is different about it risks. In: 2001 INCOSE Proceedings of a Symposium on Risk Management.. [21] Beatrice Lazzerini, Member, IEEE, and Lusine Mkrtchyan (2011) Analyzing Risk Impact Factors Using Extended Fuzzy Cognitive Maps. IEEE SYSTEMS JOURNAL, VOL. 5, NO. 2, https://doi.org/10.1109/jsyst.2011.2134730 [22] A. Tlili, S. Chikhi, (2016). Natural Immune System Response as Complexe Adaptive System Using Learning FCMs. IAES International Journal of Artificial Intelligence (IJ-AI) Vol. 4, No. 3, pp 95- 104. ISSN: 2252-8938. https://doi.org/10.11591/ijai.v5.i3.pp95-104. 142 Informatica 45 (2021) 133–141 A. Tlili et al.