Pageof 4ZOOM

MATH1317 Stochastic Processes and Applications- Class Exercise 5Name:Student Number:1. Consider the Markov chain with state 1;2;3;4;5 and transition proba-bility matrix below.P=0BBBBBBBBB@0:5 0:5 0 0 00:5 0 0:5 0 00 0 0:4 0:6 00 0 0 0:4 0:60 0 0:6 0 0:41CCCCCCCCCA(a) State 1 is reachable from state(s),but not reachable fromstate(s).(b) State3 communicates with state(s),but doesn't communi-cate with state(s).(c) State(s)forma closed set, state(s)form anonclosed set.(d) This Markov chain isreducible / irreducible.(e) State(s)aretransient, state(s)arerecurrent.(f) State(s)areaperiodic, state(s)areperiodic with aperiod of.(g) ThisMarkov chain isergodic / not ergodic.(6 marks)2.Consider the continuous time Markov process that represents the numberof users in the system. So stateimeans that there areiusers in thesystem. It has transition ratesq01=; q10=; q12=andq20=.(a)Write the transition diagram (This is the graph or net with thetransition rates on it).(b)Write down the transition rate matrixQ. (Hint: Use the providedvalues and complete what is missing).(c)Find the equilibrium distribution (i.e., the limiting probabilitiesP0; P1andP2).(d)Find the average number of users in the system (Hint: Note that itis the expected value of a discrete r.v. with values 0;1 and 2 andprobabilitiesP0; P1andP2respectively).(2 + 2 + 4 + 2 = 10 marks)3.Suppose that we have a Markov chain with states 1;2;3;4 has the fol-lowing transition probability matrix:P=0BBBBBB@1t0t0r0r00 0 1t t1 0 0 01CCCCCCA(a)What must be the value ofr? Why?(b)Draw the state diagram (i.e., graph or net) and indicate the transi-tion probabilities on it.(c)Classify the states for any possible value oftandr.(1 + 2 + 3 = 6 marks)4.A processor is inspected weekly in order of determine its condition. Thecondition of the processor can either be perfect, good, reasonable orbad. A new processor is still perfect after one week with probability0.7, with probability 0.2 the state is good and with probability 0.1 itis reasonable. A processor in good conditions is still good after oneweek with probability 0.6, reasonable with probability 0.2 and bad withprobability 0.2. A processor in reasonable condition is still reasonableafter one week with probability 0.5 and bad with probability 0.5. A badprocessor must be repaired. The reparation takes one week, after whichthe processor is again in perfect condition.Formulate a Markov chain that describes the state of the machine anddraw the transition probabilities in a network (graph or diagram).(3 marks)