All the assignment
Final Assignment (17/6/21) Instructions: • The assignment consists of four questions with multiple parts that must answered in full. • The assignment must be uploaded onto our Moodle site by 13:00 PM on June 24th, 2021. There are no time extensions. Late assignments will result in a FAIL grade. • Your assignment will not be evaluated unless it is uploaded along with a signed academic integrity statement. • Please type your work in LaTeX before submitting it. • Clearly explain your reasoning whenever necessary. When relevant, it is enough to reference a known Theorem or other results, you need not spell them out in full. The questions: 1) Let X1, X2, . . . be independent RVs following a N (θ, 1) distribution with θ ∈ Θ. We denoted the true value of θ is by θ0. The goal of this problem is to show that the maximum likelihood estimator (MLE), which we denote generically by θ̂n, differs in form and exhibits different behaviors depending on the parameter space Θ. (a) Let Θ = R. Find the MLE. Show that it is strongly consistent, i.e., that P( limn→∞ θ̂n = θ0) = 1. Also show that: lim n→∞ P(θ̂n = θ0) = 0. Comment. (b) Let Θ = {−1, 1} , i.e.; θ0 = −1 or θ0 = 1. Find the MLE. Show that: P(θ̂n 6= θ0) ≤ C0 exp (−nC1) for some positive constants C0 and C1 which you are required to find. Note that the above shows that the probability that MLE differs from its true value is exponentially small. (c) Let Θ = R+ = {θ : θ ≥ 0} . Find the MLE. Show that when θ0 > 0 we have √ n(θ̂n − θ0) d→ N (0, 1) and that when θ0 = 0 we have √ nθ̂n d→ max{0, Z} = { 0 with probability 12 |Z| with probability 12 where Z is a standard N (0, 1) RV. (d) Let Θ = Q (the set of rational numbers). Show that an MLE does not exist with probability one. Explain when the MLE exist (this, of course, is an event with probability zero). Let ε > 0 and let Xn be the sample average. Show that the set { θ ∈ Θ : L (θ) L ( Xn ) ≥ 1− ε} is not empty. Refer to this set as the set of ε −MLEs. Show that if ε ↓ 0 as n → ∞ then any ε−MLE is consistent. 2) This problem asks you to establish some elementary inequalities. 1 (a) Let |X| ≤ 1 for all ω ∈ Ω. Show that for any positive constants t: P( |X| ≥ t) ≥ E(X2)− t2. (b) Let |X| ≤ 1 for all ω ∈ Ω. Show that: V(X) ≤ 1. Find the RV with the maximal variance. (c) Let X be a RV such that P(X > 0) = 1. Show that the following inequalities hold E(X)E( 1 X ) ≥ 1, E(X) + E( 1 X ) ≥ 2 and E( max{X, 1 X }) ≥ 1. Can you find the RV which minimizes the left hand sides in the above display (do each one separately). 3) The goal of this problem is to examine mixture models. Let X1, X2, . . . be IID RVs with density f(x; θ) = { θ exp (−x) + 2(1− θ) exp (−2x) if x ≥ 0 0 x < 0 where θ ∈ [0, 1] . (a) show that f(x; θ) is a density function. find the mle of θ when n = 1 (the sample is of size one). (b) show that for any n ∈ n the likelihood function is a polynomial of degree n whose coeffi cients depend on the data. suggest a numerical or graphical procedure for finding the mle for general n. implement your method in r for n = 5. (c) find the method of moments estimator (mme) for θ. show that the mme is consistent and find its limiting distribution. (d) find the neyman—pearson test of level α test for the hypotheses: h0 : θ = 0 versus h1 : θ = 1. explicitly provide the test statistic, the rejections region and the critical value. compute the power of the test. 4) this problem asks you to establish some limit laws. (a) let x1, x2, . . . be independent n (0, 1) rvs. find the limiting distribution of n (x1x2 +x3x4 + · · ·+x2n−1x2n)2 ( ∑2n j=1x 2 i ) 2 . (b) let x1, x2, . . . be independent rvs where xk is exp (1/k!) . show that 1 n! n∑ k=1 xk d→ exp (1) . (c) let x1, x2, . . . be independent rvs where xk is exp (1/k) . show that there are sequences of constants m1,m2, . . . and v1, v2, . . . such that sn −mn vn d→ n (0, 1) where sn = ∑n k=1xk. find mn and vn. (d) can you redo (c) if xk is exp (k)? if yes do so if not explain why. 2 0="" where="" θ="" ∈="" [0,="" 1]="" .="" (a)="" show="" that="" f(x;="" θ)="" is="" a="" density="" function.="" find="" the="" mle="" of="" θ="" when="" n="1" (the="" sample="" is="" of="" size="" one).="" (b)="" show="" that="" for="" any="" n="" ∈="" n="" the="" likelihood="" function="" is="" a="" polynomial="" of="" degree="" n="" whose="" coeffi="" cients="" depend="" on="" the="" data.="" suggest="" a="" numerical="" or="" graphical="" procedure="" for="" finding="" the="" mle="" for="" general="" n.="" implement="" your="" method="" in="" r="" for="" n="5." (c)="" find="" the="" method="" of="" moments="" estimator="" (mme)="" for="" θ.="" show="" that="" the="" mme="" is="" consistent="" and="" find="" its="" limiting="" distribution.="" (d)="" find="" the="" neyman—pearson="" test="" of="" level="" α="" test="" for="" the="" hypotheses:="" h0="" :="" θ="0" versus="" h1="" :="" θ="1." explicitly="" provide="" the="" test="" statistic,="" the="" rejections="" region="" and="" the="" critical="" value.="" compute="" the="" power="" of="" the="" test.="" 4)="" this="" problem="" asks="" you="" to="" establish="" some="" limit="" laws.="" (a)="" let="" x1,="" x2,="" .="" .="" .="" be="" independent="" n="" (0,="" 1)="" rvs.="" find="" the="" limiting="" distribution="" of="" n="" (x1x2="" +x3x4="" +="" ·="" ·="" ·+x2n−1x2n)2="" (="" ∑2n="" j="1X" 2="" i="" )="" 2="" .="" (b)="" let="" x1,="" x2,="" .="" .="" .="" be="" independent="" rvs="" where="" xk="" is="" exp="" (1/k!)="" .="" show="" that="" 1="" n!="" n∑="" k="1" xk="" d→="" exp="" (1)="" .="" (c)="" let="" x1,="" x2,="" .="" .="" .="" be="" independent="" rvs="" where="" xk="" is="" exp="" (1/k)="" .="" show="" that="" there="" are="" sequences="" of="" constants="" m1,m2,="" .="" .="" .="" and="" v1,="" v2,="" .="" .="" .="" such="" that="" sn="" −mn="" vn="" d→="" n="" (0,="" 1)="" where="" sn="∑n" k="1Xk." find="" mn="" and="" vn.="" (d)="" can="" you="" redo="" (c)="" if="" xk="" is="" exp="" (k)?="" if="" yes="" do="" so="" if="" not="" explain="" why.="">