Please use python to solve these problems
Nonlinear econometrics for finance HOMEWORK 1 (Review of linear econometrics and review of methods) Problem 1 (Linear econometrics). (60 points) Household finance is a growing field in finance. Rising health costs are not just impacting house- holds’ finances, they are affecting an array of decisions, including the decision to change (or retire from) an occupation which provides favorable health in- surance subsidies. For a cross section of individuals, the file “insurance.csv” provides the following information: • age: age of primary beneficiary of health insurance • sex: gender of primary beneficiary of health insurance • bmi : this is a measure of a person’s weight relative to height. It is defined as bmi = kg/m2, where kg is the person’s weight and m2 is the person’s height measured in squared meters. A bmi between 18.5 and 24.9 is considered healthy. More would be considered “overweight”. • children: number of children covered by health insurance • smoker: whether the primary beneficiary is a smoker or not • region: the primary beneficiary’s residential area in the US (northeast, southeast, northwest, southwest) • charges: medical costs billed to health insurance. Given this information, you need to perform linear regression in Python to understand the drivers of medical costs. 1 (1) (3 points) Generate an histogram of the medical costs and compute de- scriptive statistics (mean, median, standard deviation, minimum, max- imum). Is the distribution symmetric? Why or why not, in your view? (2) (3 points) Take a logarithmic transformation of the medical costs. Plot the histogram of the log-costs. What do you notice now? How would you explain the change? Begin by excluding all categorical variables (sex, smoker and region). (3) (4 points) Run a regression of the log-costs on the non-categorical ex- planatory variables: log(costi) = θ0 + θ1agei + θ2bmii + θ3childreni + εi, where εi is an error term. (4) (3 points) Give an economic interpretation of the estimated coefficients in the regression above. What does the model say about the determi- nants of medical costs? (5) (4 points) We want to test whether the coefficient θ2 for bmi is statis- tically significant. Test the hypothesis using the relevant test statistic. Does bmi have more or less explanatory power than age? (6) (3 points) We want to test whether the coefficient θ2 for bmi is statis- tically significant. Test the hypothesis using the relevant p-value. (7) (5 points) Test the single linear restriction θ1 = 3θ2 using the relevant test statistic. (8) (3 points) Test the single linear restriction θ1 = 3θ2 using the relevant p-value. (9) (5 points) Test the multiple linear restriction θ1 = 0.04 and θ2 = 0 using the relevant test statistic. (10) (3 points) Test the multiple linear restriction θ1 = 0.04 and θ2 = 0 using the relevant p-value. 2 (11) (4 points) Using the estimated model, predict medical costs for a 50 year-old person with bmi = 36 and 4 children. Is the prediction lower or higher than the mean of the distribution of the medical costs? (Recall that the regression gives you a prediction for the log of the medical costs (say, log(y)) not for the medical costs (say, y). Hence, after you find the prediction for the log of the medical costs, you need to make a transformation to find a prediction for the medical costs themselves. Hint: if log(y) is normal, y is lognormal. What is E(y) for a log normal random variable? ) Now, take the categorical variables into account using dummy variables (https://en.wikipedia.org/wiki/Dummy_variable_(statistics)). (12) (3 points) How much more (or less) do males spend relative to females (controlling for all other variables)? (13) (3 points) How much more (or less) do smokers spend relative to non smokers (controlling for all other variables)? (14) (3 points) In which region are medical costs higher (controlling for all other variables)? (15) (3 points) What is the difference in medical costs between the northeast and the southwest (controlling for all other variables)? (16) (4 points) Are the coefficients associated with the dummies individually statistically significant? (17) (4 points) Using your model, predict medical costs for a 50 year-old male smoker with bmi = 36 who lives in the southwest and has 4 children. Problem 2 (Review of methods). (40 points) Assume an iid sample {x1, x2, ..., xT} from some distribution with expected value µ and variance σ2. A natural estimator for the true variance (i.e., σ2) of the random variable which generates the data is the sample variance, namely s2x = 1 T ∑T t=1(xt − X)2, where X defines the sample mean, i.e., X = 1 T ∑T t=1 xt. First, let us focus on the finite-T (or finite-sample) properties of s2x: 3 https://en.wikipedia.org/wiki/Dummy_variable_(statistics) (1) (6 points) Show that the sample variance s2x is biased for the true variance σ2. (2) (3 points) How would you correct the bias? (3) (3 points) What is the bias of the infeasible variance estimator s2x,inf = 1 T ∑T t=1(xt − µ)2. Why am I calling this estimator infeasible? Now, let us turn to the large-T (or infinite-sample or asymptotic) properties of s2x. Write the following: s2x = 1 T T∑ t=1 (xt −X)2 = 1 T T∑ t=1 ((xt − µ)− (X − µ))2 = 1 T T∑ t=1 (xt − µ)2︸ ︷︷ ︸ (a) − 2(X − µ) 1 T T∑ t=1 (xt − µ)︸ ︷︷ ︸ (b) + (X − µ)2︸ ︷︷ ︸ (c) (1) Now, subtract σ2 from the left-hand side and from the right-hand side of Eq. (1) and standardize by √ T to obtain: √ T (s2x − σ2) = ∑T t=1((xt − µ)2 − σ2)√ T︸ ︷︷ ︸ (a∗) − 2(X − µ) 1√ T T∑ t=1 (xt − µ)︸ ︷︷ ︸ (b∗) + √ T (X − µ)2︸ ︷︷ ︸ (c∗) (2) (4) (6 points) Show that s2x is consistent for σ 2 by applying the LLN to (a), (b) and (c) in Eq (1). (5) (6 points) Show that √ T (s2x − σ2) is asymptotically normal by ap- plying the LLN, the CLT and Slutsky’s theorem to (a∗), (b∗) and (c∗) in Eq. (2). Notice that consistency is a statement about sample averages, like s2x, con- verging (as T →∞) to expected values. Asymptotic normality is a statement about demeaned (by σ2, in our example) and standardized (by √ T , in our example) sample averages, like √ T (s2x − σ2), converging (as T →∞) to a mean-zero normal distribution. 4 (6) (8 points) Use my sample Python codes from Lecture 1 to write a code which shows consistency of s2x. You should draw your observations from a random variable which is neither exponential nor normal. (7) (8 points) Use my sample Python codes from Lecture 1 to write a code which shows asymptotic normality of √ T (s2x − σ2). You should draw your observations from a random variable which is neither exponential nor normal. 5 Sheet1 idpricestreetzipyearBuiltsqftbathroomsbedrooms 3644585741085603 N Ellwood Ave212051915123224 364335769472026 S Robinson St21224190410921.52 364335801906924 S East Ave212241904130023 36437942789732707 Jefferson St21205192091013 3643804734586436 N Linwood Ave21224192013441.53 36438050146961430 N Linwood Ave212241920134424 36438066329352818 Orleans St212241920114412 3643823655968414 N Robinson St21224192096013 3643829162782447 N Ellwood Ave212241920104013 36438325465892705 Orleans St212241920118823 3643861366367417 N Glover St212241900110012 36438769122374210 N Glover St212241900102012 3644109075724115 N Streeper St21224192091212 3644128167674229 N Curley St212241920182012 3644132341669142 N Curley St212241920102012 3644134782770161 N Curley St21224192093613 36441552123802153 N Potomac St21224192096013 3644532928983604 N Belnord Ave21205190796012 364661591409013717 Kimble Rd21218193714002.53 364577971033493951 Wilsby Ave212181925115612 364578471085723943 Frisby St212181920136826 36457908140607928 Argonne Dr212181940160023 3645806591141603 Parkwyrth Ave212181925117623 36458284126604605 E 35th St21218192015002.53 364584221525543200 Ellerslie Ave212181924148023 364650161783881202 Lakeside Ave212181937136023 364650211776611201 Windemere Ave212181937128823 364651161648561313 Lakeside Ave212181936210023 36465464110173660 E 35th St212181920126013 3646584892770600 Mckewin Ave212181923146413 3646596894970722 E 37th St21218195598623 36466000137913902 E 36th St212181930216813 36466024161391909 Mckewin Ave21218193112201.52 36466029165043906 Mckewin Ave212181931122022 36466069167861926 E 37th St212181938150023 364660921351833701 Elkader Rd212181932112023 364661001595103717 Elkader Rd212181932108813 364662251427523616 Rexmere Rd212181930136013 364663001745781108 E 36th St21218192716081.54 364663541774183614 Ednor Rd212181928149623 90008776161668813 Newington Ave212171900244034 364825491055742335 Mcculloh St212171991124223 364825741589532224 Madison Ave21217190045926.58 364826431446012301 Madison Ave212171907190735 36482986243934906 Whitelock St212171908288034 3648309565664911 Chauncey Ave212171922248625 36483097184563907 Chauncey Ave212171922198034 364831682456772520 Brookfield Ave212171918215635 36483176160566818 Brooks Ln212171917210034 36483311306504607 Reservoir St212171890243633 364833732320252034 Park Ave212171880345046 36483428135504737 Newington Ave212171905271526 364834622535792132 Mount Royal Ter21217189029762.55 364834641224692136 Mount Royal Ter212171920153813 364834651302042138 Mount Royal Ter212171928148523 36483483295413608 Reservoir St21217189527202.55 364835491162142108 Park Ave21217193213401.53 36483563797522204 Mount Royal Ter21217193516831.54 365667311242524821 Hazelwood Ave212061947121514 362677131715416012 Mannington Ave212061956122633 362679581817064503 Raspe Ave21206192610842.54 362692421432695712 Utrecht Rd212061965118423 362702811864064501 Raspe Ave212061926123224 362704091425405716 Utrecht Rd212061964177623 362738061821034854 Hazelwood Ave212061941180023 362741081234375642 Leiden Rd212061964118423 365665351875324557 Hazelwood Ave21206195118002.53 365665781227685731 Newholme Ave212061951121834 365667171870764804 Burland Ave212061967118824 365667281649124815 Hazelwood Ave212061947121514 365675821941104408 Raspe Ave212061910183324 365675961605924436 Raspe Ave212061920167224 365676271379164426 Springwood Ave212061957119213 365676831033534307 Springwood Ave212061910139423 365676962533464416 Parkmont Ave212062009224033 365677251384364500 Parkmont Ave212061940104023 365678291370614413 Parkmont Ave212061910174623 365678442088014426 Powell Ave21206191017803.54 365680291905454714 Glerm Ave21206195123122.53 900100213082761200 Steuart St UNIT 32721230200813861.51 364365031939122007 Portugal St21231190088812 364283692453036 S Collington Ave212311914166522 36428553359793115 S Collington Ave212311900215123 36428730357163215 S Collington Ave21231191820322.52 364289953156022117 Bank St212311920160032 364293842619402338 Fleet St2122419209702.52 36429605386786515 S Montford Ave212241915130233 364299573536912515 Boston St APT 407212241987125522 364299643450752515 Boston St APT 506212241987130722 36430297269798600 S Rose St212241920108022 36430427202929702 S Rose St212241920126411 36430486308117709 S Glover St212241920108022 364310224416081109 S Kenwood Ave21224198716323.54 364310642341201102 S Curley St212241900142812 364312343933103017 Odonnell St21224192013842.53 364314314390632901 Boston St APT 10121224198522602.53 364314354179452901 Boston St APT 10521224191017722.53 364314364210582901 Boston St APT 10621224198511722.53 364314422735872901 Boston St APT 204212241989137421 364314584375252901 Boston St APT 22121224198521222.53 3652286977785509 Collins Ave212291920123213 36523070137542313 Yale Ave212291920163324 3652322745755157 Collins Ave212291900118413 3652329649424156 Collins Ave212291900120013 3652336738622229 S Augusta Ave212291920150014 36542743215173374 Marydell Rd212291950250024 3654276241204337 Marydell Rd212291928115213 36542865818564465 Eldone Rd212291955102413 36542905132273757 Yale Ave212291953112223 3654298683028747 S Woodington Rd21229195389612 3654301267784722 S Woodington Rd21229195389612 365431061387174309 Cedar Garden Rd212291954112213 36543161694064432 Cedar Garden Rd21229195589612 36543189106380544 Lucia Ave212291955129623 3654321270583590 Lucia Ave21229195511221.53 3654321876303610 Lucia Ave212291955108023 3654322664531626 Lucia Ave212291955102413 3654327071544603 Lucia Ave21229195589622 36543306965164401 Parkton St212291955112233 36437346375843311 S High St212021870217633 36435122211362528 S Bethel St21231190013002.52 36437233298563215 Albemarle St212021870193024 36437252223324219 S High St21202185012401.53 36437271240292243 Albemarle St212021870120022 36437279320607234 S Exeter St212021870145622 36437312325122923 Stiles St21202187017364.53 36437347263755309 S High St212021870214022 364374092674561011 Stiles St212021900201025 36437516236407519 S Dallas St212311880110423 54846849504314704 S Bond St212312000197532 364518031023421306 N Kenwood Ave212131942110423 36442232401972528 E Madison St21205192064613 36446266374372800 Ashland Ave21205193363023 36446568330532812 E Madison St21205192058513 36451319623412309 E Oliver St21213191059713 36452006316212604 Mura St21213193657223 36452396270102416 E Oliver St21213190067213 36452689621003033 E Federal St212131950128023 36452711591231520 Edison Hwy212131945171023 36454844716721627 N Montford Ave21213192059813 36455475923861332 N Ellwood Ave212131948131223 36474394199769222 E 22nd St212181920224425 364731791244279 W 24th St212181920144324 364731902312112411 Maryland Ave21218190024662.52 364732782293012016 N Calvert St212181890225633 364735612520311631 N Calvert St212021900241043 364735642863081637 N Calvert St212021890230633 364735662509711641 N Calvert St212021900230644 36473786198593322 E Lafayette Ave212021917190023 364738432929001705 Guilford Ave212021917350554 36474159255999323 E 22nd St21218192024342.53 36474336194008336 E 22nd St212181920233523 36474344234227320 E 22nd St212181920272424 364744281887942332 Guilford Ave212181920217433 36571498601833312 Cliftmont Ave212131930114023 364558591080633412 Parklawn Ave212131940124023 36455918814093407 Parklawn Ave212131941124023 36456001727353301 Parklawn Ave212131931114013 36456239921103204 Harwell Ave212131923112013 36456468626093334 Lyndale Ave21213192489623 36456484238483422 Elmley Ave21213192589613 36456587969373101 Belair Rd212131924142214 364566181001883107 Mareco Ave212131926124823 36456876362883227 Ravenwood Ave212131920110013 36457095458933328 Ravenwood Ave21213192593212 36457111843763317 Ravenwood Ave21213192593213 36457198633613434 Ravenwood Ave21213192593213 36570659534623601 Ravenwood Ave212131953128024 36570799626753726 Bonview Ave212131954108022 36571061549823635 Lyndale Ave21213195084012 36571155685973833 Lyndale Ave212131950102413 365716241084043563 Elmley Ave212131941120023 36571688900433619 Erdman Ave212131950128023 36571718376122803 Edison Hwy212131942120023 36489741665852108 Mount Holly St212161920240024 36491907681312003 Dukeland St212161920160034 36495689931933329 Burleith Ave212151930129623 36496520887563903 Garrison Blvd212151989129623 36498558752053423 Wabash Ave212151949154613 36513602548452811 Presstman St212161920145214 365921398681817 Hamill Rd # 17B21210197274011 36602753982753015 Glen Ave21215195310871.53 366227761131982352 Nutmeg Ter21209198210081.52 36623749707625318 Lynview Ave21215194813501.53 36614152375637900 Beaumont Ave212121940300024 36614177869651006 Beaumont Ave21212193016001.55 36614768104322705 Saint Dunstans Rd21212195089622 363512611121267203 Rockland Hills Dr UNIT 206212091985117522 364071861306628424 Oakleigh Rd212341946139522 364719261026482833 Huntingdon Ave212111920106833 36483620173081729 Bay St212111846150022 364774451520003405 Greenway UNIT 51212181982134822 365716551188333501 Lyndale Ave212131943126023 365766581471974616 Walther Ave212141931158223 36577447786893606 Eastwood Dr21206194811521.53 365779641363115610 Biddison Ave212061955119623 365808661435623514 Glenmore Ave212061948120013 365837051001533612 Northway Dr212341955108013 365899242165134401 Roland Ave UNIT 402212101963130022 36606839803714708 Hampnett Ave212141920158425 36606885890784612 Elsrode Ave212141950128023 366079631159592104 Louise Ave212141955129423 366165181152201111 Gleneagle Rd212391949145224 36616717801951115 Elbank Ave212391953121613 3662062098444353 Homeland Southway APT 3A212121978111322 364701871633481101 Saint Paul St APT 812212021965134822 364732911733892022 Saint Paul St212181900126444 365892791202744312 Newport Ave21211194783212 365850201885613026 Pinewood Ave212141931191923 36580804991003403 Roselawn Ave21214194112991.53 365804561117893007 Louise Ave212141896176024 365804571829913005 Louise Ave212141948171833 365804971276263107 Royston Ave212141934108913 365806641830086007 Burgess Ave212141924124424 365806771339096015 Sefton Ave212141918118724 365808321599176107 Carter Ave212141947186034 365814641622955931 Glenoak Ave212141925134124 365814762085275925 Sefton Ave212141938124644 365815491812625909 Ed Ave212141941150423 365815601898975912 Burgess Ave212141927181534 365815692366123021 Glenmore Ave212141923189247 365815951359625910 Ed Ave212141938132425 365832231902463009 Orlando Ave212341925168233 365832742236463121 Woodhome Ave21234194114802.56 365833732115233206 Chesley Ave212341939175634 365837882029043101 Chesley Ave212341931188533 365837941351443115 Chesley Ave212341924120412 365838521075483008 Northway Dr212341929174933 365838552079943014 Northway Dr212341930200033 36583960992893019 Woodring Ave212341951120013 365846211915476421 Alta Ave212061916214634 365846221510146413 Alta Ave212061962165323 365847331749726503 Moyer Ave212061953165824 36602757994953023 Glen Ave21215195210871.53 366016261729833502 Menlo Dr212151925193526 366017221586766015 Highgate Dr21215195314562.53 366017841668505901 Winner Ave212151957112033 366019541526553406 Devonshire Dr21215195014521.54 366020171514655914 Highgate Dr212151924139824 366023661816572601 Taney Rd212091947130525 366024511550946012 Woodcrest Ave212091955125212 366024751303486005 Western Run Dr212091955143220 36602705667125615 Highgate Dr21215194710881.53 366028121293155810 Simmonds Ave212151952132024 36602814895585816 Simmonds Ave212151952132034 36602816805335801 Simmonds Ave212151948198023 366018731952663505 Menlo Dr212151948141233 366026751659125809 Highgate Dr212151922140734 36556343600751630 Malvern St21224195484012 36556308418071709 Malvern St21224195484013 36556344558911632 Malvern St21224195484012 36556383909506514 Brown Ave212241960108013 36556453690876437 Hartwait St21224195584012 36556488843326502 Hartwait St21224195384012 36556494463976514 Hartwait St21224195384012 36556629785301414 Dundalk Ave212221955102423 36556723485841139 Steelton Ave212241942108013 36556901465041511 Elrino St21224195384012 366128551454821433 Winston Ave212391945125423 36550428882132520 Terra Firma Rd212251948102413 3655014064090510 Roundview Rd212251949102423 3655014956310610 Roundview Rd212251949102413 3655051287044645 Cheraton Rd212251926102413 36517867191579102 S Poppleton St212011900157243 36398988186547944 Fairmount Ave21204195612161.53 36418229177258941 Fairmount Ave21204195812161.53 3651776815742043 S Poppleton St212011900120922 365178221143851014 W Pratt St21223190014651.53 36517939189470131 S Poppleton St212011900162023 36517963240141881 W Lombard St21201189020403.53 36518056234872841 Hollins St212011920224034 36518105120914107 Parkin St212011920145245 365213291448091405 Hollins St212231860304234 36532379169774775 W Pratt St212011900187232 36532389252033227 Scott St212301900144023 36532391239615780 Mchenry St212301900147234 36532413244900208 Otterbein St21230190019502.54 36532452207238764 Ramsay St21230190014642.53 36532502147246318 Otterbein St21230190081212 36532533134162330 S Poppleton St21230187591212 36532570276323319 S Poppleton St212302005200034 36532644217595812 Washington Blvd212301875119013 365592493872893500 Dillon St212241913225433 3642836912453036 S Collington Ave212311914166522 364285531359793115 S Collington Ave212311900215123 364287301357163215 S Collington Ave21231191820322.52 3642899513156022117 Bank St212311920160032 364296051386786515 S Montford Ave212241915130233 3642995713536912515 Boston St APT 407212241987125522 3642996413450752515 Boston St APT 506212241987130722 364304271202929702 S Rose St212241920126411 3643102214416081109 S Kenwood Ave21224198716323.54 3643106412341201102 S Curley St212241900142812 3643123413933103017 Odonnell St21224192013842.53 3643143114390632901 Boston St APT 10121224198522602.53 364314327287792901 Boston St APT 10221224198528393.53 3643143514179452901 Boston St APT 10521224191017722.53 3643143614210582901 Boston St APT 10621224198511722.53 3643144212735872901 Boston St APT 204212241989137421 364314546239392901 Boston St APT 21721224198522342.53 3643145814375252901 Boston St APT 22121224198521222.53 364314906012522901 Boston St APT 411212241985263344 364315253516142825 Hudson St21224192012452.53 36431562204309915 S Kenwood Ave21224192012001.53 36432035423108714 S Potomac St212241900255033 678692438897001609 Alicean St21231200535492.53 364352615277121621 Lancaster St21231183722842.54 364354188090411713 Lancaster St212312005300043 36435466483172929 Fell St212311860208533 9000876344709429 Yogurt Ln212312008211844 36436321269313307 S Chapel St212311900152922 364352623949001623 Lancaster St21231182027003.53 3642837228423714 S Collington Ave212311914112833 3642840625183912 S Duncan St212312007147622 3642848819477320 S Madeira St21231193684022 364284944046792103 E Lombard St212311910213033 364285023781692119 E Lombard St212311936170223 364285043680722123 E Lombard St21231191021952.54 364285053045522125 E Lombard St212311910225023 36428535281140113 S Chester St212311910177623 36428564146865120 S Madeira St21231192012401.52 36428611478473216 S Collington Ave21231191028643.53 36428656398356201 S Chester St212311910204453 364287073638692220 Gough St212311918204623 36428726253511223 S Collington Ave212311918195823 36428791219495228 S Madeira St21231191111502.52 364288013686702205 Gough St212311910227742 364288072408822217 Gough St212311910148812 36428865217831307 S Madeira St212311910108032 36428871254993319 S Madeira St2123119109601.52 36428883229134316 S Madeira St2123119107921.51 36428905258038304 S Collington Ave21231192010342.52 36428913180486322 S Collington Ave21231192081311 36428919284476334 S Collington Ave21231192017283.53 36428954267041311 S Chester St212311920169212 36428960184795302 S Duncan St21231192296812 36429749291692714 S