2 questions:- Exercise 14.1 (page 413)- Exercise 14.9 (page 414)
14 Logit and Probit Models for Categorical Response Variables T his chapter and the next deal with generalized linear models—the extension of linearmodels to variables that have specific non-normal conditional distributions: ! Rather than dive directly into generalized linear models in their full generality, the cur- rent chapter takes up linear logit and probit models for categorical response variables. Beginning with this most important special case allows for a gentler introduction to the topic, I believe. As well, I develop some models for categorical data that are not sub- sumed by the generalized linear model described in the next chapter. ! Chapter 15 is devoted to the generalized linear model, which has as special cases the lin- ear models of Part II of the text and the dichotomous logit and probit models of the cur- rent chapter. Chapter 15 focuses on generalized linear models for count data and develops diagnostic methods for generalized linear models that parallel many of the diagnostics for linear models fit by least-squares, introduced in Part III. All the statistical models described in previous chapters are for quantitative response variables. It is unnecessary to document the prevalence of qualitative/categorical data in the social sciences. In developing the general linear model, I introduced qualitative explanatory variables through the device of coding dummy-variable regressors.1 There is no reason that qualitative variables should not also appear as response variables, affected by other variables, both qualitative and quantitative. This chapter deals primarily with logit models for qualitative and ordered-categorical response variables, although related probit models are also briefly considered. The first section of the chapter describes logit and probit models for dichotomous response variables. The sec- ond section develops similar statistical models for polytomous response variables, including ordered categories. The third and final section discusses the application of logit models to con- tingency tables, where the explanatory variables, as well as the response, are categorical. 14.1 Models for Dichotomous Data Logit and probit models express a qualitative response variable as a function of several expla- natory variables, much in the manner of the general linear model. To understand why these 1See Chapter 7. 370 models are required, let us begin by examining a representative problem, attempting to apply linear least-squares regression to it. The difficulties that are encountered point the way to more satisfactory statistical models for qualitative data. In September 1988, 15 years after the coup of 1973, the people of Chile voted in a plebiscite to decide the future of the military government headed by General Augusto Pinochet. A yes vote would yield 8 more years of military rule; a no vote would set in motion a process to return the country to civilian government. As you are likely aware, the no side won the plebis- cite, by a clear if not overwhelming margin. Six months before the plebiscite, the independent research center FLACSO/Chile conducted a national survey of 2700 randomly selected Chilean voters.2 Of these individuals, 868 said that they were planning to vote yes, and 889 said that they were planning to vote no. Of the remainder, 558 said that they were undecided, 187 said that they planned to abstain, and 168 did not answer the question. I will look here only at those who expressed a preference.3 Figure 14.1 plots voting intention against a measure of support for the status quo. As seems natural, voting intention appears as a dummy variable, coded 1 for yes, 0 for no. As we will see presently, this coding makes sense in the context of a dichotomous response variable. Because many points would otherwise be overplotted, voting intention is jittered in the graph (although not in the calculations that follow). Support for the status quo is a scale formed from a number of questions about political, social, and economic policies: High scores represent general support for the policies of the miliary regime. (For the moment, disregard the lines plotted in this figure.) We are used to thinking of a regression as a conditional average. Does this interpretation make sense when the response variable is dichotomous? After all, an average between 0 and 1 represents a ‘‘score’’ for the dummy response variable that cannot be realized by any individ- ual. In the population, the conditional average EðYjxiÞ is simply the proportion of 1s among those individuals who share the value xi for the explanatory variable—the conditional probabil- ity πi of sampling a yes in this group; 4 that is, πi [ PrðYiÞ[ PrðY ¼ 1jX ¼ xiÞ and, thus, EðYjxiÞ ¼ πið1Þ þ ð1& πiÞð0Þ ¼ πi ð14:1Þ If X is discrete, then in a sample we can calculate the conditional proportion for Y at each value of X . The collection of these conditional proportions represents the sample nonpara- metric regression of the dichotomous Y on X . In the present example, X is continuous, but we 2FLACSO is an acronym for Facultad Latinoamericana de Ciencias Sociales, a respected institution that conducts social research and trains graduate students in several Latin American countries. During the Chilean military dictatorship, FLACSO/Chile was associated with the opposition to the military government. I worked on the analysis of the survey described here as part of a joint project between FLACSO in Santiago, Chile, and the Centre for Research on Latin America and the Caribbean at York University, Toronto. 3It is, of course, difficult to know how to interpret ambiguous responses such as ‘‘undecided.’’ It is tempting to infer that respondents were afraid to state their opinions, but there is other evidence from the survey that this is not the case. Few respondents, for example, uniformly refused to answer sensitive political questions, and the survey interviewers reported little resistance to the survey. 4Notice that πi is a probability, not the mathematical constant π » 3:14159. A Greek letter is used because πi can be estimated but not observed directly. 14.1 Models for Dichotomous Data 371 can nevertheless resort to strategies such as local averaging, as illustrated in Figure 14.1.5 At low levels of support for the status quo, the conditional proportion of yes responses is close to 0; at high levels, it is close to 1; and in between, the nonparametric regression curve smoothly approaches 0 and 1 in a gentle, elongated S-shaped pattern. 14.1.1 The Linear-Probability Model Although nonparametric regression works here, it would be useful to capture the dependency of Y on X as a simple function. To do so will be especially helpful when we introduce addi- tional explanatory variables. As a first effort, let us try linear regression with the usual assumptions: Yi ¼ αþ βXi þ εi ð14:2Þ where εi ; Nð0; σ2εÞ, and εi and εi0 are independent for i 6¼ i0. If X is random, then we assume that it is independent of ε. −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 Support for the Status Quo V ot in g In te nt io n 1.0 0.8 0.6 0.4 0.2 0.0 Figure 14.1 Scatterplot of voting intention (1 represents yes, 0 represents no) by a scale of support for the status quo, for a sample of Chilean voters surveyed prior to the 1988 plebis- cite. The points are jittered vertically to minimize overplotting. The solid straight line shows the linear least-squares fit; the solid curved line shows the fit of the logistic- regression model (described in the next section); the broken line represents a non- parametric kernel regression with a span of 0.4. 5The nonparametric-regression line in Figure 14.1 was fit by kernel regression—a method based on locally weighted averaging, which is similar to locally weighted regression (lowess, which was introduced in Chapter 2 for smoothing scatterplots). Unlike lowess, however, the kernel estimator of a proportion cannot be outside the interval from 0 to 1. Both the kernel-regression estimator and other nonparametric-regression methods that are more appropriate for a dichotomous response are described in Chapter 18. The span for the kernel regression (i.e., the fraction of the data included in each local average) is 0.4. 372 Chapter 14. Logit and Probit Models for Categorical Response Variables Under Equation 14.2, EðYiÞ ¼ αþ βXi, and so, from Equation 14.1, πi ¼ αþ βXi For this reason, the linear-regression model applied to a dummy response variable is called the linear-probability model. This model is untenable, but its failure will point the way toward more adequate specifications: ! Because Yi can take on only the values 0 and 1, the conditional distribution of the error εi is dichotomous as well—and, hence, is not normally distributed, as assumed: If Yi ¼ 1, which occurs with probability πi, then εi ¼ 1& EðYiÞ ¼ 1& ðαþ βXiÞ ¼ 1& πi Alternatively, if Yi ¼ 0, which occurs with probability 1& πi, then εi ¼ 0& EðYiÞ ¼ 0& ðαþ βXiÞ ¼ 0& πi ¼ &πi Because of the central limit theorem, however, the assumption of normality is not critical to least-squares estimation of the normal-probability model, as long as the sample size is sufficiently large. ! The variance of ε cannot be constant, as we can readily demonstrate: If the assumption of linearity holds over the range of the data, then EðεiÞ ¼ 0. Using the relations just noted, V ðεiÞ ¼ πið1& πiÞ2 þ ð1& πiÞð&πiÞ2 ¼ πið1& πiÞ The heteroscedasticity of the errors bodes ill for ordinary least-squares estimation of the linear probability model, but only if the probabilities πi get close to 0 or 1. 6 Goldberger (1964, pp. 248–250) has proposed a correction for heteroscedasticity employing weighted least squares.7 Because the variances V ðεiÞ depend on the πi, however, which, in turn, are functions of the unknown parameters α and β, we require preliminary esti- mates of the parameters to define weights. Goldberger obtains ad hoc estimates from a preliminary OLS regression; that is, he takes bV ðεiÞ ¼ bYið1& bYiÞ. The fitted values from an OLS regression are not constrained to the interval [0,1], and so some of these ‘‘var- iances’’ may be negative. ! This last remark suggests the most serious problem with the linear-probability model: The assumption that EðεiÞ ¼ 0—that is, the assumption of linearity—is only tenable over a limited range of X -values. If the range of the X s is sufficiently broad, then the linear specification cannot confine π to the unit interval ½0; 1(. It makes no sense, of course, to interpret a number outside the unit interval as a probability. This difficulty is illustrated in Figure 14.1, in which the least-squares line fit to the Chilean plebiscite data produces fitted probabilities below 0 at low levels and above 1 at high levels of support for the status quo. 6See Exercise 14.1. Remember, however, that it is the conditional probability, not the marginal probability, of Y that is at issue: The