1 Version: November 22, 2009 Scenario Adjustment in Stated Preference Research Trudy Ann Cameron* Mikesell Professor of Environmental and Resource Economics Department of Economics, 435 PLC 1285 University of Oregon Eugene, OR 97403-1285 USA cameron@uoregon.edu ph: (541) 346-1242 fax: (541) 346-1243 J.R. DeShazo Associate Professor Department of Public Policy 3250 Public Policy Building University of California Los Angeles, CA 90095-1656 USA deshazo@ucla.edu Erica H. Johnson Ph.D. student Department of Economics, 435 PLC 1285 University of Oregon Eugene, OR 97403-1285 USA ejohnso1@uoregon.edu * Corresponding author Acknowledgements: We are grateful for the helpful comments of Paul Jakus and Edward Morey, and participants in our session at the 2007 AAEA/AERE meetings in Portland, OR. This research has been supported by a grant from the National Science Foundation (SES-0551009) to the University of Oregon (PI: Trudy Ann Cameron). It employs original survey data from an earlier project supported by the US Environmental Protection Agency (R829485) and Health Canada (Contract H5431-010041/001/SS). Additional support has been provided by the Raymond F. Mikesell Foundation at the University of Oregon. Office of Human Subjects Compliance approval filed as Protocol #C4-380-07F at the University of Oregon. This work has not been formally reviewed by any of the sponsoring agencies. Any remaining errors are our own. 2 Scenario Adjustment in Stated Preference Research ABSTRACT Stated preference (SP) survey methods have been used increasingly to assess willingness to pay for a wide variety of non-market goods and services, including reductions in risks to life and health. Poorly designed SP studies are subject to a number of well-known biases, but many of these biases can be minimized when they are anticipated ex ante and accommodated in the study’s design or during data analysis. We identify another source of potential bias, which we call “scenario adjustment,” where respondents assume that the substantive alternative(s) in an SP choice set, in their own particular case, will be different than the survey instrument describes. We use an existing survey, developed to ascertain willingness to pay for private health-risk reduction programs, to demonstrate a strategy to control and correct for scenario adjustment in the estimation of willingness to pay. This strategy involves data from carefully worded follow-up questions and ex post econometric controls for each respondent’s subjective departures from the intended choice scenario. Our research has important implications for the design of future SP surveys. Keywords: scenario adjustment, scenario rejection, stated preference, value of a statistical life (VSL), value of a statistical illness profile JEL Classifications: C42, C35, Q51, I18 1 1. Introduction Recent interest in behavioral economics had led researchers to revisit instances where conventional empirical models of rational consumer decision-making may have failed to provide an adequate picture of choice behavior. Bernheim and Rangel (2009), for example, note that “it is often difficult to formulate coherent and normatively compelling rationalizations for non- standard choice patterns” (i.e. when consumers do not choose in the way that our utility- maximization models would predict). Researchers have also long recognized that subjective beliefs are an important determinant of consumers’ choices (Dominitz and Manski (2004); Manski (2004)). For example, individuals’ beliefs about the timing of their own risks for particular illnesses may determine their willingness to pay to reduce these health risks (e.g., to purchase organic foods), to measure their risks (e.g., to purchase a new diagnostic test not currently covered by insurance), or to buy extra insurance against undesirable health risk outcomes (e.g., to purchase Medi-Gap policies). As subjective beliefs change, individual behavior will probably change as well. The individual’s subjective beliefs are an example of “ancillary conditions” associated with a choice, as described in Bernheim and Rangel (2009). As ancillary conditions for choices, subjective beliefs are likely to play an important role when research subjects answer questions in stated preference (SP) surveys used to value non- market goods. An SP survey describes a scenario in which the respondent is offered a hypothetical opportunity to purchase one or more costly programs that yield particular sets of individual-specific consequences. When asked to make choices about health-related programs, for example, individuals may hold strong prior beliefs about many aspects of the alternatives in the choice scenario, including their own risks of particular illnesses, the time profile of those 2 illness-specific risks over their lifetimes, the effectiveness of preventive actions, the effectiveness of the probable treatments, etc. When respondents hold prior beliefs about any aspect of the scenario that diverge from the researcher-prescribed information, three possibilities arise: (1) respondents may replace their beliefs about aspects of the scenario with the information provided by the researcher; (2) they may retain their beliefs and instead reject the choice scenario as irrelevant or unrealistic, often resulting in a protest response, and (3) they may accept the scenario but “adjust” some of its informational aspects to fit their own personal situation, history or context. We define “scenario adjustment” to occur in the third case, where respondents impute or modify some aspect of a given choice scenario based upon their personal beliefs, which constitute ancillary conditions for these types of choices. This paper concerns the identification of scenario adjustment as a behavioral phenomenon affecting choices. We also illustrate one strategy for correction. We take advantage of an existing stated preference survey concerning prospective health risk reductions, described in Cameron and DeShazo (2009). This survey is designed to elicit choices that allow the researcher to infer willingness to pay for privately purchased diagnostic programs which reduce the prospective risk that respondents will experience specific illness profiles over their remaining lifespans. An illness profile consists of a description of a sequence of future health states associated with a major illness that the respondent may experience with some baseline probability. The specific type of “scenario adjustment” problem we address in this paper has to do with each respondent’s degree of acceptance of the stated latency of the illness (i.e. time until the onset of symptoms). Latency is specified as an attribute of each illness profile described in the choice sets used in the survey. 3 Our assessment of the consequences of scenario adjustment (and thus our potential correction strategy) is made possible because our respondents are asked appropriate debriefing questions after each stated choice question concerning alternative health-risk reduction programs. These debriefing questions allow us to distinguish between respondents who appear to accept the latency information given in the choice scenario (and therefore presumably answer the choice question based on the latencies described in the choice scenario) from those who subjectively adjust the latency information in the scenario (and therefore appear to have answered a somewhat different question). Some individuals underestimate the latency period—they believe that the program’s benefits, in their own case, would start sooner. Other individuals overestimate the latency period. If subjective latency affects willingness to pay (WTP) for risk reductions, then respondents’ latency perceptions can influence their estimated WTP amounts.1 If scenario adjustment is ignored, it is possible that this behavior on the part of respondents may cause the researcher to underestimate WTP for some respondents and overestimate WTP for others, to varying degrees. The opposing effects are unlikely to be exactly offsetting. In cases like this, researchers should probably calculate and compare estimates of WTP both with and without corrections for scenario adjustment. But this implies that as early as the process of survey design, researchers should try to anticipate the dimensions along which respondents may be inclined to adjust the stated choice scenario, despite the survey designer’s best efforts. Suitable debriefing questions need to be included in the survey to permit a formal assessment of the extent of this behavior. 1 “Scenario adjustment” might occur as follows. Suppose a male respondent has a family history of heart disease at age fifty. In his copy of the survey, the stated choice scenario that involves heart disease may specify that this illness would lead to moderate and/or severe pain and disability starting at age seventy. However, given his private knowledge, he might answer the question as though the benefits of the proposed risk reduction program would begin at age fifty. 4 We note that corrections for scenario adjustment must be considered in relation to the practice of “libertarian paternalism” as discussed by Thaler and Sunstein (2003) and Smith (2007). “Libertarian paternalism” involves honoring consumer sovereignty to the greatest extent possible (the libertarian part), but intervening to override some aspects of behavior when the researcher believes that these are mistakes (the paternalism part). For example, suppose the researcher is attempting to value removal of the health risks associated with a toxic waste site. The survey may state a particular objective existing risk, but one-third of the survey’s respondents may believe that the risk is ten times as large as the objective risk. The model could be estimated based on each individual’s subjective risks. However, to generate an estimate of social benefits based on the objective risks, the researcher may counterfactually simulate what this third of the sample would have been willing to pay, had they believed the lower objective risks instead. Scenario adjustment, as described in this paper, likewise involves simulating the preference parameters that would have been estimated. In this case, the counterfactual is the case where do not approach the choice using their own subjective estimates of latency, but instead “buy” the attributes of each illness profile as described in the survey. There is one final consideration when considering scenario adjustment in stated preference studies. Individuals may likewise adjust choice scenarios in real-life choice situations. If “scenario adjustment” happens with similar frequency in actual markets, then perhaps these misalignments are an unavoidable part of how consumers truly behave in real markets. If a stated preference study is designed to predict future actual choice behavior, perhaps the SP choice models should allow people to make the same “mistakes” that they would make in real life. However, if the goal is welfare assessment based on WTP under conditions of full information, then corrections are more justified. Of course, if scenario adjustment is for some reason more 5 pronounced in hypothetical choice scenarios, as opposed to real market conditions, then perhaps the researcher should correct the misalignment in order to more accurately predict respondents’ WTP under real conditions. The empirical contribution of this paper is to identify and illustrate one case of scenario adjustment by respondents to a stated preference survey, and to demonstrate the potential consequences of this behavior for the resulting estimates of WTP. The paper proceeds as follows: Section 2 reviews in more detail the related literature on perceptions and SP. Section 3 briefly describes our survey and the data it produces. Section 4 briefly reviews a utility-theoretic choice model used to analyze respondents’ program preferences. Section 5 discusses how to control for scenario adjustment and conveys our empirical results, and Section 6 concludes. 2. Related Literature Researchers certainly recognize that respondents bring their beliefs and perceptions about aspects of a choice scenario into a choice setting (Manski, 2004).2 Researchers are also aware that the information provided in an SP choice scenario may conflict with respondents’ beliefs and perceptions, in some cases because of the random assignment of attributes in efficiently designed conjoint choice sets. Some respondents may be presented with scenarios containing unrealistic or irrelevant choice alternatives, relative to the individual’s beliefs, despite these being plausible for the average respondent. A tension may thus arise between the efficient design of a choice set and respondents’ expectations regarding which kinds of choice alternatives are realistic or relevant (Louviere, et al. (2000); Louviere (2006)). 2 For example Adamowicz, et al. (1997) and Poor, et al. (2001) compare WTP estimates from choice models that use both objectively measured and subjectively perceived levels of attributes. Experimental economists (e.g. Plott and Zeiler (2005)) have examined the role of subjective beliefs in explaining the gap between WTP and willingness to accept (WTA). 6 When confronted with unrealistic or irrelevant choice scenarios, respondents may issue protest responses. Outright scenario rejection may lead a respondent to state that they prefer the status quo alternative, but they do this for reasons that have nothing to do with their preferences or the constraints they face (or they may refuse to make any choice at all). This behavior may indicate merely that they doubt the viability of the hypothetical product or proposed program, rather than implying that they would not value it if it were guaranteed to work as advertised.3 When some choices may be protest responses or belie some type of scenario rejection, then, it is important to distinguish these from “good” responses (although in practice it can be difficult to draw these distinctions).4 Bateman, et al. (2002) suggest several methods to identify these protest responses such as follow-up questions about why respondents answered the way they did. Strazzera, et al. (2003) also offer possible corrections for selection bias caused by protest zeroes in contingent valuation studies. Instead of outright scenario rejection, we address the phenomenon of scenario adjustment—where respondents find one or more of the attributes of the stated alternatives to be somewhat implausible but this problem does not derail the choice process entirely. Instead, the individuals may implicitly replace one or more of these implausible stated attributes with something that they deem more plausible and then make their decisions based on these mental edits to the choice set. Outright scenario rejection may be difficult enough to detect, but scenario adjustment—which is a matter of degree, rather than an all-or-nothing proposition—may be 3 For a more detailed description of protest responses and protest bids, see Bateman, et al. (2002) and Champ, et al. (2003). Rejection of the proposed payment vehicle (e.g. a tax or a user fee) can be another form of protest. 4 Even in real choice situations, a consumer may choose not to buy a product simply because the seller’s claims about it seem “too good to be true.” If the consumer could verify the product’s qualities, however, they would actually make the purchase. This suggests that scenario rejection (and scenario adjustment) may thus be fairly common in real markets, too. 7 more insidious and therefore even more difficult to detect. Debriefing questions asked after respondents make the key choice(s) can be invaluable for this purpose. SP researchers have long realized the potential for debriefing questions to help them understand the perceptions of the respondent during the choice process. A handful of researchers have already used specific debriefing questions for detection of scenario adjustment. Carson, et al. (1994) ask subjects whether they believed that the pollutants in question could actually cause the environmental problems stated in the choice scenario and whether they believed that natural processes would return things to normal within the stated number of years. When respondents said they did not believe the stated natural recovery time, they were asked if they thought the true recovery time was more or less than the stated time. In a similar vein, Viscusi and Huber (2006) ask their respondents for subjective assessments of the probability that the program in question will actually produce the advertised benefits.5 Researchers should put forth their best effort to make the choice scenarios in a stated preference survey as plausible as possible, for as many respondents as possible. Despite these best efforts, however, it may be impossible for researchers to fully anticipate the likely credibility of all dimensions of a randomized choice scenario from the perspective of every individual who might participate in the survey. The best strategy to deal with any residual scenario adjustments may be for researchers to anticipate that this behavior is inevitable in some proportion of cases and to plan for the option to assess and correct for it. Our paper illustrates how some carefully worded debriefing questions can be used to measure the approximate extent of one type of scenario adjustment. Our econometric model controls for these scenario adjustments, and counterfactual simulations can then be used to infer what would be the 5 Burghart, et al. (2007) extend a random utility model to include estimated scenario adjustment parameters that capture whether respondents appear to believe and/or pay attention to certain key attributes of alternatives in the choice set, conditional on the functional form of the choice model. 8 estimated preferences (and hence WTP) had each individual in the sample fully accepted these key dimensions of the stated choice scenario. 3. Available Choice Data Since market data from which to infer individuals’ demands for health risk reductions is not adequate, Cameron and DeShazo (2009) use stated preference methods to elicit preferences for programs to reduce the risk of morbidity and mortality in a general population sample of adults in the United States.6 In brief, the survey consists of 5 modules.7 The first module asks respondents about their subjective risks of contracting the major illnesses or injuries which are the focus of the survey, how lifestyle changes would change their risks of these illnesses, and how taxing it would be to implement these lifestyle changes. The second module is a tutorial that explains the concept of an “illness profile,” which is a sequence of future health states. An illness profile includes the number of years before the individual becomes sick, illness-years while the individual is sick, recovered/post-illness years after the individual recovers from the illness, and lost life-years if the individual dies earlier than he would have without the disease (or injury). Then the tutorial informs the individual that he might be able to purchase a new, minimally invasive diagnostic program that would reduce his risk of experiencing each illness profile. Each illness-related risk-reduction program consists of 6 Knowledge Networks, Inc administered an internet survey to a sample of 2,439 of their panelists with a response rate of 79 percent. 7 For more information on the survey instrument and the data, see the appendices which accompany Cameron and DeShazo (2009): Appendix A – Survey Design & Development, Appendix B – Stated Preference Quality Assurance and Quality Control Checks, Appendix C – Details of the Choice Set Design, Appendix D – The Knowledge Networks Panel and Sample Selection Corrections, Appendix E – Model, Estimation and Alternative Analyses, and Appendix F – Estimating Sample Codebook. 9 a simple finger-prick blood test that would not be covered by the individual’s health insurance plan.8 The third, and key, module of each survey involves a set of five different three-alternative conjoint choice experiments where the individual is asked to choose between two possible health-risk reducing programs and a status quo alternative. One example of a choice scenario is presented in Figure 1. Each program reduces the risk that the individual will experience a specific illness profile for a major illness or injury (i.e. one of five specific types of cancer, heart attack, heart disease, stroke, respiratory illness, diabetes, traffic accident or Alzheimer’s disease). Each individual-specific illness profile is described to the respondent in terms of the baseline probability of experiencing the illness or injury, future age at onset, duration, symptoms and treatments, and eventual outcome (recovery or death). The corresponding risk reduction program is defined by the expected risk reduction and by its monthly and annual cost. Each choice exercise is immediately followed by a set of debriefing questions designed to help the researcher understand the individual’s reasons for their particular choice. Some debriefing questions depend on the alternative chosen by the respondent. For example, there are various perfectly legitimate economic reasons why individuals may prefer the status quo— including that they cannot afford either of the risk-reduction programs which are described, they would rather spend money on other things, or they believe they will be affected by another illness before they contract either illness stated in the scenario. If respondents choose the status quo, they are asked why “Neither Program” is their preferred alternative. Included among these possible reasons are some that reveal the presence of scenario rejection, such as “I did not believe the programs would work.” 8 The cost of the program would cover the test and any indicated medications or treatments to reduce the risk of suffering the illness in question. 10 Other debriefing questions are asked regardless of which alternative the individual selects. The key question for this paper is shown in Figure 2: “Around when do you think you would begin to value highly the risk reduction benefits of each program?” We interpret this question as being equivalent to the question “When do you think the program’s benefits will start?” The benefit of the program is clearly defined on an earlier page of the survey as a reduction in the risk of suffering from the specified major illness or injury starting at the age stated in the scenario. If the respondent fully accepts the stated scenario, then the age at which the scenario states the benefits start should match the age at which the respondent believes the benefits will start. Module 4 of the survey contains additional debriefing questions which permit validation of other dimensions of the individual’s responses. Module 5 is collected separately from the survey and contains the respondent’s sociodemographic characteristics and a detailed medical history, including which major diseases the individual has already faced. 4. A Random Utility Choice Model This paper is based on an empirical specification similar, although not identical, to that used in Cameron and DeShazo (2009). In that paper, it is established that stated choices appear to be best predicted by a model that involves expected discounted utility from durations in different types of future health states. Indirect utility is also modeled as additively separable, but non-linear, in present discounted expected net income. If utility is modeled flexibly as a quadratic function of net income, the most basic specification is a five-parameter model.9 9 In Cameron and DeShazo (2009), to minimize the dimension of the parameter space, utility is modeled as a function of a simple fixed Box-Cox transformation of net income, where the transformation parameter is set at 0.48 ( roughly equivalent to a square root transformation). The remainder of this section consists of an abbreviated version of the reasoning described in Cameron and DeShazo (2009) and Appendix E associated with that paper (Model, Estimation and Alternative Analyses). 11 To understand the model, consider just the pairwise choice between Program A and the status quo alternative (N).10 Define the discount rate as r and let  1 tt r   . Let NSi be the probability of individual i suffering the adverse health profile (i.e. getting “sick”) if the status quo alternative (i.e. “neither program”) is selected, and let ASi be the reduced probability of suffering the adverse health profile if Program A is chosen. The difference between NSi and AS i is Ai , which is the risk reduction to be achieved by Program A. We assume that individuals do not expect to pay the annual cost of the risk reduction program if they are sick or dead. The sequence of health states that makes up an illness profile is captured by a sequence of mutually exclusive and exhaustive (0, 1) indicator variables associated with each future time period t . These are defined as 1( )Aitpre for pre-illness years, assumed to be equivalent to the health state under the status quo alternative. The sequence of adverse health states for which Program A reduces the risk are indicated by 1( )Aitill for illness-years, 1( ) A itrcv for recovered or post-illness years, and 1( )Aitlyl for life-years lost. The present discounted remainder of the individual’s nominal life expectancy, iT , is given by 1 iTA t i t pdvc  . Other relevant discounted spells, also summed from 1t  to it T include  1A t Ai itpdve pre  ,  1A t Ai itpdvi ill  ,  1A t Ai itpdvr rcv  , and  1A t Ai itpdvl lyl  . Since the different health states exhaust the individual’s nominal life expectancy, A A A A A i i i i ipdve pdvi pdvr pdvl pdvc    . Finally, to accommodate the fact that the individuals 10 There is an analogous choice between Program B and the status quo alternative. 12 expect to pay program costs only during the pre-illness or recovered post-illness periods, we define A A Ai i ipdvp pdve pdvr  . To further simplify notation, let  1A AS A AS Ai i i i icterm pdvc pdvp     and let A A AS A NS A i i i i i iyterm pdvc pdvi pdvl      . Adapting the model in Cameron and DeShazo (2009), the expected utility-difference that drives the individual’s choice between Program A and the status quo can then be specified as:               , 0 2 2 1 1 2 3 + + A A A A S H i i i i i i A A A i i i i i AS A AS A AS A A i i i i i i i E PDV V Y c cterm Y yterm Y c cterm Y yterm pdvi pdvr pdvl                     (1) Where 0 and 1 are the coefficients on the linear and quadratic terms in net income in the underlying indirect utility function. The five terms in braces can be constructed from the data, given specific assumptions about the discount rate.11 In our tables of results, the constructed variables in the first two sets of braces, modified by the coefficients 0 and 1 , will be referenced simply as “first income term” and “second income term.” In the sense of Graham (1981), the “option price” for Program A is defined as the maximum common certain payment that makes the individual just indifferent between paying for the program and enjoying the risk reduction, or not paying for the program and not enjoying the risk reduction. The annual option price ˆAic that makes the expression in equation (1) exactly equal to zero can be calculated as   0 11 0 1ˆ A A A i i i iA i i A i i Y yterm pterm c Y f Y cterm                 (2) 11 In this paper, we assume a common discount rate of 5%. In Cameron and DeShazo (2009), the consequences of assuming either a 3% discount rate or a 7% discount rate are explored. 13 Where   20 1 0 1( ) i i i if Y Y Y Y Y       , so that  1f   is the solution to a quadratic form. Then, the expected present value of this stream of payments must be calculated over the individual’s remaining nominal lifespan:  , ˆ ˆA A AS H i i iE PV c cterm c       (3) Finally, to convert this expected present-value option price into a measure that Cameron and DeShazo (2009) call the “willingness to pay for a microrisk reduction” ( rWTP ), we normalize arbitrarily on a 610 risk change by dividing this WTP by the absolute size of the risk reduction and dividing by one million to produce:   6, ˆ 10A Ar S H i iWTP E PV c      (4) The rWTP depends upon the entire illness profile and all of the parameters in equation (1). The value of one million microrisk reductions is the closest counterpart, in this model, to the conventional idea of the “value of a statistical life” (VSL) employed in the mortality risk valuation literature, as discussed (for example) in the meta-analysis by Viscusi and Aldy (2003). This normalized rWTP can be used to compare the relative magnitudes of willingness to pay for health risk reductions for differing age groups and illness profiles.12 Cameron and DeShazo (2009) determine, however, that the simple five-parameter model in equation (1) is dominated by a specification that is not merely linear in the terms involving present discounted health-state years. First, we factor out the probability differences in the final terms in equation (1) as: 12 For readers who may be less familiar with the literature on VSLs, we emphasize that a VSL is definitely not a measure of willingness to pay to avoid empirically relevant sizes of risk reductions, such as the modest reductions in already-small risks, achieved by most environmental, health, or safety regulations. The typical risk reduction is vastly smaller than the 1.00 risk reduction used for the conventional normalization behind a VSL estimate. 14      1 2 3 1 2 3 + + AS A AS A AS A i i i i i i AS A A A i i i i pdvi pdvr pdvl pdvi pdvr pdvl                This simple linear specification does not explain respondents’ observed choices as successfully as a model that employs shifted logarithms of the jipdvX terms (where , ,X i r l .). A form that is fully translog (including all squares and pairwise interaction terms for the three log terms) has been considered, and two of the higher-order terms bear statistically significant coefficients. If we retain only those terms for which the coefficients are statistically different from zero, this final term becomes:               1 2 3 2 4 5 log 1 log 1 log 1 log 1 log 1 log 1 A A A i i i AS i A A A i i i pdvi pdvr pdvl pdvl pdvi pdvl                     (5) The opportunity for longer durations in each health state is correlated with the youth of the respondent. Thus, it is also important to allow the  coefficients to differ systematically with the respondent’s current age wherever this generalization is warranted by the data. This leads to a model where 23 30 31 31i iage age      , and analogously for 4 and 5 . This quadratic-in- age systematic variation in parameters permits non-constant age profiles for the rWTP estimates from this model, and the data tend to produce the usual higher values during middle age and lower values for younger and older respondents.13 13 One final additional parameter is featured in these models. It accommodates a correction for sample representativeness. Cameron and DeShazo (2009), in Appendix D, estimate the determinants of membership in the estimating sample, relative to the original half-million general population panel recruitment contacts by Knowledge Networks, Inc. These models permit construction of fitted response probabilities for each consumer in the estimating sample. These response probabilities can be expressed as deviations from the central tendency in response probabilities across the recruitment pool. Only the coefficient on the term in discounted illness-years is shifted to a statistically significant extent when the subject’s response probability deviates from the average. Thus the model includes a shift variable on that coefficient by employing the interaction term    log 1AS Ai i iP sel P pdvi        . 15 5. Controlling for Scenario Adjustment with Respect to Latency Recall that after each choice scenario, respondents are asked debriefing questions about when they believe that the benefits of each proposed program would begin—for them personally. Based on the answers to each of the questions in Figure 2, we define two variables. First, 1( )jinever is an indicator variable that takes a value of one if the individual responds by checking “Never (Program would not benefit me).” Our second variable, jioverest , is an approximately continuous variable defined as the “minimum overestimate of the latency,” which measures the disparity between the individual’s subjective latency and the latency stated in the choice scenario on the survey. The variable jioverest requires a more detailed explanation. If the interval checked in the question in Figure 2 contains the stated latency for the illness from the corresponding choice scenario, then jioverest = 0. The relationship between the chosen interval and the stated latency is thus something like that shown in Part A of Figure 3. In this case, the time when benefits begin (in the opinion of the respondent) is essentially the same as the latency stated in the choice scenario. In contrast, jioverest has a positive value equal to the difference between the lower bound of the checked time interval and the stated latency if that checked interval lies entirely above the stated latency for that illness in the choice scenario, like the outcome shown in Part B of Figure 3. jioverest has a negative value equal to the difference between the upper bound of the checked interval if the checked interval lies entirely below the stated latency, as illustrated in Part C of Figure 3.14 14 In Appendix A to this paper, available from the authors, we explore the relationships between each of our two scenario adjustment variables and an array of explanatory variables specific either to the individual or to the choice scenario. In the body of the paper, however, we use the observed values of these variables, rather than fitted values. Many respondent characteristics contribute significantly to explaining variation in these two variables, but as can be 16 The usual intent within a stated preference study is to induce individuals to accept the stated choice scenario as fully as possible and for them to respond conditional on that acceptance. If respondents selectively reinterpret the question (i.e. adjust the choice scenario) before they answer, then this violates an important maintained hypothesis behind the random utility model that produces the utility parameter estimates which are the foundation of most stated preference studies. We thus use the “observed” values of 1( )jinever and j ioverest constructed from the debriefing questions associated with each of the 15,040 illness profiles presented to our respondents to control and correct for scenario adjustment with respect to the latency attribute. Descriptive statistics for the variables used in these models are presented in Table 1. We accommodate scenario adjustment by allowing each of the utility parameters in our baseline model to differ systematically with individuals’ responses to the debriefing questions about whether and when the benefits from each health-risk reduction program are likely to be realized. The working version of the model without scenario adjustments involves a total of fourteen parameters— 0 and 1 which capture the marginal utility of net income (i.e. expenditure on all other goods and services), the five basic  parameters ( 10 20 30 40 50, , , ,     ) appearing in the illness profile term in expression (5) above, plus the three pairs of coefficients on the iage and 2 iage terms that shift 3 , 4 and 5 , and the single coefficient, 13 , on the interaction term between the fitted sample participation probability and the sick-years term. To effect corrections for scenario adjustment, our two scenario adjustment variables, 1( )jinever and j ioverest , are initially allowed to shift every one of these fourteen utility seen from the R-squared values for these fitted models, it is difficult to predict accurately either of the two scenario adjustment variables based solely on observed characteristics of each individual. 17 parameters. If we represent each of these parameters generically as  , the new model substitutes a systematically varying parameter as follows: 15 0 1 21( ) j j i inever overest      (2) for a total of 52 parameters in the fully generalized specification.16 In Table 2, however, we report results for a parsimonious version that retains only those shift variables which are individually statistically significant.17 Model 1 in Table 2 gives the utility parameter estimates which result when the possibility of scenario adjustment is completely ignored during estimation. Model 2 in the same table (which actually spans columns 2 through 4) reveals the results when scenario adjustment is accommodated. The ideal situation (i.e. full acceptance of the stated latency of benefits) corresponds to 1( ) 0jinever  and 0jioverest  for all respondents and all programs. We thus label the first column of parameters for Model 2 as “Corrected,” since these are the estimated utility parameters which would apply when 1( )jinever and j ioverest are both set equal to zero— i.e. when we simulate counterfactually the latency scenarios that the survey had intended each respondent to accept. In Model 2, where we measure and correct for scenario adjustment, the magnitudes of some of the shift parameters are striking. The second column of results for Model 2 shows the significant shifts in each of these utility parameters when the respondent states that they will 15 In a set of preliminary models, we employed both 1( )jinever and a pair of indicator variables for over- or underestimation (relative to none) to shift each of the  parameters in the general model. The results were qualitatively similar to those reported here. 16 Results for these fully generalized 52-parameter models are contained in Appendix B, available from the authors. 17 We acknowledge that these variables may be, to some extent, jointly endogenous with the underlying willingness to pay for health risk reductions because they are reported by the same individuals. In Appendix A, available from the authors, we note that despite the considerable number of statistically significant coefficients in our models to explain jioverest , we are only able to explain (at best) about 35% of its variation across illness profiles using the large number of explanatory variables we have available. 18 never benefit from the program in question. The third column shows the significant shifts in these parameters for a one-unit increase in jioverest . The marginal utility of income from Model 2 serves as the denominator in the marginal rate of substitution (between each illness profile attribute and income) that gives the estimated marginal willingness to pay associated with each attribute. Overestimation of the latency appears to be associated with a higher estimated marginal utility of income, which means a lower WTP. The perception that a particular program will never benefit the respondent also appears to undo the evidence for a diminishing marginal utility of income. For these cases, WTP is no longer greater at higher incomes, whereas this appears to be the case when respondents fully accept the stated latencies. There are also a number of important differences for “scenario adjusters” among the coefficients on the illness profile terms. In two cases (for the linear term in the shifted log of discounted sick years, and for the interaction term between the shifted logs of discounted sick- years and discounted lost life-years), the discrete shift in the parameter associated with the perception that the program will never provide any benefit is sufficient to completely change the sign of the effect. In two other cases, the sign of the coefficient remains the same but the coefficient more than doubles in size. Whenever the coefficients on the interaction terms involving jioverest are statistically significant, they bear a sign that is opposite to the baseline coefficient on the same term. Scenario adjustments can thus have a clearly discernible impact upon estimated marginal utilities. The magnitudes of the shift parameters reported for Model 2 in Table 2 appear fairly large, but to appreciate the overall effects of these parameter changes on demand estimates, it is necessary to simulate distributions for the implied (normalized) willingness-to-pay estimates. 19 Bear in mind that the U.S. EPA, for example, relies upon an overall average value of a statistical life (a VSL associated with sudden death in the current period) of about $6-$7 million, whereas for transportation policies, the VSL numbers typically used have historically been closer to $3-$4 million, although they have been revised upward somewhat in recent years. In Table 3, we show selected rWTP estimates for specified individuals and conditions. We consider, in succession, an individual who is 30, 45, or 60 years old. In all cases, the individual earns an income of $42,000 per year. The illness profiles involve shorter (and longer) illnesses with “recovery,” shorter (and longer) illnesses followed by death, and sudden death with no preceding period of illness. The “sudden death” rWTP estimates, when multiplied by one million, are most comparable to conventional VSL estimates.18 Scenario adjustment in this illustration concerns illness latency, so two different latency periods are considered. In the first pair of columns in Table 3, we specify that each illness commences immediately (i.e. with no latency period). In the second pair, we specify a latency period of twenty years. In each pair of columns, the initial uncorrected rWTP estimates are calculated from the uncorrected parameters of Model 1 in Table 2. The corrected rWTP numbers are calculated using the baseline coefficients from Model 2 in Table 2, which have been corrected for any scenario adjustments reported by respondents. Table 3 shows that for the “No Latency” illness profiles, the corrected rWTP estimates are mostly higher than those produced by the model that does not take scenario adjustment into account. The most dramatic differences are for longer and fatal illnesses for 60-year-olds, where 18 The illnesses described in our choice scenarios are all major illnesses, including most of the ailments from which people eventually die. It is clear from our analysis that people do not assume that their health status “after” one of these illnesses, should they recover, will be equivalent to their pre-illness state. Thus the value of avoiding a one- year major illness includes the value of avoiding the ensuing post-illness health state. It will not be the same as the value of avoiding just that year of illness, separate from any ensuing years in an incompletely recovered state. 20 the uncorrected model suggests a rWTP of less than $1, whereas the corrected estimate is $6.93. The only exceptions (where the corrected estimates are lower than the uncorrected estimates) are for some of the illnesses which are not fatal. The difference in the corrected and uncorrected rWTP estimates suggests that if scenario adjustment is not taken into account, willingness to pay estimates for many illness profiles of this type may be biased downward. This type of bias may result in the recommendation that some programs or policies that reduce illnesses and injuries with no latency (i.e. where benefits start immediately) should not be implemented when it may actually be welfare-increasing to put these measures into effect. In contrast, the corrected estimates for illness profiles that have a latency of 20 years are almost all lower than the uncorrected estimates. The 90% simulated distributions for these rWTP measures often include zero. The only two anomalies, where the corrected estimates are higher, are for the non-fatal illness profiles for 30-year-olds. This suggests that failure to take into account scenario adjustment could cause some programs or policies that address long- latency health risks to be implemented when they are not actually welfare-enhancing from the current perspective of most age groups. These differences in the corrected and uncorrected rWTP estimates show how important it may be to acknowledge and correct for scenario adjustments in stated preference research. 6. Conclusions The absence of suitable market data sometimes forces researchers to use stated preference methods to assess demand for fundamentally non-market (or pre-test-market) goods or services. Given economists’ skepticism about the reliability of stated preference data, researchers in fields where adequate market data tend to be scarce have systematically addressed many recognized 21 problems with these alternative demand-measurement methodologies. One problem with SP research has been the occurrence of protest responses or scenario rejection, where respondents completely refuse to play along with the hypothetical choice exercise because they do not believe (or agree with) some aspect of the choice scenario. This paper addresses the related but potentially more subtle problem of scenario adjustment. Respondents do make the stated choices requested of them, but they first implicitly revise the choice scenario to better capture what would be the implications of each alternative in their own particular case. Scenario adjustment may be more likely in situations where the alternatives involved in the choice problem are less easy to perceive and appreciate. For example, it may be possible to describe, unambiguously, the relevant attributes of alternative brands of dishwashing soap, in which case scenario adjustment would be unlikely. In contrast, it may be very difficult to completely describe the relevant attributes of a program to enhance the survival of an endangered species, where even the experts cannot predict for certain whether the program will be effective. Choices that involve heterogeneous risks or uncertain outcomes, such as the reduction of health risks, may be the most vulnerable to scenario adjustment, since there is great variability in how different people perceive risks and uncertainty. Assessment and correction for scenario adjustment is easier and can be more systematic if suitable debriefing questions about each key choice scenario are posed in the survey. The specific debriefing question used in our empirical illustration in this paper is very useful, but it may still have been less than ideal. Carefully planned questions of this type, however, can help the researcher identify those individuals who acknowledge that they do not believe that the preceding choice scenario, exactly as stated, applies to them. Where possible, debriefing questions can also be used to quantify the likely extent to which individuals may have adjusted 22 the scenario. With information about the extent of scenario adjustments, researchers can explicitly model the effects of scenario adjustment on the estimated utility parameters in their choice models. This allows counterfactual simulations of the individual’s most likely response, had they answered the question exactly as it was asked. These types of simulations, with systematic correction for scenario adjustment, presumably permit more accurate estimates of demand. The data used in this study suggest that some individuals may indeed update some aspects of choice scenarios so that these scenarios better apply to their own personal situations. We use an empirical choice model that allows our utility parameter estimates to differ systematically according to the respondent’s own reports of possible scenario adjustment with respect to latency periods. Our estimation results show that our counterfactually simulated WTP- type benefits estimates—corrected for scenario adjustment—are often noticeably different from the uncorrected estimates. For example, our empirical estimates suggest that after correction for scenario adjustments, programs that benefit people now have mainly higher estimates, while programs that benefit people twenty years into the future have mainly lower estimates. These differences in estimated demands are big enough that they could potentially make the difference between enacting a policy that is warranted on a benefit-cost criterion and failing to enact it. Given our findings and the differences in demand estimates (with and without correction) in this illustration, we infer that scenario adjustment is likely to be inevitable and potentially influential, in at least some proportion of cases, in many other applications as well. Debriefing questions to permit assessment and correction of scenario adjustment should probably be a regular feature of SP surveys. Likewise, formal modeling of scenario adjustment and its impact on the final estimates of interest should probably be a routine component of sensitivity analysis 23 in empirical work using stated preferences. Researchers should at least report the extent to which their main results may be affected by this type of correction. Such information would allow the policy-makers to decide which types of “misalignments” between respondent and researcher information sets warrant correction, and therefore which demand estimates should be preferred. 24 References Adamowicz, W., Swait, J., Boxall, P., Louviere, J., Williams, M. (1997). "Perceptions versus objective measures of environmental quality in combined revealed and stated preference models of environmental valuation", Journal of Environmental Economics and Management 32, 65-84. Bateman, I.J., Carson, R.T., Day, B., Hanemann, W.M., Hanley, N., Hett, T., Jones-Lee, M., Loomes, G., Mourato, S., Ozdemiroglu, E., Pearce, D.W., Sugden, R., Swanson, J. (2002). Economic valuation with stated preference techniques: A manual. Cheltenham, UK: Edward Elgar Publishing Limited. Bernheim, B.D., Rangel, A. (2009). "Beyond revealed preference: Choice-theoretic foundations for behavioral welfare economics", Quarterly Journal of Economics 124, 51-104. Burghart, D.R., Cameron, T.A., Gerdes, G.R. (2007). "Valuing publicly sponsored research projects: Risks, scenario adjustments, and inattention", Journal of Risk and Uncertainty 35, 77-105. Cameron, T.A., DeShazo, J.R. (2009). "Demand for health risk reductions", Department of Economics, University of Oregon Working Paper. Carson, R.T., Hanemann, W.M., Kopp, R.J., Krosnick, J.A., Mitchell, R.C., Presser, S., Ruud, P.A., Smith, V.K. (1994). Prospective interim lost use value due to ddt and pcb contamination in the southern California bight: Volume II (Appendices). La Jolla, CA: U.S. Department of Commerce (NOAA) Champ, P.A., Boyle, K.J., Brown, T.C. (2003). A primer on nonmarket valuation. Dordrecht, The Netherlands: Kluwer Academic Publishers. Dominitz, J., Manski, C.F. (2004). "How should we measure consumer confidence?", Journal of Economic Perspectives 18, 51-66. Graham, D.A. (1981). "Cost-benefit-analysis under uncertainty", American Economic Review 71, 715-725. Louviere, J.J. (2006). "What you don't know might hurt you: Some unresolved issues in the design and analysis of discrete choice experiments", Environmental & Resource Economics 34, 173-188. Louviere, J.J., Hensher, D.A., Swait, J. (2000). Stated choice methods: Analysis and applications New York, NY: Cambridge University Press. Manski, C.F. (2004). "Measuring expectations", Econometrica 72, 1329-1376. 25 Plott, C.R., Zeiler, K. (2005). "The willingness to pay-willingness to accept gap, the 'endowment effect,' subject misconceptions, and experimental procedures for eliciting valuations", American Economic Review 95, 530-545. Poor, P.J., Boyle, K.J., Taylor, L.O., Bouchard, R. (2001). "Objective versus subjective measures of water clarity in hedonic property value models", Land Economics 77, 482-493. Smith, V.K. (2007). "Reflections on the literature", Review of Environmental Economics and Policy 1, 152-165. Strazzera, E., Genius, M., Scarpa, R., Hutchinson, G. (2003). "The effect of protest votes on the estimates of WTP for use values of recreational sites", Environmental & Resource Economics 25, 461-476. Thaler, R.H., Sunstein, C.R. (2003). "Libertarian paternalism", American Economic Review 93, 175-179. Viscusi, W.K., Aldy, J.E. (2003). "The value of a statistical life: A critical review of market estimates throughout the world", Journal of Risk and Uncertainty 27, 5-76. Viscusi, W.K., Huber, J.C. (2006). Hyperbolic discounting of public goods. NBER. 1 Figure 1 – One Example of a Randomized Choice Scenario19 19 A table like this one is displayed only after 24 screens of preparation, including an extensive tutorial that unfolds the information in each row of the summary choice table, one attribute at a time. The tutorial includes instructions about how to interpret the information and skill-testing questions to assess the respondent’s understanding of key points. The tutorial makes use of the same data that will appear in the individual’s first choice set. Subsequent choice sets are presented as summary tables only. Choose the program that reduces the illness that you most want to avoid. But think carefully about whether the costs are too high for you. If both programs are too expensive, then choose Neither Program. If you choose “neither program”, remember that you could die early from a number of causes, including the ones described below. Program A for Diabetes Program B for Heart Attack Symptoms/ Treatment Get sick when 77 years-old 6 weeks of hospitalization No surgery Moderate pain for 7 years Get sick when 67 years-old No hospitalization No surgery Severe pain for a few hours Recovery/ Life expectancy Do not recover Die at 84 instead of 88 Do not recover Die suddenly at 67 instead of 88 Risk Reduction 10% From 10 in 1,000 to 9 in 1,000 10% From 40 in 1,000 to 36 in 1,000 Costs to you $12 per month [ = $144 per year] $17 per month [ = $204 per year] Your choice Reduce my chance of diabetes Reduce my chance of heart attack Neither Program Figure 2 – Example of Debriefing Questions Used to Correct for Scenario Adjustment You may have chosen Program A, Program B, or neither. Regardless of your choice, we would like to know when, over your lifetime, you think you would first need and benefit from the two programs (if at all). Your answers below may depend upon the illness or injury in question, as well as your current age, health and family history. Around when do you think you would begin to value highly the risk reduction benefits of each program? Select one answer from each column in the grid Program A to reduce my chance of diabetes Program B to reduce my chance of heart attack For me, benefits would start Immediately 1-5 years from now 6-10 years from now 11-20 years from now 21-30 years from now 31 or more years from now Never (Program would not benefit me) 3 Figure 3 – Examples of “Minimum Overestimate of the Latency” Calculations; Different stated latencies, but respondent chooses “11-20 years” in the debriefing question Table 1: Descriptive Statistics – Policy Choice Models (n = 15040 illness profiles and associated risk reduction programs) Mean Std.dev. Min. Max. Program attributes Monthly program cost ($) 29.9 28.7 2 140 ji = Risk change achieved by program -.00341 .00167 -.006 -.001 Stated Illness profiles Latency (in years, stated in scenario) 19.6 12.0 1 60 - 1( )jinever (“Program will never benefit me”) .0769 - jioverest (minimum overest. of latency) -7.47 12.0 -59 29 Sick years (undiscounted) 6.50 7.17 0 52 jipdvi = Present value of sick-years 2.21 2.51 0 16.3 Recovered years (undiscounted) 26.1 13.0 0 64 jipdvr = Present value of recovered years .477 1.37 0 15.9 Lost life-years (undiscounted) 10.8 10.3 0 55 jipdvl = Present value of lost life-years 2.57 2.93 0 17.8 Attributes of individuals Annual income (in $10,000) 5.09 3.41 0.5 15.0 Age at time of choice 50.4 15.1 25 93 Systematic selection from RDD contacts ^ ^ ( )iP sel P = Difference between fitted response/nonresponse and population average .677 3.36 -.316 17.9 Table 2: Policy Choice Model (Parsimonious; alternatives = 22560) Model 1 Model 2 (Parameter) Variable Uncorrected Coefficients Corrected Coefficients 1( ) j inever jioverest   50 10 first income term  5.183 8.071 - 0.225 (8.30)*** (10.69)*** (5.14)***   91 10 second income term  -.1992 -.2109 .7656 - (4.22)*** (4.15)*** (3.05)***    10 log 1AS Ai ipdvi   -47.89 -57.32 212.7 7.083 (5.35)*** (5.04)*** (3.91)*** (7.24)***    13 ( ) log 1AS Ai i iP sel P pdvi         3.372 3.853 - - (2.34)** (2.45)**    2 log 1AS Ai ipdvr   -16.49 -57.93 - - (1.76)* (5.77)***    30 log 1AS Ai ipdvl   -580.1 -858.3 - 4.092 (3.25)*** (4.28)*** (3.26)***    31 0 log 1AS Ai i iage pdvl   20.46 43.15 - - (2.82)*** (5.41)***    232 0 log 1AS Ai i iage pdvl   -0.1874 -0.3719 - 0.0064 (2.70)*** (4.97)*** (7.39)***     240 log 1AS Ai ipdvl     199.3 281.8 395.6 - (2.41)** (3.11)*** (4.51)***     241 0 log 1AS Ai i iage pdvl     -7.786 -15.71 -5.197 - (2.32)** (4.31)*** (3.69)***     2242 0 log 1AS Ai i iage pdvl     0.0739 0.1365 - -0.0013 (2.27)** (3.90)*** (3.12)***       50 log 1 log 1 AS A i i A i pdvi pdvl          102.4 129.6 -348.0 -4.301 (1.40) (1.62) (3.77)*** (3.90)***       51 0 log 1 log 1 AS A i i i A i age pdvi pdvl          -4.484 -6.680 - - (1.57) (2.16)**       2 52 0 log 1 log 1 AS A i i i A i age pdvi pdvl          0.0561 0.0624 0.0752 - (2.10)** (2.17)** (3.28)*** Log L -11694.646 -10954.934 a Corrected utility parameters are purged of scenario adjustment as captured by systematic differences in these parameters for alternatives where stated latency was not accepted by the respondent. Table 3: Willingness to pay for a microrisk reduction (mean [5th, 95th percentiles]a) Without and with correction for illness scenario adjustment (Income = $42,000) No latencyb Latency of 20 yrs Age Illness profile Uncorrected Corrected Uncorrected Corrected 30 1 year sick, recover $ 2.49 [1.3,3.94] $ 3.20 [2.43,4.07] $ 1.54 [0.77,2.49] $ 1.94 [1.43,2.50] 5 yrs sick, recover 3.75 [2.59,5.16] 3.94 [3.13,4.86] 2.32 [1.60,3.20] 2.35 [1.87,2.90] 1 year sick, then die 4.14 [1.67,6.80] 6.52 [4.89,8.40] 4.42 [3.26,5.97] 1.67 [0.97,2.42] 5 yrs sick, then die 4.19 [1.39,7.21] 7.02 [5.05,9.12] 4.57 [3.51,6.00] 1.99 [1.42,2.65] Sudden death 4.26 [1.30,7.38] 5.74 [3.96,7.64] 4.35 [2.97,6.04] 1.42 [0.55,2.28] 45 1 year sick, recover 2.33 [1.20,3.75] 2.68 [1.93,3.48] 1.33 [0.64,2.15] 1.27 [0.82,1.72] 5 yrs sick, recover 3.56 [2.45,4.92] 3.47 [2.73,4.33] 2.08 [1.44,2.84] 1.68 [1.29,2.12] 1 year sick, then die 4.59 [2.99,6.55] 7.61 [6.39,9.09] 2.53 [1.95,3.21] -0.93 c [-1.59,-0.37] 5 yrs sick, then die 4.44 [2.73,6.66] 8.48 [7.04,10.14] 2.66 [2.16,3.32] -0.39 c [-0.89,0.04] Sudden death 4.57 [2.88,6.58] 6.10 [4.88,7.39] 2.43 [1.71,3.19] -1.37 c [-2.15,-0.70] 60 1 year sick, recover 2.21 [1.07,3.46] 2.04 [1.31,2.75] 1.11 [0.55,1.67] 0.30 [-0.08,0.63] 5 yrs sick, recover 3.26 [2.19,4.5] 2.86 [2.19,3.62] 1.66 [1.22,2.11] 0.59 [0.27,0.87] 1 year sick, then die 2.40 [0.98,4.03] 6.41 [5.26,7.82] 1.27 [0.57,1.91] -2.76 c [-3.79,-1.97] 5 yrs sick, then die 0.92 b [-0.6,2.58] 6.93 [5.65,8.48] 1.23 [0.67,1.78] -1.85 c [-2.63,-1.27] Sudden death 3.46 [1.88,5.13] 4.97 [3.83,6.18] 1.39 [0.52,2.09] -3.20 c [-4.32,-2.33] a Based on random draws from the joint distribution of the estimated parameters. b Zero latency was implausible to respondents in the illness profiles used to elicit program choices, so the minimum latency in the choice scenarios was 1 year. These values are thus extrapolated, based upon the fitted model. c Respondents were given no opportunity to express negative willingness to pay, so negative simulated values should be interpreted as zero WTP. APPENDIX A (to be made available online) In this Appendix, we carefully consider the empirical correlates of our two scenario adjustment indicators. Table A-1 gives descriptive statistics for these variables and a set of regressors we used to explain systematic variations in their magnitudes. First, we use a simple binary logit model to examine how the value of the indicator variable 1( )jinever can be explained by a wide variety of (a) characteristics of the respondent, and (b) attributes of the health risk targeted by each program. Each respondent considers ten different health risk-reduction programs, in five sets of two, with each choice set including the status quo as a third alternative. In total, therefore, 15,040 substantive illness profiles and health-risk reduction programs are considered in the 7,520 choice scenarios analyzed in this paper. For 1,156 (7.69%) of these illness profiles, respondents indicated their belief that they would never benefit from the risk- reduction program. Models 1 and 2 in Table A-2 are ad hoc binary logit models to explain these 7.69% of cases where 1( )jinever =1. Missing data for some of the explanatory variables used in these preliminary exploratory models accounts for the reduction of the number of illness profiles from 15,040 to 13,626. The logit specification suggests that people are more likely to say that a particular program will never benefit them if they are female, if they currently have a larger number of other illnesses, if they feel at greater subjective risk for getting other illnesses, if they are a member of a larger household, or if they are a single parent. People are less likely to say the program will never benefit them if they are presented with an illness profile that includes long-term pain and/or disability, if they have not attended college, if they acknowledge a higher 8 subjective risk of getting this disease, if they have (on average) more room to improve their health habits, and if they currently have children in their household. Now we explore the determinants of our approximately continuous measure of the “minimum overestimate of the latency,” in this case using an ordinary least squares (OLS) model. The jioverest for a program is known only if the individual does not state that they expect never to benefit from the program (i.e. if 1( ) 0jinever  ). Thus, we have a maximum of 15,040 – 1,156 = 13,884 potential observations on the jioverest variable. For many respondents and many programs, the interval during which the individual personally expects the benefits of the program to begin spans the onset time specified in the illness profile. For these individuals and programs, 0jioverest  , signaling minimal scenario adjustment with respect to the latency period. This happens for 4,133 of the 13,884 programs for which jioverest information is available. Latency is overestimated to some degree for 1,542 programs, and underestimated for 8,209 programs. The mean value of jioverest is -7.57 (with a minimum of -59 and a maximum of 29). 20 Models 1 through 5 in Table A-3 demonstrate the significant determinants of jioverest across a variety of alternative specifications. Missing data for some of the regressors again reduces the estimating sample, this time from 13,884 to 12,596 illness profiles. The coefficients on age and age-squared are highly significant in the first two models when latency variables for the specified illness profiles are left out of the model. When latency variables are included (as in Models 3 through 5), the coefficients on the age variables are no longer statistically significant. 20 The scenario adjustment data with respect to latency thus suggests that underestimation predominates. This may reflect opinions that acute cases of major illness do not typically come as a complete surprise. They often occur after years of decline in the individual’s general level of health. 9 It is likely that latency effects are captured by the age variables in the first two models. The insignificant age terms are dropped from the specification in Model 4. Model 5 demonstrates the consequences of using an interval-data model rather than treating jioverest as an approximately continuous variable. As is clear from in Figure 2, respondents were asked to specify the future time interval when their benefits would start, and Model 5 more explicitly captures the interval nature of these data. However, the estimates produced by Models 4 and 5 are very similar. The only notable difference is that the estimated coefficient on the respondent’s subjective risk of suffering other illnesses becomes statistically insignificant in Model 5 (although the point estimate remains similar). Models 4 and 5 suggest that individuals are more likely to overestimate the latency period when they consider an illness profile with a longer period of pain or disability, if the illness profile has pain/disability lasting more than 60 months, if they feel at greater subjective risk for other illnesses, if they belong to a two-income household, or if they will have a child under the age of eighteen in the household at the time of the stated onset of the disease. Individuals are more likely to assume that the latency in their own case will be less than the stated latency in the survey if they have not attended college, if they already have the illness in question, if they have a larger number of other major illnesses, if they feel at a higher subjective risk for this illness, if they have (on average) more room to improve their health habits, or if they have children or are single parents. The length of the latency period stated in the illness profile is also an important determinant of jioverest . Not surprisingly, a longer stated latency period in the scenario makes respondents more likely to underestimate the latency and vice versa. Table A-1: Descriptive Statistics for Correlates of Scenario Adjustment Variables Variable Mean Std. Dev. Min Max Dependent Variables Will never benefit from program*  1 jinever 0.077 Minimum overestimate of latency** j ioverest -8.12 12.3 -58 29 Minimum overestimate if latency overestimated j ioverest > 0 7.72 6.45 1 29 Minimum overestimate if latency underestimated j ioverest < 0 -15.2 10.8 -58 -1 Attributes of stated illness profile Duration of pain/disability (months if less than 60) 35.8 38.0 0 192 1(Longterm pain/disability) (>60 months) 0.288 0.453 Age/gender/income of respondent Age of respondent (years) 49.9 14.9 25 93 1(Female) 0.504 Income ($10,000) 5.18 3.38 0.5 15.0 Educational attainment 1(Less than HS) 0.104 0.305 1(High School) 0.337 0.473 1(Some College) 0.251 0.433 Objective health status 1(Have same illness) 0.040 0.195 Count of other major illness 0.294 0.578 Subjective health risks Subjective risk, same illness -0.223 1.24 Subjective risk, other illness -0.242 0.861 Avg room to improve health habits 3.446 0.831 Respondent’s household structure Size of household 2.57 1.26 1(Have kids) 0.287 0.452 1(Single parent) 0.017 0.129 1(Dualinc-w/ or w/out kids) 0.647 0.478 1(Have kid at onset) 0.029 0.169 1(Single parent & kid at onset) 0.001 0.030 1(Dual-income & kid at onset) 0.023 0.150 * To conserve space, descriptive statistics are based on illness profiles with complete data for the model to explain overest (i.e. 12,596 observations). Proportion for variable 1(never) is displayed for the 13,626 illness profiles with complete data when this is the dependent variable. ** 29.3% of the minimum overestimate of latency (overest) observations are equal to zero. Note that overest = 0 if the respondent’s subjective latency interval contains the latency stated in the survey. Table A-2: Models to explain “Never (Program would not benefit me)” 1 - Binary Logit 1( )jinever 2 - Binary Logit 1( )jinever Attributes of illness profile Duration of pain/disability (months if less than 60) 0.001 0.000 (0.57) (0.50) 1(Longterm pain/disability >60 months) -0.157 -0.155 (1.97)** (1.95)* Some demographic characteristics of respondents Age of respondent (years) -0.006 - (0.45) Age2/100 0.010 - (0.79) 1(Female) 0.375 0.381 (5.61)*** (5.71)*** Educational attainment 1(Less than HS) -0.254 -0.213 (2.09)** (1.77)* 1(High School) -0.274 -0.246 (3.27)*** (2.98)*** 1(Some College) -0.143 -0.136 (1.64) (1.57) Objective health status 1(Have same illness) 0.187 0.222 (0.99) (1.18) Count of other major illness 0.116 0.146 (1.99)** (2.61)*** Subjective health risks Subjective risk, same illness -0.342 -0.343 (10.15)*** (10.20)*** Subjective risk, other illness 0.152 0.147 (3.23)*** (3.12)*** Avg room to improve health habits -0.081 -0.094 (2.01)** (2.36)** Respondent’s household structure Size of household 0.144 0.140 (3.54)*** (3.70)*** 1(Have kids) -0.167 -0.219 (1.42) (1.96)* 1(Single parent) 0.578 0.564 (2.48)** (2.48)** 1(Dualinc-w/ or w/out kids) 0.017 - (0.22) 1(Have kid at onset) 0.064 - (0.16) 1(Dual-income & kid at onset) -0.173 - (0.37) Constant -2.720 -2.708 (6.85)*** (15.76)*** Observations 13626 13626 Log L -3550.8 -3552.8 Absolute value of z statistics in parentheses, * significant at 10%; ** significant at 5%; *** significant at 1%. Table A-3: Models to explain Minimum Over-Estimate of Latency (overest) 1 - OLS j ioverest 2 - OLS j ioverest 3 – OLS Latency j ioverest 4 –OLS Latency j ioverest 5 – OLS (Interval )* j ioverest Attributes of illness profile Pain/disability (months if <60) 0.033 0.033 0.012 0.011 0.011 (11.38)*** (11.37)*** (4.65)*** (4.31)*** (4.15)*** 1(pain/disability) (>60 months) 0.502 0.499 0.578 0.574 0.578 (2.07)** (2.06)** (2.76)*** (2.74)*** (2.61)*** Some demographic characteristics of respondents Age of respondent (years) 0.314 0.311 0.012 - - (6.92)*** (6.87)*** (0.15) Age-squared (100s of years) -0.116 -0.113 -0.078 - - (2.70)*** (2.64)*** (1.10) 1(Female) -0.205 - - - - (0.99) Educational attainment 1(Less than HS) -1.832 -1.876 -1.712 -1.813 -1.949 (4.79)*** (4.93)*** (5.21)*** (5.52)*** (5.64)*** 1(High School) -0.673 -0.701 -0.559 -0.587 -0.516 (2.56)** (2.68)*** (2.47)** (2.59)*** (2.15)** 1(Some College) -0.239 -0.256 -0.375 -0.365 -0.405 (0.86) (0.92) (1.56) (1.52) (1.59) Objective health status 1(Have same illness) -2.554 -2.542 -2.125 -2.181 -2.118 (4.70)*** (4.67)*** (4.52)*** (4.64)*** (4.29)*** Count of other major illnesses -0.567 -0.555 -0.640 -0.704 -0.718 (2.97)*** (2.90)*** (3.88)*** (4.28)*** (4.15)*** Subjective health risks Subjective risk, same illness -1.115 -1.116 -1.411 -1.397 -1.471 (10.54)*** (10.56)*** (15.42)*** (15.28)*** (15.20)*** Avg. subjective risk, other illness -0.039 -0.043 0.269 0.272 0.202 (0.25) (0.28) (2.01)** (2.04)** (1.43) Avg. room to impr. health habits -0.973 -0.974 -0.976 -0.935 -0.931 (7.40)*** (7.41)*** (8.60)*** (8.27)*** (7.79)*** Latency Period Stated latency - - -0.250 -0.204 -0.251 (2.22)** (3.09)*** (3.57)*** (Stated latency)2 - - -0.001 -0.003 -0.004 (0.78) (3.97)*** (6.36)*** (Stated latency)*(Age) - - -0.013 -0.008 -0.005 (3.50)*** (3.42)*** (2.33)** (Stated latency)*(Age2) - - 0.000 0.000 -0.000 (2.77)*** (0.58) (0.82) (Stated latency) *1(Female) - - -0.025 -0.025 -0.019 (3.25)*** (3.20)*** (2.30)** Respondent’s household structure Size of household -0.118 - - - - (0.88) 13 1(Have kids) -1.987 -2.208 -0.663 -0.673 -0.746 (5.38)*** (8.27)*** (2.81)*** (2.86)*** (2.99)*** 1(Single parent) -1.858 -1.794 -2.058 -1.993 -1.979 (2.20)** (2.15)** (2.85)*** (2.76)*** (2.60)*** 1(Dualinc-w/ or w/out kids) 0.701 0.625 0.754 0.763 0.769 (2.87)*** (2.74)*** (3.83)*** (3.88)*** (3.69)*** 1(Have current kid at onset) 14.445 14.371 2.557 3.304 3.903 (11.11)*** (11.07)*** (2.22)** (2.91)*** (3.24)*** 1(Dual-income & kid at onset) -2.681 -2.601 -2.354 -2.394 -2.679 (1.84)* (1.78)* (1.87)* (1.90)* (2.01)** Constant -17.957 -18.157 8.782 6.449 7.290 (14.36)*** (14.64)*** (3.55)*** (12.29)*** (13.14)*** Observations 12596^ 12596 12596 12596 12596 Log L -33818.9 R-squared 0.12 0.12 0.35 0.35 Absolute value of z statistics in parentheses, * significant at 10%; ** significant at 5%; *** significant at 1%. ^Sample size is smaller for models in Table A-3 than Table A-2 since they do not include those individuals who said the program would never benefit them. * Interval-data model treats jiMOEL as an interval rather than as an approximately continuous variable. This is done using the upper and lower estimates of the stated latency of the benefits of the program and using the intreg command in Stata. APPENDIX B (to be made available online) B.1 Extensive, rather than parsimonious, version of main model Table 3 in the main body of the paper gives parameter estimates from our model that corrects for scenario adjustment where all interaction terms with persistently insignificant coefficients have been dropped. Table B-1 in this Appendix provides the estimates for a model with the complete set of interactions. B.2 Extensive and parsimonious versions of a “small” model It may be important to demonstrate that the statistical significance of the interaction terms involving the two scenario adjustments variables in this study are not an artifact of the non-linear functional form of the specification in the main model. Tables B-2 and B-3 demonstrate that there are significant shifts in the estimated parameters even in simpler five-parameter versions of the specification for the program choice model. B.3 Under- or over-estimate of latency (ordered discrete variable) In addition to the interval-data model for the overest variable documented in Model 5 in Appendix A, Table A-3, we also considered a second specification for over- or under-estimating the latency. An ordered categorical variable _ jiordered latency is explored in the context of an ordered logit model. The variable _ jiordered latency is an ordered categorical variable that takes on the value 0 if the upper bound of the age interval checked among the selections in Figure 2 is lower than the stated age of onset given in the choice scenario. It takes the value 1 if the age interval checked in Figure 2 contains the stated age of onset, and take a value of 2 if the lower bound of the age interval lies strictly above the stated age of onset in the choice scenario. In 15 these data, latency is underestimated for about 54.6% of illness profiles, and it is overestimated for about 10.3% of profiles. Results for this model are displayed in Table B-2. Individuals are more likely to overestimate the latency of the illness if they have finished only high school, have temporary or long-term pain described the illness profile stated in the scenario, or will likely have a current child still in their household at the stated onset of the disease. Individuals are more likely to underestimate the length of the latency if they have a lower income, have either this illness or another major illness, have a higher subjective risk for this illness, have children, or will likely have a current child still in their household at the stated onset of the disease. 16 Table B-1: Policy choice model with all interaction terms (Alternatives = 22560) Model A1 Model A2 (Parameter) Variable Uncorrected Corrected 1( ) j inever jioverest   50 10 first income term  8.387 8.387 -2.702 0.248 (10.03)*** (10.03)*** (0.76) (4.11)***   91 10 second income term  -2.385 -2.385 10.235 -0.027 (3.86)*** (3.86)*** (2.95)*** (0.64)    10 log 1AS Ai ipdvi   -58.359 -58.359 248.650 7.233 (5.05)*** (5.05)*** (3.87)*** (7.13)***    13 ( ) log 1AS Ai i iP sel P pdvi         3.892 3.892 6.055 0.012 (2.15)** (2.15)** (0.60) (0.08)    2 log 1AS Ai ipdvr   -51.663 -51.663 -60.728 1.177 (4.52)*** (4.52)*** (1.12) (1.00)    30 log 1AS Ai ipdvl   -1019.412 -1019.412 499.341 5.900 (4.11)*** (4.11)*** (0.49) (0.36)    31 0 log 1AS Ai i iage pdvl   48.701 48.701 -19.464 -0.309 (4.80)*** (4.80)*** (0.47) (0.41)    232 0 log 1AS Ai i iage pdvl   -0.412 -0.412 0.144 0.012 (4.24)*** (4.24)*** (0.36) (1.47)     240 log 1AS Ai ipdvl     339.442 339.442 484.391 -3.979 (3.13)*** (3.13)*** (0.81) (0.41)     241 0 log 1AS Ai i iage pdvl     -17.555 -17.555 -7.705 0.308 (3.95)*** (3.95)*** (0.33) (0.72)     2242 0 log 1AS Ai i iage pdvl     0.148 0.148 0.032 -0.006 (3.44)*** (3.44)*** (0.15) (1.24)       50 log 1 log 1 AS A i i A i pdvi pdvl          141.815 141.815 -416.324 -13.371 (1.55) (1.55) (0.89) (1.42)       51 0 log 1 log 1 AS A i i i A i age pdvi pdvl          -6.993 -6.993 -0.117 0.434 (1.95)* (1.95)* (0.01) (1.07)       2 52 0 log 1 log 1 AS A i i i A i age pdvi pdvl          0.063 0.063 0.101 -0.005 (1.85)* (1.85)* (0.58) (1.20) Log L -11694.646 -10948.179 17 Table B-2: Minimal Model (Alternatives = 22560) Model B1 Model B2 (Parameter) Variable Uncorrected Corrected 1( )jinever jioverest   50 10 first income term  5.342 9.991 -1.787 0.409 (9.17)*** (12.98)*** (0.54) (7.40)***   91 10 second income term  -2.160 -2.014 9.731 -0.026 (4.61)*** (3.33)*** (2.84)*** (0.64)    10 log 1AS Ai ipdvi   -27.053 -37.493 109.601 5.348 (4.56)*** (4.99)*** (2.75)*** (7.75)***    13 ( ) log 1AS Ai i iP sel P pdvi         3.297 3.475 5.121 -0.033 (2.29)** (1.90)* (0.50) (0.23)    2 log 1AS Ai ipdvr   -21.870 -37.893 -60.407 0.993 (2.35)** (3.43)*** (1.13) (0.86)    3 log 1AS Ai ipdvl   -30.409 -36.974 190.347 6.594 (5.97)*** (5.89)*** (5.79)*** (11.12)*** Log L -11726.31 -11073.051 Table B-3: Parsimonious Minimal Model (Alternatives = 22560) Model B1’ Model B2’ (Parameter) Variable Uncorrected Corrected 1( )jinever jioverest   50 10 first income term  5.342 9.816 -1.900 0.387 (9.17)*** (14.00)*** (0.57) (10.18)***   91 10 second income term  -2.160 -1.800 9.425 - (4.61)*** (3.58)*** (2.76)***    10 log 1AS Ai ipdvi   -27.053 -37.184 103.398 5.398 (4.56)*** (4.97)*** (2.72)*** (7.98)***    13 ( ) log 1AS Ai i iP sel P pdvi         3.297 3.786 - - (2.29)** (2.39)**    2 log 1AS Ai ipdvr   -21.870 -43.664 - - (2.35)** (4.45)***    3 log 1AS Ai ipdvl   -30.409 -36.855 188.932 6.619 (5.97)*** (5.89)*** (5.74)*** (11.22)*** Log L -11726.31 -11074.305 18 Table B-4: Correlates of overest as a discrete variable 1 – Ordered logit j ioverest 2 – Ordered logit j ioverest 3 – Ordered logit j ioverest Attributes of illness profile Duration of pain/disability (months if less than 60) 0.004 0.002 0.002 (5.05)*** (1.93)* (1.99)** 1(Longterm pain/disability) (>60 months) 0.064 0.094 0.095 (0.93) (1.30) (1.32) Some demographic characteristics of respondents Age of respondent (years) 0.036 0.000 - (2.72)*** (0.00) Age-squared (100s of years) -0.029 0.003 - (2.34)** (0.15) 1(Female) 0.005 - - (0.09) Educational attainment 1(Less than HS) -0.939 -0.940 -0.936 (6.80)*** (6.68)*** (6.67)*** 1(High School) -0.040 -0.005 -0.007 (0.57) (0.07) (0.10) 1(Some College) -0.202 -0.207 -0.209 (2.62)*** (2.57)** (2.60)*** Objective health status 1(Have same illness) -0.679 -0.654 -0.651 (3.20)*** (3.01)*** (3.00)*** Count of other major illness -0.119 -0.137 -0.132 (2.08)** (2.28)** (2.23)** Subjective health risks Subjective risk, same illness -0.132 -0.200 -0.201 (4.38)*** (6.25)*** (6.28)*** Subjective risk, other illness -0.081 -0.031 -0.028 (1.86)* (0.68) (0.62) Avg room to improve health habits -0.155 -0.174 -0.178 (4.30)*** (4.59)*** (4.72)*** Latency Period Stated latency - 0.013 0.010 (0.24) (0.31) Stated latency squared - -0.003 -0.003 (6.86)*** (7.70)*** Latency and age interaction - 0.003 0.002 (1.35) (1.86)* Latency and age squared interaction - -0.000 -0.000 (3.19)*** (4.71)*** Continued... Respondent’s household structure 19 Size of household -0.011 - - (0.29) 1(Have kids) -0.284 -0.097 - (2.64)*** (1.13) 1(Single parent) -1.204 -1.319 -1.387 (2.80)*** (3.06)*** (3.25)*** 1(Dualinc-w/ or w/out kids) 0.107 0.120 0.107 (1.55) (1.82)* (1.67)* 1(Have kid at onset) 1.809 0.155 - (6.80)*** (1.03) 1(Dual-income & kid at onset) -0.330 - - (1.13) Observations 12596 12596 12596 Log L -4259.161 -3697.929 -3698.915 Absolute value of z statistics in parentheses, * significant at 10%; ** significant at 5%; *** significant at 1%.