Survey Nonresponse by Candidates: A Choice of Ambiguity?


Prepared for Presentation at the 2000 Meeting of the Midwest Political Science Association
April 27, 2000
Chicago, IL

Robert G. Boatright
Department of Political Science
The University of Chicago
robb@polisci.spc.uchicago.edu



        Those who analyze public opinion surveys are frequently perplexed about what to assume about citizens who fail to respond to survey questions, or who answer "don't know" to particular questions. It is conventionally assumed in mass public opinion surveys that these citizens either do not have an opinion on the subject or that they do not understand the question. As such, much research on "don't know" responses, or "DK's," has focused upon the linkage between education or information and survey nonresponse. What are we to make, however, of DK's among political elites? That is, what patterns can we find when those who should have substantial information about an issue decline to take a position on that issue?

        In this paper, I analyze survey nonresponse within one particular group of political elites - candidates for the US House of Representatives - who are not only expected to have adequate information to evaluate political issues, but who are generally expected to make these positions the centerpiece of their communications with the electorate. Using a comprehensive survey of all 1996 major party nominees for the House of Representatives, I evaluate possible reasons for candidates' failure to respond to the survey at all (unit nonresponse) or to individual survey questions (item nonresponse), and I analyze the connection between theories of nonresponse in public opinion surveys to the issue of nonresponse in surveys of political candidates.

        Candidates surveys differ from public opinion surveys in several important ways. The survey I evaluate, which was conducted by Congressional Quarterly and the results of which were distributed in a special, state-specific supplement to Time Magazine, was clearly intended for public consumption. Candidates were "on the record." Mass public opinion surveys, by contrast, are consumed in the aggregate; respondents are anonymous. While on some public opinion survey questions respondents may decline to answer because the question touches upon sensitive or personal issues, anonymous respondents should generally have less to lose than do respondents who will be identified, as is the case with candidate surveys.

        Furthermore, there is a cost to position-taking for candidates, while there is not equivalent cost to position-taking for other survey respondents. Even were one to imagine a public opinion survey in which anonymity was not granted, we would still expect few repercussions for most respondents on most political questions. For the average citizen, having and stating views on policy issues incurs few prospects of gain or loss in their everyday lives. For candidates, on the other hand, stating any policy positions at all ensures that support will be gained among some segments of the electorate and lost among others. Indeed, a substantial body of both theoretical and descriptive literature is devoted to studying candidates' efforts to take ambiguous positions, to be all things to all people.

        On the other hand, however, political elites are presumed to have positions on issues; failure to adopt a position may be seen by voters as a reflection of ignorance, indecision, or "waffling." Some literature on nonresponse in public opinion surveys has addressed the "forced response" issue - the appropriate amount of prodding by an interviewer that is necessary to distinguish between an initial DK response which means "wait, I'm thinking" and one that means "I don't have an opinion." Too much prodding may result in an arbitrary answer given to please the interviewer, while too little may fail to clarify the meaning of the DK response. In elite, non-anonymous surveys, however, the audience is much larger than the interviewer, and members of this audience may well come to their own conclusions about what the reasons for failure to respond were.

        Finally, the precepts of elite interviewing run quite counter to those of mass public opinion surveys. Survey interviewing literature has generally encouraged those who would interview elites to pursue open-ended questions. This format allows respondents to explain their positions, and gives them substantial leeway to create their own response categories. Such responses, however, are clearly incompatible with quantitative work, systematic comparisons of respondents, and analysis of large groups of respondents. The larger the number of respondents, the more valuable are closed-ended responses. These categories, however, may exert their own type of forced response - a respondent who might have otherwise provided a lengthy yet ambiguous answer is now prompted to give a yes/no answer - just as incumbent legislators are compelled to vote among alternatives.

        Despite these differences, however, traditional theories of survey nonresponse provide a good starting ground for studying nonresponse by political candidates. In this paper I begin by summarizing these theories and their relationship to candidate surveys. Even where lack of information is not a compelling reason for nonresponse, a desire for ambiguity or a respondent's rejection of a question's premise may be closely tied to theoretical issues that have arisen in the study of DK responses in public opinion surveys. After explicating the applicability of each of these reasons for DK responses to candidate surveys, I then identify particular types of questions or types of candidates who might be of particular interest in disentangling reasons for DK responses. I then test these theories by supplementing analysis of the DK responses, both on individual questions and in the aggregate, of candidates in the 1996 House candidates survey with demographic information on the candidates such as their vote share, overall ideology, competitiveness, and political experience.

        In detailing existing theories of survey nonresponse, I identify three strains of thought on the intent of individuals who fail to respond to survey questions. These individuals may be ignorant of the subject matter of the question, they may prefer a middle ground or a position not supplied to them among the possible responses, or they may reject the basis of the question entirely. These map onto elite surveys in a partial way; each is a possible reason for elite survey nonresponse, but the motives for nonresponse may vary within the second and third of these categories. More specifically, candidates may have responses they could offer but, because their responses will be attributed to them, be reluctant to state them. In short, candidates may be either ignorant or they may have a preference for ambiguity in their responses. Furthermore, candidates may prefer ambiguity on some types of questions, but not on others. The reasons for nonresponse can be identified through analysis of question response patterns according to candidate competitiveness. In doing so, I demonstrate that a desire for political ambiguity leads more competitive candidates to be significantly more likely to fail to respond to survey questions than are less competitive candidates. There is, however, no trend among particular types of questions - while there are a handful of individual questions where trends in the data suggest that lack of information or willful ambiguity may determine differences in response levels, ambiguity is most evident when we compare overall, non-issue specific trends.
 
 

Applying Traditional DK Theories to Elite Surveys


        Researchers have identified three major causes of nonresponse in public opinion surveys: lack of education, information, or knowledge; lack of interest in the subject matter of the question or uncertainty about that subject; or a rejection of the premise of the question. Most of the debate in public opinion literature on "don't know" responses has been focused upon disentangling these reasons and identifying remedies for them. That is, debate has centered upon the issue of whether these answers should be excluded from consideration of other question responses, or whether a DK response can be considered a middle ground. Particularly when one is seeking to scale question answers, this can be an important consideration.

        The most frequent finding in studies of DK respondents is that education is by far the most important predictive variable. Converse (1976) demonstrates that DK responses decline with education, although they fluctuate according to the subject of the question. Brehm (2000) does not point directly to education as a determinant, preferring instead to note the correlation between nonresponse and lack of political participation. Faulkenberry and Mason (1978) differentiate between uninformed DK responses - which are related to education - and undecided DK responses, which are more closely related to the salience or public prominence of an issue than they are with education. Faulkenberry and Mason's test case, a survey on wind energy, is an instance of an issue sufficiently obscure that those who do know about it are not necessarily distinctive in their education level.

        The most detailed study of the problem of categorizing DK responses is Schuman and Presser (1996). Like Faulkenberry and Mason, they seek to distinguish between the uninformed and the undecided respondent, but they address question wording and response categories as a means of doing so. Schuman and Presser note that education is by far the most important background variable in virtually all types of survey methodology issues. They emphasize, however, that it is in the hands of the survey questioner herself to identify reasons for DK responses. Too much probing may indicate to the respondent that a DK response is unsatisfactory, prompting a "mental coin flip" and a meaningless response. On the other hand, where a DK response is specifically offered in the survey, those choosing it tend to have a higher level of education than those who provide a DK response even when it is not offered as a response category. This indicates that a certain percentage of respondents will already feel pushed to make a meaningless response in any survey format. Probing, if done correctly, can also elicit more open-ended responses about the reasons for a DK response; that is, a respondent may volunteer a more nuanced answer than the question categories provided, may offer a middle ground, or may indicate confusion about the question itself. This in itself can be valuable information. For Schuman and Presser, both the offering of a DK response category and probing of DK respondents are appropriate techniques for analyzing reasons for DK responses.

        Another approach suggested by Schuman and Presser is the provision of a middle ground. In question wording experiments, they find that this can yield a ten to twenty percent differences in simple yes/no responses when offered, and can serve as a measure of intensity. If paired with a question regarding the importance of the issue to a respondent, it may reduce the number of DK responses but provide a more accurate measure of whether individuals truly do not know what to think or are uncertain about what to think. While these two issues may sound similar, they in fact shift the blame from the respondent to the issue itself, or from the respondent to those who would inform the respondent on the issue. Coombs and Coombs (1976) suggest that this is a more frequent reason for nonresponse than is often realized by survey researchers. They emphasize the possibility that DK responses indicate an issue's low priority in a respondent's schema - that respondents don't care enough to have an opinion. Converse (1976) also provides a potential metric for nonresponse according to intensity or issue salience, insofar as she provides data on nonresponse or DK response tendencies by issue type.

        This intensity problem is often difficult to measure, however, because those who would use survey results often have little input into the structure of the survey itself, and they are left to guess about the intentions of respondents. Bartels (1986) uses DK responses on candidate placement scale questions on the National Election Study to draw conclusions about the amount of information candidates have provided about themselves in their campaigns. Alvarez (1998: 56) questions Bartels' attempts to do this on the grounds that literature on DK responses is somewhat shaky in its attribution of a DK response to uncertainty or problems in information provision. Efforts such as that of Bartels illustrate, however, what should be an obvious truth - sometimes questions are asked about which we cannot expect respondents to have information, or to have an opinion. Yet sometimes respondents do still answer. Bishop, Oldendick, and Tuchfarber (1986; see also Bishop, Oldendick, Tuchfarber, and Bennett 1980) illustrate this through experiments in questions on fictitious issues.

        In addition, it is difficult to disentangle issue salience in studies such as that of Bartels. His study provides a link between mass surveys and elite, attributed surveys because it begs the question of the causes of information provision or lack thereof. Some survey respondents may not be able to place candidates on issues because the respondents do not have an interest in the issues, or the issues are of relatively low priority to them in evaluating candidates, so they filter this information out. On the other hand, candidates may have chosen not to focus on such issues for quite different reasons. They may have chosen not to focus upon them because they, too, believe these issues are of relatively low salience, or they may have chosen not to focus upon highly salient issues because they believe their positions on them will be unpopular or because they do not wish to alienate some voters by discussing them.

        A final vein of theory on DK responses is represented by Bogart (1967) and by very few others. Bogart's article is hardly a research piece; it reads more as a broadside against many of the practices of survey research. Bogart reiterates the conventional wisdom on DK responses as a manifestation of low education or information, but raises the possibility that a DK response is actually a rejection of the question on the grounds that the respondent has no control over the framing of the question itself. Rather than provide a yes/no response, the respondent declines to answer because he or she disagrees with the question itself - the manner in which it is phrased, the response categories he or she has been provided, or the closed-ended format of the question. Bogart's argument, though cited in much of the subsequent literature on survey item nonresponse, has not been extended in quantitative survey research in part because it is not particularly amenable to such research.

        These three strains of research on DK response to public opinion surveys can be conveniently mapped onto non-anonymous elite surveys. The problem of low information still remains, although education is perhaps not a convenient culprit. Although we should expect this reason to be relatively rare, it remains a possibility. Uncertainty should still figure quite prominently, although it may be different to disentangle from willful ambiguity; that is, a candidate may have a substantial amount of information about an issue and may have drawn his or her own personal conclusions but be reluctant to share them. We might expect, however, for the Bogart approach, the rejection of the question premise, to be substantially more common than in anonymous public opinion surveys. Much of the literature on elite intervewing (see Robinson 1960, Converse and Schuman 1974) has emphasize the importance of open-ended questioning in elite surveys and the potential for a well-educated or well-informed to become hostile to the survey itself if she feels that the questions do not allow her to express her views adequately.

        These three strains are also applicable both to item nonresponse and unit nonresponse. Much of the literature above solely considers item nonresponse. Research on unit nonresponse is somewhat scarce, and the research which does exist is frequently presented as a preamble for statistical techniques to measure the bias introduced into surveys by unit nonresponse (see Brehm 2000, Sherman 2000). Brehm's summary of literature on unit nonresponse shows that patterns of unit nonresponse are similar to patterns of item nonresponse. Thus, political ignorance, or lack of information, is as valid a cause of unit nonresponse as it is of item nonresponse. The relationship of the other strains of item nonresponse literature - preference for a middle ground or a rejection of the premise of the survey - would seem to be extreme cases of the phenomena found in those literatures. That is, if unit nonresponse is held to be a result of preference for a middle ground, or for ambiguity, it must be of a different scale than item nonresponse. If a respondent remains silent on one issue because of a preference for a middle ground, then this individual is only expressing a preference for a middle ground on that issue, while one who remains silent on all survey questions would be expressing a preference for a middle ground on all issues. Of course, as the number of questions asked increases, this claim would become less and less likely. On the other hand, it would seem to be more plausible to claim that just as respondents can reject the premise of one or more survey questions, so can respondents reject the premise of a survey in its entirety.

        The survey considered here certainly is not amenable to question-wording experiments in the vein of Schuman and Presser. It is, however, a survey which has sufficient breadth of question types and respondent types that relatively simple comparisons of respondents or questions can illuminate the prevalence of each of these. It is a closed-ended elite survey, so some subjectivity is necessary in imputing reasons for particular types of responses. Following analysis of the survey results, however, I do provide open-ended reflections drawn from conversations with some of these candidates or from the brief personal statements these candidates were allowed to make at the close of the survey.
 
 

The 1996 Congressional Candidate Survey


        The data I discuss below are drawn from the 1996 Time/Congressional Quarterly Candidate Survey. In this survey, all major party congressional nominees were asked to respond to thirteen yes/no questions about their views on legislation considered by the 104th Congress. The specific questions are provided in Appendix One. If one seeks to create an ideological scale using these question responses, the results are quite similar for incumbents to their 1995 ADA ratings (R2 = .955; N = 377). The questions cover a wide range of issues - from defense policy to the budget, from the environment to foreign aid. Because each of the questions covers a congressional vote from the past Congress, incumbents' answers are scarcely even necessary; their votes could simply be addressed. The true value of this survey is in its relative comprehensiveness in regard to nonincumbent candidates. Prior surveys, including those by Project Vote Smart and the previous Time/CQ survey, had a much higher survey nonresponse rate (that is, a higher number of candidates who failed to return their surveys at all) than did the 1996 survey. Seventy-three candidates, or 8.4 percent, failed to answer any questions at all, although some candidates did return their surveys blank with an explanation of why they did not answer the questions.

        Unfortunately, the survey texts given to the candidates are not uniform. Four separate versions of the survey exist, in which either 8, 10, 12, or 13 of the 13 questions were asked. In addition to this problem for analysis, the questions vary in their degree of clarity. Most of the questions are quite straightforward. Two questions, however, are sufficiently confusing that many candidates may have been rather justified in refusing to answer them. On the gays in the military question, it appears that incumbents may have viewed the "don't ask, don't tell" policy as a compromise position that was to the right of what the Clinton administration had initially proposed. A majority of Democratic incumbents, including most of those in the more liberal wing of the party, opposed the policy, while a majority of Republican incumbents supported it. It appears that nonincumbents viewed this question as a comparison of the "don't ask, don't tell" policy with the status quo ante, the ban on homosexuals in the military that had been in place prior to the Clinton administration's proposal to life the ban entirely. The differences in the environment question are somewhat less subtle; the overwhelming difference between incumbents and nonincumbents in each party suggest that, insofar as incumbents could refer to their voting on the particular bill in question while nonincumbents could not, many nonincumbents misunderstood the question. The question is phrased in a confusing manner, with a double negative that may have caused many respondents to give the exact opposite response of what they intended. Both of these questions illustrate the problem of asking incumbents to report past votes, while asking nonincumbents to respond to a one-sentence summary of the vote. On each of these questions, incumbent Democrats gave one answer while nonincumbent Democrats gave another, and incumbent Republicans gave one answer while nonincumbent Republicans gave another.

        In the discussion to follow, I supplement survey responses and nonresponses with data on the candidates. Lack of information may be crudely proxied by using a dichotomous political experience variable. A preference for ambiguity may be measured by comparing various competitiveness measures - to account for candidates who might wish to avoid alienating voters - or by analyzing competitiveness in conjunction with the type and salience of the issue upon which candidates choose to remain silent. Rejection of the question premises is best gauged by a more qualitative analysis of candidates' open-ended comments on the survey - either as provided at the end of their surveys themselves or in their recollections on candidates surveys provided to me in conversation. In the next section I analyze trends in question nonresponse in terms of candidate types and question types; following this analysis I turn to candidates comments on being surveyed.
 
 

Results


Item Nonresponse

        A preliminary investigation of the response rates for each survey question yields no significant differences; response rates for all thirteen questions range between 94.5 and 98.2 percent. Thus, it appears that there are few properties of the questions on their face that would prompt candidates to avoid responding. It should be noted, however, that incumbents almost invariably answer all of the survey questions. Incumbents have a voting record upon which to draw, however, so they can neither seek ambiguity nor plead ignorance. Nonincumbents exhibit a slightly but insignificantly greater range of variation in nonresponse across question types, from 91.5 percent to 97.2 percent.

        Incumbents' response rates were greater than 98.0 percent in this survey for all questions, and the mean incumbent response rate was 99.0 percent. Incumbents thus do not provide a particularly helpful ground for analyzing questions responses, and because their circumstances are so greatly different from those of nonincumbents - they had, after all, voted on each of these issues - I have excluded their responses in the analysis which follows. Incumbents can certainly choose what to discuss and what not to discuss in their campaigns, but failing to report a vote already taken would seem a particularly disingenuous way of treating the survey.

        Analysis of nonresponse tendencies among nonincumbents by candidate type does show significant variation in question response by candidate type on particular questions. When candidates are disaggregated by a variety of competitiveness measures - political experience, campaign funding, vote percentage, party vote percentage in the previous election, or any of several competitiveness scales - there is a slight tendency for more competitive candidates to be more forthcoming with their responses. This might be considered as support to the political knowledge or interest hypothesis, insofar as less competitive candidates may also be expected to be less serious candidates, except that each of the variables here merely shows variation because of the presence of incumbents. Separate analysis of variation for nonincumbents shows no significant variation according to any of these measures.

        This information, however, masks variation in propensity to respond to individual questions. Because the tendency in the above table is rather slight, we cannot completely attribute nonresponse patterns to candidates' level of political knowledge. As noted above, the survey questions, as public knowledge, may differ in the degree to which they are "safe" for candidates to provide an answer. Among the topics on the survey, for instance, are questions on partial-birth abortion, gays in the military, and the Clinton budget - issues many candidates may prefer to avoid or for which to give equivocal answers. As Table One shows, question responses vary somewhat by the competitiveness of the candidate.

        This table can be read in two ways. Reading across, one can obtain a measure of either the role of political experience (insofar as competitiveness is highly correlated with prior political office) in prompting candidates to answer questions, or of the role of competitiveness in prompting a desire for political ambiguity. Although there are some slight fluctuations in these tables, only one - the Clinton budget of1995 - achieves statistical significance. In few cases, however, does the propensity to answer questions actually increase with competitiveness. In the case of the Clinton budget, more competitive candidates were more likely to adopt a position than were less competitive candidates. On several other issues, however, including defense spending and welfare reform, there is a slight, though insignificant, upward trend in nonresponse as one moves from the less competitive to the more competitive candidates.


Table One: Individual Question Responses by Candidate Competitiveness

Uncompetitive

(< 35% of vote)

Somewhat Competitive

(35 - 45% of vote)

Highly Competitive

(> 45% of vote)

Total
Term Limits a 2.9% 1.9% 4.2% 2.8%
FMLA 1.3 5.5 3.0 3.3
Environment b 4.9 3.3 1.6 3.4
Gun Control 3.3 3.0 5.0 3.6
Troops in Bosnia 5.2 2.4 5.9 4.3
Clinton Budget * 7.8 3.6 2.0 4.8
Welfare Reform 4.6 4.2 9.9 5.7
Gays in the Military 7.2 7.9 5.9 7.2
Medicare Spending 7.8 7.3 6.9 7.4
Partial-Birth Abortion 9.8 5.5 6.9 7.4
NAFTA 9.8 7.3 7.9 8.4
Aid to Russia a 7.9 9.1 8.3 8.5
Defense Spending 8.5 6.1 12.9 8.6
Total N 153 165 101 419

* p < .10
a N = 389
b N = 233

        The table can also, however, be read from top to bottom, providing a measurement of the issues on which candidates are more or less likely to take positions. This may serve as a measure of issue salience, although it is hard to read the direction in which salience might take candidates. Some candidates might prefer silence on "hot button" issues, whereas other candidates might refrain from taking positions on issues they do not feel very many voters will care about. In the case of defense or aid to Russia, for instance, nonresponse rates may be high because of the relatively low priority of these issues in the 1996 election. In the case of welfare reform or NAFTA, however, it would be difficult to argue that the candidates did not, in fact, find reason to talk about these issues at some point in their campaign. In cases such as these, one might presume candidates to have a preference for ambiguity. The welfare reform question in particular exhibits a sharp increase in nonresponse among the most competitive candidates. The most frequently answered questions, on the other hand - those on term limits, the Family and Medical Leave Act, EPA standards, and gun control, seem likely to have been of relatively high salience in most campaigns.
 


Table Two: Item Nonresponse on Selected Questions by Competitiveness and by Party

Uncompetitive 

(< 35% of vote)

Somewhat Competitive (35 - 45% of vote) Highly Competitive (> 45% of vote) Total
Democrats
Medicare Spending ** 8.0% 1.1% 0.0% 3.0%
Clinton Budget * 9.3 2.1 1.6 4.3
Defense Spending * 9.3 1.1 9.8 6.1
Welfare Reform * 5.3 6.4 16.4 8.7
Democratic N 75 94 61 230
Republicans
Family and Medical Leave Act * 1.3 11.3 2.5 5.3
Republican N 78 71 40 189
* p < .05 ** p < .01

        Research on candidates' issue appeals which I conducted for a related project (Boatright 1999) confirms the general trends of Table One and of Table Two below regarding the issue focus of campaigns. Apart from the use of military force in Bosnia and term limits, the remaining issues with the highest response levels were also the issues covered in this survey which candidates were most likely to emphasize in their campaign literature. Health care, the environment, the federal budget, and guns were all among the five most frequently discussed issues in candidates' campaign literature - the only entry in the top five that does not correspond with survey response rates was education, which is not covered in the survey.

        Issue salience, or a preference for ambiguity, may also vary with party and with candidate ideology. That is, Democrats may have greater reason to keep silent on particular issues than would Republicans, and vice versa. Insofar as I am considering nonincumbent candidates -- individuals who are likely to be running in districts not favorably disposed toward their party -- one might expect that candidates would prefer silence on issues where public opinion strongly associates their party with a particular position. Table Two shows the issues where differences were strongest in one party. As can be seen in this table, the potential causes of nonresponse -- lack of political knowledge or willful ambiguity -- are still somewhat uncertain. The declining trend in nonresponse for Democrats on the Clinton budget and Medicare spending might indicate that political experience might be a factor on these questions, while the increasing numbers on the question of welfare reform could certainly be taken as an indication of a desire for ambiguity, insofar as the Democratic Party was rather split on this subject. The lone Republican issue demonstrating significant variation in nonresponse, the Family and Medical Leave Act, had unanimous Democratic support among candidates, but the trend in nonresponse here is somewhat unclear. One possibility is that the most competitive Republicans ran in more Republican districts, and were thus more likely to be able to adopt conservative positions without penalty, while the less competitive candidates might have seen more to gain through silence.

        Again, my previous survey of candidates' issue focus confirms these trends - Republicans were far more likely to emphasize welfare reform in their campaigns than were Democrats, and Republicans were actually more likely to discuss welfare reform as they became more competitive. Democrats, on the other hand, were more likely to discuss their support for President Clinton's budget proposals as they became more competitive.

        Finally, it seems possible to rather conclusively rule out confusion about the questions as a cause of nonresponse. Two of the questions on this survey are conspicuously more confusing than the others. The questions on gays in the military and on the EPA both exhibited a curious switching of allegiances, where incumbents and challengers of each party stood in opposition to each other. On the gays in the military question, it appears that incumbents may have viewed the "don't ask, don't tell" policy as a compromise position that was to the right of what the Clinton administration had initially proposed. A majority of Democratic incumbents, including most of those in the more liberal wing of the party, opposed the policy, while a majority of Republican incumbents supported it. It appears - and I have verified this in conversation with nonincumbents of each party - that nonincumbents viewed this question as a comparison of the "don't ask, don't tell" policy with the status quo ante, the ban on homosexuals in the military that had been in place prior to the Clinton administration's proposal to life the ban entirely. The differences in the environment question are somewhat less subtle; the overwhelming difference between incumbents and nonincumbents in each party suggest that, insofar as incumbents could refer to their voting on the particular bill in question while nonincumbents could not, many nonincumbents misunderstood the question. The question is phrased in a confusing manner, with a double negative that may have caused many respondents to give the exact opposite response of what they intended.

        In summary, the data on item nonresponse show variation between parties, between candidate types, and between issues, but they do not provide a particularly clear picture of the reasons behind nonresponse. In some cases, as in the case of welfare reform, a desire for ambiguity among the candidates with the most to lose from position-taking may be the culprit in searching for nonresponse reasons, while in other cases, as with the Clinton budget, political experience or competitiveness may make candidates more likely to take positions and less likely to decline to respond. These individual question data provide several issue-specific insights, but they do not lend themselves to a clear disentangling of causes of nonresponse.
 


Table Three: Unit Nonresponse by Candidate Competitiveness

Responded to all Questions Responded to Some Questions Did not Respond to Survey Total
Competitiveness **

(3 pont scale using above categories)

1.83

(.756)

1.97

(.789)

2.24

(.716)

1.91

(.772)

CQ Rating **

(5 point scale)

1.95

(1.39)

2.37

(1.63)

3.14

(1.85)

2.20

(1.57)

Mean Campaign Spending **

(in thousands)

308.87

(341.02)

394.67

(411.16)

642.23

(523.23)

371.57

(398.94)

Mean Party Presidential Vote Percent * 34.94

(8.44)

36.34

(9.28)

38.34

(9.34)

35.75

(8.86)

* p < .05 ** p < .01
 

Unit Nonresponse

        Another, potentially more illuminating, means of analyzing candidates' nonresponse propensities is by studying characteristics of candidates who provided only partial responses to the survey or of candidates who declined to respond to the survey at all. Several of the more salient differences in unit nonresponse, or what one might call "partial unit nonresponse," are shown in Table Three. Here, ambiguity seems quite clearly linked to nonresponse. On each of four measures of competitiveness, nonresponse rises as one moves from candidates who answered the survey in its entirety to candidates who answered questions at least somewhat selectively to candidates who declined to respond to the survey altogether.

        The direct competitiveness measures -- the three categories used in the above tables, as well as Congressional Quarterly's pre-election categorization of candidates ("safe for incumbent", "leaning towards incumbent", "toss-up", "leaning towards challenger", and "safe for challenger") all vary in accordance with the three unit nonresponse categories. Candidates who did not respond to the survey at all were, on average, more competitive than those who responded in part or in full. Likewise, nonrespondents spent more in their campaigns on average than partial or full respondents, and they ran in districts more favorable to their party at the presidential level. Most importantly, the partial respondents provided a middle ground on all of these questions -- they did respond to the survey, but it seems that their electoral concerns led them to decline to answer individual questions where they may have felt their views would meet with disapproval from some voters. The least competitive candidates, on the other hand, may be less encumbered by visions of victory in November, and may be running more ideological campaigns. The trends shown in Table Three do not vary by party.

        Table Three provides much stronger evidence of a preference for ambiguity among candidates - a preference either not to take a position at all or for a compromise, middle ground position. If we consider unit nonresponse as a matter of scale, with item nonresponse by candidates as a midpoint on this scale between full disclosure of issue positions and a desire to avoid disclosure of any information on the survey, we see a steady trend in a desire for silence on the survey as we move across candidate types. While this logic is not shown in the data on individual issues - perhaps because the issue focus of campaigns varies from one district to the next or across candidates merely because of their personal preferences - it is apparent when the response tendencies of candidates of different levels of competitiveness are compared. It is important to note that the question yield figures presented in considering item nonresponse do not include candidates who failed to respond to the survey entirely, as these candidates would dilute our ability to compare trends across questions. Most of the candidates who I place in the "responded to some questions" category responded to all but two or three questions, so it ought not to be said that they represent a true halfway point in the number of questions answered. In terms of intentions, however, they do present a separate category - they are selectively silent on the survey, while other candidates are either not silent at all or completely silent. Put together, these different candidate tendencies demonstrate that survey nonresponse tends to be a strategic choice.
 
 

Candidates' Perspectives on Issue Surveys


        Is the above analysis making too much of what is, after all, only one of many batteries of questions candidates are asked? The tendencies above do not indicate that the decision about whether to respond or not to respond to one particular survey will have momentous implications for the competitiveness of a campaign. In fact, the more one downplays the importance of surveys such as this, the greater is the evidence provided here that these candidates consider even the smallest decisions very carefully. Just as research on incumbent members of Congress has noted that their tendency to "run scared" is manifested even in position-taking on issues substantially removed from public view, so the survey nonresponse tendencies here indicate a level of detailed calculation on the part of candidates that extends even to surveys such as this.

        Yet perhaps the candidates' own comments provide evidence of the reasoning behind candidates nonresponse decisions. A total of eight candidates in the House and Senate surveys chose not to answer individual survey questions, yet returned their blank surveys with an explanation of their decision. Four of these candidates were victorious in their campaigns - a remarkable ratio in an election year with a 94.5 percent incumbent re-election rate.

        All of the candidates who provided reasons for nonresponse indicated a dissatisfaction with the premise of a yes/no survey - despite the fact that they would have to vote up or down on such issues once elected. Among the victors, Ohio challenger Dennis Kucinich wrote that "YES and NO is too simplistic," Mississippi open seat candidate Chip Pickering stated that the questions "are too hypothetical to answer," South Dakota open seat candidate John Thune responded that the survey questionnaire " did not adequately represent the effect or the intent of the legislation" (a claim that is certainly true in the EPA and gays in the military questions discussed above), and Georgia Senate open seat candidate Max Cleland noted that the similar questionnaire given to Senate candidates "is so skeletal and oversimplified as to be distortive." The less competitive candidates provided similar, although occasionally more idiosyncratic, reasons. These candidates wrote that the survey questions "are too complex for a yes-or-no response", "often put words into a candidate's mouth", or "focus only on what a candidate would have done in the past." Finally, one of these candidates refused to take any positions until polling voters more extensively.

        As further anecdotal support, two candidates whom I interviewed (again, for a related project) following the election brought up the survey without prompting during the course of a lengthy, open-ended interview. Their quotes are each worthy of consideration at length:

In a way I regret having filled out the Congressional Quarterly survey, because it does give them ammunition, but I wasn't there involved in the debate, you know? [My opponent] tried to seize on that. But no, I wouldn't say that I would have done anything different. I think we should have done a better job of picking fewer items and making them stick.

You miss a lot of things in those statistics. There are more than two sides to most of those issues. NAFTA, for example, is more simple than "are you for or against it?" And some of those issues just aren't relevant - I mean, how many candidates really cared that much one way or the other about aid to Russia, unless you were running in a district with a lot of Russian immigrants. You pick the two or three issues you think are going to matter in the election, but the others, you just say what you think when people ask you and don't bring them up the rest of the time.

        Such responses do indicate that the candidates tended to weigh the costs and benefits of filling out their surveys quite carefully, and that in cases where they failed to respond to questions, they tended to have strategic reasons for doing so - they were not ignorant of the ramifications, and often thought themselves more attuned to the nuances of the issues than their survey responses could possibly convey.
 
 

Conclusions


        In studying mass public opinion surveys, nonresponse is frequently attributed to a lack of political information or for a preference for a middle ground. When those analyzing survey data have access to demographic information on the survey respondents, they are able to identify the correlates of nonresponse and are often able to conduct question-wording experiments to identify the reasoning behind nonresponse. Surveys of candidates follow a similar logic in that the basic causes and types of nonresponse possible seem to be relatively comparable, but causes of nonresponse differ dramatically when question answers are placed "on the record." The data presented here indicate that we can impute a certain degree of political calculation to candidates when they leave survey questions blank, or when they fail to respond to surveys at all. As such, analysis of survey nonresponse tendencies among this group can render survey analysis which considers these nonresponses far richer than an analysis which simply excludes failures to respond from the data considered.

        These findings indicate that candidate surveys can provide fertile ground for identifying areas in which candidates seek to avoid position-taking or to establish ambiguous positions. They also indicate, however, that candidates are not particularly different from many members of the general public in their strategies for answering interview questions. Survey researchers frequently bemoan declining survey response rates; these data indicate that analysis of the reasons for this decline can be illuminating in and of itself. When we are able to map nonresponse decisions onto characteristics of the nonrespondents, and when we are able to supplement them with the reasoning given by these nonrespondents, we cannot help but to come to a better understanding of the calculations that go into the responses to closed-ended survey questions. Especially in for-attribution surveys of political actors, nonresponse is not random noise but a conscious choice, a choice which can be identified and analyzed in itself.



Appendix:

CQ/TIME Survey Questions and Coding


BUDGET: Would you have voted for the budget-reconciliation bill aimed at balancing the federal budget by fiscal year 2002? The bill had provisions reducing expected federal spending $ 894 billion over seven years, including major restraints on Medicare, Medicaid and welfare spending, and cutting expected tax revenue $ 245 billion.

MEDICARE: Would you have voted for the bill to reduce expected Medicare spending $ 270 billion over seven years?

DEFENSE: Would you have voted for the amendment to cut $ 493 million provided for continued production of the B-2 Stealth bombers?

TERMLIMITS: Would you have voted for the "term limits" constitutional amendment imposing a 12-year lifetime limit on congressional tenure in either the House or the Senate?

ABORTION: Would you have voted for the bill to ban so-called partial-birth abortions, in which the doctor removes the fetus's brain tissue after bringing the fetus into the birth canal? Under the bill, doctors who perform the procedure could be subject to criminal and civil penalties.

ENVT: Would you have voted for the amendment striking 17 provisions from a fiscal 1996 spending bill that funds the Environmental Protection Agency? The provisions would limit the EPA's enforcement of certain environmental regulations.

FMLA: Would you have voted for the Family and Medical Leave Act, which requires many businesses to provide workers up to 12 weeks of unpaid leave for the birth or adoption of a child or a medical emergency?

GUNCONTROL: Would you have voted for the "Brady Bill," which requires each would-be purchaser of a handgun to wait five days while local law-enforcement officials conduct a personal background check on the purchaser?

GAYS: Would you have voted for the amendment to allow the Pentagon to implement its "don't ask, don't tell" policy, continuing the existing ban on known homosexuals in the military, but preventing military officials from asking about service members' sexual orientation?

BOSNIA. House and Senate: Would you have voted for a bill to bar the use of federal money for the deployment of U.S. troops in Bosnia-Herzegovina?

NAFTA: Would you have voted for legislation to implement the North American Free Trade Agreement, which links the United States, Canada and Mexico in a free-trade zone and requires each country to eliminate numerous tariffs and trade barriers?

RUSSIA: Would you have voted for the foreign spending bill that included $2.5 billion in direct aid for Russia and other former republics of the Soviet Union?

WELFARE: Would you have voted for the bill overhauling the federal welfare system, ending welfare as an entitlement, turning it into a block-grant program to be run by states and placing certain work and behavioral requirements on welfare recipients?



Literature Cited
 

Alvarez, R. Michael. 1998. Information and Elections. Ann Arbor, MI: University of Michigan Press.

Bartels, Larry M. 1986. "Issue Voting Under Uncertainty: An Empirical Test." American Journal of Political Science: 709-728.

Bishop, George F., Robert W. Oldendick, Alfred Tuchfarber, and Stephen E. Bennett. 1980. "Pseudo-Opinions on Public Affairs." Public Opinion Quarterly 44: 198-209.

Bishop, George F., Robert W. Oldendick, and Stephen E. Bennett. 1986. "Pseudo-Opinions on Public Affairs." Public Opinion Quarterly 50: 240-250.

Boatright, Robert G. 1999. Invisible Politics: The Ideological Strategies of Congressional Challengers. Ph.D. Dissertation, the University of Chicago.

Bogart, Leo. 1967. "No Opinion, Don't Know, and Maybe No Answer." Public Opinion Quarterly 31: 332-345.

Brehm, John. 2000. "Alternative Corrections for Sample Truncation: Applications to the 1988, 1990, and 1992 Senate Election Studies." Political Analysis 8: 183-200.

Converse, Jean. 1976. "Predicting No Opinion in the Polls." Public Opinion Quarterly 40: 515-530.

Converse, Jean, and Howard Schuman. 1974. Conversations at Random: Survey Research as Interviewers See it. New York: John Wiley and Sons.

Coombs, Clyde H., and Lolagenie C. Coombs. 1976. "Don't Know: Item Ambiguity or Respondent Uncertainty." Public Opinion Quarterly 40: 497-514.

Faulkenberry, G. David, and Robert Mason. 1978. "Characteristics on Nonopinion and No Opinion Response Polls." Public Opinion Quarterly 42: 533-543.

Robinson, James A. 1960. "Survey Interviewing Among Members of Congress." Public Opinion Quarterly 24: 127-138.

Schuman, Howard, and Stanley Presser. 1996. Questions and Answers in Attitude Surveys. Thousand Oaks, CA: Sage Publications.

Sherman, Robert P. 2000. "Tests of Certain Types of Ignorable Nonresponse in Surveys Subject to Item Nonresponse or Attrition." American Journal of Political Science 44: 362-374.