Lacy: A theory of nonseparable preferences in survey responses

Lacy. 2001. A theory of nonseparable preferences in survey responses. AJPS 45 (2).

Puzzle

Though Lacy credits Zaller and Feldman's (1992) effort to explain question-order effects and response instability, he criticizes their inability to predict which subjects will have the strongest question-order effects. (The closest Z&F come is an argument that those with more information about politics will have less instability.)

Solution and Example

Generally

On some pairs of issues, a segment of the population may have "nonseparable" preferences: preferences on A depend on the outcome of B. Question order effects occur when you ask about these issue A, then issue B, as if preferences between A and B were independent, not interactive.

Examples

• For example, preferences about the partisan control of Congress depends on who controls the Presidency. Although 40% want Republicans in Congress either way, and 23% want the Democrats in charge, 18% have a preference for divided government and 19% prefer unified. Thus, 18% + 19% have nonseparable preferences: their preference for Congress depends on who is likely to win the presidency.
• Another famous example: Many Americans want US journalists to have free access to the Communist bloc (no matter what), but they only want to allow Communist journalists free access to the US if American journalists also get free access to the bloc. Thus, these preferences are weakly separable: We always want American access, but Communist access is nonseparable.

Prediction

The larger the portion of the population that has nonseparable preferences on issues A and B, the larger the question-order effects of questions about A and B on the same survey. (To the extent that our memories are not perfect, these effects depend on how far apart the questions about A and B are on the survey.)

Empirics

Design

• Survey of Franklin County, Ohio, to test the hypotheses

Findings

• Table 2 (p 244): Pairs of issues listed in descending order of how many people had nonseparable preferences
  • Nonseparable preferences can interact positively ("positive complements") or negatively ("negative complements").
  • The "Order Effect" tells you whether responses about issue B change significantly when B is asked immediately after A.
• The issues near the top of Table 2 have the strongest order effects, as predicted.

Competing Hypotheses

Lacy tests his idea against Zaller and Feldman's suggestion that information matters, but his Lacy's measure of information is too civics-based. There are only three questions: what is Al Gore's current job, which party controls the House, and which party is more liberal. Lame. No wonder Table 3 doesn't find that information matters much.