Green and Gerber: Reclaiming the experimental tradition in political science

Green and Gerber. 2002. Reclaiming the experimental tradition in political science. in Political Science: State of the Discipline. Ira Katznelson and Helen Milner, eds. New York: W.W. Norton.

INTRO: The experimental tradition of the 1920s-1950s gave way to surveying techniques in the 1950s, b/c surveys seemed to give more insight into why people do what they do and b/c surveys are easily re-used to answer other questions. Further, experiments couldn't address big variables like civic culture, modernization, etc. (pg 807). We can't tell how well our nonrandom (statistical) studies reflect reality unless we actually compare them to experiments.

VALUE: Experiments have greater internal validity --> better causal inference. Unlike with other designs, experimental designs don't require a complete statistical model in advance. (page 810). No risk of data mining due to running scores of regressions. Page 811 lists threats to internal/external validity. You don't need a perfect list of control variables to statistically control for.

IN PRACTICE: In statistical studies, our concerns are about legitimacy and spuriousness. In experimental studies, our concerns are external validity: would the same thing happen somewhere else? At a different time? So we repeat the experiment (page 814). Nonexperimental studies might be replicable (external validity), but never be able to present satisfying evidence of internal validity. See example on pages 815-6. **Downstream randomization: "The dependent variables in lab experiments make for highly informative independent variables in subsequent research." (see page 818) --> if you can intervene in a way that alters the independent variable (randomly, as a dependent variable), you can study its effects on some other dependent variable. (page 818).

LIMITATIONS: There are some things that we can't study experimentally. But try to think of one for every subject you study, even if sometimes there isn't one. "Experimentation is possible whenever decision makers face constrained resources and are indifferent between competing ways of allocating them." (page 821). (E.g. if a candidate's campaign has a list of 10,000 supporters it hopes to call before election day but doesn't know whether it will get through the whole list, ask permission to randomize the order of the list. If some people don't get called, they are a random control group. If the whole list gets called, oh well--no experiment this time.) Other limitations also discussed.