Disclaimer. Don't rely on these old notes in lieu of reading the literature, but they can jog your memory. As a grad student long ago, my peers and I collaborated to write and exchange summaries of political science research. I posted them to a wiki-style website. "Wikisum" is now dead but archived here. I cannot vouch for these notes' accuracy, nor can I even say who wrote them. If you have more recent summaries to add to this collection, send them my way I guess. Sorry for the ads; they cover the costs of keeping this online.
Smith. 1980. The levels of conceptualization: False measures of ideological sophistication. American Political Science Review 74: 685-.
Campbell et al (1960) proposed a notion of "levels of conceptualization," using four levels. Some people think about politics at the highest level (ideology, broad principles), while others think only in narrow, issueless terms ("Bush is a nice guy"). In the middle, some people think mostly in terms of group benefits and such.
In response, several authors later proposed measurement schemes to quickly classify survey respondents according to their level of conceptualization. This article addresses two common measures, and finds that neither is reliable or valid. In fact, these measures don't reflect how people evaluate politics at all; they only reflect the changing words we use to explain our evaluations.
Field and Anderson (1969) have three levels of evaluation, Nie, Verba, and Petrocik (1976) have seven. Any survey respondent should have a stable level of conceptualization over time, since (the theory says) that cognitive abilities are a key determinant of one's level. However, Smith uses panel data from 1956 and 1960 to show that the 'same' respondents have very different levels of conceptualization in the two different years (see pg 689 for tables). The correlations between 1956 and 1960 are much lower than we would expect (in the 0.2-0.3 range).
Thus, he concludes that these measurement schemes aren't reliable. They aren't measuring people's enduring level of sophistication (which Smith does believe exists, even though he thinks we aren't measuring it well); they're only measuring how we talk about evaluations. When a surveyor asks you to explain why you support a person or policy, you just spit back something you've heard in the news lately, or something recent that is on your mind; thus, what you say can suggest a different level of sophistication than what you really have.
Research on similar subjects