Disclaimer. Don't rely on these old notes in lieu of reading the literature, but they can jog your memory. As a grad student long ago, my peers and I collaborated to write and exchange summaries of political science research. I posted them to a wiki-style website. "Wikisum" is now dead but archived here. I cannot vouch for these notes' accuracy, nor can I say who wrote them.
Battista, James Coleman. 2011. Formal and Perceived Leadership Power in U.S. State Legislatures. State Politics and Policy Quarterly 11 (March): 102-118.
Those who study Congress have engaged in long arguments about the importance (or lack thereof) of Congressional leaders in influencing outcomes. Among others, see Cox and McCubbins 1993 and 2005, Krehbiel 1993 and 1998, Binder 1996, and so on. But in a recent article published in SPPQ, Battista asks an important prior question: Do we even know what "leadership" is? All these theories argue whether policy outcomes change depending on the strength of leadership. Battista wonders whether we even know how to measure the strength of leadership in the first place.
Battista looks to the literature on state legislatures, where we've seen two general approaches to measuring leadership strength.
If I read correctly, Battista didn't calculate either of these measures. He wisely collected them from researchers who had used them in the past, and then he compared them. If both measurement approaches are getting at the same underlying construct--leadership strength--then they should correlate.
Trouble is, these two measures don't really correlate, especially when we exclude southern states from the analysis. The correlation coefficients are close to zero, and they fluctuate between statistical insignificance and borderline significance depending on the specification Battista uses.
As such, if we want to study the effects of leadership strength, we need to think carefully about how to measure it appropriately. Perhaps formal leadership powers have one set of effects, whereas perceived leadership power has a different set of effects.
For those who study leadership effects in legislative bodies, this is an important paper and worth reading. However, the comment I find most compelling comes near the end of Battista's article (page 114):
"While it is admittedly possible that one of these measures is right and the other wrong..."
I have a hard time accepting that the survey measure is valid. Simply put, state legislators have no basis for comparing the strength of their own leaders to the strength of leaders in other legislative bodies. Battista acknowledges that legislators lack "any fixed basis for comparison" (page 105) as one of several problems with the survey measure, but it strikes me that this problem is so severe as to undermine use of this variable.
Those who study state legislatures are supposed to be the experts. They observe many, many state legislatures over many, many years. If anybody has the perspective necessary to compare the strength of legislative leadership in one state to another, it ought to be the political scientists who study state legislatures. Yet we as political scientists (rightly) acknowledge that we are unable to rank the states using our judgment alone, so we grasp for other measures.
If we're willing to acknowledge that we can't rank the states by leadership strength despite our multistate expertise, why would we expect that legislators, who are experts on their own state legislature but know nothing about other state legislatures, can do it for us?
Measurements of formal power are, of course, incomplete. Much of "leadership" exists outside the bounds of formal rules. Still, formal rules ought to at least pick up part of the variance from state to state in leadership power. By contrast, I'm not convinced that a legislator survey picks up any of the variance from state to state in leadership power.
Research on similar subjects