Lies, damn lies, & statistics (continued)

As a follow-up to this bit of inside dope from Dilbert (which I posted yesterday), a friend who asks to be identified as a "dissident social psychologist" offers these additions and elaborations.  —Jeff Weintraub
--------------------------------------------------------------------------------

I have a note taped above my desk which is a quote from [former U.S. Senator] Pete Domenici: "If you torture numbers long enough, you can get them to confess to almost anything."

And then there is Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corrupting pressures and the more apt it will be to distort and corrupt the social processes it is intended it monitor." (The "Campbell" here is Donald Campbell, whose field was quantitative, policy-relevant social psychology.) If you want to see Campbell's law in action, take a look at the mad rush to provide "evidence" to claim that various psychotherapies and psychological interventions are "evidence-based."

An even more discouraging discussion of social "scientists" manipulating numbers is the one that 3 Wharton professors (Simmons, Nelson, and Simonson) produced last year under the title "False-Positive Psychology" [described here—JW]. They showed that by using the everyday, taken-for-granted post hoc fudging of samples, data, hypotheses, and statistical tests that social psychologists routinely use (but do not report), they could demonstrate that singing a Beatles song would result in the subject living longer. The article is quite a tour de force. But if history is any indication, it will have NO effect on researchers' practices.

[JW; It's probably safe to predict that this will also prove to be true for the rigorous debunking of "Voodoo Correlations in Social Neuroscience " by Harold Pashler and his collaborators.]