Jason Mitchell uses the example of “black swans” to argue that there is a fundamental asymmetry between positive and negative findings in psychology experiments, such that positive findings are the only meaningful findings and negative findings should not be published. The idea is that no matter how many white swans you observe, you don’t know if black swans exist; whereas if you observe one black swan, you know they do (full quote at bottom).
The problem: findings from behavioral science experiments aren’t like being able to hold a black swan by the neck and shout to everyone, “See! I told you they existed!”
Instead, you are presented with papers in which you have to trust researcher reports of what they did to produce a finding that an observed swan was darker than would be expected under the white-swan null (p < .05).
In this respect, positive experimental findings are somewhere on a continuum between Bigfoot and black swans. Continue reading