Jason Mitchell uses the example of “black swans” to argue that there is a fundamental asymmetry between positive and negative findings in psychology experiments, such that positive findings are the only meaningful findings and negative findings should not be published. The idea is that no matter how many white swans you observe, you don’t know if black swans exist; whereas if you observe one black swan, you know they do (full quote at bottom).
The problem: findings from behavioral science experiments aren’t like being able to hold a black swan by the neck and shout to everyone, “See! I told you they existed!”
Instead, you are presented with papers in which you have to trust researcher reports of what they did to produce a finding that an observed swan was darker than would be expected under the white-swan null (p < .05).
In this respect, positive experimental findings are somewhere on a continuum between Bigfoot and black swans.
There’s all sorts of positive evidence about Bigfoot’s existence, but what does not exist is inarguably decisive evidence: “Mom! Can we go to the zoo and feed the bigfoot?” Whatever the rate that new smudged Bigfoot photos appear in the cryptozoological literature, the continued elusiveness of a decisive demonstration weighs more and more as negative evidence that Bigfoot doesn’t exist the longer it goes on.
When evidence is statistical and demands audience trust, you do not have a black swan in your hand to wave at skeptics. As long as there is this possibility for doubt-in-the-first-place, both positive and negative evidence obviously remain in place for someone trying to figure out what to believe.
Mitchell: “Suppose I assert the existence of some phenomenon, and you deny it; for example, I claim that some non-white swans exist, and you claim that none do … Whatever our a priori beliefs about the phenomenon, from an inductive standpoint, your negative claim (of nonexistence) is infinitely more tenuous than mine. A single positive example is sufficient to falsify the assertion that something does not exist; one colorful swan is all it takes to rule out the impossibility that swans come in more than one color. In contrast, negative examples can never establish the nonexistence of a phenomenon, because the next instance might always turn up a counterexample. Prior to the turn of the 17th century, Europeans did indeed assume that all swans were white. When European explorers observed black swans in Australia, this negative belief was instantly and permanently confuted. Note the striking asymmetry here: a single positive finding (of a non-white swan) had more evidentiary value than millennia of negative observations. What more, it is clear that the null claim cannot be reinstated by additional negative observations: rounding up trumpet after trumpet of white swans does not rescue the claim that no non-white swans exists.”