Beckieball is a new sport sweeping the country. Beckieball performance is the result of three things: height, skills, and desire, all of which are uncorrelated with one another. Let’s fire up Stata and show what the correlation matrix would look like for the population:
| perform height skills desire -------------+------------------------------------ perform | 1.0000 height | 0.5751 1.0000 skills | 0.5751 -0.0072 1.0000 desire | 0.5774 -0.0000 -0.0005 1.0000
Unfortunately, scouts cannot really assess desire well, so when they are picking players for professional beckieball leagues, their assessment is just based on equal parts height and skills. What happens?
Say Premier League Beckieball is comprised of the top 5% of beckieball prospects selected on this basis. Here’s what the correlation matrix looks like:
| perform height skills desire -------------+------------------------------------ perform | 1.0000 height | 0.1616 1.0000 skills | 0.1561 -0.7603 1.0000 desire | 0.8842 0.0060 -0.0125 1.0000
Height and skills both become less associated with performance, while the association of desire with performance increases. Also, notice that the correlation between height and skills is now strongly negative: taller pro beckieball players have less skills.
Nevertheless, even in the Premier League, it is the case that taller and more skilled players are better at beckieball than shorter and less skilled players, although, relatively speaking, this difference is much weaker than it is in the overall population.
Now say Minor League Beckieball consists of the 10% of players selected on height and skills who did not make it into the Premiere League. This is what their correlation matrix looks like:
| perform height skills desire -------------+------------------------------------ perform | 1.0000 height | 0.0464 1.0000 skills | 0.0347 -0.9421 1.0000 desire | 0.9705 0.0040 -0.0049 1.0000
In the Minor Leagues, it looks like performance is almost entirely a matter of desire, and that height and skills don’t matter at all. And height and skills even more strongly negatively associated with one another.
(I could carry the example to Semipro Beckieball, which takes the next 15% of people; the result is just a bit more extreme version of the matrix above.)
So, the idea I was talking about by analogy to the NBA, in which the association between height and performance weakens when you study it by looking at a group selected on multiple characteristics including height, is even stronger if you imagine a league that has people who are above average on these characteristics but not the people at the very top. And for these less-than-fully-elite leagues, being very strong on one of the selected characteristics necessarily implies weakness on a different selected characteristic (or else one would be in the elite league).
Also, if you have characteristics that are difficult or impossible to judge in the process of selecting players, but is subsequently available to observe in terms of performance, then it will look like those characteristics are overwhelmingly important for who succeeds and who fails.
But if a team concludes from this information that they should just stop caring about height and skills and just drafting their beckieball players at random–say, on the basis of written statements of how they came to be beckieball enthusiasts–the team might feel very righteous and data-driven, at least up until the point it starts getting crushed in beckieball leagues by teams that did select on height and skills and, as such, have much higher average performance.
(Again, if there is any connection of the foregoing to the matters for which the NBA analogy in previous posts was used, it is left as an exercise for the reader.)