sabermetrics comes to college rankings

Two items in my feed today with one theme: using the new College Scorecard to create college rankings that account for the quality of inputs, so to speak.*

First, from the Economist, we get college rankings based for “Four-year non-vocational American colleges, ranked by alumni earnings above expectation.” The methodology is described in a bit more detail as a multiple regression with the many controls: “average SAT scores, sex ratio, race breakdown, college size, whether a university was public or private, and the mix of subjects students chose to study…” and several others including the Economist’s own “Marx and Marley” index of political leftism and stoner slacking. So, one part serious, one part silly, like the famous Big Mac Index. Congrats to Washington and Lee for coming out on top.

At the same time, Brookings published a separate ranking with a similar methodology, though very different analytical choices. Thirteen schools tied for top marks at Brookings, but North Central Institute in TN was listed first so, by the logic of college rankings, they are the big winners.

The Economist usefully describes the differences:

the Brookings numbers regard a college’s curriculum as a significant part of its “value add”, causing the top of its rankings to be dominated by engineering schools, and the bottom by art and religious institutions. In contrast, we treated fields of study as a reflection of student preferences, and tried to identify the colleges that offer the best odds of earning a decent living for people who do want to become artists or study in a Christian environment. Similarly, the Brookings rankings do not appear to weight SAT scores nearly as heavily as ours do, if they count them at all: colleges like Caltech and Yale, whose graduates earn far more money than those of an average university but significantly less than their elite test results would indicate, sit at the very bottom of The Economist’s list, whereas Brookings puts them close to the top.

Alas, no one has yet created a Fantasy University League for us to draft and sell universities, so for the moment this information is relatively useless except maybe to a very savvy college applicant deciding between Harvard, North Central Institute, and Washington & Lee… Leave your thoughts on the methodological choices and what this all means for academia in the comments.

*The inputs are made of people!

Author: Dan Hirschman

I am a sociologist interested in the use of numbers in organizations, markets, and policy. For more info, see here.

1 thought on “sabermetrics comes to college rankings”

  1. There’s an unspoken linearity assumption that goes into these rankings (even if we buy the premise that these are causal estimates of value-added). Could North Central maintain its large value-added if its students were the same ones that now go to Harvard? That’s a tall order.

    Liked by 1 person

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s