bleg: feats of operationalization

I’m writing some stuff related to measurement, and specifically about operationalizing concepts as measures. Anybody have any favorite examples of clever ideas in measurement?

A economist colleague here at Northwestern (David Figlio) has a paper in which he was interested in whether African-American kids with first names like Da’Quan or Jacquizz had worse outcomes (possibly due to differential treatment by teachers) than African-American kids named Michael or Stephen. My recollection is that it was nicely designed in that the effects were identified off sibling variations: that is, families with a kid named Michael and a kid named Da’Quan. For measures, he used features of names like whether they included an apostrophe and prefixes (e.g., Lo-) or suffixes (e.g., -isha), but this didn’t take into account the apparent common feature of longer names with low-frequency consonants. So, he also included the Scrabble score of each name, which did indeed prove to be an informative measure.

Author: jeremy

I am the Ethel and John Lindgren Professor of Sociology and a Faculty Fellow in the Institute for Policy Research at Northwestern University.

6 thoughts on “bleg: feats of operationalization”

  1. ps, another good example of creative operationalization is chris bail (who is finishing at Harvard and about to start at Mich RWJ) used plagiarism software to identify when journalists quoted or paraphrased from press releases.

    Like

  2. I’m not sure this qualifies as “clever” and it certainly isn’t novel as it has been around a long time, but the Berry et al “citizen ideology” scale in political science correlates surprisingly well with a lot of stuff despite (to my mind) utterly lacking face validity in its construction. It is complicated in the details but the basic idea is using the ADA and COPE ratings of members of Congress, weighting them by the number of people in a state who voted for them, assigning the proportion of the electorate who voted for someone who was never in Congress the average score for loser’s party in the state, and calculating the weighted average for each state/year.

    Their “institutional ideology” scale follows a similar procedure for state legislatures and governors, who are allocated the average ideology of their state’s Congressional representatives from the same party and then weighted by a complicated measure of political control. So far it does not seem to correlate very well with anything I’m studying.

    Like

  3. Figlio is great… I like a lot of his work, e.g. his clever instrument for the effect of disruptive students on student test scores… If a boy in your class has a gender-ambiguous name he’s more likely to disrupt. Admittedly, I haven’t read this carefully and its almost too good to be true, but it sure is smart.

    I could probably come up with 100 different examples, but I’ll name just one more: Jesse Shapiro’s work on political slant of newspapers.
    http://www.econ.berkeley.edu/users/webfac/gilbert/e221_f07/gentzkow.pdf

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s