the (non)polarization illusion

Occasionally social scientists become interested in whether Americans are becoming “more polarized” in their opinions. The obvious strategy for considering this question is to take a bunch of survey items that have been asked of comparable samples in the past and now, and to look at whether people hold more divergent views now than they did then. Studies that use this method can be expected to find that American opinion is not becoming more polarized, allowing social scientists to tut-tut journalists and whoever else has been saying otherwise. The only problem is that the research strategy on which this conclusion is based is deeply, perhaps irredeemibly, flawed.

Why? Well, consider the following sentence quoted by Ken over at Marginal Utility:

By 1996 the percentage willing to vote for a black candidate reached 93 percent, so close to unanimity that the survey dropped the question.

Ken mocks the statement because of the difference between 93% and actual unanimity. Something the sentence reveals, though, is that people buying survey time are typically interested in questions that vary. If they are asking a question that doesn’t vary, it’s for some reason, like perhaps because it has been asked repeatedly in the past. There are questions on the General Social Survey, for instance, that have been retained despite not yielding much variance anymore, precisely because they are items about which people did vary in the past. Rewind that and consider again. Way back when, if the issue was one that had as much consensus as it does now, the question never would have been asked in the first place, and it would be unavailable for comparing whether opinion has become more polarized. So, items that would provide evidence of polarization — consensus then, divergence now — are disproportionately less likely to be part of the universe of available items for comparison over time, while items that provide evidence of no polarization — divergence then, consensus now — are disproportionately more likely. And thus researchers claim to producing findings about the world of public opinion when the patterns in their data actually reflect the world of public opinion surveys.

I’m not even sure it’s possible to draw conclusions about whether people are or are not becoming more polarized in general in their opinions, just because what constitutes the universe of opinions over which one is making such a claim seems a deeply murky concept.

Author: jeremy

I am the Ethel and John Lindgren Professor of Sociology and a Faculty Fellow in the Institute for Policy Research at Northwestern University.

4 thoughts on “the (non)polarization illusion”

  1. I wonder whether “divergence of opinion” and “polarization of opinion” are really the same thing? Doesn’t the latter imply that people a) adopt more extreme positions and/or b) become more entrenched in their positions? So, we might see an increase in consensus in that a greater proportion of the population hold a particular opinion, but also an increase in polarization in that the consensus position and/or the minority position are more extreme or more vociferously held?

    Like

  2. one implication is that you could triangulate this bias by imputing consensus to questions about which there is now controversy. for instance, you could include observed values for the last ten years on gay marriage and prior to the mid-90s you could impute opposition to gay marriage either at some very high number or based on attitudes to items that are highly correlated in later periods (eg HOMOSEX).
    this won’t necessarily work for everything (eg “immediately withdrawal from Iraq” would have been a meaningless question in 1995) but would go a great deal towards countering the kind of bias Jeremy is describing.
    btw, this is a much more sophisticated critique than the usual conflation of party-sorting with polarization (as seen for instance in JQ Wilson’s comment and reply with John Evans in Commentary).

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: