a citation analysis of piketty

The following is a guest post by Marko Grdesic.

The work of Thomas Piketty needs no introduction. Piketty is that rare case – an academic superstar. What has been the reception of his work in the social sciences? This blog post will present the results of a citation analysis.

Although his 2014 book Capital in the Twenty-First Century caused the biggest splash, his most cited work is actually another piece, the article “Income inequality in the US, 1913-1998”, co-written with Emmanuel Saez and published in 2003 in the Quarterly Journal of Economics. In mid-November of 2015, Capital was cited 2185 times, “Income inequality” 2345 times. In terms of the number of citations, that is more than twice what the third most cited piece has gathered (Atkinson, Piketty and Saez, “Top incomes in the long run of history”). Combining Capital and “Income inequality” makes it possible to investigate the different receptions of Piketty: (1) the Piketty of Capital with its controversial theory of long-run doom and gloom (r>g) and (2) the largely neutral data-gatherer Piketty of “Income inequality.”

How was the citation analysis conducted? First, through a Google Scholar search, I located all the works that cite Piketty’s Capital. I included the first 100 works, ranked by how many times the works themselves were cited. Second, the same procedure was repeated for “Income inequality.” Next, I conducted a search for “Piketty” in several of the main academic databases: EconLit for economics, SocINDEX for sociology and Worldwide Political Science Abstracts for political science. Only articles which were themselves cited at least once were included. Articles by Piketty and his main co-authors (Saez and Atkinson) were excluded. I also excluded non-academic publications such as The Economist, NY Times or blogs. In the end, this led to a database of 255 works.

What kind of information could be extracted from this? After a lot of reading and a largely fruitless search for Piketty memes, I decided to code the following: (1) Is the work of Piketty evaluated as positive? (2) Is the work of Piketty evaluated as negative? (3) Is the citation of Piketty essentially a courtesy citation, so that it could have been replaced with another reference? (4) Is Piketty’s work referred to primarily as primary data, i.e. to substantiate an empirical point such as: “The wealth of the top 1 percent has risen sharply in recent decades.” I also tracked the number of citations that each work in the database had (as reported by Google Scholar) as well as the number of times that Piketty’s name appears in the text. This last piece of information is a useful indicator of how detailed the examination of Piketty’s work is. Each item was double-coded and reliability scores were high: Cohen’s Kappa and Krippendorff’s Alpha were around 0.93.

So what do the data say? The first table (pdf here) breaks down the descriptive statistics. Overall, the percentage of works that had a positive reading of Piketty was about the same as the percentage of works that had a negative reading (around 20 percent). But, the difference between Capital and “Income inequality” is interesting. Capital has a higher percentage of both negative and positive evaluations. In other words, it has proven more divisive. The ratios are 29 percent positive and 37 percent negative for Capital and 10 percent positive and 5 percent negative for “Income inequality.” In other words, the perception of Capital has leaned slightly towards the negative. “Income inequality”, on the other hand, is not so controversial, though its perception leans slightly towards the positive. For example, when discussing “Income inequality” Gregory Mankiw says that the article of Piketty and Saez presents “the best data we have on the upper tail of the income distribution.” But when he turns to Capital, Mankiw signals his disagreement with Piketty already in the flippant title of his essay: “Yes, r>g. So what?”

Some authors both critique and praise Piketty’s work at the same time. For example, out of the 39 works that present Capital in a positive light, a total of 29 also presented it in a negative light. In other words, the two need not be exclusive. A good example of this is Deirdre McCloskey’s essay: “Nothing I shall say – and I shall say some hard things, because they are true and important – is meant to impugn Piketty’s integrity or his scientific effort.” This suggests that many authors gave Piketty’s work serious consideration and weighed its strengths and weaknesses carefully. But there are also many casual readers of Piketty. For example, about 30 percent of the works that cite Capital really do so as a courtesy, such that the reference to his book could easily be replaced with a reference to any other work about inequality. “Income inequality”, on the other hand, was most often cited as data: about two thirds of the works that cite it refer to it in that way.

Differences across disciplines are also interesting. Out of the 52 works that were gathered from the economics database EconLit, a total of 20 had positive readings while 27 had negative readings. This suggests a slightly negative reception of Piketty’s work in economics. This is driven primarily by the reception of Capital, which has sparked passions more than “Income inequality.” The number of works drawn from the sociological database SocINDEX is not high, so interpretation needs to proceed with caution: a total of 19 works were gathered. Within this group, it seems that Piketty’s work evokes stronger reactions: around 84 percent positive and 80 percent negative. Once again, this is mostly driven by the reception of Capital. In other words, both economics and sociology have had some rough things and some nice things to say about Piketty. On the whole, sociology seems to be a bit more supportive. But again, it should be re-iterated that not all economists are hostile to Piketty. Some prominent names have come to his defense. For example, Branko Milanovic wrote in his review of Capital that “we are in the presence of one of the watershed books of economic thinking.” For the political science database WWPSA, the numbers are even smaller so no conclusions can be offered.

Disciplinary differences can also be examined in a multi-variate context. The second table (see pdf) shows three models. The first model looks at the determinants of positive evaluations of Piketty’s work, the second at the determinants of negative evaluations. Perhaps the most interesting finding is that a positive reading was more likely if the work that cited Piketty appeared in SocINDEX. On the other hand, a negative reading was more likely if the work appeared in EconLit. But the variable for SocINDEX is very close to conventional thresholds of statistical significance in the second model too (p=0.052). This is in line with the descriptive analysis: both economics and sociology had both negative and positive things to say about Piketty but the balance tips towards the positive in sociology and towards the negative in economics. Of course, the overall number of observations is not large (especially for SocINDEX) and so the statistical findings should not be pushed too far. Here, a few examples of sociological praise can flesh out the picture. For instance, Mike Savage writes that Piketty’s book is “of profound significance for sociology” while Robert Antonio admits that “I find the work compelling and cannot stop thinking about it.”

In addition, it seems that works that mentioned Piketty’s name more frequently were more likely to present both negative and positive points. In other words, detailed discussion of Piketty led readers to a rather even-handed assessment of his work. The coefficient for the variable that tracks the number of times his name appears is statistically significant and positive in models 1 and 2. This finding was replicated in the third model: the variables that track positive and negative assessments are both positive and statistically significant predictors of the number of times his name is mentioned in the text (this was double-checked with an interaction term too). In other words, the closer one reads Piketty, the harder it is to make simple judgments about his work. Consider David Soskice’s essay, which mentions Piketty’s name about a hundred times. Soskice writes that Capital is “one of the most compelling books I have read in a long time, lucidly and easily written and exceptionally interesting”, but he also adds that Piketty’s “parsimonious mathematical model (neat and clever though it is) almost completely ignores the interacting relations between politics, history and technological change.” In other words, the more deeply one reads Piketty, the more likely it is that one will present the positive and the negative together.

In summary, here are the main conclusions of this citation analysis of Piketty:

• The academic reception of his work is split: as many see his work in a positive light as in a negative light.
• Capital has evoked more passionate responses, compared to his more descriptive but oft-cited article “Income inequality in the US.”
• The reception of his work in both sociology and economics is split, but the balance tilts towards more positive in sociology and more negative in economics.
• Positive and negative evaluations combine in those works that treat Piketty in most detail.
• Although the existence of r>g t-shirts is welcome, we are still in dire need of a Piketty meme.

 

Author: Dan Hirschman

I am a sociologist interested in the use of numbers in organizations, markets, and policy. For more info, see here.

1 thought on “a citation analysis of piketty”

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s