performance management, threshold effects, and asa sections

Every year, the end of September brings a peculiar class of emails from American Sociology Association section chairs and membership committees. ASA sections (e.g. “Economic Sociology,” “Sex and Gender,” etc.) organize much of the activity at the annual meetings. Each section is awarded a certain number of sessions based on the size of its membership on September 30th. If you have 399 members, you get 2 sessions; if you have 400 members, you get 3, and so on. As you would expect, sections routinely scramble in September to try to exceed the next threshold. The form of this scrambling includes offers to subsidize graduate student members (who pay a much smaller amount in dues, but “count” the same towards the session thresholds), book raffles, and even drawings to win coffee with senior scholars. After receiving another such email, I got curious about the effectiveness of these strategies. ASA conveniently posts membership data back to 2009 on its website, and so it’s easy to plop that data into R and produce a quick histogram of year-end membership counts for 2009-2013.*

As expected, we see sharp jumps around major cutoff points: 300, 400, 600, and 800. We see similar trends when looking at publicly traded firms’ earnings data vs. analyst forecasts, or when looking at the size of courses offered by universities trying to game their USNWR ranking (see Espeland and Sauder’s work). So, it seems like all the emails are working – at least, working for the sections trying to get their numbers just above the threshold. Whether or not this particular system is collectively rational I will leave for you all to judge.**

* Thanks to the @ASANews twitter account for the links!
** One clunky but effective solution would be to transition from a pure threshold system to one that awards the final session to each section probabilistically based on how far past the previous threshold it went, with each member being worth about half a percent of a section.

kurzman, missing martyrs

I’m teaching my colleague Charlie Kurzman’s book The Missing Martyrs for the second time this semester in my Sociology 101 course. It’s a great book, and the students appreciate both its counterintuitive (to them) claims and its accessibility. (It doesn’t hurt that the book opens with a recounting of the all-but-forgotten botched attack on UNC’s campus in 2006.)

the revolution will not be invite-only

Still, an Ello invitation would be nice.

UPDATE: Thanks to Tina Fetner, I’m now on Ello, as jeremyfreese.  Friend me or whatever the appropriate verb for that service is.

overheard

“Few things portend a protracted mess as powerfully as an insane person with a totally legitimate grievance about which little can actually be done.”

his honor wants more truck drivers

Our governor, bless his heart, has come out with his latest education-is-overrated statement:

“We’ve frankly got enough psychologists and sociologists and political science majors and journalists. With all due respect to journalism, we’ve got enough. We have way too many,” McCrory said to laughter from the audience.

He said we have too many lawyers too, adding that some mechanics are making more than lawyers.

“And journalists, did I say journalists?” he said for emphasis.

My favorite neocon friend/mentor/correspondent wrote me to ask:

What say you to your Governor about this? In fact, he is always partly right. In fact, your Univeristy [sic] Entitled Ones are always more wrong than right.

Here’s my answer:

the jellybean problem

I’m not as big a fan of xkcd as many geekly friends are, but, in my mind, this cartoon remains the most incisive depiction of the basic problem of low-sigma null hypothesis significance testing in practice.

(Was reminded of it because of Matt’s comment yesterday about how he uses Twenty Questions as an example while teaching. While this use of twenty questions isn’t at all like what Matt was saying, the jellybean comic is the idea that you get to ask twenty hypotheses, the universe will probably lie to you once, and as long as you get one “YES” you can publish it like it is the only question you ever asked.)

replicating the future

Gelman post on meta-analysis of the Daryl Bem research on precognition (yes, precognition):

The ESP context makes this all look like a big joke, but the general problem of researchers creating findings out of nothing, that seems to be a big issue in social psychology and other research areas involving noisy measurements. … I have a feeling that the authors of this paper think that if you have a p-value or Bayes factor of 10^-9 then your evidence is pretty definitive, even if some nitpickers can argue on the edges about this or that. But it doesn’t work that way. The garden of forking paths is multiplicative, and with enough options it’s not so hard to multiply up to factors of 10^-9 or whatever. And it’s not like you have to be trying to cheat; you just keep making reasonable choices given the data you see, and you can get there, no problem. Selecting ten-year-old papers and calling them “exact replications” is one way to do it.

I think the parapsychology research is actually extremely useful, especially if one is willing to take as incorrigible the proposition that parapsychological phenomena aren’t real. Because then parapsychology serves as a kind of control group for science practice, and what’s striking about the Bem research is how much it looks like ordinary psychological science–even psychological science that goes above and beyond the norm–and yet the findings are what they are.

headlong

David Mitchell, author of the sublime Cloud Atlas and this new book that’s gotten mixed reviews but my wife says I would enjoy, on focus and writing:

The internet—it’s lethal, isn’t it? Maintaining focus is critical, I think, in the presence of endless distraction. You’ve only got time to be a halfway decent parent, plus one other thing.

For me, that one other thing is: I’ve got to be writing. I have a few ways to make sure I can carve out time.

did facebook and okcupid experiments violate maryland law?

According to James Grimmelmann, professor of law at the University of Maryland, the recent controversial studies by Facebook and OkCupid violated Maryland’s research ethics law. For past posts on the studies see here and here. The long version of Grimmelmann’s argument is up on Medium. The short version is:

Maryland’s research ethics law makes informed consent and IRB review mandatory for all research on people, even when carried out by private companies. As we explain in a letter to Maryland Attorney General Doug Gansler Facebook and OkCupid broke Maryland law by conducting experiments on users without informed consent or IRB review.

Not only does Maryland require IRB and informed consent for research, it also requires that IRB meetings provide documentation on request, which both FB and OkCupid have failed to do. Grimmelmann and a colleague, Leslie Meltzer Henry, have asked the Attorney General of Maryland to enjoin the companies from conducting further research until they enter into compliance.

fun with a purpose: no longer fun enough?

From Kieran on Twitter, I learned that the Neal Stephenson Kickstarter project Clang! has been abandoned. The idea was to break out of existing videogame conventions and provide a realistic depiction of longsword fighting. As it turned out, apparently, the most fun thing about it proved to be the name. As Stephenson says, “I probably focused too much on historical accuracy and not enough on making it sufficiently fun to attract additional investment.”

I was reminded of the failure of Arden: The World of Shakespeare, which once upon a time was supposed to be an instructive MMORPG that was based on Shakespeare’s world and that would also provide a platform for behavioral scientists to run experiments (on things like, say, pricing of in-game goods). The project leader’s exegesis of the failure involved two things: (1) designing a videogame that can successfully compete for player attention involves a scale that is hard to imagine, but, more notably, (2) after building a big prototype, it became clear that “it’s no fun.”

Broader upshot that I wonder about is how society is evolving with not just increasing entertainment options but, with the implied competition, increasing refinement and specialization on being entertaining itself. In other words, if you aren’t in the first instance–and second instance, and third instance–entertaining, you are done. To take an entirely different example, I enjoy the weekly John Oliver clips that circulate on Facebook, but, man, that guy has to try really, really hard with all kinds of digressions just to be able to hopefully strike home with his main point.

Chris Kennedy, no brainer

So Chris Kennedy, chair of the UIUC BoT, says the decision to fire Steven Salaita would be a “no brainer” outside academia. But since Salaita was actually fired inside academia I find myself asking who would think “this is how it is done outside academia” ought to be a winning claim here.

david vs. goliath

A small woodland creature takes on the man responsible for the technique-revolution in competitive eating. Whatever, I found it inspiring. [HT: RCM]

paula england response

I sent a short note to ASA President Paula England on the letterhead issue. She wrote that I was free to share the information on scatterplot, so here it is:

Thanks for your note.  The policy of the Executive Officer, Sally Hillsman, stretching back a number of years has been that elected officers of ASA may use ASA letterhead when writing on ASA business if they wish.  As you may know, the question of whether Salaita’s treatment abrogated academic freedom and whether ASA should speak out was raised at the end of the Council Meeting in August.  Council could have voted an official ASA position, but there was no time for that, so members informally encouraged me to write as President if I decided it was appropriate.  I drafted a letter, but decided to see if a few other officers agreed.  Not everyone did, so at first my call was to say nothing.  Later, however, the three presidents and secretary decided we did want to speak out. We used ASA letterhead, as we were writing as presidents and secretary of the ASA, but I was careful to word the letter not to imply that this was an official ASA position (that would be appropriate only if Council had voted).  Two elected officers who didn’t agree with our letter decided to write a letter taking a different position, and approached the Executive Officer about whether they too could send it on ASA stationery.  Sally consulted with me, and articulated what the policy had been. Given the past policy, I felt the fair thing to do was grant their request.

Regarding your question about postings under “Advocacy” on Member News and Notes, I’m told by ASA staff that “Advocacy” is just an internal file name (a bit of a residual category), not a title used in the text of News and Notes.

does asa speak for sociology? do they even try?

Sociologists will often profess especial concern for inclusiveness and minority rights. So, when it comes time for sociologists to set up a democratic system of their own, how do they do it? The ostensible governing body of the ASA Membership, its Council, has 20 or so sociologists in the room. By the system that sociologists have devised, it is possible for 49% of voting ASA members to have voted for none of those 20 people. How can that possibly be morally defended?

(Note: in the United States, the winner-take-all system at least comes about as a by-product of the constraint that individuals only vote for one person in a particular election, so it’s hard then to see how the system could be changed without a radical overhaul of what being a Congressperson means. Sociologists have set up their systems so that multiple Council-Members-At-Large are elected via the same election, and sociologists have chosen to run the election so that voters each vote for X candidates and the top X vote-getters are chosen–the most “tyranny of the majority” method of doing so.)

letterhead

Upon returning to blogging, I told myself I would not write posts about the main professional organization for American sociologists. Nevertheless, the front page of the ASA website currently links to a letter hosted in the “Advocacy” section folder of the site that is presented on ASA letterhead, which opens with the following statement:

We write as elected leaders of the American Sociological Association to express our support for your decision not to hire Dr. Steven G. Salaita as a faculty member at the University of Illinois.

This is not the association expressing a concern about the perceived importance of either protecting free speech or protecting students from speech-of-certain-sorts, but instead this is ostensibly the association straight-up taking a position regarding a decision made by the Chancellor on this question. The letter does proceed to note that “some” in the organization may not agree with this position. Likewise, subsequent text–as well as the fact that it is signed only by a Vice-President Elect and one Council member of the association–creates some ambiguity about its status as an official Association document. In this respect, one might also wonder about the preceding letter signed by persons including the outgoing, current, and incoming presidents of the association. At least in that case, the standing of these persons as “leaders” of the Association is stronger and the letter is carefully framed as a statement of concern about a particular principle and not as an explicit position on a decision.

I have known of instances in which individuals make unauthorized use of organizational letterhead to make statements of personal opinion that are not statements made in their official capacities. But of course being presented on the website would suggest this was, in some sense, authorized by the organization (see response from ASA president). Regardless of whether one believes the University of Illinois did or did not do the right thing in this case, the process by which this organization has come to issue its statements on the matter appears peculiar, to say the least.