congress and science funding

In the latest battle in the war on science, the Chair of the House Committee on Science, Space and Technology has asked the NSF to explain the peer review process that lead to five grant applications being funded.

This time the attack hits a little closer to home for sociologists. Although the focus seems to be cultural anthropology, as four of the five grants received at least part of their funding from that program, two of the five PIs are sociologists. The Directory of the NSF is being asked to defend awards given to Michael Macy (Cornell) and Linda Kalof (Michigan State).

The lead Democrat on the committee, Representative Johnson, has penned a pretty militant letter in favor of NSF autonomy, and is asking Representative Smith to withdraw his request. (h/t to @howardaldrich for the two letters.)

conflicts

Over at OrgTheory, Philip Cohen asked about norms of retraction when a reviewer has an undisclosed conflict. Here is a test case.

Walter Schumm (Kansas State) is the author of an article in Social Science Research defending the New Family Structures Survey (NFSS) and the Regnerus article that uses the data. Dr. Schumm was also paid by the Witherspoon Foundation to consult on the, “early stages of the development of the NFSS”. His non-peer-reviewed article* makes no mention of this relationship. In an email to me, Dr. Schumm wrote, “I don’t recall if it did come up.” Jim Wright, the editor of Social Science Research, told me, “This was never revealed, at least not to me. This is the first I have heard of Schumm’s involvement.”

Ball is in your court, Social Science Research Editorial Board.

* The article is included in a “Commentary and Debate” section of SSR on the Regnerus and Marks articles. In his introduction, the editor writes, “This ‘Commentary and Debate’ section contains several items pertinent to the controversy. They are published here so that the journal’s readers, authors, editorial board members, and reviewers will have the full story as well as some of the larger context in which the story unfolded.” If you looked at Schumm article without reading the Wright preface, you would likely think it was a normal SSR article.  It does not say “Commentary” anywhere and provides “Article Info” including the “Article History.”

Update: I missed this before, but Mark Regnerus cites both his SSR followup and the Schumm article in the Supreme Court brief he co-authored. They write:

…what is clear is that there remains much to be studied in this  domain, and hence confident assertions of “no difference” ought to be viewed with suspicion. As the study author [Regnerus] indicated, [long quote from the Regnerus sequel]  See also Walter R.  Schumm, Methodological Decisions and the Evaluation of Possible Effects of Different Family Structures on Children: The New Family Structures  Survey, 41 Soc. Sci. Research 1357-66 (2012) (validating methodological decisions made in New Family Structures Study, and noting similar decisions in other large-scale surveys).

A reasonable person who followed the citation to the Schumm article would have no idea that (1) Schumm was a consultant on the NFSS, or that (2) neither article was not peer-reviewed. Setting aside the issue of whether or not the Schumm article should have ever been published, I think  SSR has an ethical obligation to clarify both of these issues ASAP.

Update 2: Both the Schumm and Regnerus articles in the, “Commentary and Debate” section are labeled, “Original Research Article.”

 Schumm

 

None of the others have this designation. For example, here’s the listing for the Gary Gate’s piece:

Gates

 

academic caste system 2013

Recent discussions about department rankings and picking a department for grad school had me wondering how my own department is doing in placing our graduate students in top departments (Spoiler: Pretty good.) I had my undergraduate RA look at the faculty listing web pages for all the sociology departments with a rank of 20 or better in the current US News & World Report rankings. For each of the assistant professors, we noted where they went to graduate school and what year they earned their PhD.

I’ll say up front that this measure is not perfect for determining placement in top departments over the last six or so years (the period I consider). For example, if you earned tenure early, you aren’t in the dataset. I’m also not 100% sure all the people received their PhDs from a sociology department. Because of the small number of graduates from each department, these errors can have a substantive impact on the placement rank of individual departments, so I’m not going to assign ranks to all the departments. Regardless of these caveats, I’m pretty sure that the data capture the big picture of placement in top 20 departments, but feel free to argue that point.

Anyway, here’s what I found: Continue reading “academic caste system 2013”

scatterplot competition: guess the nra’s membership

You know what sounds fun? A Scatterplot competition! I know it’s not exactly a sequel to the epic Mario Kart races of 2009, but it might still be fun. This time it is a guessing game. How many members will the NRA have in June of 2013? Continue reading “scatterplot competition: guess the nra’s membership”

from paper to article

Now that you’ve submitted your paper to the ASAs, how can you turn it into a publication? Two ideas. First, if it is nifty and about social movements, please consider submitting it a special issue of Mobilization that I’m putting together. Deadline is Friday, January 11th.

Second, you should stop calling it a “paper” and start calling it an “article.” Seriously. You might also want to ditch wishy-washy words like: seeks, attempts, looks, and presents. Instead, have hypotheses, analysis and results with consistent, positive effect sizes. And certainly delete the word preliminary. At least that is what the data suggests we find.

Continue reading “from paper to article”

the most cited works in sociology, 2012 edition

Because December is the season for making lists, I’ve updated my sociological navel-gazing to create a 2012 edition of the most cited items in sociological journals.

Since there is no widely accepted list of sociological journals, I include those journals* where the majority of authors who list a department in their mailing address list one that includes the word “sociology” and which have a significant US editorial presence.† This totals 47 journals and includes all the ones you would expect along with some less widely-known journals, like Social Politics. I downloaded the 1,563 research articles published in these journals between December 1, 2011 and November 30, 2012 from Web of Science. That took nine–just nine–clicks, which isn’t so bad. For each article, I counted up which books or articles they cited, and then summed it all up. Journals that published frequently and publish lots of articles (like Social Science Research) or journals where authors tend to cite lots of things (like AJS) probably have undue influence by this measure, but simply counting the number of times something has been cited is a pretty good first pass at seeing what is being commonly referenced.

Here’s the top 25 (as an image, because I can’t do tables in WordPress):

top25_2012

If you want to waste some time, here’s the full list.

So, one in 33 articles cites Distinction. The majority at the top of the list are books along with a pair each from AJS, ASR and the Annual Review, along with one article from Social Forces. The authors and titles are truncated by Web of Science, so don’t blame me. Remember that the lists only counts citations in this group of sociology journals, so being famous in other worlds doesn’t get you on the list.

Fun fact: 2/3 of things that were cited last year were only cited once, and 95% of things cited were cited less than five times. And, unless one of your articles was cited nine or more times in one of these journals last year, you can consider yourself, like me, one of the 99%.

One thing that struck me was how old everything  on this top list was. The median publication year in the top 100 was 1992. Of the top 100, only one piece was published in the last five years. The author ended up at boarding school for future investment bankers, so there is a price to pay for influence.

More generally, things aren’t that bad. It turns out that the average thing we cite is ten or eleven years old. There is a lot of factors that go into what items get cited and how many times, such as the number of papers published in an area or the degree to which there is a common puzzle or cannon in a subfield.  But we don’t seem to be in a hurry to cite new stuff. Or alternatively, we aren’t easily swayed by the newest research trend.

when

I don’t know how that has changed over the long term, but I just ran the numbers for articles published in 2009 in the same set of journals and got the same median lag of ten years.

On a side note, it’s my understanding that Journal Impact Factors are often computed using citations to articles published in the last two or five years. Last year, 93% of the stuff we cited was more than two years old and 78% more than 5 years old, further complicating these measures.

* Social Science Research (110 articles); Sociology of Health & Illness (77); Journal of Marriage and Family (73); Social Forces (52); Demography (48); Journal For the Scientific Study of Religion (47); Sociological Forum (45); Deviant Behavior (44); American Sociological Review (40); Population Research and Policy Review (35); Poetics (33); Sociological Spectrum (31); International Migration Review (31); Sociological Quarterly (30); Social Networks (30); Journal of Health and Social Behavior (30); Review of Religious Research (29); Sociological Inquiry (28); Criminology (28); Social Compass (27); Symbolic Interaction (26); American Journal of Sociology (26); Gender & Society (25); Mobilization (25); Annual Review of Sociology (25); Socio-economic Review (25); Theory and Society (24); Teaching Sociology (23); Journal of Contemporary Ethnography (23); Ethnography (23); Punishment & Society-international Journal of Penology (23); Rural Sociology (23); Sociological Methods & Research (22); Social Politics (22); Homicide Studies (22); Qualitative Sociology (21); Sociology of Education (20); Social Problems (18); City & Community (17); Feminist Criminology (17); Sociology of Religion (16); Sociological Theory (16); Theoretical Criminology (16); Work and Occupations (15); Social Psychology Quarterly (12); Journal of Mathematical Sociology (10); Sociological Perspectives (10). I might be missing the last issue from a couple of journals because they haven’t showed up in Web of Science yet. You snooze you lose.

† My method of identifying sociological journals doesn’t really work for non-US journals as it appears non-US folk are much more likely to lists themselves in things like “School of Social Policy.”  Sorry. You can ballpark the effect of including these journals by adding 327 cites to Giddens’s Modernity and Self-Identity and including a couple of works by John Goldthorpe to the list.

constructing quantitative data

I don’t envy those researchers who collect and analyze retrospective family history surveys. The data can be a real mess.

Mark Regnerus recently released the raw data from his New Family Structure Study (NFSS). I thought that many researchers would want to reanalyze the study, so I attempted to put together a Stata do file that replicates the original analyses that were published in Social Science Research. I’ve also posted my version of the full regression tables, which weren’t in the original article. The article is quite clear on how most of the items were constructed so it wasn’t too difficult to put this together for most of the variables.

The items that were most difficult were those dealing with family structure growing up. I haven’t been able to get my numbers to match Regnerus’s for all the different types of families. Largely this is because the item being measured is quite messy with multiple overlapping categories. If someone else can figure out how Regnerus got his numbers, specifically the step-parent and single parent categories, please send me the code and I’ll incorporate it in my publicly available do file.

An additional problem when analyzing the data is that an individual’s answers are often quite contradictory.  While sometimes this can create sociologically interesting questions, like under what conditions do people change their self-identified ethnic/racial classification, other times you just want a decent operationalization of the concept, like how many respondents lived with two moms?

Continue reading “constructing quantitative data”

2012 sociology job market

If you are one of the twelve people who follow me on Twitter or one of the three people who follow the RSS feed for my website (Hi, mom!), you’ll know that I’ve been tracking the sociology job market this year. My method is pretty crude. I download all the job postings to the ASA Job Bank each month and count which ones have variations of the phrases “tenure track” and “assistant professor.”

As of the end of November, here’s what my monthly count of advertised tenure-track sociology jobs looks like:

tenure_track

By my count, 329 advertisements have been posted for these types of positions so far this year. This is up 5% from where we were last year, up 15% from 2010, and up a whopping 73% from 2009. That said, this year is down 8% from 2008, a year that was a horrible market according to a 2009 ASA study.

In prior years, about 80% of jobs that were going to be listed were posted by the end of November. Based on that, I estimate that we’ll come in at about 410 jobs this year, which is 15 jobs less than my forecast based on July, for those who care about that sort of thing.

My best guess is that this is about how many jobs we can expect to see posted in the coming years. I doubt we’ll see a surge in money for hiring in the social sciences given the current political and economic climate.