Chris Winship has written a post on orgtheory regarding the amicus brief submitted by ASA in the Walmart case. It should be noted that my colleague and friend Laura Beth Nielsen, who was centrally involved in drafting the brief, has written a response (in collaboration with our department’s graduate students Amy Myrick and Jill Weinberg) to the article by Mitchell et al. that Chris cites.
I haven’t read the exchange carefully and given the thicket of personal and professional relationships I have with people with different views of the matter–not to mention my own complete lack of interest in ever being an expert witness or otherwise ever setting foot in a courtroom–I am staying out of this one.
Incidentally, a background conversation led me to consider: I think quarrels various people have with ASA actions can be usefully divided into two types. One type is rooted in asserted misalignments between ASA staff interests and ASA member interests. Whatever their merits, I would count concerns people have expressed about the dues increase and the job bank as examples of this type. The other type involves frustrations one portion of ASA members has with the preferences/will/convictions of a seemingly larger portion of ASA members. The Iraq War resolution is one example, and the Walmart brief is probably another.
scatterplot 
Jeremy,
You are blurring an important distinction in the second sort of grievance, which is that one can have a purely procedural objection to ASA getting involved in an issue on the grounds that it is a professional service organization not a political action committee. Obviously such a circumscribed role is especially attractive to people in the minority, but in principle (and I think often in fact) the procedural objection could include people who agree with the majority on the merits. For instance, I am not a fan of the Arizona SB 1070 law but I still think ASA as an institution (as compared to individual sociologists) has no business making a statement about it.
This reticence about creeping beyond the organization’s professional service mission was the attitude taken by ASA Council in reaction to the membership petition for an Iraq War resolution — under the bylaws they had to put it on the ballot but they attempted to undermine it by amending the language to allow an “anti-war but none of ASA’s business” option. Since then more recent council’s haven’t shown any reticence at all, as can be seen by the ever accelerating rate of official council statements, currently issued at a rate of about 3 a year, and some of which are directly related to the discipline’s interests (eg, supporting the Bureau of Justice Statistics) but others of which are much more closely related to the median sociologist’s policy preferences than to the discipline’s interests (eg, endorsing card check and opposing SB 1070).
LikeLike
I meant what you describe as my second type–lo, as almost the prototype of my second type–so I didn’t say it right. In the Iraq War case, there was a division between members who wanted the resolution and a smaller number of members (myself included) who felt it wasn’t something ASA should really be passing resolutions about. In the cases of the official council statements you cite, I could be wrong, but my quess is that most members would support not just the sentiment of those statements but ASA issuing those statements.
In any case, none of this really has much to do with ASA practices related to revenue expansion or staffing levels, which is the primary source for the complaints of the first type. For instance, I think a majority of ASA members would oppose ASA using its listserv policies to prevent members from learning about some jobs (a policy about which there was member complaint and which might have changed in the last year). Meanwhile, with the dues increase, all you need to see are the original footnotes article and the way the issue was presented on the ballot to recognize that ASA staff have a clear interest in it passing, and the task is more selling it to members than presenting it to members.
LikeLike
just to be perfectly clear, this particular amicus brief was in response to the defense assertion that sociology basically is not science. The plaintiffs offered statistical evidence and sociological case-study evidence and the defense waved their hands and said the case was built on nothing more than “statistics, sociology, and anecdote.”
The ASA brief makes the oh so bold argument that “Sociological methods allow researchers to use aggregate data to form testable hypothesis about specific cases.”
We may disagree about what is good or bad sociology and what methods are appropriate for what questions (and yes, there we have actual disagreements), but in this case, but this amicus brief primarily was saying that sociology allows us to know some social things and how to analyze some things.
Now, Mitchell et al/Winship may disagree about how Bielby did it, but that is argument #2.
Argument #1 — and the one ASA was most interested in making — is that we are a science that may possibly have something reliable to add to the case.
Defending ourselves as a discipline with scientific norms seems centrally related to the discipline’s interest.
LikeLike
Agreed. I almost said something along these lines in my first comment but neglected to. Sorry about that.
LikeLike
Laura,
Until this morning, I had never read the ASA brief. I was quite curious to read the section on causality and legitimate forms of explanation in sociology.
The key section is on pages 6 and 7, following a quote from an Annual Review of Sociology piece by Gangl (though his name is misspelled as Gangul). Then, the ASA offers an opinion: “A researcher’s ability to draw causal conclusions depends on the scope of information available, the type of data she collects, and the methods she employs to analyze data. Equally valid for some research purposes are techniques that focus on correlations between social phenomena instead of on causal relationships. Sociologists can determine with a great deal of certainty, given a strong body of empirical evidence, when particular conditions or practices lead to common outcomes.”
Two questions occurred to me at this point, and I’d love to get your perspective as one of the authors of the brief:
1. What “research purposes” deem correlations “equally valid” as “causal conclusions”?
2. What is the difference between “lead to” and “cause”?
LikeLike
for phenomenon where you can’t get a causal explanation (the area in which most of us work, I think), sociologists show relationships. Maybe I am missing something, but most of us work in areas where we can’t (and mostly eschew) showing causation. And that research (most of what is in ASR all the time) is a valid part of sociology’s scientific method.
If a court chooses to say that they will only admit things if causal links are shown, they want the gold standard only, then OK fine. But to only allow that kind of evidence in because the rest of what sociology does is “not science” seems wrong to me.
I’m all for a vigorous debate on the merits of the/this/any method (and so is Chris, I should point out) but that should happen in the District Court under the rules of Daubert (which may happen if the case is remanded).
Here the defense just shooed the entire discipline of sociology away and lumped us in with anecdote. I think that even when we are not showing causality, we are more than anecdote.
LikeLike
I agree completely that lots of work in sociology is noncausal and is worthwhile (indeed vital to the discipline). Since it’s worthwhile, it serves a purpose for sociology.
Nonetheless, the ASA brief seems to be speaking on behalf of all sociologists without (a) making it clear to the court for what purposes noncausal research is worthwhile [when it is presumably absolutely clear to the court what the purpose of the expert testimony is supposed to be in this particular trial] and (b) acting as if “lead to” is not a causal relation [which is a preposterous position, neither in accord with scholarship on causality nor ‘the woman on the street’].
So, it may be that the ASA brief is there to defend sociology, but it sure reads like it is trying to argue that noncausal associations can, for this court case, be treated as causal evidence because sociologists have a different standard for causal claims than does Walmart. And since sociology is a science with its own traditions, it has more credibility than Walmart, which is merely a litigant. Or Winship, and his ilk, because he surely doesn’t know much about causality in sociology.
You’d think if you were going make that sort of argument, you’d support your position with a bit more than an improper citation to a review article that does not at all support the general position.
My guess is that the supreme court will not side with Bielby or the ASA on this matter, and the result will be that the ASA has put forward a weak argument that was ineffective and gave Walmart an excellent defense. If so, then sociology and women who are discriminated against by large corporations (and there surely are many) will have been harmed.
LikeLike
Are you seriously claiming that Chris Winship doesn’t know much about causality?!?
LikeLike
no. please see below.
LikeLike
Okay, then what is your explanation for why you didn’t contact him while preparing your brief?
LikeLike
Laura: I’m with janecausal on this one. That line about “Equally valid for some research purposes..” made me wince. What is the research purpose that a sociologist might have that makes it OK to infer causality from correlation? Sure, correlations are fine if no claim about causality is being made or if the researcher is very clear that no claim of causality can be concluded. But I can’t believe that sociologists want to endorse the view that it is OK to claim that causality has been shown even all that has been shown is correlation.
It seems to me that if you work in an area where showing causation is impossible, then there is an easy solution: don’t claim that you’ve shown causation. And if you want to “eschew” showing causation, there is another alternative: only make non-causal claims. The no-no is saying that you’ve making an empirical claim (whether causal or not) and not backing it up with sufficient evidence. (And if you have partial evidence [the typical case], discuss the study’s limitations; this is a big part of what’s missing in expert testimony and makes it very problematic for us to endorse as science).
In this case, it seems that Bielby’s claim *is a causal claim*. As such, it seems quite reasonable for them to ask whether he has provided enough evidence to support a causal claim. And they are charging that he has not. Personally, I think they might be right. And so there is no reason for me to feel that sociology is being attacked.
LikeLike
I would be wise not to weigh in on the “causal” debate with respect to whatever either Bielby or the ASA said, as I have not read those.
But it seems like Jane C. and Ezra Z. have gone down the path of implying that research is useless if it is not causal, which seems to me to be a limb out on which I do not want to sit, and I would hope they don’t want to be out there, either. Apart from the generic usefulness of some kinds of correlational research with no causal aspirations, there is also substantial useful evidence to be adduced from correlations with respect to possible causal relations, even if pure one-way causality cannot be nailed down. Surely you cannot be seriously arguing the contrary?
In a discrimination case, I don’t see how a high level of causality is required anyway. Or, rather, WHETHER you need to demonstrate a high level of proof that intentional discrimination and only intentional discrimination created the observed empirical patterns is precisely the point at issue. One side thinks that an observed descriptive pattern can and should be taken as prima facie evidence of discrimination unless a good alternate explanation can be offered, while the other side thinks discrimination should never be offered as an explanation unless there is some sort of “smoking gun” proving intention. Or, if there is no smoking gun, the second side proposes that discrimination might possibly be factor if there has been an exhaustive and comprehensive procedure of testing for all possible alternative explanations.
People can and do discriminate unconsciously — this is, I would trust, a relatively uncontroversial summary of scientific research. Seemingly-neutral rules and procedures can have implicit discriminatory impact. Also pretty well accepted among social scientists.
And couldn’t a pattern of past discrimination cause a corporate culture that creates seemingly neutral rules and generates unconscious presuppositions that have ongoing discriminatory impact? Wouldn’t it be impossible to disentangle causal direction in this case? And wouldn’t you still think that discrimination had something to do with what was going on?
Again, this is not a comment on the original point, but on the way the discussion has gone.
LikeLike
OW:
1. I think you should indeed read those documents before you weigh in.
2. Please read more carefully. Neither I nor janecausal “have gone down the path of implying that research is useless if it is not causal”? Janecausal said in her first line that “Janecausal wrote “I agree completely that lots of work in sociology is noncausal and is worthwhile (indeed vital to the discipline).” And I have published a paper in the American Journal of Sociology that does not make a causal claim [and is based largely on qualitative data]. (And I would also endorse your view that correlations can be useful, so long as the proper caveats have been made– see what I wrote about study limitations). When I said that– “if you work in an area where showing causation is impossible, then there is an easy solution: don’t claim that you’ve shown causation”– I did not say, nor did I imply, that such claims were useless. There are all kinds of claims about the world that are of value if they say something new. Some are causal. Some are not.
I’m siging off these discussions for a few days and I hope that people will not just imagine what they read before responding to it, but actually read it. I will try to do the same when I return.
LikeLike
Obviously I don’t agree that noncausal work is unimportant. After all, I started that remark with “I agree completely that lots of work in sociology is noncausal and is worthwhile (indeed vital to the discipline). Since it’s worthwhile, it serves a purpose for sociology.”
I am also not an expert on discrimination, and so I take no position whatsoever on social science consensus on discrimination or on the merits of this case against Walmart. (In fact, I have no love of Walmart and so I am probably predisposed to believe they are guilty anyway. Certainly I believe that discrimination against women is pervasive in society and have no reason to doubt that it is present at Walmart too.)
My comments are rather narrowly about how confused the ASA brief is on what constitutes a causal claim, and what explanatory status a non-causal claim has in sociology (and most importantly, whether there is consensus on this latter point). I think the brief is pitiful in conveying the depth of thinking on this issue that can be found in the recent literature in sociology, as well as the broader social science and philosophy literature. A single citation to a Gangul [sic] piece? This is our association’s best effort?
My guess is that if I studied Bielby’s conclusions carefully, I would determine that I share his _opinion_ completely. But, I wouldn’t be comfortable asserting that opinion as confidently as he did, but I’d have to read carefully and think before determining for sure. But, with that being said, I am 100 percent certain that I don’t regard the ASA brief as a model of clear thinking worthy of submission to the supreme court. And, when the supreme court rules in a way that goes against the ASA, no one is going to argue that the brief was good for anyone. It just makes sociology look even more fuzzy headed than it is, and the plaintiff won’t have been helped. (And, if Clarence Thomas writes for the majority that the ASA brief convinced him to take Bielby seriously, etc., then I suppose I will happily eat my hat … or even that one that Beatrice wore.)
And that brings up a larger issue, now that I think about it: What was the process by which the brief was put together? Was Markus Gangl consulted about how his article would be used? What about other sociologists who write on causality from other perspectives, like Charles Ragin or Ken Bollen? After all, this is a document from the ASA, right? Is there an obligation to consult with a range of experts, not simply trot out a single article and then claim (mistakenly) that it supports the point for which it seems to be used?
LikeLike
p.s I should have written “There are all kinds of claims about the world that have value if they say something new… and they are supported by appropriate evidence.”
LikeLike
Call me crazy, but I think lots of things “lead to” other things when the “causal” story is not clear. Maybe this is a “mechanisms” distinction.
I don’t know exactly how discrimination at Wal-Mart happens, but I’m pretty confident that in contemporary America, when big, male-dominated bodies of managerial employees are turned loose to hire, promote and fire workers without stringent anti-discrimination oversight, it usually “leads to” women not getting as far up the corporate ladder as you’d expect given other known factors.
In the 18th century a doctor figured out that being a chimney sweep apprentice led to scrotum cancer – though he’d never heard of cells, proto-oncogenes or DNA. He couldn’t say how soot caused scrotum cancer, but he was pretty sure he didn’t want his son doing that job. (This story is in The Emperor of Maladies.)
The one frequently, predictably, precedes the other. I’d call that “leads to” but stop short of “causes.” Is that wrong? (It’s OK with me to be wrong on that terminology.)
LikeLike
Well, what do you mean by “lead to”? Do you mean “I can’t give you the exact size of the causal effect, or even fully explain every causal pathway, but I am confident that the effect is not zero and in fact is in a particular direction [in this case, leads to discrimination]”?
Is that what you mean? If so, do you really want to maintain that that is not a causal claim? Surely no. You want to say it is, but just qualify your claim with a ‘I can’t give you every number along the pathway from cause to effect.’
LikeLike
OK, I’m good with that. Thanks.
LikeLike
I would like to ask Janecausal and Laura Beth to explain in what sense they think most research published in the top journals is noncausal. It seems to me that most of it is, though words and phrases like affect, contributes to, leads to, etc. often get used instead. How often do we do purely descriptive analyses?
I think the confusion here is over what is meant by a causal analysis. As far as I am concerned we can never show causality definitively. In that sense I am a Humean. We look out at the world see various associations, have various theories about how the world works and this leads us to causal inferences. The key question is how much confidence we should have in those causal inferences. This then gets us into all types of interesting questions about what type of evidence is useful for backing up a causal claim. Correlations of various types are key to any causal analysis. The key is interpreting them.
I would say the Bill is making a causal claim in his expert report. To quote from the end of the report
Discretionary and subjective elements of Wal-Mart’s personnel system and inadequate oversight and ineffective anti-discrimination efforts contribute to disparities between men and women in their compensation and career trajectories at the company.
Saying they “contribute” is a causal claim in my book. Saying A causes B is not some stronger version of A affects B, contributes to B, leads to B. These are all causal claims – changing something about A will change B (at least some of the times). One of my complaints with this and other reports of Bill’s is that he makes no effort to rule out other possibly explanations for the evidence he has examined. In addition, he makes no attempt to assess the confidence we should have in his claim. To me, both are essential to any scientific analysis.
But back to causality. Let me make a stronger claim. I believe that unless you are an extreme naïve positivist, any description must be understood as at least a pre-causal analysis in that in describing something either implicitly or explicitly one is making a statement about what entities are important and thus what entities are potentially causally related to each other. In my book, pure description is never a possibility. Spent the last few years reading a lot of cognitive psych. They have basically empirically proven that pure description isn’t possible.
So Jane (whoever you are) if you are going to slam me for knowing anything about causality “Or Winship, and his ilk, because he surely doesn’t know much about causality in sociology” could you explain. Don’t think much of pure ad hominines. Before you decide that you really know who I am and what I know, you might look at my website and see the rather varied work I do. You might read “In Defense of Foxes” to get some sense of who I am. All the said, I liked your last comment.
LikeLike
Chris,
We agree completely on the essentials (with one caveat later). But … you have misunderstood my sarcastic comment because of my sparse punctuation and brevity. This is what happens when one is typing blog comments quickly in between meetings and other commitments.
Let me add some punctuation and verbosity to the paragraph in question in order to clarify for you (and any readers still paying attention).
I wrote (but now with additional punctuation and verbosity): “So, it may be that the ASA brief is there to defend sociology, but it sure reads like it is trying to argue: ‘Noncausal associations can, for this court case, be treated as causal evidence because sociologists have a different standard for causal claims than does Walmart. And since sociology is a science with its own traditions, it has more credibility than Walmart, which is merely a litigant.’ And the brief implies: “Winship, and his ilk, have less credibility because he and his ilk surely don’t know much about causality in sociology.'”
So … what I am trying to say is that the brief implicitly argues against you, without addressing you, because the authors know that you are enmeshed in this type of litigation as well. So, I am not saying that you don’t know anything about causality, but instead claiming that the brief is, in not addressing you directly, implying that you don’t know what you are talking about. That is obviously wrong.
I apologize for seemingly attacking you, and that was never my intention. Clearly, anyone who has read your work knows that you are among a very small group of sociologists that has written on the modern literature on causality.
But … we do disagree a bit, at least in what you wrote in response to my earlier comment. My point that noncausal analysis is quite worthwhile simply reflects my position that there is worthwhile sociology that is only minimally explanatory. So, I agree that explanatory efforts cannot avoid causality, but there is much ‘this is how the world is’ that is not explanatory in a deep sense and is still worthwhile sociology. Surely that is consistent with your position in the last chapter of Morgan and Winship (2007). So, I don’t really think we disagree. But, I should read that ‘foxes’ article nonetheless!
LikeLike
I am staying out of the debate about the amicus brief per se, but although I would agree that A “contributes to” B is a causal claim in a counterfactualist sense, it’s not the case that A “contributes to” B is synonymous with A causes B. “Contributes to” implies multiple causes and, at least to my ear, insufficiency. “Causes” includes such cases but also sufficient and otherwise simple causal relations. Compare: “A gunshot wound caused JFK’s death” to “A gunshot wound contributed to JFK’s death.”
LikeLike
Not synonymous, yes, if you want to talk about event causation in an overdetermined world. But … this is clearly a case where counterfactuals are the only way to go. In other words, this is not a ‘what was the cause of event A,’ wherein all manner of preemption issues must be considered. This is really a quite simple causal structure of: ‘what would have been the case had Walmart not behaved thusly’. Nothing at all complex about the structure here, and thus the “leads to” syntax is not some sort of deep escape from the obvious. This is just sloppy writing without any effort to get up to speed on the causality literature that many of us know, including you.
LikeLike
It seems like by introducing terms like “intentional discrimination,” we are confusing a scientific or logical notion of causality with a legal or political notion of culpability or intentionality. It is possible to have notions of causation that don’t involve agency and this is basically the argument I see Bielby making.
Here is the summary paragraph from the original Bielby report:
“Centralized coordination, reinforced by a strong organizational culture, creates and sustains uniformity in personnel policy and practice throughout the organizational units of Wal-Mart. Subjective and discretionary features of the company’s personnel policy and practice make decisions about compensation and promotion vulnerable to gender bias. Finally, I have concluded that there are significant deficiencies in the company’s policies and practices for identifying and eliminating barriers to equal employment opportunity at Wal-Mart. The basis for my conclusions is explained below.”
My reading is that Bielby is basically saying that (macro) corporate culture exacerbates (micro) unconscious stereotyping that in turn creates discrimination given the (macro) policy of discretion and the absence of (macro) affirmative action policies. This is a causal argument, as suggested by words like “create,” “sustain,” and “make.” On the other hand, it does not appear to be a claim about intentionality or malice. This all strikes me on the face of it as a perfectly plausible causal story about how Wal-Mart’s internal labor markets work, though I don’t know (and can’t say without reading the Bielby report very closely) that I would accept it as a peer reviewer. (FWIW, I am quite comfortable using a practical scientific realist criteria for causal inference rather than a hardcore Humean critique of the problem of induction).
LikeLike
I agree with Gabriel’s comment above. I posted the following comment at orgtheory,
but people here might also reply.
Various people keep claiming here and on Scatterplot that Bill does not make any causal claim. The last sentence in his report is:
“Discretionary and subjective elements of Wal-Mart’s personnel system and inadequate oversight and ineffective anti-discrimination efforts contribute to disparities between men and women in their compensation and career trajectories at the company.”
How is the meaning of this different from:
“Discretionary and subjective elements of Wal-Mart’s personnel system and inadequate oversight and ineffective anti-discrimination efforts are a cause (among possible others) of disparities between men and women in their compensation and career trajectories at the company.”
Please help me understand why these two statements are not saying the same thing. Clearly, the second statement is a causal claim.
Correlations (associations) are about things we see in the real world. Causal statements are claims about how we think the world works. They have entirely different epistemological statuses. When Bill says that certain conditions in an organization contribute to disparities, he is making a claim about how things work in the world in general and Walmart in particular.
If this reasoning is wrong please explain. BTW, Judea Pearl has developed a very neat theory for examining the relationship between observed associations and causal claims based on directed acyclic graphs, basically path models. His approach shows when an observed association supports a causal claim or not. Chapter 3 of Morgan and Winship provides an introduction.
Several people have indicated that they read Janecausal’s comment about my not knowing anything about causality in sociology as sarcastic. If so, my apologies for being defensive.
LikeLike
Chris and I actually have a new piece that you can read on Pearlian DAGs, etc., and that doesn’t require you to buy our book. It is forthcoming in the Oxford Handbook of Philosophy of the Social Sciences but is here now:
Click to access Morgan_and_Winship_2011.pdf
LikeLike
I agree the Bielby statement is equivalent to the restatement. Let me just add that if you’ve read the report, and know much about the voluminous research in the area of gender and work, the statement is also so obviously true — and such a mild understatement — that this whole conversation is quite excruciating.
LikeLike
As I get up to speed in this debate, it sounds there are four positions:
A. Against the ASA statement
A.1 ASA should not support Bielby but should, instead, issue a public statement repudiating his work entirely as not scientific and (presumably) drum him out of the discipline. These folks say this would improve the reputation and status of sociology.
A.2 Don’t agree with Bielby’s statement and thus want ASA to stay out of the case entirely, rather than wade in on his side.
B. For the ASA statement
B1. ASA should straightforwardly back Bielby’s statement presumably because, like Phil, they see it as basically stating what is obvious to anyone who thinks sociologically
B2 Does not necessary back Bielby’s methods or argument but sees the statement as saying that sociology belongs on both sides of the case, that expert arguments for and against Bielby’s frame analysis are “science” on both sides, and that sociology as a field cannot let the “sociology is not science” claim stand.
Oh, I guess there is a fifth group,
B3C1 who believe in principle that sociology is a science that can be on both sides of a court case, but are critical of the specific language in the ASA statement and think no statement would be better than the one that was written.Do I have the players right now?
LikeLike
you said it. excruciating.
LikeLike
I wrote my last comment having missed some of the comments above. My apologies.
I have no problem with noncausal research. Describing what is observationally true about the world impresses me as quite worthwhile. I would argue that more often than not as sociologists we are making claims about how the world, and in particular, the social world works. To my mind, we are then making causal claims.
LikeLike
Just confirming what I believe you have been saying here and elsewhere, do I correctly understand that you are NOT arguing that it is illegitimate to advance causal claims on the basis of partial or circumstantial evidence? I.e. we all know the elementary school mantra “correlation does not prove causation” but you would not be part of the camp that has turned this into “it is unscientific and illegitimate to make causal arguments on the basis of correlational evidence”?
In legal jargon, I code you as arguing that causation should be inferred only if the evidence for causation is “beyond the shadow of a doubt” rather than “preponderance of the evidence.” I.e. that the burden of proof is on demonstration causation, otherwise no causation should be inferred? Or does the burden of proof vary depending on the issue?
LikeLike
First, I never ever said Chris Winship knew nothing about causality. Ever. Chris and I are having good civil conversations on the side because we both believe this is important for our discipline and it would be great if people would not make this snarky and personal (like blogs tend to do) so that some people are afraid to weigh in.
Second, this argument about method is the one that should happen in a Daubert hearing (in my humble, legally-trained opinion). That is really where Chris and I diverge. I think sociologists have something to say sometimes (so does Chris). I think sociologists should testify on both sides and the court can decide if it is admissible. Chris thinks the discipline should stay out of it. And he testifies to that effect in cases on the side of defendants primarily. And, he thinks that is a neutral place and that he is arguing for the good of science (I am willing to give the benefit of the doubt that this is all true – he seems perfectly nice.)
But from my vantage point, that “neutrality” further reinforces the existing gigantic disparity between plaintiffs and defendants in the legal system generally and especially in civil rights cases. As such, I think the “neutral” position is for courts to decide probative value and admissibility. And if Chris can beat Bill every time so be it (or vice versa). But our discipline deserves to be in the game.
As I said to Council in one exchange — we might disagree in a medical malpractice case about what one doctor or the other says, but nobody thinks doctors shouldn’t be involved. Right? That is ALL the brief says. If you can find a sentence or two that suggests more, then there it is. Legal briefs are not written over years like academic articles. And by the way, the court is mostly going to focus on the Rule 23 jurisprudential question here.
LikeLike
Laura: Is there some way to get you to defend your claim that the scientific status/legal credibility of sociology as a science was under attack? I have repeatedly asked you this question, but cannot find an answer from you or anyone else. I am sorry that this is “excruciating” for you, but as the primary author of the ASA’s amicus brief, I don’t see why it should be painful to respond to this question– since it is the main basis by which you and the ASA leadership justify the writing of the brief.
LikeLike
see here: http://orgtheory.wordpress.com/2011/05/20/wal-mart-and-beyond-can-social-science-be-itself-in-court-response-to-professor-winship/
LikeLike
I agreed with someone that it was excruciating given what we know in the field – that was regarding the substance of the findings in the field not the procedural questions which are really Council’s to answer, not mine.
second, I was an author along with others. Maybe primary maybe not — probably primary.
Third, we answered on org theory. But, you may still after reading all the briefs (and you should read both sides) still not agree that it was a smack against sociology. That is your read. Apparently Council also agreed since they asked us to write it. You will still have your read, I don’t really have an answer except that I know how legal briefs are written and how lawyers argue. All the lawyers int eh case and many of the justices agreed it was at stake (insofar as they bothered to ask about it in the very circumscribed oral argument).
I’m leaving the process questions to Council.
LikeLike
I’m sorry that you are taking so much of this heat, Laura. Certainly the other authors of the report are as accountable for the brief as you are. And certainly you are not accountable for the process at Council or the misrepresentation of the petitioner’s brief to ASA members in Footnotes. You deserve our appreciation for getting online to defend how you approached the brief. I hope the members of Council (who have been apprised of these discussions) are very appreciative of your willingness to be the lightning rod. I know I am. I also hope they step up and join you in presenting their account.
(I also hope they have apologized for giving you the impression that the vote was unanimous, leading you to write in your SSRN reply to Mitchell that “ASA’s elected, 19-member Council voted unanimously to sign on to the brief.”)
Two additional notes:
(a) There is no daylight between your representation of the petitioners’ brief as an attack on sociology as junk science (“Do you think of research that you do as science or anecdote?”) and Hillsman’s characterization in Footnotes (to be sure, you did not misquote from the brief as Hillsman did). And there is no daylight between your characterization and that of colleague and co-drafter of the brief Amy Myrick (“Wal-Mart attacked Sociology. The ASA responded.”) This leads me to infer that this characterization of Wal*Mart’s brief enjoyed wide currency among those who were involved in drafting and approving the brief. And I am having a very hard time understanding how all these eyes on the process misread an attack on Bielby’s method (as used to support the certification of the class) as an attack on sociology. So again, you (and Amy) have been the lightning rod. But rest assured that the questions are posed to Council (and your other coauthors as well). I hope they take up the idea of holding a panel at ASA to discuss. I can’t see any reason why they would not want to.
b. Unsurprisingly, you do not concede that you and the others misread the petitioner’s brief. But I guess I’m stunned but you offer nothing by way of rebuttal (“don’t have an answer”) to my analysis of the petitioner’s brief (http://orgtheory.wordpress.com/2011/05/18/walmart-and-the-asa-a-guest-post-by-chris-winship/#comment-96034)other than “that (you) know how legal briefs are written and how lawyers argue.” Really? Have I really waded into an area that is so hard for a non-lawyer to understand, that it is not worth trying to explain it? That is really too bad because I really do not understand what you are telling me, and I would really like to. You say that “all the lawyers in the case and many of the justices agreed it was at stake (insofar as they bothered to ask about it…).” But as I pointed out in that comment and the earlier one (http://orgtheory.wordpress.com/2011/05/18/walmart-and-the-asa-a-guest-post-by-chris-winship/#comment-95993), the justices did not in fact give any indication that the scientific status/legal credibility of sociology was at stake (rather, they made very reasonable inquiries on substantive issues for which none of us [including Bill Bielby] have clear-cut answers), nor do the other amicus briefs suggest that sociology’s status was at stake. So essentially you want the case (for how to interpret the petitioner’s brief) to rest on an argument from authority. [**FN below] You are a lawyer, and that’s how you and other lawyers see it. Indeed, even if it looks to this layman like none of the lawyers are seeing it this way (These law.com interviews with the lawyers who argued the case before the SC are instructive [http://www.law.com/jsp/nlj/PubArticleSCI.jsp?id=1202487546136&slreturn=1&hbxlogin=1; and http://www.law.com/jsp/nlj/PubArticleNLJ.jsp?id=1202486349252%5D the word sociology does not show up in either of them), you assure us that this is indeed the case.
But if we really think that non-lawyer/sociologists cannot understand these legal issues, it leads to all kinds of thorny questions. Shall we just take it on faith when lawyers tell us that a legal matter bears on our professional reputation? And then how do we expect the ASA to decide whether to file an amicus brief on a given case– should it just listen to whatever lawyer it has retained? What if it happens to talk to two lawyers and they disagree? Sorry, but just as social science enters the court on the court’s terms, the law must enter social science (and that of the layperson generally) on laymen’s terms. If you want us to work with you, you have to educate us so that we can interact over that divide. I am still eager to be educated. I understand that you do not want to be the lightning rod anymore, so I will not expect you to engage further on this– but I would be eager to read anything you write. I do hope that *Council* sees this as an opportunity for us all to become better educated and to improve the process by which such issues are approached and decisions taken.
**FN: I’m not sure what to make of you and Phil finding it excruciating to explain things to those of us who apparently don’t have the command of “the field” that you enjoy. This is again an argument from authority, and a patronizing one at that. I would hope that if we are defending sociological science (and we are now in a conversation among scientists with no jurisdictional divide to worry about), it is inappropriate to say “if you knew the literature like I knew it, then you couldn’t possibly utter such nonsense.” If it’s nonsense, please spell out why. We can take it– and indeed, are eager to be educated.
LikeLike
Ezra, this seems obvious from the case against the class certification. The petition (I think that’s what you call it, http://bit.ly/mR5RAw) reads:
Plaintiffs moved for class certification, alleging
(as they had in their complaint) that Wal-Mart “fosters or facilitates gender stereotyping and discrimination, … and that this discrimination is common to
all women who work or have worked in Wal-Mart
stores.” … They were unable, however, to
identify “a specific discriminatory policy promulgated
by Wal-Mart.” … In “the absence of”
such a policy (ibid.), plaintiffs premised their motion
on statistics, sociology, and anecdotes.
How do you interpret that argument?
LikeLike
To me, this isn’t that much of a slap against sociology. It is not as if they wrote: statistics and sociological anecdotes.
I guess we need to see the context. Based on this, it seems they have stuck us in between something respectable (statistics) and something else (anecdotes). I’d be convinced if they also sneer at “statistics” elsewhere.
LikeLike
I think I am with C1 in OW’s earlier post. My quick position as I type this in:
(1) sociology is a science, and indeed lots of science is observational and wrestles with how to make causal claims
(2) Bielby and Winship have scientific judgment, and they are trained as scientists, and should feel free to offer their judgments as scientists in court cases
(3) the ASA should not side with one group of scientists rather than another group of scientists UNLESS the ASA does a great deal more to reach out to all competing sides than was the case here
(4) the brief itself is incoherent on the important issue of causality that is at the heart of all of this
Were I on council, I would have argued: The ASA should write a short brief that doesn’t engage the merits of Bielby’s position at all. Instead, if the ASA wants to write a brief, then the ASA should write a brief that says that sociology is a science and nearly all sciences have internal debates about causal effects. In fact, the ASA could praise sociologists who engage in this sort of work as an important form of public sociology. And the ASA could use other examples (and surely there are many from psychology and economics) to show that what is going on in this case is probably very similar to what goes on in lots of cases. Social scientists come in on both sides a lot, express alternative judgments, all of which may or may not be helpful to anyone.
LikeLike
typo in my own message: replace “The ASA should write a short brief that doesn’t engage the merits of Bielby’s position at all. Instead, if” with just “If”
LikeLike
Hi Jane – if you were on council you probably would have gotten that brief because we were very interested in unanimity. Though someone recently asked why on one of these blogs and I don’t have a good answer for that except that it seemed to represent the disciple better. And the lawyers probably would have taken that brief.
I/we consulted lots of scholars. I did check in with chris on one point writing the brief. But anyone on council could have forwarded him the whole thing. As far as I saw there was never any confidentiality admonition. But why not Laurie Edelman or Frank Dobbin or Bob Nelson or Robin Stryker or Erin Kelly or Elizabeth Hirsch or Paula England or Barbara Reskin or Jim Baron or Devah Pager or (you get the idea)?
Asa asked ME. I asked loads of people who work in the area to help and told them to forward to “anyone who would be interested in working on this” because I was already working on the IWPR brief and the conditions of these things are that you have about 3 weeks (maybe 4). And getting 19 people on board and making all the (often contradictory) compromises takes a lit of time.
LikeLike
“Why not Laurie Edelman or Frank Dobbin or Bob Nelson or Robin Stryker or Erin Kelly or Elizabeth Hirsch or Paula England or Barbara Reskin or Jim Baron or Devah Pager”
I agree that they should have been given a chance to consider such an important statement from the ASA too. Did you give them a chance?
This whole discussion has gotten Winship focused because he is out in front in being forthright, and because he knows more about causality than anyone on your list (or Bielby or Tomaskevic-Devy or you or your graduate student). And you surely knew that he would have objected to the brief, and so it looks like you didn’t show it to him for that reason.
But, I agree that this is not any one person’s responsibility. Council represents the discipline, and it didn’t consult widely enough. Perhaps you were misled by council into thinking that no one would object to such a brief, but you seem pretty smart to me! So, the end result is that it looks horribly one sided, and sociology is now out there in front of the supreme court looking lame. I think we are going to get smacked down, and I think that does more damage to the image of the discipline than the Walmart brief.
LikeLike
The idea that losing makes this wrong strikes me as quite petty. I venture to speculate that anyone who thinks that makes the discipline looks bad already thinks the discipline looks bad. But also, isn’t the Council elected?
LikeLike
Philip Cohen (sorry, I don’t know you, and don’t know if you should be Phil):
Remember that my position is that the ASA shouldn’t have written a brief, or if they felt compelled to do so, it should have either been a broad-based one and/or contained a narrower argument. So, it is not petty to say that wading into something, and having it backfire, is not a reason against having stupidly waded in. It surely is a reason to stay out. The ASA simply made a possibly bad reputation situation into a certainly bad reputational situation.
Anyway, that’s my view, and I guess we can all judge after Roberts and Alito and Thomas have their say. We can then ask: Did the ASA brief help, hurt, or have no effect (and hence was a waste of time and money)? I suspect it will either hurt or have no effect, so at best it was a waste of time and money.
But … you are absolutely correct on one point. Surely most of the people annoyed by the brief are annoyed by the falling scientific standards of the discipline, and that the ASA leadership seems to be exacerbating the problems. My overall position is that (a) all is not well in sociology but (b) Walmart is probably still guilty. What’s the solution? The ASA is not going to have any effect on the outcome in the Walmart case, in my view. But, we can fix sociology, and so let’s stop this lame ass intellectual weakness we are suffering from and get serious about upping our game.
LikeLike
yes, jane, I emailed a ginormous set of sociologists including, I believe everyone on that list, on this and asked them to please feel free to forward my plea for help to ANYONE they thought would be interested in working on it. This was WAY before we had even an outline. Frankly I was begging for help – it is an extremely time-consuming thing to do.
I’m not going argue the Council process points — I am not on Council; I was asked to do some very hard and time consuming work for the organization of which I am a member and I said yes because I thought it was important (still do).
Again, fundamentally the brief says that the question of scientific consensus should be before a court. NOT for ASA council, not for Chris Winship, not for me (therefore the quite snarky remark about what I do and do not know about causality seems completely off the topic).
All we are asking is for the court to decide. And trust me, Chris or someone else will not only get to make that point if and when the case is remanded, they will get PAID to make that point! Lucky them.
Keep in mind, this is the same argument made by the many of the defense side briefs.
But really nothing I am going to say will put your mind at ease and I guess you fundamentally disagree that ASA should ever file a brief. I get that. OK. Run for Council. Vote for people that agree with that position.
But when Wal-Mart, consulting firms, and high power members of the defense bar are trying to say that sociology is not relevant because case study methodology is not scientific in a question like this, and Council asks me to work on something, I think it is important for our discipline to say something about what we can and cannot do.
You disagree. that’s really ok. but I did not do this in secret behind anyone’s back. So please, if you could try to stay calm and not accusatory, that would really help everyone have this important conversation.
LikeLike
Laura:
So, just a final clarification: Are you saying that you sent an appeal for help to lots of people, but not Winship. Then, after that, you only shared drafts and solicited comments from council, which includes Tomaskevic-Devey, apparently also a frequent expert witness. But you didn’t share drafts with the larger group and never asked Winship for input at any point?
Now, a note of clarification from my side: you say above that I would never support the ASA ever filing a brief. I know it is hard to keep up with these blog discussions, but I never said that. I’ve said in several places here that if the choice were between the the brief that was submitted and no brief, it should have been no brief. I can imagine other briefs that could have been written that would have been worthy of submission.
On running for council, the funny thing is that no one has nominated me. Go figure.
LikeLike
Thanks, janecausal, for pointing out the, ahem, *awkwardness* of claiming that because something was approved by our Council and Council members are elected, it is therefore representative of American sociologists (as you indicated was stated in first paragraph of the amicus brief), let alone ASA members. Sorry, but there are many aspects of how ASA is governed that deviate from what is considered best practice in organizational governance. I have mentioned the issue of staggered board terms. The issue that janecausal raises– that candidates for office are restricted to those selected by the nominations committee– is at least as problematic. No doubt that there are reasons for having such a committee, especially in the old days when it was impossible for members to caucus outside the annual meeting. But having no process by which a member might become a candidate outside the nominations committee is an obvious deviation from democratic principles. Kanter’s term “homosocial reproduction” comes to mind. At the very least, ASA statements should be clear that ASA only represents its members rather than all U.S. sociologists (many of whom choose not to join ASA and are therefore not represented by it) and given its governance structure and its processes, there are serious questions as to how well it represents us members.
LikeLike
As a friendly aside, you actually can run for ASA office without being nominated by the nominations committee. A mere 50 petitioners can put you in the race for Council! See by-law II.2.b.
A friendly aside to that aside: As long as I am an ASA member, I suspect I would sign for anyone who wishes to run for office via this route. Only 49 to go!
Also, I’m pretty sure Don T-D’s time on Council ended when his secretary/treasurer term did.
LikeLike
Thanks Jeremy. That is an important point. I should have read the bylaws carefully before making that charge. And now that I know that it is possible to do this, I am making up “janecausal for ASA President” T-shirts, with the tagline:
“We can fix sociology, and so let’s stop this lame ass intellectual weakness we are suffering from and get serious about upping our game.”
LikeLike
janecausal perfectly expresses the key problems here. There are at least two separate debates: 1) whether the ASA should have written an amicus brief in this particular case; and 2) whether the brief that was written was a good one.
Apparently some find it excruciating that some people should think that the answer to either or both of those questions is no. But it is worth considering that some find it excruciating that the Council’s answer to both of those questions was yes.
One can support the notion of writing an amicus brief to defend the status of sociology as a science without thinking that it had to be done in this case, and while thinking that the brief could have been much better, exactly in the ways outlined by janecausal. I am not totally convinced that sociology was under attack here, and I am very skeptical that this is the battle that I would pick. But perhaps I could have been convinced. I do however find the quality of argument in the brief embarrassing, do not like being associated with it, and fear that it will do more harm than good.
To return to Jeremy’s original post, this raises real questions about the functioning of the ASA Council. Consider the counterfactual. What if a sociologist had testified for the defense, and no sociologist had weighed in for the plaintiff, but the plaintiff had dismissed the sociologist’s testimony in the same way. How many people honestly think the ASA would have weighed in to defend sociology as a science in that case?
LikeLike
Imagine it is 1990. What if some group sued the Department of Sociology at the University of Wisconsin-Madison, alleging that they have been harmed because Wisconsin’s male-dominated, big-ten, beer-on-the-lakeside culture has led to lower rates of hiring of women, compared to pipeline, from 1901 to 1990? (If you are offended by picking good ole Wisconsin, go ahead and pick a department where the gender disparity was and still is greater. You won’t have a hard time thinking of one. Chicago, anyone?)
The plaintiffs then hire Barbara Reskin as an expert witness (Bielby, of course, had a conflict of interest, having been out of that PhD program at Wisconsin only a few years earlier and had probably been seen at the union drinking beer and playing in some all-male band).
The smarmy lawyers for Wisconsin go after Reskin’s opining in exactly the same way that Walmart’s lawyers went after Bielby. But this is a trial now, and so Wisconsin hires a statistician of the highest order (say, in fact, that they are so brilliant that they pick David Blackwell). He is then brought in to argue that Reskin’s testimony goes too far. He argues that her “contributes to” statements are based on assumptions with only theoretical support and empirical results never shown to hold in sociology departments, and that she provided no analysis of Wisconsin’s hires, applicant pools, and so on. In fact, when pressed, she admitted that she couldn’t say whether 15% or 85% of Wisconsin’s hiring decisions were tainted by discrimination. She simply said that she was sure that the number is greater than 0% based on her professional judgment and her knowledge of the impact of male-dominated cultures.
Should the ASA offer a brief defending Reskin against Wisconsin’s lawyers and their statistician? Or should ASA stay out of the fracas and let the court decide the merits on its own? If the ASA decides to write a brief, should they talk only to Reskin and other sociologists who have testified like her in other cases?
LikeLike
I’m not sure, but I think the point of JaneC’s Wisconsin story is about how to write a better anti-discrimination brief. It did remind me of what happened in 1970. This history is pretty well buried and I did not remember the details exactly right except that it involved the University of Michigan. My Google search turned up little except biographies of feminists until I found this account on an MLA web site which sums up what was happening:
http://www.adfl.org/bulletin/V03N3/033043.HTM
(There are probably better accounts somewhere in the academic literature. EDIT: For those of you wondering whether to click over, the link has a very long story in the second half about the University of Wisconsin in 1970-72.)
Short summary of the story: women academics filed complaints with Dept of Labor demanding that the executive order requiring federal contractors to avoid discrimination be enforced with respect to university treatment of women. The case was transferred to HEW, which started demanding salary data from universities broken down by sex. This threat of loss of federal funding got female academics’ salaries and the wages of university clerical staff increased virtually overnight.
Unbeknownst to universities, their employees were secretly giving federal investigators information about what questions to ask. In 1970 the line-up of who thought this was a good thing and those who thought it was terrible pretty much coincided with one’s demographic status among academics.
Notice that the question of the cause of the pay differentials was not even raised. The differentials themselves were taken as problematic. Just asking for the data under threat of loss of federal contracts was enough to generate a fast response on the part of universities as employers. These days, refusing to collect data and then claiming that there are no data to support allegations of discrimination is the first line of defense against possible charges of discrimination.
BTW 1990 is not far off. Quoting from the Wisconsin State Journal, February 2, 1993:
“UW-Madison female faculty members should get an additional raise this year averaging 3.8 percent, to resolve gender inequities, the Faculty Senate decided Monday.
The motion, which passed easily in a quick show of hands, needs only the approval of Interim Chancellor David Ward. He and former Chancellor Donna Shalala have been calling for gender equity pay since last September, when a study showed female faculty members made between 1.6 percent and 6 percent less than their male counterparts.
The higher figure represents the gender pay gap when differences in rank are ignored. Some researchers believe that is appropriate because female faculty often are hired at lower rank and advance slower than their male counterparts.
The Faculty Senate arrived at the 3.8 percent figure by splitting the difference between 1.6 percent and 6 percent.
. . .
Finally, an overall gender committee for UW-Madison will be appointed to study whether this year’s equity raises fully remove the bias in faculty salaries and whether they remain unbiased. An adopted amendment also calls on the committee to study how to resolve the lower pay found throughout departments dominated by female faculty members.”
Edit: My point is that the “causes” issue comes up only if you dispute that descriptive evidence of differences is not sufficient to justify remedy and instead insist that there be evidence of discrimination as a cause. Whether you think descriptive evidence is enough is often correlated with (although certainly not perfectly) one’s own interests in the distribution in question.
LikeLike
I have read the Bielby report, and parts of the Wal-Mart petition and the Supreme Court oral arguments. What is striking to me is how LITTLE issues of causal inference are at stake in the legal proceedings.
Wal-Mart is basically saying that the plaintiffs need to prove either that (a) sex-based animus was involved in hiring/promotion decisions (see page 29 in Wal-Mart’s petition (http://bit.ly/gns0fM) or (b) that Wal-Mart had some specific practice or criteria that specifically penalized women (see pages 12-13 in the SCOTUS oral arguments, the comparison of Wal-Mart’s practices to AT&T’s “total person test”, http://bit.ly/hfvQfp).
Some people here are saying that Bielby did not do a good job of proving his claim that Wal-Mart’s practice of giving discretion to managers resulted in disparate pay/promotions along gender lines. Fine. But let’s say he did prove his claim, by whatever standard we deem appropriate.
Even in that case, it looks to me like Wal-Mart would be arguing the same thing it is arguing now, because Bielby’s argument does not talk about the role of conscious sex-based animus, it does not talk about specific discriminatory practices. As Myrick points out over at orgtheory, Wal-mart is arguing for an impossibly high standard of proof that could only be met if (a) the plaintiffs had mind-readers or (b) if the employer/manager was incredibly dumb and somehow went on the record describing sexist attitudes/practices.
Nielsen and Myrick have a point when they say that Wal-Mart is attacking sociology, although I’d phrase it differently: Wal-Mart is trying to define the rules of evidence so that sociological arguments are considered irrelevant and invalid.
LikeLike
— Josh,
I am sort of back on line. Hard to be involved in extended discussions on two blogs while traveling.
I hope one of the lawyers will respond. My understanding is that punitive damages are only possible if one can show intent — that is one has a disparate treatment case, as opposed to a disparate impact one. Plaintiffs are asking for punitive damages and Walmart is objecting that they haven’t shown intent. If this is correct, Walmart is not arguing for an impossible standar; it is just arguing in the framework of the law as it is now written. That, of course does not mean that the standard within the law for punitive damages is reasonable.
LikeLike
Phil: My answer to your question is here: http://orgtheory.wordpress.com/2011/05/18/walmart-and-the-asa-a-guest-post-by-chris-winship/#comment-96034 (along the lines of janecausal’s comment in 14).
I agree with Jimmy (i.e., the disgruntled sociologist) that janecausal perfectly expresses the issues. I have tried to summarize them in that comment cited above, where I propose that the ASA hold a panel on the matter in August.
I also love janecausal and Jimmy’s counterfactuals. Very important food for thought there. The ASA should obviously have stayed out of this.
LikeLike
I am working on a new post for orgtheory. Not sure when it will be finished. I wonder if someone wants to try to summarize the conversations on both the orgtheory and scatterplot posts about where there is agreement and where there is disagreement.
There has been an email exchange among ASA Council members that some of us have been copied on. I am not sure that I or anyone else has the right to share it publicly. It does make clear:
1. That at least two perhaps more individuals abstained from the vote.
2. Apparently, no one the Council read Bielby’s expert report.
3. At least some Council members felt that the process was rushed and that the Council did not have the time fully consider it.
4. Some people on the Council have qualms about what was done.
In terms of the extended dialogue on the two posts it is clear that some individuals believe that sociology was being attacked by Walmart, and as a result it was necessary that ASA submit an amicus brief defending sociology as a science. Others can find no evidence that demonstrates to them that sociology was being attacked.
People disagree that about whether the ASA should every file briefs.
We have yet to have a detailed description of the process by which the brief was written, who participated, how did the authors interact with the Council, who else was asked to review the brief and comment on it. In addition, we have yet to have a detailed description of the Council’s deliberation process. Those individuals who believe that what was done was fully legitimate owe the discipline these detailed descriptions. If what was done was legitimate then there should be nothing to hide. (I will describe my somewhat complicated views on this in my next posting).
LikeLike
Thanks Chris. I think another critical question is who on Council, if anyone, had read the petitioner’s brief, and how “statistics, sociology, and anecdotes” became “sociology is ‘statistics’ and mere ‘anecdotes’.”
LikeLike
We have yet to have a detailed description of the process by which the brief was written, who participated, how did the authors interact with the Council, who else was asked to review the brief and comment on it. In addition, we have yet to have a detailed description of the Council’s deliberation process. Those individuals who believe that what was done was fully legitimate owe the discipline these detailed descriptions. If what was done was legitimate then there should be nothing to hide.
This is the second time in two months that we are having a discussion about A Supposedly Unanimous ASA Council Vote That Maybe Wasn’t, which is a bit tiresome although it’s not clear where to assign blame.
I have a hard time understanding without more context why multiple people would abstain on a vote like that; seems like something for which it would have been especially important to get votes of YES or NO. I would like to hear a rationale from an abstainer that allayed my suspicion of it just being a vote of trying-to-have-it-both-ways.
In any case, I agree that what Chris asks for above is desirable, and not just for ASA members, but it seems like it would be only fair to Laura Beth and others who spent so much time working on the brief, for there to be a clear and public account of how Council conducted its responsibilities all along this process. I would feel especially surly if there are undermining remarks going around from Council members suggesting more dissensus than what had been communicated before, if these had not been fully reported earlier to those who were involved in drafting the document.
LikeLike
Jeremy: I don’t get why you have such a problem with people who abstained. In my view, that was preferable than voting for going ahead. What is so bad about abstaining if you don’ think you have had enough time or information to make a determination?
LikeLike
I don’t get why you have such a problem with people who abstained. In my view, that was preferable than voting for going ahead.
You present this as though voting yes or abstaining were the only options. Why not vote “NO”, or vote “This is really important; I have mixed feelings and haven’t had enough time to think it through; until I have, NO”?
Am I asking for unrealistic levels of pluck here? Have I spent too much time reading Profiles in Sociological Courage?
The reason I have a problem with abstentions is that it’s the easy vote if one doesn’t want to be a squeaky wheel, but wants to retain the option value to disavow the decision to others later. Importantly: I’m not saying that was the rationale here–especially as I don’t even know who the abstainers were–but I’d be interested in hearing what the rationale was.
My understanding–feel free to correct–is that this brief was initiated by an ASA Council action, that Laura Beth and other nonelected members put a considerable amount of pro bono work into it (including work precisely toward producing a consensus document), that Council had every right of revision and refusal over it, that Council certainly could have contacted other people known to have views on these issues, that Council approved it in a way that could be characterized as unanimous, and that now some Council members are talking about their “qualms” and the “rushed” process.
Granted, I’m not entirely sure with whom I should be angry with about this–I don’t know anything about what the voting process was like, and I’m not privy to these backchannel conversations–but it does feel like Laura Beth has been treated badly here.
LikeLike
(This is a reply to this–https://scatter.wordpress.com/2011/05/19/discrimination-briefly/#comment-12121– post of Jeremy’s. It looks like scatterplot is going to post it over it…)
Jeremy: I’m in full agreement with you that Laura Beth has been mistreated. (I have repeatedly try to post a comment in which I say this myself– it must be stuck in the queue. Could you please release it?) In particular: (a) Council could make this a lot easier on her if they were to provide a forum at the ASA in which the substance of the amicus brief and the process by which it was generated would be discussed; and (b) I think someone at Council should apologize to her for having mischaracterized the vote as unanimous. You say that a vote with two abstentions “could be characterized as unanimous,” but it darned well can’t be characterized this way (from Laura Beth et al’s SSRN response to Mitchell): “ASA’s elected, 19-member Council voted unanimously to sign on to the brief.”
Otherwise, I’d say that you have given good reason, given what is publicly known, to be tough on the abstainers. You don’t want to leave them off the hook. Fine. But let’s be sure to remember that the other 17 are already on the hook, and that there are some serious questions for them to answer.
LikeLike
Kudos to OW for a spot-on analysis. 15 years ago, when I was the “statistical expert” for the faculty union (along with a statistician representing the administration) at a different school, we both agreed that the patterns we were finding — though not passing through the narrow statistical needle’s eye to qualify as unambiguously “causal” — were shocking, and violated the norms of what a “fair” system should be producing. In particular, there was an abrupt drop in the women’s distribution of salaries just around the point where the long tail of “elite” men’s salaries started (controlling of course for all the obvious and quite a few non-obvious causal alternative explanations) and vice versa at the bottom. This “description” generated not merely a salary adjustment (since the means of the whole distributions were also skewed) but also a targeted set of rewards for high-achieving women and a conscious decision not to (a) try to go backward and fix what were undoubtedly YEARS of underpayment and (b) do anything about the over-rewarded men at the bottom, but to set a floor for each rank (which as I understand it, is now often the case in many universities) to prevent the double-whammy of being a low performer AND a disrespected group member ending up holding some people at unconscionably low salaries without having to find some acts of “discrimination” or argue that they deserved “mean” salaries for sub-mean performance. Not that there WERE more women at that low end; the numbers of women and men at the respective ends of their distributions were proportionate, and when the bottoms and tops were removed from both, the means for those in the -2 to +2 range of the distributions were still skewed in favor of men (with all the many controls). So I am still willing to argue that the distinction between what is “causal” and what is “descriptive” in these matters is at least as much a matter of how one thinks that organizations left to their own devices usually operate in a climate of status attribution inequity favoring men (as demonstrated by Ridgeway and others experimentally/causally). Since I think sociology (not statistics alone) has had a lot to offer to show both how status attributions and organizations work, I think there is warrant to generalize (which is what I understand moving from a principle to a specific case includes) even if the principle won’t give a number in a specific case where there are other unknowns in the equation. But to me, to argue for “it’s got to be either strong causal evidence of a specifically quantifiable effect or accept the null” seems to reject the most plausible, systematic and sociologically grounded hypotheses of known causal processes out of hand, despite “descriptive” evidence of their working in the specific case. That seems to elevate technique over substance (aka theory) in an ultimately unscientific way. If sociology is indeed scientific, it should be made up of more than a set of statistical techniques, and Council should be prepared to say so. It is not “just politics” that leads sociologists to believe that there are processes underlying how the world works, even when our statistical techniques and the (politically withheld/uncollected) data are not always up to the task of getting into all the causal mechanisms in detail.
LikeLike
I think there’s an assumption with some of these claims that by doing something the ASA engaged in political behavior, and thereby its decisions were not “scientific.” Here there is an implied opposition between (assumedly political) action and science.
This might be correct. I don’t know how council made its decision. However, I would point out that NOT doing something is also a political decision. So while I think it’s useful to think through how council comes to its decisions, I think it should be noted that the claim that ASA should do nothing is as political a claim as that it should do something.
In my view, the question as grouchosis and OW have noted, is largely over a set of epistemological claims as to what constitutes sociological knowledge, and not whether or not the ASA is a politicized organization.
LikeLike
That other ASA did not write a brief defending statistics as a discipline or set of methods. Was its inaction also a political act? Or is inaction by (our) ASA only a political decision because it has engaged in political action in the past and some members/leaders/staff of the ASA see political action as part of the organization’s mission?
My sense is that part of the disaffection over the Wal-Mart brief is because it’s viewed, whether fairly or not, as another example of the ASA’s drift away from being a scientific association and toward being the organizational agent of “public sociology” in the critical / activist sense of the word.
My own view, not that anyone cares, is that this mission creep is a mistake, and not just because being it justifies / requires an expensive Washington DC address and staff. The ASA can, and should, advance the cause of sociology as a science by (a) facilitating and supporting more rigorous science, even if that means being less supportive of weak science or non-scientific approaches than it currently is; and (b) doing the hard work of communicating the scientific research of members to the general public and other constituents (recognizing that not every sociologist has the ability or resources to do this work themselves). It should not jump up and down and say “we’re a science, we’re a science” in one breath, and, in the next, write science-free statements condemning political activities or views that clash with the typical political preferences of members. (I’m thinking less of the Wal-Mart brief here than of the resolution against the Iraq War or the statement condemning Glenn Beck/Fox News’s vitriolic reaction to Piven’s editorial in The Nation.)
LikeLike
Laura suggests above that I was really quite unfair to her in suggesting that she knew nothing about causality. I don’t recall writing that, and my point is that the brief gives an impoverished view of what counts as a valid explanation in sociology. I really have no idea who wrote the brief and I don’t mean at all to make statements on any particular individuals.
Let’s focus, instead of on the authors and their knowledge, just on the content of the brief that represents all of us sociologists (indeed, as is implied in the opening paragraph, where there is a nice shout out to all 14,000 of us including nearly all of us with PhD’s at accredited institutions!).
Just consider this paragraph and what its purpose is supposed to be:
“In sum, contrary to the defendant’s claim that Dr. Bielby’s methods “lack reliable, scientific basis for linking general research to the corporate setting,” Pet. Br. at 30, the methods Dr. Bielby used are those social scientists rely on in scientific research that is published in top-quality peer-reviewed journals. Dr. Bielby’s analysis of Wal-Mart is presented as an example of the application of sociological knowledge in the newest edition of an important sociology textbook. A. Giddens, M. Duneier, R. P. Applebaum, and D. C. Carr, INTRODUCTION TO SOCIOLOGY (7th ed. 2009).”
This is the closing paragraph of the section that argues the point in heading I “SOCIAL SCIENCES IN GENERAL, AND SOCIOLOGY IN PARTICULAR, PROVIDE SYSTEMATIC AND SCIENTIFIC RESEARCH METHODOLOGIES FOR ANALYZING INDIVIDUAL AND ORGANIZATIONAL BEHAVIOR, AND SHOULD BE CONSIDERED ALONG WITH OTHER SOURCES OF EVIDENCE TO INFORM LEGAL ANALYSIS”
Really? A recent addition as an example in an introductory textbook is something that the supreme court should know about? I know this paragraph is preceded by what is otherwise a quite reasonable review of literature on corporate culture (as far as I can tell), but closing with a citation to an introductory textbook as a source of legitimacy of some form?
LikeLike
Really? A recent addition as an example in an introductory textbook is something that the supreme court should know about?
Um, yes, really. What better way to argue that something is mainstream than to say that it’s taught in the simplest of places: a textbook? It’s so uncontroversial we teach it to undergrads (and even high school students) when we introduce them to our discipline.
I don’t mean at all to make statements on any particular individuals.
I would encourage this. Rather than make broad assumptions about who knows what about causality, I’d encourage you, as Laura Beth does, to tone down the rhetoric.
LikeLike
Shakha: I will let janecausal respond to you on the textbook issue since I’m sure she will respond well.
As for toning down rhetoric, sounds good, but I have trouble with the implication that it was janecausal who needs to tone rhetoric down. Someone sent me the following from Facebook: “I especially love that I am a manipulative bitch in this story. yay me!” followed by “Oh a manipulative bitch who gave 6 weeks of her leave year because I think my discipline is worth a crap.” Really? I think it is just horrible to throw around the b-word with such abandon in a situation like this, and I would ask Laura to retract that. Meanwhile, no one has accused her of manipulating anything– if anything, it seems more likely that Laura was manipulated– but who knows. And as for giving a crap, that is really rich. I’m sure that Laura does give a crap about sociology. But why exactly do you think Chris or janecausal or I are spending time on this? The issue is not who cares more about sociology. The issue is what is the right way to care for it.
(I find it interesting that the facebook post , which reportedly includes Bielby and Phillip Cohen making snide remarkers, is made under the assumption that the only people are on the “right side” in all of this could be friends or friends of friends of theirs. Sorry, but this is indeed a debate among friends, and why not keep it on a fully public forum?)
LikeLike
@ezra: there’s a difference between a conversation between friends (which is basically what facebook wallposts are) and a discussion on a public forum. People vent to their friends. That’s perfectly reasonable. To bring that up as if you have been aggrieved is a cheap shot. I get that you want to win an argument. And I guess that trying to undermine others by what they have said to friends in other places is a strategy for doing that. But it strikes me as a low blow.
LikeLike
EZ: Wait a minute, you are now claiming to be offended about rhetoric because someone forwarded to you a message from a different forum that was never sent to you in the first place in which that person expressed frustration and used a word you don’t like??? Especially as the context makes it sound like the word you don’t like was being applied to the person herself. Come on, get a grip. People are allowed to have their own private opinions. If you get upset at the idea that people talk about what you said behind your back when they are speaking with their friends, then you don’t want to say things in public. That’s what happens in public: you say stuff, and other people talk about it. (I personally get taken down on right wing talk radio every time I get quoted in the newspaper.)
On a sociological note, some of us think that any reasonable sociologist would think that the ASA memo is just summarizing what any reasonable sociologist would say and that the people who are criticizing it (that is, the content of it) have a political or personal axe to grind. Remember, the point at issue is whether Wal-Mart should be treated as a class in the face of statistical evidence of differential probabilities of promotion for women in Wal-Mart relative to other comparable firms, not whether Wal-Mart has been proven to discriminate. The idea that a national company can be assumed to have a common organizational culture and context that should be treated as a single entity is not exactly radical and certainly not unsociological.
I do recognize that there are people like Jeremy who are taking the position that a memo about which actually existing sociologists disagree should not have been signed on behalf of the ASA as a body.
As a Congregationalist, I am familiar with this problem and sympathetic to that position. No Congregationalist can speak for any other Congregationalist. This does not prevent Congregationalists from taking a wide variety of strongly political positions, however. They sign them as themselves. So, for example, the ASA Council could have signed the brief as “Members of the ASA Council” or even as “The ASA Council” and a Congregationalist would have been satisified. I wonder whether this would satisfy Jeremy.
The debate about whether a scientific or professional association should or should not take “political” positions, and whether a friend of a court brief counts as a political position, is a different debate than the one EZ and JC have mostly been addressing. That sort of debate would not involve criticisms of the brief itself, discussions of causality, or strange scenarios.
Another very interesting sociological debate would be about why (as LBN points out) individual plaintiffs lose discrimination cases, while class action suits win. The answer, of course, is that you can always tell a story about why a particular plaintiff is a nut job — I’ve known a few myself — while statistical patterns are much more damning.
LikeLike
I do recognize that there are people like Jeremy who are taking the position that a memo about which actually existing sociologists disagree should not have been signed on behalf of the ASA as a body.
Whoa! I said in the original post that I was staying out of this one. I meant it. My contributions to this thread have been on side issues only.
LikeLike
oh, sorry I misunderstood your position.
LikeLike
Shakha: I don’t participate in faceebook and so was unaware of the etiquette. If it is truly understood to be a private forum (I guess all the talk about facebook being used to coordinate collective action, etc, put me in that frame of mind) then I apologize for having repeated something publicly that was said in private.
LikeLike
I accept your apology. Coordinating collective action on FB is usually done on Fan Pages or Group pages. This was a personal wall post. I have clarified it on facebook (I wonder if that will get forwarded to you) by saying that I have been feeling a little beat up (which is true) and especially don’t like the implication that I somehow manipulated people into this brief. Council wanted it, I believe most of what is in there (though there were concessions made to appease some Council folks that I would put differently).
And, in the event that you ever know me in person, you will find that I can outswear the bluest trucker potty mouth you ever met. That’s me. And on FB, I am chatting with my friends as myself.
LikeLike
should have said “I apologize *to Laura.* Sorry Laura.”
LikeLike
OW: Perhaps we were cross-posting. I have already apologized to Laura for repeating something in private (if that’s what it was) that was said in public.
[BTW, you are misconstruing her use of the b-word. Think about it]
(Could you or or your colleagues please release a reply to Laura that I repeatedly tried to post yesterday? I think it might be hung up bc it has some links in it. Thanks)
LikeLike
EZ yes we were cross-posting. Re the comment, Jeremy unspammed it but I guess that didn’t “take.” I just approved it.
LikeLike
Thanks Jeremy for releasing the post of my reply to Laura. It is here: https://scatter.wordpress.com/2011/05/19/discrimination-briefly/#comment-12123.
And sorry for filling up the comments. I’m sure many following this will be glad to know that I’ll be offline now for awhile.
LikeLike
Laura (and I guess Shahka): Could you please point out for me where I wrote that you do not know a sufficient amount about causality? I am trying to find it. I can see all sorts of things I have written: sarcastically that Winship was assumed not to know anything about causality, that the brief is incoherent on causality, and so on. But, I honestly cannot find where I wrote something about you knowing or not knowing anything about causality. I apologize if I did, but I’d prefer to know what I am apologizing about. I stand by the claim that the brief is incoherent on the issue, and strangely mute on the dominant perspective on causality, even to the extent that it misrepresents the one piece on causal explanation that it cites (Gangul [sic]). In fact, the Gangl piece cites Winship in its abstract. In its abstract! It then goes on to adopt exactly Winship’s position on the matter, and yet the brief lifts a quotation that even misrepresents [by ommission] the first paragraph of that article.
Shahka: What do you mean by “rhetoric”? [I’ll grant you sarcasm.] Feel free to offer examples of where my thinking and writing rides on rhetoric rather than genuine points of argumentation.
Olderwoman: Do you think that Winship is doing this because he has a personal axe to grind? I think that is what you said. Maybe you should apologize! (Joking here: I don’t really think you need to.) Regrading my musings on pre-1990 Wisconsin, it was mostly food for thought. I do think that women have been discriminated against in sociology in the past, and surely even at Wisconsin. Worse yet, I can think of some prominent sociology departments today, where I would bet that they are discriminated against in every job market cycle. My point is that it is easy to throw stones at Walmart, not so easy at your friends.
Jeremy: Your unwillingness to take a stand sure looks like an abstention! (Joking here too. I think abstentions are fine, and there are times when abstentions are better than ‘no’. Individuals do sometimes need to dial down on obstructionism. The abstention is a way to do that but to let everyone else know that when the shit hits the fan, they won’t be with you. If that is what is going on right now, then I hope those members sufficiently explained what their abstentions meant. If they didn’t, then your criticism of them stands, and I’d agree with it.)
The merits of this case: As I think I have also said at many points, I truly believe that Walmart is discriminatory. The statistics alone are sufficient for me to see a preponderance of evidence, and as a juror or judge I would vote against them. I don’t need the sociology or the anecdote. I take no position on the class action issue, since I don’t know a sufficient amount about how classes are certified. If adopting Bielby’s position is what is required to certify the class, then I think the ASA should have either not submitted a brief or submitted a better one that reflected the true complexity of the issues.
Why I am engaged in this: I think the ASA submitted a flawed and embarrassing brief, which will make sociology less likely to be taken seriously by the court in the future and that enshrines, in a very public document, an impoverished position on valid explanation in sociology. That’s it. I hope the plaintiffs win, whether at the class action level or in some smaller scale way later. Didn’t Laura say that when SCOTUS rules against the class, they will ratchet down the class definition and pursue it at a regional level?
LikeLike
I vented to friends because it is being implied or inferred that I somehow manipulated Council. I am kidding on FB with my friends who know that I am kidding. I feel accused of trying to manipulate something and was joking about it. Come on.
LikeLike
Ezra – I will work on trying to show you how it was an attack on sociology. It will take me a long time. I don’t mean to say “only lawyers can understand and there is no hope for you” – I can try to explain but it really does involve a serious primer on legal argument.
Re excruciating – I am sorry that felt condescending. I meant it is excruciatingly difficult and time-consuming. Especially the idea that this was not an attack on sociology. It was a clear strategy by the defense that was immediately recognized by the plaintiff’s side and discussed ad nauseum in legal blogs, journals, etc. I will round up some of that stuff for you. Adam Liptak had a NYTimes piece about it (in which I was quoted and in our conversations, I referred him to Chris for a picture of the other side).
http://www.nytimes.com/2011/03/28/us/28scotus.html?_r=1&scp=1&sq=%22laura%20beth%20nielsen%22&st=cse
some of the legal work and discussion is cited there.
I think the question is — can you conduct sociological analysis (a case study) using documentary and secondary interview data to inform analysis of whether the company discriminates? I think you can. I think sociologists do it all the time. Please note, I say inform, not determine, if the company discriminated. It would be one part of a larger case in which a fact-finder would make the ultimate determination.
I’m sure there is tons more I have not answered. Sorry. But I gotta get KID to school/
LikeLike
Laura:
Apology accepted. And I would be more than happy to buy you a martini at the ASA too. Maybe I can then get you to tell me who janecausal is. Boy do I wish she felt comfortable revealing her identity. Same with OW and TDS/Jimmy. I understand people have their reasons for wanting to be anonymous, and that many people out there know their offline identities. But in my opinion, it makes a big difference when someone is willing to stake their names on something. And I appreciate Laura putting herself on the line in the way she has.
As for your other points: As I said, I will be eager to read anything you write on why a better understanding of the legal issues does support the idea that the scientific status/legal credibility of sociology was at stake. I’m sure you will understand why I will continue to be skeptical until that point. (The NYT article doesn’t help me, in my opinion. It says that the credibility of a particular sociologist and his opinion was at stake, but says nothing about the scientific status/legal credibility of sociology as a field. The only place where you might read the article as consistent with your interpretation is if you read the title “Sociology issue…: as “Sociology is the issue” or some such. But if you read the text of the article, the more accurate reading is something like “An issue regarding a sociological analysis” or some such. The article is also problematic because it makes it seems like social framework analysis is an approach that the ASA defends when in fact the brief makes it clear that there is no such sociological method. The last thing I’d say is that I tend to agree with Mitchell that the kind of case study that Bielby did would not be regarded by any serious qualitative sociologist as meeting disciplinary standards (I also don’t think anyone uses the term “case study”). I hope that qualitative sociologists will read the Mitchell et al piece, and voice their opinion on this.
(Amazingly, this is an issue that hasn’t been raised in these debates. But if you are a serious qualitative sociologist reading this, I’d highly recommend that you read the Mitchell et al piece (http://generaltospecificinference.wikispaces.com/file/detail/MMW_ReplytoASA_SMRVersion.5.docx) and Laura and colleagues’ reply (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1844550). I expect that some of you will want to weigh in on the question of what kind of qualitative work the ASA should be endorsing as meeting disciplinary standards.)
LikeLike
Okay, now that I have dropped my kid off at school (hat tip to Laura), back to the substance.
In an earlier comment, I noted how an introductory textbook was used to support the claim that Bielby’s analyis of Walmart is accepted scientific work in sociology. This was parried by Shahka as evidence that Bielby’s activity and conclusions are mainstream, though the point of that section is that Bielby’s conclusions are of the sort one finds in peer-reviewed journals of the highest order. Indeed, the immediate prior sentence says so: “the methods Dr. Bielby used are those social scientists rely on in scientific research that is published in top-quality peer-reviewed journals.” Wouldn’t it be better if the brief offered an example where a peer-reviewed publication asserts a causal statement in a case study in an analogous fashion, using general literature to assert that a causal effect exists for a single case that the researcher did not have access to? The problem is that you will have a hard time finding one.
Now, on to the next section. Here we have a quite amazing set of non sequiturs:
“Proper social scientific research does not draw unsupported conclusions about specific cases based on general patterns in aggregate data, but rather formulates testable hypotheses based on existing research. The probability that a given case will conform to predicted patterns varies with the strength of past findings as well as available data on the specific case. Social scientists thus necessarily consider both “general” and “specific” information within a rigorous structure that acknowledges variable certainty. Indeed, the use of the results of prior social science research in analyzing case-study or survey data is an essential component of social scientific research. Dr. Bielby’s unwillingness to draw causal inferences is consistent with sociologists’ prudent reluctance to assert causality in the absence of appropriate methodological safeguards. … It is not true that social scientists cannot rigorously use general data to understand specific cases. The court should look beyond petitioner’s opposition to legally defined “social framework analysis” and to the actual social science methods that inform research.”
Now we have a series of sentences that move from the fallacies of conclusions drawn from deductions from general to specific [which is true] directly to a non sequitur about developing “hypotheses” rather than conclusions. Then, a statement about probabilities is offered, without noting that the claims only follow from an assumption that the probability relation has to be built on an underlying set of assumptions about homogeneity of cases. This is then followed by a [false] laudatory claim that Bielby does not draw a causal conclusion [when, in fact, he merely says that he cannot quantify how strong the causal effect is, although I think this thread has shown clearly that he states that the effect is greater than zero and certainly substantial]. Then, the passage shifts back to what, I believe, is a quite false accusation that Bielby’s critics have claimed that specific cases cannot be understood by considering general data. I don’t recall the petitioners’ brief making such a point, and the use of “understand specific cases” shifts the meaning of the sentence in such a way that it becomes a non sequitur relative to the argument at stake [which is not about “understanding” but rather about drawing conclusions].
Now, I know some of you are now thinking, “But, hey, we can make probabilistic causal statements, and in any case the sentence above was very strategic in making it clear that the probabilities asserted were for ‘undersstanding’ and not asserted as causal anyway.” And “We are sociologists, and we try to develop general explanation that are not case specific, which we then use for all manner of understanding.” That is all true, but we need to be very careful in how we do such things. Humanists beat us up all the time, and for good reason, when we reason from generalities to specific cases. This is a central tenet of the postmodern critique of our reductive positivist ways. There are all manner of aggregate relationships that would be incredibly unethical and unscientific as guides for behavior if used in an individualistic way. For example: It is a known fact that the SAT scores (or their equivalent) are dramatically lower for African-American students than Asian and white students at the US News top 100 undergraduate colleges. Would it be appropriate for sociology professors to regard a specific African-American student, who comes in for office hours to ask a question following a lecture in introductory sociology, as ‘below the college mean’ and treated as needing remedial instruction relative to her peers? Would we develop a special answer so that the African-American student can understand the material that other students are more prepared to accept by virtue of their stronger preparation for college? We can all agree that that would be horrible, since there is a quite large probability in fact that she is well above the mean SAT score in the college, and we work hard to make sure that we don’t behave in a way that treats the student as less capable than her peers of answering her own question or understanding the material. We try hard to see her as equal to her peers in every way, and as someone who walked in to our office to ask a question that is totally unrelated to the fact that she happens to be African-American.
When something lines up with our politics [Walmart is evil for lots of reasons!], we let our guard down. It is easy to do. My position is simply that our general standards for reasoning are then impoverished, and others [like our students] mistakenly infer that such thinking is mainstream and acceptable. This is why the ASA should not write such confused things speaking on our behalf, even though Bielby may be 100% correct that his reasoning nails a true causal effect.
LikeLike
re: the introductory textbook.
A couple things: first, I am not going to say who on council moved us certain directions in the brief. It was a collaborative process with shared drafts.
Second, and more importantly, you have to think about the audience here. I understand that may not be compelling proof to we sociologists, but it is the kind of persuasive authority that courts think is valid. Fundamentally, they are not going to learn sociology. We have to talk to them in ways they understand. And that is done in conjunction with the lawyers about what is persuasive.
You and I might think an ASR article by, say, Chris Winship, is the be all end all of genius method. But the court does not have the whole picture of the field and all of the short-hands that we use to talk to one another, to know who he is, what ASR is, etc. So we tried to make the case in a way that that lawyers and judges recognize — here is that way of thinking (I do not endorse it as my own, but I recognize it because I was trained in it). The chapter article was vetted/chosen by important decisionmakers in the field (which is part of what Daubert requires). The editors of the textbook (and, since it is well-used, the professors who teach the course).
So on that, we are talking about what constitutes convincing evidence to different audiences.
On the hypothesis/conclusion thing – I refer you back to Amy’s post. You might think of Bielby’s report as a report — we are not talking about the standard for peer-review on this REPORT – it is preliminary and would need to be argued about thoroughly. (I remain unclear on why sociologists would object to arguing this in court)
You are right when you say that Bielby’s statement is causal in that he is claiming certain practices are more likely than not to result in discrimination more than zero and less than 100. If that is a causal claim, then yes, he is making a causal claim – and in not quantifying how much, I would (did) argue, that he is doing so in a responsible way.
The legal standard is not causality or 100%. It is whether it is more likely than not (51%) that these practices are (in some part) responsible for the disparities. If he had said 49% or 51% we would all recognize that as ridiculous. The defendant’s lawyers saw the (I think intellectually honest) attempt by Bielby to be cautious and not give a number as meaning it is worthless (should not be considered at all — a demurrer). We are trying to say that the caution does not make it useless, it makes it something that should be considered.
The legal question is whether (it is more likely than not) that discrimination played any part in producing the disparities between men and women at Wal-Mart. That is a causal question. Bielby talks in terms of vulnerabilities in part because the court could require companies to reduce vulnerabilities. And in part to try not to be pinned down to some % of causality that would make us all flinch.
So I get that you don’t think ASA should have endorsed this version of the brief. Do you think sociology should play a role in cases like this? In other words, do you think that, if done well, a case study of the type Bielby did, using the available data is/is not “sociology” or is/is not reliable? And, should courts consider it? I don’t know your area of research, but here at least the court is considering the effects of the kinds of things we have been studying for 50 years or more. I think it is at least worth putting that research before the court.
The brief was trying to say (and we worked very hard) to not reduce Bielby (or sociology) to crude causal claims. We just can’t say that these practices are 22% (or 49% or 85%) likely to result in discrimination. Without that number, the defendants argue that it is not even worth thinking about/considering. I think it is (that is what I meant by giving a crap about the discipline — I think we have a lot to say on a lot of issues in a lot of areas I don’t know anything about. But if we have knowledge that will add to how the world works (which is really what these courts have to decide all the time) WE are the experts on that. We know how to go out and research that. We ARE worth a crap. And there is room for our knowledge to enter these debates. Indeed, I think of it as a imperative. We go out and interview all these people and do research out in the world to help understand the social world. Shouldn’t we give back some of that knowledge when it is needed?
I think of the amicus brief as doing that – trying to translate what we know and how we think about it so the courts can decide about it. That translation is not easy. And we would never get all the ASA members to agree to one version. But, we did our best under the constraints we have (which means assigning portions of this to be written by certain people, cites to be checked by others, and other material working conditions that lead to some misspelled cites and perhaps not the perfect sentence).
SO that may be where we part ways. I think we should try to participate when we can contribute knowledge. And there is a LOT of knowledge in that literature. It doesn’t mean we are vouching for every article we cited or Bielby’s analysis here. It means that there is a body of knowledge worth considering here.
LikeLike
Thank you for explaining.
You have asked “Do you think sociology should play a role in cases like this?” Here is my answer to that, stated within my broad set of positions on this whole situation:
1. I have no problem with Bielby stating his judgment, and I think his role in this case is fine. In fact, I think he is probably correct, in the sense that his judgment is probably correct.
2. Insofar as individual sociologists want to serve the courts and plaintiffs and defendants, they are free to do so. I think they have a role to play, within the adversarial system where their claims can be disputed.
3. I see lots of ways for sociology to get into this, beyond just expert testimony. Research itself is, on its own, getting into it, and I see lots of good research out their on discrimination.
4. I object to this particular brief for all of the reasons stated.
5. I can conceive of an ASA brief on this particular issue that I would have supported, but it would not have taken the incoherent positions on explanation that it has, and it would have tried harder to represent the sort of view that Winship has legitimately brought into similar cases.
6. I think the statistics alone on Walmart are convincing enough that Walmart is guilty, and so I hope the class action goes forward. And, if it doesn’t, I hope the plaintiffs try through a narrower class.
7. And, although I have never said it, I agree with Don Tomaskevic-Devey and you that sociology should be making the case that experts need better access to employers’ data if we want to make genuine progress on these broad societal issues. I could be wrong, but I don’t think the current brief is going to make that any more likely to come to pass.
LikeLike
#7 is the most important part — the P lawyers are working on this through a creative strategy using wage and hour litigation. If you tell me your real name I’ll tell you all about it over a martini at ASA.
LikeLike
Re motives: The idea was not mine, but a sociologist colleague in a private conversation (who has been reading but not participating in this debate) opined that the blog “debate” about the ASA memo is all in the service of the defense in the Wal-Mart and similar cases, who have an interest in challenging the merits of “framework analysis” by showing that it is controversial. I doubt this is true as regards overt intentions but do not know any of the players well enough to have an independent opinion.
So, not questioning JC’s motives but simply the consequences in the textual data before us, we are now treated to the claim that arguing that Wal-Mart is a single business that should be treated as such in a lawsuit is equivalent to assuming that a Black student who walks into our offices is unintelligent. The mind boggles. It is an act of discrimination and prejudice against Wal-Mart to say that it is a single business that can be treated as a coherent legal entity!
If we got that kind of argument in an undergraduate paper, we’d have to work hard to kindly explain why it makes no sense. I teach my students to pick apart those kinds of rhetorical devices designed to delegitimate a reasonable argument by equating it with a stigmatized argument. I am running out of hypothesizes for what could lie behind such phrase work that don’t violate the rules of decorum. Do we really have to treat this kind of stuff as if it were being offered as serious sociological analysis?
Wal-Mart is not a single individual. It is a big complex business. The statistical data demonstrate (according to the documents summarized in this debate) that Wal-Mart’s patterns of promotion are more gender discriminatory than other similar big businesses. So Wal-Mart the specific entity is being accused of being discriminatory on the basis of that data about Wal-Mart. Wal-Mart is trying to get out of that accusation by claiming that it is not a single entity, but a random collection of decision-makers that Wal-Mart as an entity has no control over. Only if Wal-Mart loses this part of the case will it need to counter the statistical data which, on their face, seem quite damning. Note that it is the statistical data about their own practices as a business that Wal-Mart is seeking to avoid addressing.
The anti-ASA-brief commenters seem to be defending the turf that a literature that explains how businesses in general operate (i.e. that they are not random collections of decision-makers making random decisions) should not be applied to any specific business. Because the central claim that businesses are not random collections of decisions is so obviously true [it seems to me], the critics are not addressing the central claim, but are criticizing the details of what was written in the brief to defend the claim. It is my impression that the main motive for this is frustration with ASA rather than a direct concern with the Wal-Mart case. As I have said previously, I do understand the ASA political issues.
LikeLike
Laura: I did not know that the SCOTUS was more likely to be convinced by the contents of textbooks, but I am not an expert on Daubert criteria. I am writing on this only with the desire to have sociology put its best foot forward.
Olderwoman: My motives are nothing more than what I have stated they are. And I don’t at all believe in the equivalence you state, and I am sorry if it reads that way.
LikeLike
Jane,
The degree to which disciplines value textbooks is often a function of how strong the paradigm is and Kuhn used textbooks as an indication of this. Contrast a strong paradigm field like econ where they heavily rely on textbooks and theoretical concepts abstracted from their original authors, to a weak paradigm field like sociology where we seldom use textbooks and tend to teach theory as intellectual history. That is, economists still talk about “rents” but almost none of them still read Ricardo, whereas all sociologists have read at least some Marx even if most of us don’t really care about “commodity fetishism” or “alienation.”
Courts seem to be assuming a strong paradigm model which is why they find textbooks to be convincing evidence. My hunch is that Laura Beth was smart to cite a textbook, given her primary audience.
LikeLike
“I doubt this is true as regards overt intentions but do not know any of the players well enough to have an independent opinion.”
Olderwoman: it seems now that it is your turn to cause offense. Are you truly saying that those who are upset about the ASA amicus brief are subconsciously trying to defend Wal-Mart? That we are stooges for Wal-Mart, and don’t even know it? At least your colleague had the courtesy to think that we know why we’re doing what we’re doing.
janecausal did not at all claim that “It is an act of discrimination and prejudice against Wal-Mart to say that it is a single business that can be treated as a coherent legal entity!” (Hopefully you read your undergraduate papers more carefully before explaining to the students why they are misguided.) In fact janecausal was very explicit about her motives and her beliefs about Wal-Mart. Yet you insist that she cannot possibly be telling the truth, since apparently it is obvious that being opposed to the amicus brief as written necessarily implies being pro Wal-Mart.
Perhaps not filing the amicus brief would have resulted in a win for Wal-Mart on this issue. But from that outcome it does not follow that those who oppose filing the brief hope that Wal-Mart will win.
Perhaps we should allow for the possibility that people know what they’re saying, and why they’re saying it. I wouldn’t be surprised if almost everyone who is upset by the amicus brief would stipulate to the claim that Wal-Mart discriminates against women; janecausal does so explicitly. But the fact that Wal-Mart discriminates does in no way mean that it is logically necessary that the ASA file an amicus brief; nor does it follow that the ASA should file the amicus brief that it did file (as opposed to a superior one as described elsewhere by janecausal); and nor does it follow that the ASA should support Bielby’s original testimony.
People who agree about Wal-Mart’s discrimination can have an honest debate about those three questions. And notice that in such a debate, there is no need to claim that some people are secretly pro-Wal-Mart. And doing so is costly, because if we retreat to the comfort of the idea that people are either pro- or anti-Wal-Mart, then we don’t actually have to engage with the hard questions raised by the amicus brief, Winship’s post, etc.
LikeLike
I quoted a person who does believe that this discussion will have the consequence of helping Wal-Mart. I have no qualifications for assessing this causal claim. I also said that this person asserted intentionality; I said I doubted that claim. I can see where you could take offense for quoting that other person and implicitly raising the issue, even though I myself said I did not believe it. I think most people have animus against ASA for which this brief has become a symbol and don’t really care where the chips fall with respect to the Wal-Mart case. In fact, quite a few (like JaneC) claim to want the case to win.
JaneC did in fact bring up an extended example about a hypothetical Black student in a comment about the Wal-Mart case. If that hypothetical example was not supposed to be relevant to the Wal-Mart case, it is not clear why it was in the comment at all. It was a sociologically mindless example in this context and I called it out. If JaneC is making extended blog comments claiming to find bad sociology in a court brief, calling attention to JaneC’s illogic seems entirely appropriate. If you think the example was relevant to the Wal-Mart case, then I suppose I call you out too. Any characterization of Wal-Mart as a victim of discrimination or prejudice is theoretically and substantively off-base and deserves criticism.
It seems to me that in THIS space, the anti-ASA people have become aggressive and argumentative. They are willing and able to write long diatribes challenging the professional competence and motives of sociologists who wrote a document they disagree with and become extremely defensive when people challenge the core assumptions they are operating under, even to the point of trying to defend indefensible arguments. I have never disputed that the core energy is anger with ASA, that seems obvious.
LikeLike
Thanks to olderwoman and disgruntledsociologist on fighting this out!
I don’t think my example was a cheap shot. It was a statement about inferring an individual conclusion from a general relationship. I don’t believe something silly like ‘Bielby is discriminating against Walmart,’ as one might get from an unfortunate UW undergrad. I think the ASA made a general statement about inferring individual conclusions from general relations, and I picked an example that I thought would make it clear why we don’t in general do this, just to make the point that we can’t possibly see this as a valid general claim on the part of ASA. Kind of a ‘speeding train with a baby on the track’ example from philosophy, though using real data. Clearly, it touched a raw nerve. Olderwoman has “called me out,” and I guess feels good about that, and I can live with that. I certainly didn’t mean to offend anyone at all, and I stand by the point that the example makes. I suppose it is nice to give others an opportunity to score some political points with their likeminded colleagues, and so olderwoman can thank me for providing such a nice opportunity. Others can judge for themselves whose side sound reasoning is on.
I feel I have said my piece on this entire issue, and so I will now happily sign off for good on this. My final act will be to reveal my identity to Laura and buy her that martini she deserves (see above). I can’t say when I’ll do this, but one day I promise I will, though feel like I should let things cool down for now.
LikeLike
OW:
Earlier on in this debate, you said that you hadn’t actually spent any time reading the briefs and other materials (Supreme Court oral arguments), and were reluctant to wade in. But that reluctance has disappeared. Pray tell, does this mean that you have now read those materials and it is on this basis, that you feel comfortable asserting that those who criticize the brief and the process behind it are making “indefensible arguments.” See, from where I sit, there is nothing in your recent comments that suggests that you have actually done this homework. But I’m sure you wouldn’t say what you’re saying if you had. So would you please do me the favor of explaining
why you apparently think that my analysis of the WalMart brief was wrong (see here: http://orgtheory.wordpress.com/2011/05/18/walmart-and-the-asa-a-guest-post-by-chris-winship/#comment-96034). Laura herself has admitted that she has “no answer” for my analysis because of the legal intricacies involved, though she hopes to have the time to prepare one. But yet you apparently don’t need the help that I do. Since you’re not a lawyer, perhap you can explain why my criticism of the brief was indefensible. It would be great if you’d also tell me why you think why it is OK that Hillsman’s report on the amicus brief to ASA members misquoted from the Wal-Mart brief in a way that made it sound like sociology was being attacked.
And while you’re at it, I’d love to hear you back up your charge that I (or others) are “anti-ASA”, that we wrote “diatribes challenging the professional competence and motives of sociologists,” and that we have been “extremely defensive when people challenge the core assumptions they are operating under.” From where I sit, none of those accusations stand up to scrutiny. (Hmm… but right now I am indeed being extremely defensive. And maybe it is because someone is indirectly challenging a core assumption of mine– i.e., that a senior colleague of mine would not make such wild accusations, especially in a public forum. So maybe you’re right about that one).
LikeLike
Someone I might or might not know allegedly told someone I do not know that I made a “snide remarker [sic]” on a Facebook wall post. That’s true, though I thought it was really “sarcastic” (which seems like a positive distinction to me, but I could be wrong). I definitely didn’t mean my remark to be private, since it was made in the presence of 993 friends and friends of friends, most of whom I don’t know. But I can’t repeat it without compromising the confidentiality rights of the Wall owner.
LikeLike
This comment is not a rebuttal to anybody, it is a thought that I believe might advance general understanding of the underlying issues in the court case, not the ASA case, although it might indirectly reflect on the debate about the ASA brief. LBN or others can correct me if I’m wrong on this point. The legal understanding of “cause” is “blame”, which is very different from the sociological understanding of causation. Under some circumstances, people can be blamed for causing events they did not intend. The example that comes to mind is the attractive nuisance. I can be sued if a child drowns in a pool I have left unfenced, even though nobody thinks my purpose in building the pool was to drown children and even if I put a big sign up that tells children to stay away from my pool. There is widespread legal understanding that I am responsible for not creating a condition which leads to children drowning, even though such accidents always involve the additional culpability of other people and are never caused just by me.
What I’m getting from the discussions is that “framework analysis” is a legal concept similar to the idea of an attractive nuisance, and is an argument about why it is not enough for a company to issue a non-discrimination statement, but needs to do more of a corporate equivalent to building a fence. To pursue the fence analogy, someone might climb my fence and drown in my pool anyway, but if I’ve taken the reasonable step of building a fence, my culpability is no longer in question. The Wal-Mart court case is about what kind of “fence” a corporation is responsible for building. OR so I’m thinking based on what LBN has said.
The sociological debates about how you construct counterfactuals and establish causation are really irrelevant to the idea of an attractive nuisance. And perhaps irrelevant to the idea of framework analysis. Or so I’m thinking.
LikeLike