perverse incentives in grading – exhibit a

A student forwarded me this email last week, which s/he received unsolicited from MyEdu, a company that specializes in helping students exploit grade inequality between departments and instructors:

res ipsa loquitur.

Author: andrewperrin

University of North Carolina, Chapel Hill

13 thoughts on “perverse incentives in grading – exhibit a”

  1. I think it’s fair enough that students would want to take classes in which they are more likely to get higher grades. What I have always found bizarre, and much more problematic than students shopping around for classes based on grades, is the simple fact of professors being able to decide for themselves how many students will get what grade. I’m amazed, given grade inflation concerns, the decision isn’t just taken out of their hands by a blanket rule across the university that every course is curved and every class can only give out a certain percentage of each grade: 10% of the class getting an ‘A’ or whatever. So long as a given class is not exceptionally small, then this system should be fair – and certainly would be fair for most classes in public schools, since they’ll have 150 students or whatever.

    Like

    1. regardless of the existence of such a blanket rule on a formal curve, instructors decide how hard their class is by setting policies & assessments how they see fit. the problem of grade inflation would have to be solved by a substantial discussion of pedagogy and learning objectives…

      Like

    2. There is vast heterogeneity across disciplines and instructors as to what grades mean and whether such curves are intellectually defensible. That’s probably a conversation we (meaning The Academy) should have, but it’s not likely to happen in the near future.

      Like

  2. One thing this doesn’t take into account is that the two professors may very well be teaching different groups of students, and *that* may be the cause of the disparity. Say that there is an engineering prereq at the same time as one professor’s course, such that one professor gets all the engineering students, and the other gets the math majors. Or say that students who register early (and get the “good” time slot) are better students than those who register late and get stuck with a class that meets late on Friday afternoon. Many professors have had the experience that the same course will have very different distributions from semester to semester.

    That is also the problem with curves. Even if 15% of students overall deserve As, there is no reason to believe your course is a random sample of all students at your school.

    Like

    1. This is a great point (along with OW’s below). Andrew – does the new UNC grading system that calculates the median/distribution of grades account for some component of time over which courses are aggregated. For a class like Calculus 101, this is unlikely to be much of a problem since there are likely many sections. But for a smaller class (maybe Soc Theory at the 300 or 400 level), there might only be one or two sections which would lead to a high degree of (potential) heterogeneity.

      FWIW, I think that this is a great way to teach students about sampling distributions and potential problems with samples. The overall grade distribution is broken up into smaller samples (sections) that may be random or may be completely dependent on different selection probabilities (i.e., Andorinha’s and OW’s comments).

      Like

  3. The interesting thing is that you could easily get two different distributions like this from two sections of the same class that take common exams. But notice that in this case, you make exactly the opposite attribution. Instead of thinking that the prof with the higher grade distribution is a pushover who is grading too easy, you think that the prof with the higher distribution must be a better teacher, if his/her students do better on common exams. Or you back up and realize that you could have had a different mix of students in the two sections. But all of this should give you pause when you make attributions to a prof based on grade distributions, in the absence of further information.

    Like

  4. Still, for a class of a big enough size (20+? 30+?), I would happily conform to some preset grade distribution. I think that I might find conforming to such a distribution easier than A) giving out “too many” Cs without such a requirement and risking course evaluations that condemn me as the “mean new soc prof”; or B) antagonizing over whether my expectations are too low and that I’m “giving out” too many unearned As. The former was my concern in fall term, and now I have the latter concern.

    Like

  5. now that we’re in full on outcomes assessment regime, i wonder if grade deflation will take hold. i started using a written rubric this term (before it was just in my head) and found my grades dropped through the floor — hardly an A’s, tons of C-, D, and F’s.

    Like

  6. syed1ali: I like written rubrics a lot. I also provide myself (and the students) qualitative descriptions of each quality level. That helps a lot. Then you just have to decide what letter grade to peg to each quality level. I prefer absolute scales and detest forced curves. I prefer to tell a class that I am prepared to give them all As and prepared to give them all Cs — nobody will get an A just because everyone else in the class is worse and nobody will get an undeserved bad grade just because they happen to be in a class full of good students. But you can only resist grade inflation with absolute scales if you give hard enough tests/assignments and use high enough standards for an A that you can observe the difference between work that is ok and work that is really good.

    cperchesk: 30 is too small to force a curve onto. You still have a significant probability of a “good” class or a “bad” class or a bimodal class (some good, some bad, nobody in the middle). Anything smaller than that is more likely than not to exhibit substantial between-class variations in the mix of students. I personally would not be comfortable forcing a curve on a class smaller than 100. And even in that case, I would think it was immoral to force some people into low grades just because of a curve if students are allowed to drop the course after the midterm.

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s