Bonuses for Teaching Evals
Texas A&M is going to offer $10k bonuses to profs who get good teaching evals?
Interesting.
The blogospheric reaction has been (strongly) negative, but I think that this is not a terrible idea. There are a lot of small things that profs can do to make classes better. It's just that many profs are busy enough not to think about (or do) those things most of the time. I've been guilty of this, myself, some semesters when I've been overextended. It's easy to let class prep slip, to not update lectures, to hold fewer office hours and to phone it in. It's easy to demonstrate your lack of effort or caring to the students. And they get it.
So some incentive to do better, great.
I would be interested in the thoughts of others on this. I'm assuming TMcD won't like it?
Update: If you're not reading the comments, then you're missing the insights of some of America's brightest minds.
posted by Number Three @ 8:04 AM
17 Comments:
I, too, am guessing that TMcD won't like it.
I don't like it either.
It's not that I'm opposed to the idea of giving incentives to teach better, to think carefully about teaching, to devote more time to it at R1 Universities. I'm all for all of that. But all the evidence we have suggests that teaching evaluations given out within a US tenure-track college/university system are TERRIBLE measures of the quality of teaching. Instead, they are excellent measures of the teacher's gender/race/age, and they are outstanding causal agents for grade inflation.
I need to go read the link, but is TA&M doing anything to deal with balancing out student evals with the grades received in the class?
If I was both running a university and really into statistics, I'd try to come up with some sort of index that shows how much the students like you within the context of the grades they get (not individual grades, but distribution in the class, and not absolute grades, but relative to the university averages).
Still, it should be interesting to see how this experiment works at TA&M and what effect it has elsewhere in academe
You know me. You really, really know me.
I wish I had something smart to add, but I think fronesis has said it all quite eloquently. I just don't trust those things for all the reasons fro mentions, especially the grade distribution issue. I've always thought that evals should be paired with grade distribution data. But at least here, I've never even seen that data. I assume I'm a pretty tough grader myself--and I've got some anecdotal evidence to support that idea--so I've got a stake in this, but it still would serve as useful info. Maybe fancier universities hand that info out, but I don't know.
That said, the TAM reform would probably help me personally, since my evals have always been good (although who the hell knows what the averages are for those things). Did I tell you guys that I won the university's Teacher of the Year back at opening ceremonies this fall? How? Evals. Actually, robo-evals with all kinds of specific and even bizarro Qs. Good thing they didn't actually visit a class! Maybe I should look to cash in.
tmcd: Congratulations!!!
(I've never heard of any college/university providing the type of data that we'd like to see. I think it would embarrass too many faculty.)
Good points. I would like a great deal of additional information on fro's points, especially. Are there studies that show that gender/age/race are correlated with evaluation scores? Interesting. As a hip (youngish) white dude, I'm totally oblivious.
On specific points:
(1) I think that the bonuses should account for class size. In my experience, class size is strongly correlated with evaluation scores (smaller class, better scores). But I generally receive high marks in large classes, which I think reflects my mad skilz.
(2) Grade inflation may be a problem. But I've decided that grade inflation is not MY PROBLEM.
Grading is essentially sorting students for the benefit of corporate America. Students with high grades are more desirable, students with lower grades, less so. I've decided that it's not my job to sort students for employers.
Grades can be incentives for students. True that. But here's a hypo based on real life. Last semester I had two students in Con Law . . .
Student A already knew a great deal about Con Law and thus learned very little in the class. Student A's work was excellent from the start.
Student B had little knowledge or background in the subject at the beginning of the semester. First paper was not very good. But student B met with me, took my feedback into account, and wrote a much improved second paper. Third paper, after additional feedback, even better. Still not as good as student A's, but definitely learned a great deal. Final exam demonstrated the mastery of the subject that I expected.
Now student A clearly gets an A, because student A's work was excellent. But what grade does student B get? S/he learned a great deal and developed a range of new skills; student A didn't do that. S/he worked very hard. Student A certainly worked hard, but I don't think he sweated the assignments in quite the same way as student B.
Student B had a bad grade on the first paper, and a mediocre grade on the second. So if you simply average the scores, she gets a B.
Does that mean that the student who learned more in the class--the student for whom the class made the bigger difference--gets a lower mark than the student for whom the class was not particularly challenging?
That almost seems like punishing the student who needed the class.
Almost. It is!
I don't see it as my responsibility to punish learning. And that's what strict grading tends to do. I will gladly punish slacking. But I find it hard to punish the students who have to work (i.e., learn) because they don't know it all on day one.
Especially when the actual beneficiaries of grading are employers and their middlemen, law schools. In my case, law schools. I don't feel like sorting students into "elite," "very good," "good," and "not so good" law school cadres. Let the LSAT do that.
BUT, and this is the but, I don't think grades are that correlated with student evaluations. I think that if you're an awful teacher, you can't get GREAT evaluations by being an easy grader. And vice versa, if you're a GREAT teacher who gives hard grades, I don't think students punish the hard grades at eval time. In the mushy middle, my guess is that there may be a relationship--mediocre teachers can probably increase evals by signaling easy grades at the end of the semester. But I'd be surprised if the numbers showed that they could push their evaluations into the bonus money.
MORE COMMENTS PLEASE!
On race/gender/age, I don't have the studies in front of me, but I remember reading that the bias works in this way, from best evals to worst:
1) young men
2) older men
3) older women
4) young women
That jibes with my own limited, anecdotal view too, and I think it reflects the problems students often have with female authority. Now I don't think that's determinative of the final scores--just a persistent factor.
As for your dilemma, you are the prof. You are responsible for giving grades that reflect performance, regardless of what you think about who will use that information and how (what, are you resisting "the man"? fighting injustice? And don't tell me that LSATs are a great measure, either--they're often worse than grades). Where I am, law school's NOT the only purpose--simple graduation often is. Once you stop taking grading seriously for the "elite" students (b/w whom it IS possible to distinguish) you slowly lose the rationale for grading anyone. And then you become a joke.
Your specific scenario is not that tough. Just make the call. I always give myself the freedom to bump a student up to the next 1/3 letter grade when they've shown the kind of progress you describe. Put that together with later assignments counting more than early assignments, and that student is usually going to come out pretty well.
On grade inflation, I think you're partly right: hard grading doesn't necessarily hurt you on evals. But it can. And profs overwhelmingly think it matters to their evals. If you're a borderline teacher, and you want a boost for performance bonus purposes, what's the quickest way up? Work to improve your teaching, or go Santa? Door #2! That then puts pressure on everyone else to follow along, lest they lose out to Prof. Lame-O. So this is as much about professor psychology as about student perceptions.
For my own part, hard grading hasn't generally hurt my evals. But the one class (at VU) that did once give me weak scores was clearly, from comments, obsessed over hard grading. And whenever I have a student slag me in comments, they always let slip that they're pissed off that I gave a super-genius such as them a poor grade. If that's a generalized experience, grade inflation is a natural result of eval emphasis.
That's a lot of vitriol from a teacher of the year. I stand by my statement that strict grading is sorting for corporate America. If the LSAT is no better than grades, at least it's not ME doing it. And that matters.
Wow, things are hopping over here today. A couple of things:
1. tmcd has the order (and the theory behind it as well, I think) correct. There have been many studies that show this.
2. I don't grade strictly for corporate America. I grade strictly because part of being a good teacher, I think, is challenging your students. It is VERY hard to get a flat A in my class (not so hard to get an A-). I also grade strictly because I think rubbish work should get a rubbish grade. And there are VERY FEW students out there who will work hard simply because they want to learn. They will almost always slide to the minimum level of work required to get the grade they think they want. I think it's my job to not let them be comfortable, to make them work for whatever grade they earn - and often they end up getting a higher grade than the minimum they desire, precisely because they are more afraid in my classes that they'll get a lower grade.
3. It's obviously true that giving high grades is not sufficient for getting good evals. But that's not the point. The point is: how much HARDER is it to get good evals if you don't give high grades. I think it's MUCH harder. If I have to judge teaching just by the numbers (and this is just an assumption we are working with in this thread, because we know this is a lousy way to judge teaching), then I would definitely favour the person who gets, say, a 4.1 (out of 5) on their eval and gives out a median grade of B- over the person who gets a 4.5 but gives out a median grade of B+. I think it's also true that students aren't very likely to complain or give out very low evals to the prof that gives all A's. No, they probably won't respect this person and they won't nominate them for teacher of the year, but they also won't hammer them. But...bring in a 27 year-old woman fresh from grad school, who is really rigorous, and you may have a recipe for eval disaster. And I think that *only* a great teacher can get away with being a really hard grader, and this means that everyone else is going to inflate just to cover their ass, and this puts more pressure on the hard graders to inflate as well...etc. I usually grade harder than many of my colleagues, but I also think I grade easier than I would do if I weren't worried about getting that one set of evals that spells tenure disaster.
3, if I expressed a bit of vitriol, it was for the professors who don't take grading seriously as part of the job--as you seem to be doing. Yes, it gets under my skin. I've had numerous frustrating discussions with young faculty that have gone along these lines: "Oh, who am I to judge their work? I really like my students, and it's just tough to have them work hard and not feel good about themselves at the end of semester. The law school admissions are just a racket anyway!, etc. etc." (Actually, my first such encounter was with a young prof in grad school--you may remember her--who had the same attitude.)
Excuse me, but this is a cop out, even cowardice. Teaching is neither a popularity contest nor a self-esteem seminar. And you are an expert or you wouldn't (or shouldn't) be in front of a classroom. Suck it up and evaluate their work. Don't be a Nazi about it, make allowances for improvement, intellectual spark, what have you, but don't just abdicate. You're not doing anyone any favors. And all that crap about corporate America is just bullshit rationalization for not doing your job.
Once again, fro has hit the nail on the head. We grade b/c students need us to, whether they like it (or us) or not. It pushes them to work harder, and better. Even if the grade in one class is an unfair measure of their promise, it is just ONE class--put them all together and you get a pretty decent picture of performance. At least, that is, if we profs are all doing our jobs.
It is true that grades can, theoretically, provide extrinsic motivation for students to learn. I.e., we think that we're inducing them to learn b/c of the threat of sanctions. I am skeptical of this. Students will definitely respond to the threat of sanctions in a number of ways (including cheating and grade grubbing), but whether they actually learn anything (that they retain for any length of time) as a result is an open question.
I still don't grasp why I care about providing "a pretty decent picture of performance" through grades. I really don't care about "performance." That sounds like sorting for corporate America to me. I don't think that's bullshit. Face up to the fact that you are part of the System and you play an important role in its recruitment processes. The more seriously you take grading, the more important a role you play.
I care that my students LEARN something and develop skills, but grades are an awful way to measure LEARNING.
Grading is tertiary, at best, to my job as a teacher. Primary is my role in facilitating student learning. I have to facilitate it, because they have to do the work. I can talk, but they have to think. That is very hard to do. Grades may help, at the margins. Secondary is my role in challenging them against complacency and prior knowledge. Grades don't help with that at all. Only attitude and approach can do that.
Finally, the university requires that I assign a grade to students based on . . . ? It is rarely, if ever, explained, what the grades are supposed to mean. And don't reference the faculty or student handbook. Those descriptions are worthless.
Late to the party, my excuse being that I am, er, half way around the world. A couple of things:
1. I thought it was old men and young women together at the bottom of the ranking for evals.
2. I've taught in departments with art studio faculty, and been a part of assigning grades for students' creative work. In many ways this is the litmus test for the attitude of: oh, they tried real hard, and I like them a lot, and their work is good--who am I to judge? isn't it all subjective anyway? The answer? No. it's not subjective in the least. Not even when you're dealing with the creative arts and assigning a grade to a painting. It is part of a conversation that you have with the student(s) involving wider feedback and the like, and in that sense I think that grades play a hugely important role for helping students to get to the next level in their thinking/creating. But it's certainly an artform, the use of grades within pedagogy. To think otherwise is to misunderstand grades, or to use grades as marks in the British context: solely as an end-of-term assessment of the student's knowledge.
My additional 2 rupees, to add to the excellent points already made. Oh, and I want my 10K, please?
A few quick thoughts:
--The texas a&m idea seems like a marketing idea more then anything else. "Hey, we care about teaching! Parents, send your kid here!" With the "12th man" tradition dying out, a&m probably needed a new gimmick.
--Paying for performance is a good idea, particularly in elementary and secondary schools, but evaluations are a bad way to do it, for the reasons stated above.
--What we should pay for is learning, or improvement in knowledge/skills. Using Number Three's example, the professor should get a performance bonus for student B, but not one for A (at least as those cases are described).
--Universities certainly sort people for corporate America (that's the main societal function, although not the most valuable). But they do it inherently . . .I'm not sure grades change the equation much.
--Grading does serve to motivate students, if not in every case, at least overall.
--given the insane class stratification of our society, preserving the value of at least some grades at the top is one of the few ways to help those who don't come from privilege.
--But, then again, if student evaluations of professors are biased, why wouldn't the reverse be true as well? Not in every case, of course, but at least in many cases and on average. (it would be very interesting to compares those biases to those in corporate america. I suspect they are close . . . people like those who: are similar to them; are attractive; can get of give them stuff, can advance their career, etc.
Nice points from Tekne and DK, the latter of whom preempted some of what I was going to say about the "corporate" charge here. Sure, companies look for smart graduates with good grades. What, exactly, is wrong with that? It certainly seems better than the old system of family privilege and country club connections. Whatever ills corporations may cause, hiring college graduates with records of achievement is not one of them.
3, a question for you: if grades don't matter, why not just give everyone in your class a C, making them all "average"? OK then, how about, in the age of inflation, all Bs? You could just announce it on Day 1 as a way of "taking the pressure off." Plus, you'd weed out all those students who were in it for the "wrong" reasons. Then you'd have your little utopian community of pure learners. I expect, of course, that what you'd really get are a lot of bad students who just want to goof off, knowing that they can't possibly fail. But why not try it? If grades don't matter, the grade that you give everyone shouldn't matter either, should it? A good old fashioned team-C might even get them off the corporate track altogether and for the good of mankind.
Can you give me a good reason why a professor supposedly "indifferent" to grades would want to only give high grades? I can't come up with anything other than sucking up (or down, maybe, in this case). On the other hand, I'd argue that a prof who gives only high grades cheapens the A for the rest of us, since all the hard work a student puts in to get an A from me can easily be replaced by the lazy A from someone else. And aren't our young consumers likely to buy the product/grade they can get at cheaper cost? Ironically, you're the one playing into the market logic by encouraging students to comparison shop on perverse incentives.
My position on many of these issues may not be completely clear. Here's one of my students from www.ratemyprofssor.com:
"Excellent. I learned SO MUCH. He has subtle way of challenging you and making you think at a level you never thought possible. Hilarious. If you aren't willing to work your butt off, you'll think he doesn't care about you (he will never tell you the answer directly). It is only when you REALLY work hard that you see how great he is. One of the best"
My students, my better students, work hard. I challenge them and, apparently, make them think at an impossible level (!!). I don't tell them the answers because, honestly, I don't think that there are answers. Only arguments. And I am, as even TMcD must admit, I am hilarious.
But I don't fetishize the grade book. Tekne's right--grading in a very real sense is not subjective. But one can certainly put too much emphasis on "measures of performance."
And I think DK is right that this is largely a marketing play by TAMU. And, I think, a brilliant one.
Chronicle of Higher Education: here
I'm way late, but I wish to offer a student's point of view, I'll offer one. In the interest of full disclosure, I'm a former student of tenaciousmcd.
One summer, I was so bored with my classes that I went to mcd's office and, knowing I was signed up to take his class in the fall, begged him to make it an intellectually challenging one. "You mean you want me to kick your ass?" he said. "Yes," I said. One of my test questions in the summer class was "Who broke the color barrier in major league baseball in 1947?" I've only known that since 4th grade, folks.
More full disclosure: I fall into the "older women" demographic--virtually decrepit, in fact. So I guess I'm a nontraditional student in more ways than one.
I'm also a university employee. My public relations work for the university brought me into contact with the people charged with revising the forms that students use to evaluate teachers. The prior instrument was really inane and stupid and allowed no room for nuance. Some questions were not even applicable, depending on what kind of class the student was taking. They switched it to an instrument developed at Berkeley. There are many more questions on it, 33 in fact. Some make sense to me ('encourages class discussion"); some don't ("is a dynamic and energetic person"--I tend to like that personality type, but suppose some other student responds better to a laid-back professor?)
Anyway, the professors working on the change told me the faculty who are new and young and the professors who are near retirement are fine with the change, but the ones in the middle are nervous about it--so much so that the "change committee" (for lack of a better term) wanted me NOT to spread the word that chairs and deans could use the results of evaluations to suggest and/or order professors to go to workshops to "improve themselves."
On a related subject, what does Number Three think about the new superintendent of D.C. public schools and her plan? From what I heard on NPR, it sounds like signing away tenure in hopes of working up to a six-figure salary is sort of a Faustian bargain without a clear understanding of what she means by "academic excellence."
way way way late to the game. fronesis pointed me here after i blogged about this issue yesterday. Certainly grade inflation can be a concern, but where I am, students do the evaluations before finals week and thus before final grades are given out. Yes, if they've been beaten down by bad grades throughout the semester, they may reflect that in their evaluations, but they will ideally not have a direct connection between the grade they received in class and the evaluation of the instructor.
What I'm more concerned about is how this affects pedagogy in the classroom. For example, at one place I taught we had this horrible question about "challenge" on our evaluations. It was horrible because it was poorly phrased, because for some students, a challenging class is good, while for others it isn't. Anyway, I'm trying to draw a line in the sand here against edutainment, and to me, this might provide an incentive for faculty to make students happy instead of making them learn (not that the two are mutually exclusive of course, because we had some serious fun with Samuel Johnson today, but that's not hard to do).
Post a Comment
<< Home